Technology, Food, Life

Our Network Engineers at work have been trying to troubleshoot a site-to-site VPN connectivity issue with a remote office for a few weeks. They need to figure out whether or not the connection issues are on our provider’s side or not. As a result, I’d configured our Smokeping installation to monitor the firewall and switch at our remote office, alerting us when we saw a loss of packets. Our network guys asked if smokeping had a trouceroute utility built-in. I replied that it did not.

After some research, I realized that you can configure Smokeping alerts to run a script. I created a Python script that runs an MTR report against the offending host and emails the results.

Here are the instructions for installation and configuration:

MTR Installation

Download and install MTR per their README.

Script Installation and Configuration

Download the mtrtrace.py script (right click and save link as)

Copy it to a location of your choice. I stuck it in my smokeping/bin directory:

$ scp mtrtrace.py user@host:/path/to/smokeping/bin

SSH to your smokeping server:

$ ssh user@host
$ cd /path/to/smokeping/bin
$ chmod 755 mtrtrace.py

Edit the email addresse(s) and append more as needed:

# List of email address to send report to
emailList = ['user1@mydomain.com', 'user2@mydomain.com']

Edit the number of seconds the MTR trace runs (leave blank for default of 120 seconds).

# Number of seconds to run MTR report, default is 120
reportSeconds = ""

Smokeping Configuration

Edit your somkeping configuration at:

/path/to/smokeping/etc/config

If you don’t have an *** Alerts *** section declared, add it to your config. My alerts section lives below my *** General *** section.

Configure it with an alert that fits your needs:

*** Alerts ***
to = user@mydomain.com
from = smokeping@mydomain.com

+lossdetect
 type = loss
 # in percent
 pattern = ==0%,==0%,>20%,>20%,>20%
 comment = suddenly there is packet loss

I don’t configure the script to fire in my original alert definition. I prefer to configure the script at the node definition level so I get the report for the hosts I care about.

Make sure the alert is active in your *** Targets *** section:

*** Targets ***
alerts = lossdetect

Here’s how to configure the script for your node(s):

+ REMOTE-OFFICE
menu= REMOTE-OFFICE
title= Remote Office
alertee=|/path/to/smokeping/bin/mtrtrace.py

++ FW1
menu= FW1
title= FW1-xxx.xxx.xxx.xxx
host= xxx.xxx.xxx.xxx

++ SW1
menu= SW1
title= SW1-xxx.xxx.xxx.xxx
host= xxx.xxx.xxx.xxx

Make sure there is no space between the ‘=’ and the ‘|’ in the ‘alertee’ section. If there is, it will be interpreted as:

' /path/to/smokeping/bin/mtrtrace.py'

instead of:

'/path/to/smokeping/bin/mtrtrace.py'

This will throw an error in Smokeping and your script won’t run.

I also configure the script as an ‘alertee’ for each definition I want the report to run on.

That’s it! The script will be run when loss is detected according to your alert pattern.