While the setup we described in these article worked out great, there were a few things that we didn’t really like about the setup:
The reason why we didn’t like to have the Cacti server connecting to the servers was simply that we needed one more user account on the remote servers. If these are production servers, it’s desirable to keep the publicly accessible user accounts to a minimum.
As it turned out, replacing the ‘-L’ with a ‘-R’ in the tunneling command turns the tunnel around. Instead of opening a port on the local machine, it opens a forwarded port on the remote server. By doing this, we can connect from the remote server to the Cacti server and still fulfilling the same purpose (but without creating an additional user on the remote server).
One of the major problems we were having with the setup was that the tunnels died for one reason or another. We initially solved this by writing a bash-loop that automatically reloaded the tunnel if that occurred. However, we were still experiencing some problems with dead tunnels.
The solution to the problem was autossh, a simple front-end to SSH that keeps the tunnel alive.
With autossh, we could simply launch the tunnels at boot-time on the remote server (in rc.local) without having to worrying about them dying. As we were implementing this on a number of servers, we wrote a small bash-script that launches autossh with the server-specific settings. The script looks like this:
#!/bin/sh
REMOTEPORT=2001
MONITORPORT=29001
/usr/bin/autossh -M $MONITORPORT -q -f -N -R
127.0.0.1:$REMOTEPORT:127.0.0.1:161 [email protected]
This script creates a tunnel on port 2001 ($REMOTEPORT) on the Cacti host (cacti.server.tld) that goes to port 161 on the local machine (in this case, the server we want to monitor).
In order to make our monitoring both more secure and portable, we felt that we wanted to isolate the monitoring to a separate Virtual Machine. This was easily done by creating a new VM under VMware Server. If you’re lazy, there are ready-to-use VMware images to download on the Cacti forum.
While reading the Cacti forum the other day, I ran across this article that talks about SNMP Proxies. By adding an extra entry in the SNMP config file, it’s possible for a single host to relay SNMP information about the other hosts on the network. This is very useful if you’re trying to monitor more than one host on the same network, as you don’t need one tunnel per server (beware that this creates a single-point-of-failure though).
After implementing these changes, we feel much more comfortable with our Cacti setup. Not only is it more reliable with more robust tunnels, but it’s also more secure.
The next Cacti-related task we will be looking at is to design custom plug-ins to monitor our own apps.
]]>