Into.the.Void.

On my previous port about netroute2 I said that it was really easy to grab some stats with mrtg about line activity. This has a small drawback, you must have your collector/server that will run mrtg always open so that it can constantly get the snmp statistics from the modem/router. So my quest was to figure out a way to keep the statistics on the modem and be able to get them when I want and create the graphs that I like. This is not something mrtg can do, this is an rrdtool job 🙂

Most things said here are based on my previous post about netroute2: Intracom netroute2 hacks. You will definitely need to have read it before continuing…

Tweaking the Netroute2:
The first thing was to create a script to give me information about incoming and outgoing traffic on ppp0 interface. I also to export these stats on a file “parsable” by rrdtool. The following script does exactly that:


#!/bin/sh
ifconfig="/sbin/ifconfig"
grep="/bin/grep"
date="/bin/date"
cut="/usr/bin/cut"
tail="/usr/bin/tail"
delim=":"
OUTPUT="/tmp/var/for_rrd.log"
RX=`$ifconfig ppp0 | $grep -A 6 ppp0 | $tail -n 1 | $cut -d":" -f2 | $cut -d" " -f1`
TX=`$ifconfig ppp0 | $grep -A 6 ppp0 | $tail -n 1 | $cut -d":" -f3 | $cut -d" " -f1`
echo `$date +%s`"$delim$RX$delim$TX" >> $OUTPUT


What this script does is parse the output of ifconfig ppp0, find the number of bytes for incoming and outgoing and store them to /tmp/var/for_rrd.log. Remember that netroute2 has /tmp writable, so we can use/abuse it 🙂
The output of the script is something like this:

TIMESTAMP:OUTGOING:INCOMING

1160651881:7938452:1310405

So now we can have the stats stored someplace. It’s time to tell netroute2’s cron to do it every X minutes.

# cat /etc/cron.d/cron_rrdscript
*/1 * * * * root /etc/rrdscript.sh > /dev/null 2>&1


Now, every 1 minute our log is going to be filled with our traffic. You did remember to run /etc/init.d/checkpoint … didn’t you ? If you don’t know why you should do it…DON’T continue reading on!!! Go back and read this: Intracom netroute2 hacks

RRDTool time now…
I suppose you have already installed rrdtool on your favorite distro somehow (emerge,apt-get,rpm,blahblahblah). First thing we need to do is create an rrd database to keep our stats. Save the following in create_netroute2_rate.txt

rrdtool create netroute_rate.rrd --start 1159455804 \
DS:IN:COUNTER:600:0:12500000 \
DS:OUT:COUNTER:600:0:12500000 \
--step 60 \
RRA:AVERAGE:0.5:1:2000 \
RRA:AVERAGE:0.5:6:2000 \
RRA:AVERAGE:0.5:24:2000 \
RRA:AVERAGE:0.5:288:2000 \
RRA:MAX:0.5:1:2000 \
RRA:MAX:0.5:6:2000 \
RRA:MAX:0.5:24:2000 \
RRA:MAX:0.5:288:2000


BIG FAT WARNING!! You HAVE to change the timestamp –start 1159455804 with the current timestamp if you want to get accurate stats. 1159455804 was when I started getting my stats. You MUST change this with your current one. The command “date +%s” is your friend in this…

Now execute sth like this: “/bin/sh create_netroute2_rate.txt”. You should have a beautiful netroute_rate.rrd file inside your current dir. Time to fill up this database with information from the file “for_rrd.log”.
If you don’t have yet public key authentication with your netroute2, now is the crucial time to do it. Paragraph F) of my previous post (Intracom netroute2 hacks) is exactly that.

(timelapse)

I am supposing you already did it…To check if it’s working issue something like this:
ssh -l root -p 22 PUT.IP.OF.NETROUTE2.HERE uname -a
Hopefully you’ll get prompted with something like: “Linux netroute- 2.4.30-netta2 #2 Tue Mar 21 10:51:17 EET 2006 ppc unknown”. If not check again what went wrong…

Now a command like the following will grab “for_rrd.log” from netroute2 and put it in /var/rrds/ (that’s the dir I am using to store my rrd databases, logs, scripts, etc)
scp root@PUT.IP.OF.NETROUTE2.HERE:/tmp/var/for_rrd.log /var/rrds/for_rrd.log

Now..parsing time! This is my netroute-rrd.sh:

#!/bin/sh
cd /var/rrds/
NEWFILE="for_rrd.log"
OLDFILE="for_rrd.log.old"
if [ ! -f $NEWFILE ]; then
echo "No new file found"
exit 1
fi
diff $OLDFILE $NEWFILE | grep -v "," | sed 's/^>[ \t]\+//' > to_parse
cat to_parse |while read line;
do
echo "${line}";
rrdtool update netroute_rate.rrd $line
echo "$?"
done
rm -f to_parse
mv for_rrd.log for_rrd.log.old


If you execute this script it will compare this script with the older one you had downloaded, diff them and parse the output with rrdtool. I do that to ensure that rrdtool only parses what was not parsed before. It’s really stupid to parse the log file everytime from the beggining…

Now you have your database filled up with values…so what ? how can you see the results ? With a script that takes advantage of rrdgraph 🙂 This script is quite big, more than 150lines so I will not copy/paste it here but I will provide you with a tarball of it and all the previously mentioned scripts ment to be stored on your box.
Netroute2 RRDTool Scripts

Combine all these together with a cron job from your pc like the following:
*/5 * * * * scp root@PUT.IP.OF.NETROUTE2.HERE:/tmp/var/for_rrd.log /var/rrds/for_rrd.log ; /etc/scripts/netroute-rrd.sh; /etc/scripts/netroute-rrd-graph.sh

and you will have a /var/rrds/index.html with 5 graphs: Hourly, Daily, Weekly, Monthly and Yearly.

Now you can monitor your Internet traffic even when your stats collector is offline.
There is a chance that you will get some spikes when netroute2 reboots. To eliminate them read this: HOWTO remove spikes from RRD graphs

If you are greek and want to know some more stuff about rrdtool there’s a tutorial here: RRD [HOWTO]

Filed by kargig at 15:16 under Linux,Networking
2 Comments | 5,350 views