Cherry Cymotion Master Linux with Fluxbox

Some tips in getting Cherry Cymotion Master Linux (german layout image) to work with fluxbox (with Gentoo of course). This keyboard’s keys have a great feeling and a great “clickety click” sound. The 20% extra surface on the keys makes typing a lot easier but you need a couple of hours to get used to them. It’s worth it though.

First of all, don’t try to use the keyman software that Cherry provides. Just DON’T. Unless you ‘ve got LOTS of patience and some skills to fix the horrible configure/make scripts that it comes with. It’s horrible to set it up and you need to patch your kernel for it to work properly. What do you lose ? Just the K-key next to your Penguin Key (bye bye Windows Key) and the combinations that you can with the K-Key. This keyboard has SO many extra keys that you won’t need it (yet).

0) Create /etc/init.d/cherry with the following contents:
#!/sbin/runscript
start() {
ebegin "Setting up Cherry Keyboard CyMotion Master on PS/2"
# Search Eject Home An Aus Media
# Up/Right Down/left Cut Paste Copy Desktop1
# Desktop2 Desktop3 Redo Undo
#setkeycodes e065 136 e070 161 e032 172 e05f 143 e063 145 e06d 171 \
# e00b 177 e012 178 e017 137 e00a 135 e018 133 e071 148 \
# e02c 149 e072 202 e007 129 e008 131 e05b 200
setkeycodes e065 136 e070 161 e032 172 e05f 143 e063 145 e06d 171 \
e00b 177 e012 178 e017 137 e00a 135 e018 133 e071 148 \
e02c 149 e072 202 e007 129 e008 131
eend $?
}
stop() {
ebegin "Restoring defaults for PS/2 keyboard"
# Search Eject Home An Aus Media
# Up/Right Down/left Cut Paste Copy Desktop1
# Desktop2 Desktop3 Redo Undo
setkeycodes e065 217 e070 000 e032 150 e05f 142 e063 143 e06d 226 \
e00b 000 e012 000 e017 000 e00a 000 e018 000 e071 000 \
e02c 000 e072 000 e007 000 e008 000 e05b 125
eend $?
}

Make it executable: chmod +x /etc/init.d/cherry
Start it: /etc/init.d/cherry
and add it to your default runlevel: rc-update add cherry default (on gentoo)

1) Take a look at your /usr/share/X11/XKeysymDB. This file includes all the available keyboard mappings you can have.

2) Change your Keyboard Inputdevice in /etc/X11/xorg.conf to include these (works in my current Xorg 7.1):
Option "XkbModel" "cymotionlinux"
Option "XkbVariant" "nodeadkeys"

Restart your X server.

3) Create a .Xmodmap file in your home dir and fill it with mappings from /usr/share/X11/XKeysymDB. Mine looks like this:
! Internet-Keys (Back, Forward, Stop, Reload, Search)
keycode 234 = XF86Back
keycode 233 = XF86Forward
keycode 232 = XF86Stop
keycode 231 = XF86Refresh
keycode 122 = XF86Search
! Multimedia-Keys (Play/Pause, Stop, Back, Forward, Eject)
keycode 162 = XF86AudioPlay
keycode 164 = XF86AudioStop
keycode 144 = XF86AudioPrev
keycode 153 = XF86AudioNext
keycode 204 = XF86Eject
! XPress-Keys left (Up, Down, Cut, Paste, Copy)
keycode 220 = XF86ScrollUp
keycode 143 = XF86ScrollDown
keycode 188 = XF86Cut
keycode 192 = XF86Paste
keycode 248 = XF86Copy
! XPress-Keys right (List 1, List 2, List 3, Redo, Undo)
!keycode 159 = XF86Launch0
!keycode 151 = XF86Launch1
!keycode 171 = XF86Launch2
!keycode 138 = XF86Launch3
!keycode 135 = XF86Launch4
! Premium-Keys (Calculator, Email, Browser, On/Off)
keycode 161 = XF86Calculator
keycode 236 = XF86Mail
keycode 130 = XF86HomePage
keycode 199 = XF86LogOff
! Main-Keys (-, +, Mute, Media-Player)
keycode 174 = XF86AudioLowerVolume
keycode 176 = XF86AudioRaiseVolume
keycode 160 = XF86AudioMute
keycode 129 = XF86AudioMedia
! @-Key
keycode 116 = XF86Terminal

Do a:
xmodmap ~/.Xmodmap

You can test your multimedia keys using xev.
If you want fluxbox to automatically load these binding each time you logon, put “xmodmap ~/.Xmodmap” somewhere (NOT in the end) inside your ~/.fluxbox/startup

4) Edit your ~/.fluxbox/keys and map the new keys to actions like that:
None XF86Launch1 :Workspace 1
None XF86Launch2 :Workspace 2
None XF86Launch3 :Workspace 3
!
None XF86AudioPrev :ExecCommand audacious--rew
None XF86AudioPlay :ExecCommand audacious --play-pause
None XF86AudioNext :ExecCommand audacious --fwd
None XF86AudioStop :ExecCommand audacious --stop
None XF86Eject :ExecCommand audacious -j
!
None XF86AudioLowerVolume :ExecCommand amixer sset PCM 2-
None XF86AudioMute :ExecCommand amixer sset "Audigy Analog/Digital Output Jack" toggle
None XF86AudioRaiseVolume :ExecCommand amixer sset PCM 2+
None XF86AudioMedia :ExecCommand gmplayer
!
None XF86Calculator :ExecCommand gcalctool
None XF86Mail :ExecCommand /usr/bin/thunderbird
None XF86HomePage :ExecCommand /usr/bin/firefox
!
None XF86Terminal :ExecCommand mrxvt --loginShell

Restart Fluxbox. You should now be able to start applications using your multimedia keys on top and on the bottom. If you are using Gnome or KDE you can do all this with their own programs. I think the proper program in KDE is called khotkeys.

5) In order to use the side keys for more advanced functions like copy,paste,unde,redo,etc do the following:
Create /usr/share/X11/xkb/compat/cymotion with what you like. Mine looks like:
// $XFree86$
// XFree86 special keysyms
default partial xkb_compatibility "basic" {
interpret.repeat= True;
// Ctrl+C for Copy
interpret XF86Copy {
action = Redirect(Key=<AB03>, modifiers=Control);
};
// Ctrl+V for Paste
interpret XF86Paste {
action = Redirect(Key=<AB04>, modifiers=Control);
};
// Ctrl+X for Cut
interpret XF86Cut {
action = Redirect(Key=<AB02>, modifiers=Control);
};
// Ctrl+Pos1 for ScrollUp
interpret XF86ScrollUp {
action = Redirect(Key=<HOME>, modifiers=Control);
};
// Ctrl+End for ScrollDown
interpret XF86ScrollDown {
action = Redirect(Key=<END>, modifiers=Control);
};
// Ctrl+Shift+Z for Redo
interpret Redo {
action = Redirect(Key=<AB01>, modifiers=Shift+Control);
};
// Ctrl+Z for Undo
interpret Undo {
action = Redirect(Key=<AB01>, modifiers=Control);
};
// Backspace for Back (Firefox)
interpret XF86Back {
action = Redirect(Key=<BKSP>);
};
// Shift+Backspace for Forward (Firefox)
interpret XF86Forward {
action = Redirect(Key=<BKSP>, modifiers=Shift);
};
// ESC for Stop (Firefox)
interpret XF86Stop {
action = Redirect(Key=<ESC>);
};
// Ctrl+R for Reload (Firefox)
interpret XF86Reload {
action = Redirect(Key=<AD04>, modifiers=Control);
};
// Ctrl+F for Search Page (Firefox and most other applications)
interpret XF86Search {
action = Redirect(Key=<AC04>, modifiers=Control);
};
};

6) Edit /usr/share/X11/xkb/compat/complete and add inside it:
augment "cymotion"

Restart X.

And you should be done.

The process is a bit complicated but the result is very rewarding. The Cut/Copy/Paste with just one click is amazing, especially for Greek users because when someone with another keyboard switches the keyboard language to greek, “ctrl+c” becomes “ctrl+ψ” (ctrl+greek_psi) and copy/paste does not work any more (ctrl+insert and shift+insert are not very efficient shortcuts) . You can easily program Cymotion Master Linux keyboard though to send the keystrokes you want (that was what I did in /usr/share/X11/xkb/compat/cymotion) because there are so many extra keys to use 🙂

Here comes the result of all these (click to maximize):

References:
a) http://forums.gentoo.org/viewtopic-t-261126-highlight-cymotion.html
b) http://www.ubuntu-forum.de/thread.php?threadid=11319
c) http://my.opera.com/netzpirat/blog/show.dml/505978

Zenwalk vs iloog on an ancient Compaq laptop

Testing Zenwalk version 4.2 on an ancient Compaq Presario 1260 Laptop (AMD K6@333 with 64Mb RAM).

0) Install CD Boots pretty fast
1) Autopartition does not work, or it did not work for me. I’ve tried it two times both giving me an empty partition table thus being unable to continue the installation.By the way, why does it need at least 1Gb of swap space ?
2) Why can’t netpkg use more than one mirror concurrently since they’ve got different packages ? I’ve tried to select 2 mirrors from the menu but I didn’t work. Am I doing something wrong ?
3) XFCE was not light enough for this ancient laptop. I tried disabling various stuff from the panels…but it was still very slow to work with. So I used netpkg (from the command line) to add Fluxbox. Fluxbox was a lot faster than XFCE but still applications were pretty slow to start.
4) There was no APM support on the kernel. No battery status for the laptop 🙁

Overall I think that zenwalk is an “ok” distro but it’s still pretty rough around the edges. It still need some slackware-ish hacks to behave like it should.

The good part about Zenwalk testing was that it made me “install” iloog on a hard disk for the first time, so I could compare the two. Zenwalk and iloog.

I booted the iloog livecd,
a) rsynced /(root) to /mnt/hda2,
b) mount /proc, /sys to /mnt/hda2 and chrooted to it,
c) changed /etc/fstab to suit the laptop’s hard disk (beware to remove some of the noexec, nosuid options),
d) created a /etc/lilo.conf from /etc/lilo.conf.example and ran “lilo”
e) disabled some init scripts needed by the livecd for autoconfiguration
f) rebooted

and … the result was very very good. Applications start in a lot less time than they start with zenwalk and are generally more responsive. Firefox, for example, is actually usable in iloog in contrast with zenwalk where there was no way to use it…it took around 1 minute just to start it! I didn’t expect to see such a difference because zenwalk is supposed to be a “light” distro specifically for old machines (ok, maybe not as old as mine :P)
The bad thing is that since iloog is based on gentoo, it is almost impossible to emerge new applications on that laptop while it is very easy to do so with zenwalk using netpkg. It may be possible to install new application on iloog with a) distcc but it requires another strong PC to act as the “compiler PC” , b) by getting binary packages from another user’s repo (that sounds more like science fiction) debianizing the gentoo/iloog.

If anyone has an ancient laptop, like the one I have, he/she might want to give iloog a shot. It may revive it…

P.S. All that may be a bit biased towards iloog since I am the one developing it. I still think though that iloog is more suited as a livecd for educational/scientific purposes than as an installable meta-meta-distro (since gentoo is already a meta-distro). Once you actually install it to your hard disk it’s not iloog any more…but gentoo.

P.S.2. The latest iloog beta is here: http://195.130.120.70/livecd.0124-1242.iso . If anyone tests iloog livecd please leave a comment.

ATI-8.32.5 blank screen fix

I’ve just upgraded my ATI drivers from 8.29.6 to 8.32.5, restarted my X and was greeted with a very beautiful blank/black screen. Playing a bit with my ModeLine in xorg.conf though I was able to make it work again.

I just needed to change:
Modeline "1280x1024_75.00" 138.54 1280 1368 1504 1728 1024 1025 1028 1069 -HSync +Vsync

to:
Modeline "1280x1024_75.00" 138.54 1280 1368 1504 1728 1024 1025 1028 1069

This Modeline created by gft makes my monitor work at 75Hz instead of the 60Hz that ddc provides to xorg.

I don’t know why but it looks like the new drivers don’t like the polarity options that the previous drivers had no problem with…

Anyway, here’s the part of my xorg.conf referring to my Monitor and Graphics Driver that currently works:

Section "Monitor"
Identifier "Monitor0"
VendorName "LG"
ModelName "L1970H"
HorizSync 30.0 - 83.0
VertRefresh 56.0 - 75.0
# 1280x1024 @ 75.00 Hz (GTF) hsync: 80.17 kHz; pclk: 138.54 MHz
Modeline "1280x1024_75.00" 138.54 1280 1368 1504 1728 1024 1025 1028 1069
Option "DPMS"
EndSection
Section "Device"
Identifier "ATI 9500 Pro"
Driver "fglrx"
Option "no_accel" "no"
Option "no_dri" "no"
# === misc DRI settings ===
Option "mtrr" "off" # disable DRI mtrr mapper, driver has its own code for mtrr
Option "DesktopSetup" "Single"
Option "HSync2" "unspecified"
Option "VRefresh2" "unspecified"
Option "ScreenOverlap" "0"
# === TV-out Management ===
Option "ForceMonitors" "tmds1,notv"
Option "TVStandard" "NTSC-M"
Option "TVHSizeAdj" "0"
Option "TVVSizeAdj" "0"
Option "TVHPosAdj" "0"
Option "TVVPosAdj" "0"
Option "TVHStartAdj" "0"
Option "TVColorAdj" "0"
Option "GammaCorrectionI" "0x06419064"
Option "GammaCorrectionII" "0x00b0000b"
# === OpenGL specific profiles/settings ===
Option "VideoOverlay" "on"
# === OpenGL Overlay ===
Option "OpenGLOverlay" "off"
# === Center Mode (Laptops only) ===
Option "CenterMode" "off"
# === Pseudo Color Visuals (8-bit visuals) ===
Option "PseudoColorVisuals" "off"
# === QBS Management ===
Option "Stereo" "off"
Option "StereoSyncEnable" "off"
# === FSAA Management ===
Option "UseFastTLS" "1"
Option "UseInternalAGPGART" "yes"
Option "FSAAEnable" "on"
Option "FSAAScale" "2"
BusID "PCI:1:0:0"
EndSection

And the glxgears result:
17418 frames in 5.0 seconds = 3483.523 FPS
17341 frames in 5.0 seconds = 3468.088 FPS
17428 frames in 5.0 seconds = 3485.311 FPS
17548 frames in 5.0 seconds = 3509.505 FPS

Vivodi Full LLU και Packet Filtering ?

Χθες προσπαθούσα να βοηθήσω κάποιον στο IRC να βάλει gentoo (δεν χρειάζονται σχόλια 😛 ). Είχε διαβάσει το gentoo handbook φτάσει στο σημείο που πρέπει να κατεβάσει το stage3 tarball (Παράγραφος 5a του handbook). Αρχίζει να κατεβάζει όλο χαρά το stage3 tarball από το mirror του ntua (http://ftp.ntua.gr/pub/linux/gentoo/releases/x86/current/stages/stage3-i686-2006.1.tar.bz2) και ξαφνικά κολλάει στο 72%. Το ξαναβάζει…τα ίδια. Δοκιμάζει με ftp αντί http….τα ίδια. Του δείχνω το wget -c ώστε να κάνει resume…τίποτα…δεν γινόταν resume. Του λέω ότι εγώ στο σπίτι μου το κατεβάζω κανονικά (Vivodi με γραμμή ΟΤΕ)…και του δίνω το mirror του uoi. Αρχίζει το download από την αρχή, στο 72% πάλι κόψιμο. Δοκιμάζει να κατεβάσει το αρχείο από windows ώστε να το περάσει μετά με usb flash στο gentoo, πάλι κόλλημα στο 72%. Δοκιμάζει mirror από την Γερμανία, τα ίδια…ενώ εγώ να το κατεβάζω κανονικότατα σε 2-3 διαφορετικές dsl. Του λέω δεν γίνεται…κάποιο δικτυακό πρόβλημα έχεις. Του ζητάω να μου ανοίξει ssh πάνω στο PC που θα γινόταν το install, δοκιμάζω και εγώ να το κατεβάσω…τίποτα…ούτε με links ούτε με wget, πάντα κόλλημα στο 72%. H κάρτα δικύου έπαιζε κανονικά…collisions, errors όλα μηδενικά.

Λέω…δεν γίνεται, το modem σου πρέπει να έχει πρόβλημα, κάποιο time-out γίνεται…δεν βγάζω άκρη. Τον ρωτάω αν έχει 2ο adsl modem, μου απαντάει θετικά και το αλλάζει κατευθείαν. Αρχίζει πάλι το download, στο 72% πάλι τα ίδια. Η κατάσταση ήταν πλέον για κλάμματα.
Τον ρωτάω τη σύνδεση έχει και μου απαντάει Vivodi Full LLU στην Πάτρα. Τον ρωτάω έπειτα αν έχει κανένα άλλο φίλο/γνωστό με full LLU στην Πάτρα και μου απαντάει θετικά. Βάζει το γνωστό του να κατεβάσει το ίδιο αρχείο…και ναι…κολλάει στο 72% !!!! Παράλληλα ο γνωστός του αυτός του αναφέρει πως ακούγεται πως η Vivodi έχει στήσει filtering σε κάποια σημεία του δικτύου της…
Η τελευταία ελπίδα ήταν να πειράξω το mirror του uoi ώστε να απαντάει και σε https ώστε τα πακέτα να είναι encrypted και να μην μπορούν να τα πιάσουν τα οποιαδήποτε φίλτρα. Δοκιμάζει να το κατεβάσει με https…και όντως δούλεψε! Πέρασε το 72% και ολοκληρώθηκε χωρίς κανένα πρόβλημα…

Το συμπέρασμα είναι πως σίγουρα το dslam της Vivodi στην Πάτρα είναι προβληματικό. Πολύ πιθανό η Vivodi να εφαρμόζει κάποιο packet filtering, και το συγκεκριμένο αρχείο στο “72%” να κάνει trigger ένα από τα filtra της vivodi και να γίνεται corrupt ύστερα. Εγώ δεν μπορώ να το εξηγήσω διαφορετικά. Αν κάποιος μπορεί και έχει μια άλλη εξήγηση…ευχαρίστως να την ακούσουμε…Αν επίσης κάποιος με full LLU από Vivodi έχει λίγο χρόνο ας δοκιμάσει να κατεβάσει το αρχείο:

http://ftp.uoi.gr/mirror/OS/gentoo/releases/x86/current/stages/stage3-i686-2006.1.tar.bz2

και να μας πει αν κολλάει στο 72%. Αν ναι ας πει και σε ποια πόλη/περιοχή μένει…μήπως και βγει κάποια άκρη…

Εννοείται πως σήμερα όταν ο άνθρωπος αυτός πήρε τηλέφωνο στην Vivodi να ρωτήσει πως και γιατί…δεν είχαν να του πουν τίποτα. Ποιος όμως θα τον αποζημιώσει για τις 4-5 και παραπάνω ώρες που έχασε και κόντεψαν να τον φέρουν σε κατάσταση υστερίας ?

Thoughts on the evolution of Operating Systems

Linux is a multiuser operating system “designed to be secure by design”. Each user has it’s own home directory and can only execute applications that the Administrator (root) of the system has allowed him to. That means that users that want to run priviledged applications must either have root’s permission to do so or are asked for a password to escalate their priviledges. So every linux user not only knows the difference of a simple user and the “root of all evil”, but is well aware of where/when to use passwords, what are file permissions and so on and so on. A linux user has (or had, until recently) given up pretty graphics in favor of a more stable, more secure and more “free” operating system.

Windows is a (multiuser?) operating system with emphasis on usability. Since the first windows versions, the users of windows got used to being able to do almost everything without ever being asked for another password than the one at the login screen. Sometimes there wasn’t even such a login screen. A windows user is used to doing administrative tasks with his every day account. Most windows users don’t even know about file permissions and how to use them on their system. That makes life both easier and riskier.

Until broadband came to our lives at the very end of the 20th century, when Windows 98 and ME ruled the IT universe, windows users had very little to fear. While they only exchanged files with their friends on floppies and cdroms, and their computers were not 24/7 online, remote exploits, internet worms and trojans were unknown words to them. A decent antivirus was the only thing required. Windows 98/ME did not even have “services” running on them by default (apart from shared folders). During that time Microsoft only had to worry about making their users’ OS easier and more beautiful. And they were pretty successful in that.
Their server (NT) version though had tons of problems. It was very incompatible with a lot of software and was easily attacked by internet worms. The number of service packs for NT reached a ridiculous number, and still Microsoft couldn’t handle all the problems. Their enhanced server edition, Windows 2000 was a lot better than NT, but the security weaknesses remained. What made Windows NT and Windows 2000 so insecure was that they were supposed to be 24/7 online. The bad guys attacked WinNT and Win2K because they could then use them for their own purposes. A hacked win98 box behind a dialup was useless compared to an always online windows server. While more and more windows 2000 servers were getting online and worms hammered them, more and more people started bitching Microsoft about it. Microsoft tried to fix problems those problems with numerous service packs, but that wasn’t enough.
Then came Windows XP, the first Microsoft OS for users that had various services enabled by default. At the same time more and more people started having broadband at their homes. Now the bad guys had something new and more powerful to fiddle, and it wasn’t long until the first remotely exploitable problems appeared. The bad guys created worms and trojans that attacked WinXP, the OS of millions users. Users’ machines are millions more than server machines, and they were all probable targets/victims for those worms. If a remotely exploitable vulnerability was somehow found for Win98, the impact would be a lot smaller because the number of online PCs was a lot smaller back then . Every remotely exploitable hole found for XP drove users crazy. XP was not designed to be secure, it was to designed to be a _lot_ more beautiful than 98/ME. It was designed with multimedia, games, office work, etc in mind. People started asking for more than an antivirus for their PCs and a new word came to every windows user life, “Firewall”. Then came service pack 1, then service pack 2…and problems still existed and will exist even if a service pack 3 is launched sometime in the future. One of the basic problems with Microsoft XP is that it doesn’t help users understand how and when to use the Administrator account. Most users create a user with Administrator privileges during XP’s installation process and then run their machines with that admin account. This is _plain_ wrong. Every windows XP user feels that he can do everything he wants with his PC without ever being asked for a single password. So when something infects their PC and runs with their user’s privileges, it actually runs as the administrator of the machine. This leads to total destruction.

Vista is said to have a different approach to security though. It has supposedly been built from scratch with security in mind. Users are now ‘just users’ and their default account does not have administrative privileges. So every time they want to do an administrative task dozens of warnings appear before them. That supposedly ensures that nothing can run with administrative privilege unless the user explicitly wants it to be that way. Someone I know who used Vista told me that Vista makes you think that it’s for more advanced users than XP was. All these notifications that pop up asking you for permission to do this and that, makes you feel a bit uncomfortable. New users of computers, that first stumble upon Vista will feel much more uncomfortable with that OS than they would feel if their first OS was XP. And that’s bad for a “Windows OS”, every version until Vista was easier to use than the previous one, apart from Vista…which is harder.
Microsoft with Vista acts as parent who has spoiled their children for a veeeery long time, giving them free chocolates and candies even though they knew that by doing this they hurt their children’s health, and are now trying to put them on a diet. And this just can’t be done. Windows have spoiled users for more than 10-15 years and it’s too late to start telling people, “Hey mind your security!”. “Your account is now on a diet, no more candies for you”.

When Vista starts spreading among users (maybe in 1 year from now?), users themselves will eventually understand more and more about security concepts. They will start to understand why it is so important that the administrative account is something completely different than theirs and why they should only “touch” it occasionally. Vista might be a lot more beautiful as an OS but it will be a lot more difficult for users to “manage”. Nag screens will be all over the place. Passwords might be frequently asked to change something fundamental for the system.

Where does this lead ? Users will get more and more acquainted with the whole administrator’s rights idea and Linux will not look so much like an alien OS to them. The transition from Windows to Linux for users will get easier and easier as linux becomes (slowly and painfully) more good looking and windows becomes (slowly and painfully) more secure.

Every OS has a different beginning and a different approach on evolution, but they tend to meet at some place in the future. They have just taken different roads to reach their goal. Windows prefers user-friendliness over security and stability and linux prefers it the other way around.

Traffic shaping TorrentFlux

*Update on peers supporting encryption at the end of the post*

TorrentFlux is a great program/interface to download your torrents remotely on a linux machine. It is based on php and it uses a modified bittornado client to download the torrents.

The problem: The bittornado client is able to put specific limits on a per torrent basis. That means that you can put a 100kb/sec download and 50kb/sec upload limit per torrent through torrentflux’s web interface. If you have 20 torrents though, this easily becomes 20*50=1Mb/sec upload “limit”. There are cases that you don’t want this to happen and you want both a per torrent limit (eg 50kb/sec) and a global limit (eg 300kb/sec).

The solution: My solution is based on iptables, layer7 filter and tc (iproute2). I am using layer7 filter to pick out the bittorrent packets, iptables to mark those packets with specific values and tc to shape those marked packets into categories. Beware that the method I am using works mostly on the “uploading” part (outgoing traffic). It is not that hard to make it work for the incoming traffic as well, but it is my personal view that downloading with a few Mb/sec is not as harmfull as uploading with a few Mb/sec. I usually have my downloads seeded over many weeks…so it’s good for my ratio to have the torrent downloaded as fast as possible and then seed it endlessly. I usually like to seed until i get a ratio over 1000% per torrent (that means 10 times as much uploaded traffic than downloaded). The following example configs are created for use on a 100mbit line and keeping in mind that outgoing torrent traffic should not exceed 2-2.5Mbits (~250-300kb/sec).

The procedure:
0) Before you begin make sure you have the kernel sources on /usr/src/linux.

1) Then, you need to patch your kernel for layer7 filtering and enable marking. On gentoo linux you only need to:
#emerge -avt net-misc/l7-filter net-misc/l7-protocols
and then configure your kernel for marking.

Here’s how my netfilter configuration looks like:

CONFIG_NETFILTER=y
# CONFIG_NETFILTER_DEBUG is not set
# CONFIG_BRIDGE_NETFILTER is not set

#
# Core Netfilter Configuration
#
# CONFIG_NETFILTER_NETLINK is not set
CONFIG_NETFILTER_XTABLES=y
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
# CONFIG_NETFILTER_XT_TARGET_CONNMARK is not set
CONFIG_NETFILTER_XT_TARGET_MARK=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
CONFIG_NETFILTER_XT_MATCH_COMMENT=m
CONFIG_NETFILTER_XT_MATCH_CONNBYTES=m
CONFIG_NETFILTER_XT_MATCH_CONNMARK=m
CONFIG_NETFILTER_XT_MATCH_CONNTRACK=m
CONFIG_NETFILTER_XT_MATCH_DCCP=m
CONFIG_NETFILTER_XT_MATCH_ESP=m
CONFIG_NETFILTER_XT_MATCH_HELPER=m
CONFIG_NETFILTER_XT_MATCH_LENGTH=m
CONFIG_NETFILTER_XT_MATCH_LIMIT=m
CONFIG_NETFILTER_XT_MATCH_MAC=m
CONFIG_NETFILTER_XT_MATCH_MARK=m
# CONFIG_NETFILTER_XT_MATCH_POLICY is not set
CONFIG_NETFILTER_XT_MATCH_MULTIPORT=m
CONFIG_NETFILTER_XT_MATCH_PKTTYPE=m
# CONFIG_NETFILTER_XT_MATCH_QUOTA is not set
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_SCTP=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
# CONFIG_NETFILTER_XT_MATCH_STATISTIC is not set
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_NETFILTER_XT_MATCH_TCPMSS=m

#
# IP: Netfilter Configuration
#
CONFIG_IP_NF_CONNTRACK=m
CONFIG_IP_NF_CT_ACCT=y
CONFIG_IP_NF_CONNTRACK_MARK=y
# CONFIG_IP_NF_CONNTRACK_EVENTS is not set
CONFIG_IP_NF_CT_PROTO_SCTP=m
CONFIG_IP_NF_FTP=m
CONFIG_IP_NF_IRC=m
CONFIG_IP_NF_NETBIOS_NS=m
CONFIG_IP_NF_TFTP=m
CONFIG_IP_NF_AMANDA=m
CONFIG_IP_NF_PPTP=m
CONFIG_IP_NF_H323=m
CONFIG_IP_NF_SIP=m
CONFIG_IP_NF_QUEUE=m
CONFIG_IP_NF_IPTABLES=y
CONFIG_IP_NF_MATCH_IPRANGE=y
CONFIG_IP_NF_MATCH_LAYER7=m
# CONFIG_IP_NF_MATCH_LAYER7_DEBUG is not set
CONFIG_IP_NF_MATCH_TOS=y
CONFIG_IP_NF_MATCH_RECENT=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_DSCP=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_MATCH_OWNER=m
CONFIG_IP_NF_MATCH_ADDRTYPE=m
CONFIG_IP_NF_MATCH_HASHLIMIT=m
CONFIG_IP_NF_FILTER=y
CONFIG_IP_NF_TARGET_REJECT=y
CONFIG_IP_NF_TARGET_LOG=y
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_IP_NF_TARGET_TCPMSS=y
CONFIG_IP_NF_NAT=m
CONFIG_IP_NF_NAT_NEEDED=y
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_TARGET_REDIRECT=m
CONFIG_IP_NF_TARGET_NETMAP=m
CONFIG_IP_NF_TARGET_SAME=m
CONFIG_IP_NF_NAT_SNMP_BASIC=m
CONFIG_IP_NF_NAT_IRC=m
CONFIG_IP_NF_NAT_FTP=m
CONFIG_IP_NF_NAT_TFTP=m
CONFIG_IP_NF_NAT_AMANDA=m
CONFIG_IP_NF_NAT_PPTP=m
CONFIG_IP_NF_NAT_H323=m
CONFIG_IP_NF_NAT_SIP=m
CONFIG_IP_NF_MANGLE=m
CONFIG_IP_NF_TARGET_TOS=m
CONFIG_IP_NF_TARGET_ECN=m
CONFIG_IP_NF_TARGET_DSCP=m
CONFIG_IP_NF_TARGET_TTL=m
CONFIG_IP_NF_TARGET_CLUSTERIP=m
# CONFIG_IP_NF_RAW is not set
CONFIG_IP_NF_ARPTABLES=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m

You can clearly see layer7 being enabled as a module: CONFIG_IP_NF_MATCH_LAYER7=m
Rebuild your kernel and install the proper modules. If you need to reboot your machine to apply the new kernel do it now.

2) Now it’s time to install iptables and iproute2 if you don’t have them already. On gentoo linux:

#echo "net-firewall/iptables extensions l7filter" >> /etc/portage/package.use
#emerge -avt net-firewall/iptables sys-apps/iproute2

3) Now it’s the iptables marking time. I am going to show you (some of) the output of my iptables-save command. Change it to fit your neeeds:

# Generated by iptables-save v1.3.5 on Fri Jan 12 20:50:52 2007
*mangle
:PREROUTING ACCEPT [1102387:193393325]
:INPUT ACCEPT [1102372:193390208]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [2100485:2922693566]
:POSTROUTING ACCEPT [2100483:2922690566]
-A PREROUTING -s IP.OF.MACHINE -p tcp -m multiport --sports 22,80 -j MARK --set-mark 1001
-A PREROUTING -d IP.OF.MACHINE -p tcp -m multiport --dports 22,80 -j MARK --set-mark 1001
-A PREROUTING -m layer7 --l7proto ssh -j MARK --set-mark 1001
#-A PREROUTING -m layer7 --l7proto bittorrent -j MARK --set-mark 11090
-A PREROUTING -m mark --mark 1001 -j RETURN
-A POSTROUTING -s IP.OF.MACHINE -p tcp -m multiport --sports 22,80 -j MARK --set-mark 1001
-A POSTROUTING -d IP.OF.MACHINE -p tcp -m multiport --dports 22,80 -j MARK --set-mark 1001
-A POSTROUTING -m mark --mark 1001 -j RETURN
-A POSTROUTING -m connmark --mark 0x0 -j MARK --set-mark 11030
-A POSTROUTING -m layer7 --l7proto dns -j MARK --set-mark 11010
-A POSTROUTING -m layer7 --l7proto ssh -j MARK --set-mark 11010
-A POSTROUTING -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j MARK --set-mark 11010
-A POSTROUTING -p icmp -j MARK --set-mark 11010
-A POSTROUTING -m layer7 --l7proto bittorrent -j MARK --set-mark 11090
COMMIT
# Completed on Fri Jan 12 20:50:52 2007
# Generated by iptables-save v1.3.5 on Fri Jan 12 20:50:52 2007
*nat
:PREROUTING ACCEPT [407:30699]
:POSTROUTING ACCEPT [111:6662]
:OUTPUT ACCEPT [111:6662]
COMMIT
# Completed on Fri Jan 12 20:50:52 2007
# Generated by iptables-save v1.3.5 on Fri Jan 12 20:50:52 2007
*filter
:INPUT ACCEPT [266369:32040284]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [479227:676859047]
COMMIT
# Completed on Fri Jan 12 20:50:52 2007

You need to change IP.OF.MACHINE with the IP of your linux box.

4) And now the traffic shaping part:

# Main Link
LINK=100000
SHAPEDLINK=50000
# High Priority
HIGHPRIO=10000
HIGHPRIO_MAX=$SHAPEDLINK
# Normal
NORMAL=512
NORMAL_MAX=$SHAPEDLINK
# Downloads
TOR=512
TOR_MAX=2048
# del old
tc qdisc del dev $DEV root 2> /dev/null > /dev/null
# add root
tc qdisc add dev $DEV root handle 100: htb default 1
tc class add dev $DEV parent 100: classid 100:1 htb rate ${LINK}kbit
tc qdisc add dev $DEV parent 100:1 handle 1: htb
tc class add dev $DEV parent 1: classid 1:1 htb rate ${SHAPEDLINK}kbit
# some more rules
tc class add dev $DEV parent 100: classid 100:1 htb rate ${LINK}kbit
tc qdisc add dev $DEV parent 100:1 sfq perturb 10
tc filter add dev $DEV parent 100:0 protocol ip prio 1 handle 1001 fw flowid 100:1
tc class add dev $DEV parent 1:1 classid 1:10 htb rate ${SHAPEDLINK}kbit ceil ${SHAPEDLINK}kbit prio 5
tc qdisc add dev $DEV parent 1:10 sfq perturb 10
# High priority
tc class add dev $DEV parent 1:10 classid 1:1010 htb rate ${HIGHPRIO}kbit ceil ${HIGHPRIO_MAX}kbit prio 0
tc qdisc add dev $DEV parent 1:1010 sfq perturb 10
tc filter add dev $DEV parent 1:0 protocol ip prio 0 handle 11010 fw flowid 1:1010
# normal
tc class add dev $DEV parent 1:10 classid 1:1030 htb rate ${NORMAL}kbit ceil ${NORMAL_MAX}kbit prio 5
tc qdisc add dev $DEV parent 1:1030 sfq perturb 10
tc filter add dev $DEV parent 1:0 protocol ip prio 5 handle 11030 fw flowid 1:1030
# bittorent
tc class add dev $DEV parent 1:10 classid 1:1090 htb rate ${TOR}kbit ceil ${TOR_MAX}kbit prio 10
tc qdisc add dev $DEV parent 1:1090 sfq perturb 10
tc filter add dev $DEV parent 1:0 protocol ip prio 10 handle 11090 fw flowid 1:1090

The rules are pretty straightforward…so I am not going to fully explain them. The basic concept is that you create a “shaped” partition of your bandwith and you add classes (high priority, normal , bittorrent) there. The trick is that you can skip anything you don’t want shaped by marking it with iptables 1001 mark.

In my iptables example above, I mark as 1001 the outgoing ssh and http traffic. This way I can shape the seeding of my torrents using TorrentFlux but I can download via http without any traffic shaping the torrents to my PC at home. I can also ssh to the machine without any latency caused by the shaping because the sshd port (22) is marked with 1001.

The only problem I faced with those scripts was that sometimes the layer7 filter for bittorrent let’s some torrent traffic pass by. My solution to that was to change NORMAL_MAX=$SHAPEDLINK to NORMAL_MAX=2048 for example. Then, even “normal traffic” was shaped. Remember that anything I didn’t want shaped, was marked as 1001 on the iptables script…so the machine was still very responsive even after shaping the “normal traffic”.

To check how your scripts are doing in terms of shaping you can download this excellent perl script: tc-viewer. Click here for a screenshot: tc-viewer htb screenshot

The above example configs are very very generic. If you have a server that serves many other duties apart from ssh, http and bittorrent, then this script might not work out of the box for you.

*Update*
It looks like the problem I had with layer7 bittorrent filter missing packets was not actually a layer7’s “problem”, but rather a new feature of the latest version of bittornado. I was using bittornado version 0.3.18 (experimental) which is the first bittornado version that comes with Message_Stream_Encryption. What this means: whenever bittornado finds another peer with encryption capabilities, it encrypts all traffic between you and the other peer, so the layer7 filter cannot understand that these flows are torrent traffic anymore, and categorizes them as “normal” traffic. That’s why I needed to “shape” normal traffic as well.
There are three ways to cope with encrypted bittorrent traffic. The first one is the one I described above without even knowing about it (shaping normal traffic). The second way is to go back to a version without encryption (0.3.17), which I think is a _really_ bad idea. Encryption came to help us hide our traffic from ISP filters, and is a step we can all take to protect ourselves. The third way is to mark the port range that torrentflux uses (check the admin panel of torrentflux for it) as torrent traffic by our iptables script. If the port range is high enough it can be almost certain that no other service will use those ports, so no priority traffic will be mis-matched as “torrent traffic”. If, for example, you have defined your port range to be from port 61000 to port 63000, then inject a command like:
-A POSTROUTING -p tcp --sport 61000:63000 -j MARK --set-mark 11090
just below the
-A POSTROUTING -m layer7 --l7proto bittorrent -j MARK --set-mark 11090
command of the iptables script above.

Enjoy shaped encrypted bittorent uploads! Keep seeding…

No space left on device

Last night I decided to do some tests with portage inside a virtual machine using iloog livecd. I fired up vmware player (it’s free) and created one 1Gb partition, type Linux and one 1Gb partition, type swap with fdisk. Then I issued :

#mke2fs -j /dev/hda1
#mkdir /mnt/hda1
#mount /dev/hda1 /mnt/hda1
#mkdir /mnt/hda1/portage
#mkdir /usr/portage
#mount --bind /mnt/hda1/portage /usr/portage
#emerge --sync

The syncing with gentoo portage had started.. After a while though, my screen got full off errors about not having enough space left on the device. That was a bit weird since 1Gb is more than enough for just the portage without any distfiles or packages… Issuing a “df -HT” gave me something like this:

/dev/hda1 ext3 985M 557M 379M 60% /mnt/hda1

So, I do have space free…where was the problem ? Luckilly some friends on IRC gave me the answer…

#df -i
/dev/hda1 120000 120000 0 100% /mnt/hda1

I was out of inodes…The portage has so many small files that made me run out of inodes and the only solution was to run mke2fs again but with options that look like the following, in order to increase the inode number by decreasing the block size:
mke2fs -j -b 1024 -i 1024 /dev/hda1

That gave me enough inodes to store the portage 🙂

dynamic hostname updates with DHCPd + BIND

The following text describes the way to make DHCPd create DNS records of dhcp clients. This scenario is usually called a DDNS server.

We will first setup up BIND and then DHCPd.
1) We need to create a “key”. A key is a password that allows dhcpd to update bind… imagine something like an authentication scheme. In order to create such a key file to the following:
# rndc-confgen -a
A key file will be created. On Gentoo Linux this file is put inside /etc/bind/ and is called “rndc.key”.
Now open up your named.conf file and put these line inside:
include "/etc/bind/rndc.key";
controls {
inet 127.0.0.1 allow { localhost; } keys { rndc-key; };
};

If you already have a controls section just modify it to include the keys { } directive.

Now, in your zones that you want to be dynamically updated modify them to look like this:

zone "0.168.192.in-addr.arpa" {
type master;
file "pri/192.168.0.zone";
allow-update { key "rndc-key"; };
notify yes;
};
zone "home-sweet-home.gr" {
type master;
file "pri/home-sweet-home.gr.zone";
allow-update { key "rndc-key"; };
notify yes;
};

The “allow-update” is the crucial part here.
If you’ve done that changes we are finished from the BIND configuration. Restart the service and you are done.

2)Time for DHCPd configuration.
Remember your rndc-key file ? Open it and paste it contents inside your dhcpd.conf file so it looks a bit like this:
key "rndc-key" {
algorithm hmac-md5;
secret "goa6aw7a9WwapCyai0naZQ==";
};

Among others you have to put the zone files that you want to be updated. Following my previous zone declaration in named.conf and supposing that the primary nameserver is 192.168.0.50, that should be:

zone home-sweet-home.gr {
primary 192.168.0.50;
key rndc-key;
}
zone 0.168.192.in-addr.arpa. {
primary 192.168.0.50;
key rndc-key;
}

Now some generic configuration stuff to allow dynamic dns updates:

ddns-update-style interim;
ddns-domainname "home-sweet-home.gr.";
use-host-decl-names on;
allow client-updates;
option oe-key code 159 = string;
option oe-gateway code 160 = ip-address;
on commit {
if (not static and
((config-option server.ddns-updates = null) or
(config-option server.ddns-updates != 0))) {
if exists oe-key {
set ddns-rev-name =
concat (binary-to-ascii (10, 8, ".",
reverse (1, leased-address)), ".",
pick (config-option server.ddns-rev-domainname,
"in-addr.arpa."));
set full-oe-key = option oe-key;
switch (ns-update (delete (IN, 25, ddns-rev-name, null),
add (IN, 25, ddns-rev-name, full-oe-key,
lease-time / 2)))
{
default:
unset ddns-rev-name;
break;
case NOERROR:
on release or expiry {
switch (ns-update (delete (IN, 25, ddns-rev-name, null))) {
case NOERROR:
unset ddns-rev-name;
break;
}
}
}
}
}
}

(weird huh ? Nope that’s not mine … I just copied it from someplace on the web)

I will now show you my subnet configuration. Change it to fit your needs:

subnet 192.168.0.1 netmask 255.255.255.0 {
range 192.168.0.100 192.168.0.200;
option domain-name-servers 192.168.0.50;
option domain-name "home-sweet-home.gr";
option routers 192.168.0.10;
default-lease-time 600;
max-lease-time 7200;
update-static-leases on;
host PC1 {
hardware ethernet 00:12:34:56:78:ab;
fixed-address 192.168.0.21;
option host-name "PC1";
DDNS-hostname "PC1";
}
}

With this configuration, all clients on the subnet will be given IP addresses in the range 192.168.0.100 – 192.168.0.200, except from the PC with MAC address 00:12:34:56:78:ab that will be given IP 192.168.0.21. The ” update-static-leases on;” is very important for PC1. Unless it is defined no updates to DNS records would be created for it.

Now restart DHCPd, make a client DHCP request and check your log file (/var/log/messages probably) for the results.

You can read more about the ddns server subject at the following places:
http://kristijan.org/?q=node/69
http://alex.kruijff.org/FreeBSD/Dynamic_DNS.html
http://www.ops.ietf.org/dns/dynupd/secure-ddns-howto.html
http://www.southwestern.edu/pipermail/netreg/2005-October/001074.html

It’s quite usefull if you have a need for it 🙂

Netroute firmware 577

After asking some people, I have finally decided to post netroute’s firmware version 577 on my blog.

I hope I won’t be forced to remove it any time soon…so here it is: netroute-firmware-577.tar.gz

I won’t publish any update instructions. You are on your own…I don’t want to be held responsible if you bork your router…

Audacious 1.2 encoding problems with gentoo

Yesterday I updated audacious on my gentoo box from ver 1.1 to ver 1.2.1. The result that was all my songs with greek tags became unreadable.
It seems that in the 1.2.1 version a new use flag was introduced: “chardet”.. So the solution is:

echo "media-sound/audacious chardet" >> /etc/portage/package.use
echo "media-plugins/audacious-plugins chardet" >> /etc/portage/package.use
emerge -avt audacious audacious-plugins

then open audacious–>Preferences–>Playlist–>Metadata–>”Fallback Character encodings” ISO-8859-7

hope it helps some of you…

Guli LiveCD

It’s been a while now, since May 2006, that me and some others from the local Ioannina LUG are trying to create a new livecd, the Guli LiveCD. This livecd is geared mostly towards students, scientists and developers. It’s geared towards us…it’s something that will make our lives easier while travelling or while visiting a friend or … or … or …

It is Gentoo based, severy influenced by DSL (damn small linux) as well as other minimalistic livecds, but it’s size has grown to a full CD due to the hundreds of applications we wanted to add. There is of course X windows support, with fluxbox as a window manager, but it is not started by default, yet.

There are a few main categories of applications:

  • Networking
  • There are a lot of networking tools inside this livecd what will help you identify and correct problems inside your network. These include scanners, packet sniffers, tunneling software, and so on…

  • Scientific
  • Scientific applications were included in order for some of us to be able to travel and still have a way to implement a new idea that strikes us. The main interest so far is Chemistry, Mathematics and Physics. Full tex/latex support is included.

  • Developing
  • Since many of the ILUG members are students of the Computer Science department it was inevitable that many development tools would be included. There is support for many languages (C, fortran, Java, Haskell, prolog, php, python, perl, ruby,etc), with their compilers and debuggers. The full man pages are also kept so that they serve as a reference for those who want them.

  • Disaster recovery
  • Last but not least we have included a lot of tools that could salvage your system in case of an emergency. These are file recovery tools, password reset tools (even for windows) and lots of others.

    There are of course applications like firefox, sylpheed, abiword, gnucalc, gaim, skype, etc included on Guli for everyday tasks.

    The total number of executables on this LiveCD is quite large, a double tab pressing on the console will give you something like this:
    Display all 3355 possibilities? (y or n)

    And there are more to come…

    The current version was released yesterday and it is an anniversary edition for ILUG’s 1 year of existence. You can find more information, full list of packages and the download link about this live cd at ILUG’s forums (currently only in Greek, until Guli gets a proper website).

    This liveCD is not geared towards new users who want to see a fancy livecd with XGL and stuff like that. People who have never before used linux might find it a bit diffucult to navigate through the livecd.

    We would appreciate any comments/bugs either on the forum or on the bug tracker

    A disk cataloger with sql support!

    I’ve written before about me looking for a disk cataloging tool in linux. I think I’ve finally found what I wanted. It’s called gcdkatalog.

    The good things about it:
    1) It has mysql/sqlite support
    2) It’s FAST
    3) does not crash while searching (at least did not crash yet 😛 )

    The bad things:
    1) No documentation AT ALL
    2) I have no idea whether the author stills maintains it or not. You can’t even find a link to download the latest version from his homepage. I’ve found it by guessing…
    2) The code is written by a polish guy and the code comments are also in polish. This can be a problem if you want to change something in the code.

    Anyway…I’ve posted an ebuild of gcdkatalog on Gentoo bugzilla. The ebuild is currently only for ~x86 since I have no amd64 machine to test if it compiles, if you do own one and have 2 spare minutes, please test it and inform me about it. Let’s hope it will eventually make it inside the portage tree. Until then grab the ebuild from bugzilla.

    gcdkatalog ebuild for Gentoo

    P.S. Before gcdkatalog I used Gwhere, but it is lacking sql support. If you don’t mind about it…it’s by far the most advanced cataloging tool for linux I’ve seen.

    Netroute2 stats with rrdtool

    On my previous port about netroute2 I said that it was really easy to grab some stats with mrtg about line activity. This has a small drawback, you must have your collector/server that will run mrtg always open so that it can constantly get the snmp statistics from the modem/router. So my quest was to figure out a way to keep the statistics on the modem and be able to get them when I want and create the graphs that I like. This is not something mrtg can do, this is an rrdtool job 🙂

    Most things said here are based on my previous post about netroute2: Intracom netroute2 hacks. You will definitely need to have read it before continuing…

    Tweaking the Netroute2:
    The first thing was to create a script to give me information about incoming and outgoing traffic on ppp0 interface. I also to export these stats on a file “parsable” by rrdtool. The following script does exactly that:


    #!/bin/sh
    ifconfig="/sbin/ifconfig"
    grep="/bin/grep"
    date="/bin/date"
    cut="/usr/bin/cut"
    tail="/usr/bin/tail"
    delim=":"
    OUTPUT="/tmp/var/for_rrd.log"
    RX=`$ifconfig ppp0 | $grep -A 6 ppp0 | $tail -n 1 | $cut -d":" -f2 | $cut -d" " -f1`
    TX=`$ifconfig ppp0 | $grep -A 6 ppp0 | $tail -n 1 | $cut -d":" -f3 | $cut -d" " -f1`
    echo `$date +%s`"$delim$RX$delim$TX" >> $OUTPUT

    What this script does is parse the output of ifconfig ppp0, find the number of bytes for incoming and outgoing and store them to /tmp/var/for_rrd.log. Remember that netroute2 has /tmp writable, so we can use/abuse it 🙂
    The output of the script is something like this:

    TIMESTAMP:OUTGOING:INCOMING

    1160651881:7938452:1310405

    So now we can have the stats stored someplace. It’s time to tell netroute2’s cron to do it every X minutes.

    # cat /etc/cron.d/cron_rrdscript
    */1 * * * * root /etc/rrdscript.sh > /dev/null 2>&1

    Now, every 1 minute our log is going to be filled with our traffic. You did remember to run /etc/init.d/checkpoint … didn’t you ? If you don’t know why you should do it…DON’T continue reading on!!! Go back and read this: Intracom netroute2 hacks

    RRDTool time now…
    I suppose you have already installed rrdtool on your favorite distro somehow (emerge,apt-get,rpm,blahblahblah). First thing we need to do is create an rrd database to keep our stats. Save the following in create_netroute2_rate.txt

    rrdtool create netroute_rate.rrd --start 1159455804 \
    DS:IN:COUNTER:600:0:12500000 \
    DS:OUT:COUNTER:600:0:12500000 \
    --step 60 \
    RRA:AVERAGE:0.5:1:2000 \
    RRA:AVERAGE:0.5:6:2000 \
    RRA:AVERAGE:0.5:24:2000 \
    RRA:AVERAGE:0.5:288:2000 \
    RRA:MAX:0.5:1:2000 \
    RRA:MAX:0.5:6:2000 \
    RRA:MAX:0.5:24:2000 \
    RRA:MAX:0.5:288:2000

    BIG FAT WARNING!! You HAVE to change the timestamp –start 1159455804 with the current timestamp if you want to get accurate stats. 1159455804 was when I started getting my stats. You MUST change this with your current one. The command “date +%s” is your friend in this…

    Now execute sth like this: “/bin/sh create_netroute2_rate.txt”. You should have a beautiful netroute_rate.rrd file inside your current dir. Time to fill up this database with information from the file “for_rrd.log”.
    If you don’t have yet public key authentication with your netroute2, now is the crucial time to do it. Paragraph F) of my previous post (Intracom netroute2 hacks) is exactly that.

    (timelapse)

    I am supposing you already did it…To check if it’s working issue something like this:
    ssh -l root -p 22 PUT.IP.OF.NETROUTE2.HERE uname -a
    Hopefully you’ll get prompted with something like: “Linux netroute- 2.4.30-netta2 #2 Tue Mar 21 10:51:17 EET 2006 ppc unknown”. If not check again what went wrong…

    Now a command like the following will grab “for_rrd.log” from netroute2 and put it in /var/rrds/ (that’s the dir I am using to store my rrd databases, logs, scripts, etc)
    scp root@PUT.IP.OF.NETROUTE2.HERE:/tmp/var/for_rrd.log /var/rrds/for_rrd.log

    Now..parsing time! This is my netroute-rrd.sh:

    #!/bin/sh
    cd /var/rrds/
    NEWFILE="for_rrd.log"
    OLDFILE="for_rrd.log.old"
    if [ ! -f $NEWFILE ]; then
    echo "No new file found"
    exit 1
    fi
    diff $OLDFILE $NEWFILE | grep -v "," | sed 's/^>[ \t]\+//' > to_parse
    cat to_parse |while read line;
    do
    echo "${line}";
    rrdtool update netroute_rate.rrd $line
    echo "$?"
    done
    rm -f to_parse
    mv for_rrd.log for_rrd.log.old

    If you execute this script it will compare this script with the older one you had downloaded, diff them and parse the output with rrdtool. I do that to ensure that rrdtool only parses what was not parsed before. It’s really stupid to parse the log file everytime from the beggining…

    Now you have your database filled up with values…so what ? how can you see the results ? With a script that takes advantage of rrdgraph 🙂 This script is quite big, more than 150lines so I will not copy/paste it here but I will provide you with a tarball of it and all the previously mentioned scripts ment to be stored on your box.
    Netroute2 RRDTool Scripts

    Combine all these together with a cron job from your pc like the following:
    */5 * * * * scp root@PUT.IP.OF.NETROUTE2.HERE:/tmp/var/for_rrd.log /var/rrds/for_rrd.log ; /etc/scripts/netroute-rrd.sh; /etc/scripts/netroute-rrd-graph.sh

    and you will have a /var/rrds/index.html with 5 graphs: Hourly, Daily, Weekly, Monthly and Yearly.

    Now you can monitor your Internet traffic even when your stats collector is offline.
    There is a chance that you will get some spikes when netroute2 reboots. To eliminate them read this: HOWTO remove spikes from RRD graphs

    If you are greek and want to know some more stuff about rrdtool there’s a tutorial here: RRD [HOWTO]

    Intracom netroute2 hacks

    Since I now own my own dsl I am using an intracom netroute2 as a dsl modem/router. What really rocks about it is that it is linux (busybox) based, so one can change a lot of how stuff works in it. The bad news about it is that Intracom no longer produces it! The good news is that someone might be able to get some second hand of those modems really cheap. I wonder who was the idiot to stop the production of those modems in Intracom…

    Anyway…I’ll post here some of the changes I did to it . I am using firmware version 577 so everything I say here might not work on older ones.

    A) Fix ntp.
    The modem/router on boot reads /etc/date and changes the date according to that file. My version dated from april 2006 made the modem on boot to think that it was april. It is easy to fix this using an ntpclient cronjob. So create a file inside /etc/cron.d/ named cron_ntpclient and paste the following inside it:
    0,15,30,45 * * * * root /usr/bin/ntpclient -s -h ntp.grnet.gr > /dev/null 2>&1
    This will sync the time on the modem every 15minutes.
    If you want to be even more accurate like I do check out the following. I had noticed that my modem takes about 1 min and 20 secs from the time I plugged it in till the time it got IP from my ISP. So if I could make it to ntp sync immediately after getting an IP it would be pretty cool. So, I edited /etc/date and replaced the contents of the file with the following:
    091918002006
    This makes the modem to set its date at 18:00 19-September-2006.

    Then change the cron script to this:
    02,15,30,45 * * * * root /usr/bin/ntpclient -s -h ntp.grnet.gr > /dev/null 2>&1

    Now the modem syncs the date on the next minute after it gets IP from the ISP. This is really helpfull if you use syslog (read on for this).

    BEWARE though that BEFORE you reboot you MUST run /etc/init.d/checkpoint. This command is VITAL! It saves the changes you do inside the configuration files so they are not overwritten on the reboot by the stuff inside /etc-ro/.

    B) This version has snmpd installed but not started by default. So let’s make it start when netroute2 boots.
    The only thing that needs to be done for this is add an entry inside /etc/net.conf. You need to add somewhere around the end an entry like this:

    SNMP_RO_COMMUNITY=YOUR_SNMP_COMMUNITY

    Change ‘YOUR_SNMP_COMMUNITY’ with a password of your choice. You can use vi to do it. Run /etc/init.d/checkpoint afterwards to save your changes.

    You can now reboot Netroute2 and if you type ‘ps aux’ on it’s shell you will probably see something like this:
    408 root 840 S /sbin/snmpd -c /etc/snmpd.conf -l /dev/null
    That means snmpd is now listening for connections. You can now monitor the traffic with mrtg if you want from another PC. I’ll post my own mrtg.conf for the netroute2 but it does not necessarily mean that it will work for you too. It depends on which ethernet devices you use to connect it to your network. I use eth0.
    Here’s the mrtg.conf for my netroute2: netroute2-mrtg.conf

    Use it with your other mrtg scripts, I won’t tell you how here…if you don’t know…google for it. It’s realy simple. Here’s a HOWTO from the Gentoo Wiki HOWTO_SNMP_and_MRTG_Made_Easy for the lazy ones.

    C) Syslog activation.
    A syslog daemon is also included but not started in netroute2. It is primarily thought to be logging remotely to another machine. But netroute2 has 10Mb of free space at /tmp…so we can make it store some logs there. On reboot these logs are lost, so it’s only usefull if you netroute is on a UPS or if you want to monitor what’s going on while netroute is working. It’s not for watching what happened and your netroute2 crashed. If you want such behaviour you must activate remote syslog logging, and if you want that you must be able to easily find out how to do it, you don’t need my advice 🙂

    First of all fire up vi and edit /etc/syslog.conf. Change all entries that start with /var/log to /tmp/var/log/. Then go find /etc/init.d/rc-syslog and edit it with vi. Find line 13. It should say something like this:

    /bin/echo "Invalid Syslog Server IP!"
    exit 1

    Change it to:

    /bin/echo "Invalid Syslog Server IP!"
    /bin/echo "Logging Locally!"
    mkdir -p /tmp/var/log/
    #exit 1

    What happens here is this, the script searches /etc/net.conf for a variable named SYSLOG_SERVER_IP. If it finds it it enables remote syslogging to that IP (damn!.. I’ve said it), but we want to enable local logging so we have to bypass it. We first create the /tmp/var/log directory and then put a “#” in front of the exit command so the script does not stop when it can’t find SYSLOG_SERVER_IP.

    /etc/init.d/checkpoint again….reboot…and if all went fine if you “ps aux” you should see something like:

    166 root 336 S /sbin/syslogd
    169 root 264 S /sbin/klogd -c 3 -x

    If you cd to /tmp/var/log you can find the log files. You can more them, tail them, grep them, whatever…it’s up to you now.

    D) Enable iptables logging.
    Netroute2 features iptables firewalling and tc for QoS!!! You can easily modify the firewall script to make it suit your needs better. What I am going to describe here is how to make iptables log to syslog so you can view what is being blocked by iptables.

    First thing you have to do is go to the web interface (yes it IS sloooooooow but who cares ? You won’t be needing much anyway 🙂 ), go to “Wan Connections” and click on Firewall. Go where it says “Expert Configuration” (don’t you already feel l33t ? heh) and add a dummy line like this:

    iptables -A INPUT -i ppp0 -p tcp --dport 135 -j DROP

    Click on apply and wait a bit until it refreshes the firewall webpage.
    This line, even it is totally useless (if you know a bit of iptables and see the rest of the firewall listing I am sure you already know why), it will force netroute2 to create a seperate firewall file for your connection and not use its default one.

    Now go back to netroute2's shell. If you don't know/remember the name of your connection do an
    ls /etc/wan/current/
    and it will show you it's name. Then go to /etc/wan/firewall/ and edit the file with your connection name. You will see inside it the full listing of iptables rules.

    Lines 25-27 should a bit like these:

    :INPUT ACCEPT [0:0]
    :FORWARD ACCEPT [43:9993]
    :OUTPUT ACCEPT [73:33579]

    Add below the last one a line like the following:

    :LOG_DROP - [0:0]

    Around line 65 there should be something like this:

    -A INPUT -i ppp0 -j DROP

    change it to:

    -A INPUT -i ppp0 -j LOG_DROP

    Then finally find the last line of the file that says COMMIT and add above it these two lines:

    -A LOG_DROP -j LOG
    -A LOG_DROP -j DROP

    The editing is finished. DON’T forget to /etc/init.d/checkpoint….reboot and you are ready.

    If you had previously enabled syslog logging correctly you can do a:
    tail -f /tmp/var/log/kernel.log

    and if you firewall blocks something you might see a line that looks like this:
    Sep 24 02:16:57 (none) kernel: IN=ppp0 OUT= MAC= SRC=XX.XX.XX.XX DST=YY.YY.YY.YY LEN=48 TOS=0x00 PREC=0x00 TTL=128 ID=30976 DF PROTO=TCP SPT=3935 DPT=135 WINDOW=65535 RES=0x00 SYN URGP=0

    You can change all the other “-j DROP” settings inside your firewall config with “-j LOG_DROP” to gather more information about what gets dropped. If you do that though it might be necessary to refine your /etc/syslog.conf to create more accurate log files..Which log files keeps what kinds of logs…play with it.

    E) Netroute has some advanced commands that are not inside the PATH.
    Look for some very interesting binaries like tcpdump, tc, etc inside /usr/sbin

    F) Public key auth with dropbear:
    on client machine:

    ssh-keygen -t rsa
    scp .ssh/id_rsa.pub root@netroute.ip:

    on netroute2:

    mkdir .ssh
    mv id_rsa.pub .ssh/authorized_keys
    chmod 400 ~/.ssh/authorized_keys
    chmod 700 ~/.ssh
    /etc/init.d/checkpoint

    If everything is done properly you will now be able to ssh from your client to netroute2 without password.

    I wish I could find more of these great machines. 2 Ethernet ports, usb, serial console, 2 FXS, it’s own small sip compatible PBX … what else can one need ? A usb port for an external disk and a torrent client probably… 🙂

    XTerm*saveLines possible problem ?

    I just noticed a somewhat strange behaviour of xterm. When savelines option is set to a high number, say above 10000, the output of xterm is REALLY REALY slow.

    Check the following examples on maximized xterm windows:
    File: .xsession-errors 200074 (200kbytes)

    % wc -l .xsession-errors
    4103 .xsession-errors
    % xterm -v
    XTerm(218)
    % mrxvt --help
    Mrxvt v0.5.1
    Options: XPM,Jpeg,PNG,transparent,fade,tint,textshadow,utmp,menubar,XIM,multichar_languages,scrollbars=rxvt+NeXT+xterm+sgi+plain,
    xft,Greek,frills,linespace,24bit,selectionscrolling,256colour,cursorBlink,pointerBlank,session management,Resources

    XTerm*saveLines: 500
    % time cat .xsession-errors
    cat .xsession-errors 0.00s user 0.00s system 0% cpu 0.933 total

    XTerm*saveLines: 1000
    % time cat .xsession-errors
    cat .xsession-errors 0.00s user 0.00s system 0% cpu 1.029 total

    XTerm*saveLines: 2500
    % time cat .xsession-errors
    cat .xsession-errors 0.00s user 0.01s system 0% cpu 1.286 total

    XTerm*saveLines: 5000
    % time cat .xsession-errors
    cat .xsession-errors 0.00s user 0.00s system 0% cpu 1.382 total

    XTerm*saveLines: 10000
    % time cat .xsession-errors
    cat .xsession-errors 0.00s user 0.00s system 0% cpu 1.969 total

    XTerm*saveLines: 11000
    % time cat .xsession-errors
    cat .xsession-errors 0.00s user 0.00s system 0% cpu 3.038 total

    XTerm*saveLines: 20000
    % time cat .xsession-errors
    cat .xsession-errors 0.00s user 0.00s system 0% cpu 8.341 total

    XTerm*saveLines: 32767
    % time cat .xsession-errors
    cat .xsession-errors 0.00s user 0.01s system 0% cpu 12.798 total

    On my faithfull mrxvt with 32767 buffer lines this gives me:
    mrxvt*saveLines: 32767
    % time cat .xsession-errors
    cat .xsession-errors 0.00s user 0.01s system 0% cpu 1.494 total

    I hope you notice the difference…

    “bug” or “feature” ? who knows…