One of the most painful deployment issues for Linux Traffic Control on a commodity ADSL link is the variable overhead relative to IP for each packet you transmit, as the underlying transport is actually ATM. Each ATM cell is a 53 byte fixed length with a capacity for 48 bytes of actual data. Unfortunately, much of that 48 bytes is consumed by overhead from PPP, LLC, AAL5, and so forth. This is especially bad for small packets, like TCP ACK packets which are only 40 bytes over Ethernet, but with overhead across an ATM network actually require two full ATM cells to transmit in most cases.
A resolution was deduced in great detail by Jesper Dangaard Brouer in his master thesis, who best describes the scenario therein:
An important finding is that, for all but one encapsulation method two ATM cells are needed to transport a TCP/IP control packet (of 40 bytes). That is, to transport a packet of 40 bytes, 106 bytes of ADSL capacity is used giving a[n] overhead of 66 bytes, which is an overhead of 62%. For larger packets the overhead has less impact as it constitutes a smaller percentage of the total packet size. The overhead cost per packet makes smaller packets more expensive to send.
Thankfully, he and others, most recently Russell Stuart, have made various patches available to address the situation. As effectively explained by Stuart, both a userspace patch to the iproute2 package and to the Linux kernel are necessary for best results, although a properly patched userspace tc binary is quite effective on its own.
To take advantage of these developments, you need to fetch Brouer’s kernel patch (optional, but highly recommended) and Stuart’s patch for the tc binary for which he has made available a Debian package that installs cleanly on Debian Sarge as of this writing.
With the patched tc binary, you need only determine your specific ATM overhead, which could be a variety of values. If you’re lucky, your DSL modem will cough up the information you need to make a determination yourself. If not, you can try the four values suggested by Stuart (later corrected):
The complete table, for the _outbound_ direction going over an Ethernet link is:
PPPoA + VC/Mux: tc class add htb … overhead -4 atm
PPPoA + VC/LLC: tc class add htb … overhead 4 atm
PPPoE + VC/Mux: tc class add htb … overhead 20 atm
PPPoE + VC/LLC: tc class add htb … overhead 28 atm
Presently, none of this helps with any scheduler other than htb. As its a popular qdisc (queuing discipline) this probably won’t be an issue for too many people. (It was for me, since I actually use tbf, but I’m simply moving to htb instead since I don’t know C and can’t patch tbf myself.)
Additionally as demonstrated in Brouer’s thesis, you will want to increase your HZ to a larger value than than 100, such as 1000 which is easily selectable in recent 2.6 series kernels. (Choices being 100, 250, and 1000 for 2.6.x.) Equally important, you will also want to disable htb’s hysteresis option, which can only be done at compile time. The hysteresis option trades accuracy for speed when enabled and is for htb on links far faster than a typical ADSL connection, but hurts latency on the typical ADSL uplink of 384Kbps or 512Kbps, both far slower than 100Mbps.
diff -ruN net.orig/sched/sch_htb.c net/sched/sch_htb.c --- net.orig/sched/sch_htb.c 2006-03-12 19:18:05.000000000 -0500 +++ net/sched/sch_htb.c 2006-03-12 21:14:46.000000000 -0500 @@ -73,7 +73,7 @@ #define HTB_EWMAC 2 /* rate average over HTB_EWMAC*HTB_HSIZE sec */ #undef HTB_DEBUG /* compile debugging support (activated by tc tool) */ #define HTB_RATECM 1 /* whether to use rate computer */ -#define HTB_HYSTERESIS 1/* whether to use mode hysteresis for speedup */ +#define HTB_HYSTERESIS 0/* whether to use mode hysteresis for speedup */ #define HTB_QLOCK(S) spin_lock_bh(&(S)->dev->queue_lock) #define HTB_QUNLOCK(S) spin_unlock_bh(&(S)->dev->queue_lock) #define HTB_VER 0×30011 /* major must be matched with number suplied by TC as version */