[Michael T. Babcock] Linux Quality of Service and Traffic Control


Network Quality of Service is a concept is easy to understand, but can be difficult to configure optimally. Quality of Service offerings from various networking products do different things and are aiming for different markets. One product may simply prioritize VoIP traffic to try and keep your inter-office phone calls from getting bumped by someone's Windows Update download. It may be more complex and guarantee a certain amount of traffic for each specific segment of your network for each specific network protocol. You can see that at such a level, configuration could become overwhelming.

What and why would I use this?

Quality of Service controls allow you to specify in what order and when packets are allowed to leave a network interface. Typically, packets arrive at the kernel from some application (Firefox, a ping) or forwarded through the machine, like your online gaming session, and then simply exit the machine on the appropriate network device (ppp0 or eth1 or something like that).

If you've ever started downloading a large file with BitTorrent and then tried to play an online game (on the same or another computer), you've probably realized that you're lagging and gone to pause or shut off the download while you play. Quality of Service and Traffic Control allow you to configure the system in such a way that the more important data gets sent first, or that various connections are all given more fair treatment than usual.

What can Linux do?

Linux has the ability to classify any packet that it sees into queues which can then have various disciplines associated with them. These disciplines can, in lay terms, reorder the packets based on priority, drop or delay packets to make more room for others, and much more.

There are a number of ways to classify outbound data into queues. Some types of queues allow for sub-classes (like putting ICMP packets as a sub-class of a small high-priority class -- see the example further below), and some are self-describing and have their own algorithm for then managing packets their own way.

Two of the more well-known queue types include RED (random early detection) which drops packets randomly as the queue gets closer to its peak performance to help level out maximum performance instead of hitting a brick wall of sorts and the SFQ (stochastic fairness queue) which provides a somewhat more fair way of dividing data usage between connections.

Basic design goals

You'll want to decide how you want to segregate your packets (by machine, by data type, by time of day, etc.) and then filter them one or more queues. You could simply assign SFQ as the default queue type for your network card to make for more fair sharing of the connection between applications on your PC for example (tc qdisc add dev eth0 root sfq) or create a more complex tree of classes and leaf queues as in the examples below.

Application-specific examples

Example configuration

Here's an example configuration I actually use at one site to prioritize VPN, ICMP and NTP packets over bulk data and then give priority to a few known more important computers than the rest.

#!/bin/sh

NEWCLASS="/sbin/tc class add"
NEWQUEUE="/sbin/tc qdisc add"
DELQUEUE="/sbin/tc qdisc del"

EXT="dev eth0"

EXTSPD=768
#EXTSPD=640

$DELQUEUE $EXT root 2>/dev/null
$NEWQUEUE $EXT root handle 1:0 htb default 199 r2q 5
$NEWCLASS $EXT parent 1:0 classid 1:1 htb rate 100mbit
$NEWCLASS $EXT parent 1:1 classid 1:2 htb rate ${EXTSPD}kbit \
        ceil ${EXTSPD}kbit quantum 10240

# High priority small packets
$NEWCLASS $EXT parent 1:2 classid 1:100 htb rate 5kbit \
        ceil 128kbit quantum 24 prio 2
$NEWQUEUE $EXT parent 1:100 handle 100 prio

# NTP, DNS, VPN management (UDP 123, 53, 500)
# 1:101 -  364bps average, 4-5kbps peak
$NEWCLASS $EXT parent 1:100 classid 1:101 htb rate 4kbit \
        ceil 48kbit quantum 24 prio 2
$NEWQUEUE $EXT parent 1:101 handle 101 prio
# ICMP subset of above
$NEWCLASS $EXT parent 1:100 classid 1:102 htb rate 5kbit \
        ceil 16kbit quantum 24 prio 3
$NEWQUEUE $EXT parent 1:102 handle 102 prio

###
# Known hosts
$NEWCLASS $EXT parent 1:2 classid 1:170 htb rate 64kbit \
        ceil ${EXTSPD}kbit prio 6
$NEWQUEUE $EXT parent 1:170 handle 170 sfq

# Important people
$NEWCLASS $EXT parent 1:170 classid 1:171 htb rate 48kbit \
        ceil ${EXTSPD}kbit prio 4
$NEWQUEUE $EXT parent 1:171 handle 171 sfq

# The rest of the office
$NEWCLASS $EXT parent 1:170 classid 1:172 htb rate 32kbit \
        ceil ${EXTSPD}kbit prio 7
$NEWQUEUE $EXT parent 1:172 handle 172 sfq

###
# Encrypted data
$NEWCLASS $EXT parent 1:2 classid 1:150 htb rate 256kbit \
        ceil ${EXTSPD}kbit prio 5
$NEWQUEUE $EXT parent 1:150 handle 150 sfq

# External SSH sessions (subset of encrypted data)
$NEWCLASS $EXT parent 1:150 classid 1:151 htb rate 96kbit \
        ceil ${EXTSPD}kbit prio 6
$NEWQUEUE $EXT parent 1:151 handle 151 sfq

# For support connections (subset of SSH)
$NEWCLASS $EXT parent 1:151 classid 1:152 htb rate 32kbit \
        ceil ${EXTSPD}kbit prio 3
$NEWQUEUE $EXT parent 1:152 handle 152 sfq

# Default class
$NEWCLASS $EXT parent 1:2 classid 1:199 htb rate 32kbit \
        ceil ${EXTSPD}kbit prio 12
$NEWQUEUE $EXT parent 1:199 handle 199 sfq

The data is classified into those queues using the following iptables lines:

*mangle
:QoSHandle - [0:0]
-A POSTROUTING -o eth0 -j QoSHandle 
-A QoSHandle -p icmp -j CLASSIFY --set-class 0001:0102 
-A QoSHandle -p icmp -j RETURN 
-A QoSHandle -p udp -m udp --dport 123 -j CLASSIFY --set-class 0001:0101 
-A QoSHandle -p udp -m udp --dport 123 -j RETURN 
-A QoSHandle -p udp -m udp --dport 53 -j CLASSIFY --set-class 0001:0101 
-A QoSHandle -p udp -m udp --dport 53 -j RETURN 
-A QoSHandle -p udp -m udp --dport 500 -j CLASSIFY --set-class 0001:0101 
-A QoSHandle -p udp -m udp --dport 500 -j RETURN 
-A QoSHandle -p udp -m udp --sport 500 -j CLASSIFY --set-class 0001:0101 
-A QoSHandle -p udp -m udp --sport 500 -j RETURN 
-A QoSHandle -p ipv6-crypt -j CLASSIFY --set-class 0001:0150 
-A QoSHandle -p ipv6-crypt -j RETURN 
-A QoSHandle -d $supporthost -p tcp -m tcp --sport 22 -j CLASSIFY --set-class 0001:0152 
-A QoSHandle -d $supporthost -p tcp -m tcp --sport 22 -j RETURN 
-A QoSHandle -p tcp -m tcp --sport 22 -j CLASSIFY --set-class 0001:0151 
-A QoSHandle -p tcp -m tcp --sport 22 -j RETURN 
-A QoSHandle -s 192.168.26.21 -j CLASSIFY --set-class 0001:0171 
-A QoSHandle -s 192.168.26.21 -j RETURN 
-A QoSHandle -s 192.168.26.22 -j CLASSIFY --set-class 0001:0171 
-A QoSHandle -s 192.168.26.22 -j RETURN 
-A QoSHandle -s 192.168.26.23 -j CLASSIFY --set-class 0001:0171 
-A QoSHandle -s 192.168.26.23 -j RETURN 
-A QoSHandle -s 192.168.26.1 -j CLASSIFY --set-class 0001:0170 
-A QoSHandle -s 192.168.26.1 -j RETURN 
-A QoSHandle -s 192.168.26.10 -j CLASSIFY --set-class 0001:0170 
-A QoSHandle -s 192.168.26.10 -j RETURN 
-A QoSHandle -s 192.168.26.32/255.255.255.224 -j CLASSIFY --set-class 0001:0172 
-A QoSHandle -s 192.168.26.32/255.255.255.224 -j RETURN 
*commit

BitTorrent classification

BitTorrent typically uses TCP ports 6881-6889 for data transfer. If you wish to classify these packets specifically, try the following U32 filter:


tc filter add dev eth1 parent 1:1 protocol ip prio 10 u32 \
     match ip sport 0x1AE0 0x1FF0 flowid 1:10
You'll need to change the device, parent and flowid values, but the rest is the important part; port 0x1AE0 is 6880 and with mask 0x1FF0 it matches all ports from 6880 to 6895. Because of binary netmask limits, this is probably the best you can do -- I'd just change my BitTorrent configuration to use these ports, but its default ports fall within that range nonetheless.

More importantly, for any application where you can specify a port range, feel free to specify it in such a way as to be able to easily mask off those ports. For example, 1164-1192 is a range of 32 ports that is easily masked whereas 1160-1190 is not.

See also

A few very good resources exist already for Linux QoS controls.

Stumble it! XFN Friendly Powered by DJBDNS Powered by Zope Valid CSS! Website Security Test

Served by:  Zope 2.7.6

Page Copyright © 2014, Michael T. Babcock. All Rights Reserved.

To contact me, send an E-mail to sawyoursite at this domain.

If you'd really like your mail server reported for spam, send me some junk mail to junk-yum@mikebabcock.me or devnull@mikebabcock.me. This site powered by djbdns