Back to my home page

dummynet

1. Description

dummynet is a flexible tool originally designed for testing networking protocols, and since then (mis)used for bandwidth management.

It simulates/enforces queue and bandwidth limitations, delays, packet losses, and multipath effects. It also implements a variant of Weighted Fair Queueing called WF2Q+. It can be used on user's workstations, or on FreeBSD machines acting as routers or bridges.

Just to get the idea of what you can do with dummynet, e.g. by using dummynet on your workstation, or putting a PC with two ethernet cards between your network and your router and booting from the floppy-image below, here are a few examples :

These rules limit the total ICMP traffic (inbound+outbound) to 50Kbit/s

    ipfw add pipe 1 icmp from any to any
    ipfw pipe 1 config bw 50Kbit/s queue 10

These rules limit inbound traffic to 300Kbit/s for each host on your network 10.1.2.0/24.

    ipfw add pipe 2 ip from any to 10.1.2.0/24
    ipfw pipe 2 config bw 300Kbit/s queue 20 mask dst-ip 0x000000ff
If you want all machines to share evenly a single link, you should use instead:
    ipfw add queue 1 ip from any to 10.1.2.0/24
    ipfw queue 1 config weight 5 pipe 2 mask dst-ip 0x000000ff
    ipfw pipe 2 config bw 300Kbit/s

And these rules simulate an ADSL link to the moon:

    ipfw add pipe 3 ip from any to any out
    ipfw add pipe 4 ip from any to any in
    ipfw pipe 3 config bw 128Kbit/s queue 10 delay 1000ms
    ipfw pipe 4 config bw 640Kbit/s queue 30 delay 1000ms
dummynet works by intercepting packets (selected by ipfw rules - ipfw is one of the FreeBSD firewalls) in their way through the protocol stack, and passing them through one or more objects called queues and pipes, which simulate the effects of bandwidth limitations, propagation delays, bounded-size queues, packet losses, multipath. Pipes are fixed-bandwidth channels. Queues represent instead queues of packets, associated with a weight, which share the bandwidth of the pipe they are connected to proportionally to their weight.

Each pipe and queue can be configured separately, so you can apply different limitations/delays to different traffic according to the ipfw rules (e.g. selecting on protocols, addresses and ports ranges, interfaces, etc.). Pipes and queues can be created dynamically, so using a single set of rules you can apply independent limitations to all hosts in a subnet, or to all types of traffic, etc. You can also configure the system to build cascades of pipes, so you can simulate networks with multiple links and paths between source(s) and destination(s).

2. Performance, status and availability

Unlike other traffic shaping packages which run in userland, dummynet has a very little overhead, as all processing is done within the kernel. There is no data copying involved to move packets through pipes, just a bit of pointer shuffling, and the implementation is able to handle thousands of pipes with O(log N) cost, where N is the number of active pipes.

The WFQ variant we implement, called WF2Q+, has a complexity which is O(log N) in the number of active flows, so again it is able to handle efficiently thousands of flows. dummynet is part of FreeBSD since Sept.1998. It has been recently (Jan.2000 and June 2000) rewritten, so the most recent, feature-rich and robust versions are in FreeBSD 3.4-STABLE and newer releases.

You don't need to install FreeBSD on your hard disk to use it, as below you will find a bootable single-floppy version of FreeBSD which includes dummynet, bridging, and a lot of other goodies.

Dummynet is being heavily used by lots of people, and the code seems to be extremely stable and robust, especially in the 3.4-STABLE version and above. Bug fixes are generally applied to the FreeBSD source tree and are available from the CVS tree or in newer snapshot/releases of FreeBSD. From time to time i update the floppy image on this site as well.

3. Support

If you have found some bug, please report it to me by email, but don't forget to include information on which version of FreeBSD and dummynet you are using, your rules (ipfw show; ipfw pipe show), your configuration (bridge or router) etc.
If you have a simple question, again just email me and i generally try to reply as soon as possible. Again, please supply details!
For more complex things (like "i have no time to learn how to use it, i just want this work done"), or customizations and additions of new features to dummynet/ipfw, I am available (through my department) for doing support on a contract basis.
Email luigi@iet.unipi.it for discussing details.

This said, FreeBSD users should be able to use dummynet without the need for support.
The relevant manpages (ipfw(8), dummynet(4), bridge(4)) are a great source of information, so please read updated version of them before asking questions.
You can also try posting on the various FreeBSD mailing lists or newsgroups, they are usually a very good source of information.

4. Using dummynet

Dummynet is entirely controlled by the ipfw commands and a set of sysctl variables.

4.1 Basic ipfw commands

The basic structure of ipfw commands is
   ipfw add [N] [prob X] action PROTO from SRC to DST [options]
where N is the rule number ;
X is a number between 0 and 1 that, when present, indicates the probability of getting a match on this rule if all other fields are correct. The default is deterministic match;
action is one of the actions executed on a match, which can be any of allow, deny, skipto N, pipe N and others. To send a packet to a dummynet pipe, we have to use pipe N; PROTO is the protocol type we want to match (IP, TCP, UDP, ...);
SRC and DST are address specifier (we can use addresses with netmasks and optionally followed by ports or port ranges);
options can be used to restrict the attention to packets coming from/to specific interfaces, or carrying some TCP flags or ICMP options, or bridged, etc.

4.2 Sysctl variables

The following are the main sysctl variables to control the behaviour of ipfw, bridging and dummynet:

Controlling ipfw

The firewall is mostly controlled by ipfw, and the sysctl variables only serve to give global configuration and default parameters.
net.inet.ip.fw.enable: 1
	enables firewall in the IP stack

net.inet.ip.fw.one_pass: 1
	Forces a single pass through the firewall. If set to 0,
	packets coming out of a pipe will be reinjected into the
	firewall starting with the rule after the matching one.
	NOTE: there is always one pass for bridged packets.

net.inet.ip.fw.dyn_buckets: 256 (readonly)
	Current hash table size used for dynamic rules.

net.inet.ip.fw.curr_dyn_buckets: 256
	Desired hash table size used for dynamic rules.

net.inet.ip.fw.dyn_count: 3
	Current number of dynamic rules. (readonly)
	
net.inet.ip.fw.dyn_max: 1000
	Max number of dynamic rules. If you exceed this limit, you will
	have to wait for a rule to expire before being able to create
	a new one.

net.inet.ip.fw.dyn_ack_lifetime: 300
net.inet.ip.fw.dyn_syn_lifetime: 20
net.inet.ip.fw.dyn_fin_lifetime: 20
net.inet.ip.fw.dyn_rst_lifetime: 5
net.inet.ip.fw.dyn_short_lifetime: 5
	Lifetime (in seconds) for various types of dynamic rules.

Controlling dummynet

Also dummynet is mostly controlled by ipfw, with the sysctl variables serving mostly for default parameters.
net.inet.ip.dummynet.hash_size: 64
	Size of hash table for dynamic pipes.

net.inet.ip.dummynet.expire: 1
	Delete dynamic pipes when they become empty.

net.inet.ip.dummynet.max_chain_len: 16
	Max ratio between number of dynamic queues and hash buckets.
	When you exceed (max_chain_len*buckets) queues on a pipe,
	packets not matching any of these will be all put into the
	same default queue.

Controlling bridging

Bridging is almost exclusively controlled by sysctl variables.
net.link.ether.bridge_cfg: ed2:1,rl0:1,
	set of interfaces for which bridging is enabled, and cluster
	they belong to.

net.link.ether.bridge: 0
	enable bridging.

net.link.ether.bridge_ipfw: 0
	enable ipfw for bridging.

4.3 Pipe and queue configuration

The following ipfw commands control dummynet pipes The following parameters can be configured for a pipe, adding the command in the pipe config... line: To use WF2Q+, packets must be passed to queues which in turn must be connected to a pipe.

The following ipfw commands control dummynet pipes

The following parameters can be configured for a queue, adding the command in the queue config... line:

5. Using dummynet for testing protocols

Dummynet was originally created to test network protocols and applications, possibly even on a standalone system. As a consequence, some of its features such as delay emulation, random loss etc. are explicitly designed for that purpose.

There are a few things you should take in mind when doing such tests, to avoid getting incorrect results. They are all obvious things, still it is better to have them in mind.

5.1 Simulating multipath

One nice feature of the new version of dummynet is the ability to simulate multiple paths between sender and receiver. This is done using probabilistic match, e.g.:
    ipfw add prob 0.33 pipe 1 ip from A to B
    ipfw add prob 0.5  pipe 2 ip from A to B
    ipfw add           pipe 3 ip from A to B
    ipfw pipe 1 config ...
    ipfw pipe 2 config ...
    ipfw pipe 3 config ...
Given the right packet, the first rule will match with probability 1/3; in the remaining 2/3 of occurrence we move to the second rule, which will match with prob 1/2 (so overall 1/2*1/3 = 1/3), and the remaining 1/3 of occurrence will move to the third rule, which has a deterministic match. We can then configure the three pipes as desired to emulate phenomena such as packet reordering etc.


6 Related links

Here i collect some info on how to do various ipfw-related things. Most of this is just URLs collected from the mailing list so the reliability of the info might be different (for good or bad) from what is in this page.

A PicoBSD floppy

To conclude... if you want to try dummynet, here is a bootable floppy image of a system with FreeBSD, bridging, ipfw, dummynet, natd, ppp, drivers for a few interfaces, and accessible via telnet.

To setup this system, download the 1.44MB image, pico.000608.bin and copy it to a floppy using dd under FreeBSD, or rawrite under DOS/Windows.

Then put the floppy into a machine with hopefully at least one interface, and wait for it to boot. When the system comes up, login as root, password "setup", and you can play with bridging, ipfw and dummynet using the above commands.


Luigi Rizzo
Dipartimento di Ingegneria dell'Informazione -- Univ. di Pisa
via Diotisalvi 2 -- 56126 PISA
tel. +39-050-568533 Fax +39-050-568522
email: luigi@iet.unipi.it