Building Dreamhack, part three

DHCP design for IPv4 on Dreamhack

I will describe the protocol DHCP in general and specific the DHCP design that we use at Dreamhack for IPv4.

DHCP for IPv4
DHCP is a layer 3 protocol used for dynamic assignment of ip addresses and options to clients. The client device sends a layer 3 broadcast to on the local network destination UDP port 67. This message is called a DHCP discovery and it is a request for a free ip with options. The server answers the broadcast on UDP port 68 with a DHCP offer. This offer contains information about IP, subnet mask, lease time, options and the ip address of the DHCP server. The client then sends a DHCP request to the DHCP server accepting the offered lease. When the server receives the DHCP request it sends back a DHCP acknowledgement with lease duration and options.
When half of the lease time has gone the client tries to renew its lease by sending a DHCP request message to the DHCP server. If the client does not get a response from the server it will continue to send DHCP request messages to the specific DHCP server on a regular interval. When the lease time ends the client will begin the process from start by sending a DHCP discover.

DHCP design at Dreamhack
At Dreamhack all the clients use DHCP for configuring IPv4, subnet mask, default gateway, SMTP, TFTP, DNS and NTP servers.

For hardware redundancy we have three DHCP servers. For operating system redundancy we run Debian and FreeBSD. We have one active/primary server that syncs its lease file to the two passive/secondary DHCP servers. If the primary goes down or a severe OS related issue occurs then we can start using one of the secondary.

DHCP monitoring and statistic
We have our own developed DHCP scope monitoring and statistic system written in ruby by me 🙂 The system has two daemons and a web application.

Daemon one tails and parses the DHCP lease file, and parses the scope information. Daemon one then sends the parsed output to MySQL and MongoDB datastores. Daemon two analyzes the data in the datastores and creates statistics and graphs. This information is then made available through a web application developed with the Sinatra framework.

Building Dreamhack, part two

The next generation network or IPv6

Why change to IPv6
Every device connected to the internet need to have a IP address to be able to communicate.
Today on internet the main OSI layer 3 (network layer) protocol used is IPv4. The IPv4 addresses is 32-bit, in total that makes 4294967296 addresses. The main problem with IPv4 is that it is running out of addresses. By using NAT (Network Address Translation) the problem has been managed.

But the more internet grows, the more you need a long term solution. The long term solution is IPv6. The length of an IPv6 address is 128 bits, in total that makes 4,8 × 10^28 addresses, which is a gigantic amount. One other advantage over IPv4 is that IPv6 has the multicast specification. Multicast is the transmission of a packet to multiple destinations in a single send operation.

Configuring IPv6 address
There are three ways to set a IPv6 address:
-SLAAC(Stateless address autoconfiguration)
I will describe those three options in detail below:

You configure the IPv6 address, netmask, gateway and DNS servers manual. This method is preferable on servers and routers where the address needs to be consistent over time.

The client ask for a IPv6 address, the DHCPv6 server gives the client its IPv6 address, netmask and options like gateway, DNS servers, NTP servers etc. The DHCPv6 server know about which MAC address has what IP and for how long, which means DHCPv6 is statefull.

SLAAC(Stateless address autoconfiguration)
Every IPv6 enabled device has a link-local IPv6 address, this address is used to communicate on the local network. This link-local address is set if the client has IPv6 enabled. The link-local address always begin with FE80:: and is calculated on the client. When the client connects to a network physically, the client will use its link-local address as a source and send a router solicitation message to a specific multicast group on the local network. Every router on the local network will listen to this specific multicast group and answer the clients with a RA (Router Advertisement) message. In the RA there is a prefix specified, the client will use this prefix to calculate its own IPv6 address. The client will use the routers link-local address as the default gateway.

In the initial IPv6 specification there where no way of setting the DNS server using the SLAAC method. Instead there are a M-FLAG in the RA package specifying a DHCPv6 server that the client can connect to get options like DNS, NTP etc. Lately support for setting DNS server using the SLAAC method has been developed, but it is not supported by all operating systems.

IPv6 implementation at Dreamhack
At Dreamhack the client set its IPv6 address, prefix and gateway using the SLAAC method. In the RA there are a DHCPv6 server specified that the client use to set the DNS server.

1: Client connect to network.
2: Client calculate its own link-local address.
3: Client send out a router solicitation package on the specific multicast group using the link-local address as source.
4: Router listen on the specific multicast group and answer the client with a RA(Router Advertisement) containing the network prefix using its own link-local address as source.
5: The client calculates its own IPv6 address using the prefix in the RA.
6: The client sets the gateway to the routers link-local address.
7: Client ask the DHCPv6 server for the DNS server.
8: DHCPv6 server response with a list of DNS IPv6 servers.
8: Client is IPv6 ready 🙂

Problems with IPv6
Even if your computer, your ISP and you destination is fully IPv6 functional you can have problems with IPv6 because all ISP/routers etc where the IPv6 package travel from, source to destination needs to be IPv6 functional.

1: Client gets a IPv6 address.
2: Client now has dualstack IPv4 and IPv6 addresses.
3: Client asks the DNS server for the ip of
4: DNS server answer with the IPv6 address for
5: Client tries to communicate with the IPv6 address of
6: If IPv6 is not configured correctly form source to destination the request will timeout when it reaches the none IPv6 ready router/network.
7: Client fallbacks to IPv4.
8: Client is unhappy because request took a long time.

Building Dreamhack, part one

Dreamhack is the world’s largest digital festival and holds the official world record as the world’s largest LAN party in the Guinness Book of World Records. Last event (november 2011) the network had 13 292 uniqe devices connected.

The Dreamhack network team is responsible for planning, building, development, operations and teardown of the network. The team consists of 30 people with a great passion for technology from different companies and universities. The team is divided into four subgroups: core, services, access and logistics. I’m a member of the services group which is responsible for the services required in the network.

Part one: building anycast DNS system supporting IPV4 and IPV6
Anycast is a technology where an (anycast) IP is announced on more than one location using a routing protocol. By doing this the routing protocol thinks that it has multiple routes to the (anycast) IP when in fact there are two different endpoints with the same (anycast) IP. The routing protocol will send the client packages to the endpoint with the shortest path from the client. To achieve high availability you need to be able to remove an endpoint service when errors occur. You can do this by removing the specific route to the broken endpoint from the routing table.


In the example image above, the client computer’s request to the anycast IP will be routed to the adns server with the IP because that is the shortest path to the anycast IP If the route saying is reachable via is removed the client’s request will be routed to the server adns with IP instead.

To build our anycast DNS infrastructure at Dreamhack we use Debian GNU/Linux, Bind, iptables, ip6tables and quagga with the routing protocol BGP. We have two anycast DNS servers connected to two different Cisco ASR 9000 routers. On the servers we have loopback interfaces that have the anycast IPV4 and IPV6 address configured. We are then using iptables to forward DNS requests from the interface connected to the routers to the loopback interface. On the servers, bind is handling the DNS requests. To achieve high availability we have built a service which checks if a DNS server is unable to answer 5 different DNS request in a row. If it is, the route to that specific DNS server will be removed from the routing table making all the clients’ DNS request go to the other working DNS server.

Dreamhack anycast DNS design.

During Dreamhack winter 2011, me and my colleague Karl Andersson held a lecture where i discuss the Dreamhack anycast DNS implementation. You can find this presentation on Youtube: Dreamarena Orange – Dreamhack Behind the Scenes.


Employees at Basefarm are building the network for DreamHack

This weekend it is once again time for DreamHack – the world’s largest LAN-party. The event runs from 18th to 21st of June and for 72 hours gamers, coders and hackers from all over the world will gather in Jönköping, Sweden to compete in e-sports, creative competitions or just to hang out with likeminded people. Last winter the network hosted 12 757 unique MAC-addresses which once again meant a new Guiness world record. The network which is designed, implemented, operated and disassembled by the Dreamhack Network Crew, consists of 30 people: two of these are consultants at Basefarm.

The crew is represented by top IT-companies in Sweden and students from IT-universities and our skills and backgrounds are varying from operations, development, engineering to electricians. The core network is built using Cisco enterprise routers and switches and is connected to Internet by 2×10 Gbit connections from Telia. The access-layer consists of over 400 access-switches where the visitors connect their computers. The entire area is also covered with wi-fi and more than 20 internet streams cover the event live, including Swedish Public Service. The design process starts four months before the event and the reoccurring thing we discuss in our meetings is how to improve the network since last time.

This is in my mind the best equipment and foundation to build an enterprise network on. And by happy accident this is the exact same equipment we use in our datacenters at Basefarm. Building the network at Dreamhack is therefore like disassembling, analyzing and rebuilding our entire datacenter-core at Basefarm twice per year. After every event we sit down and analyze what we want to do better and what we want to learn for our next event.

For more information about the event, go to