The Hetzner Cloud network stack – history and technical overview
The Evolution and Architecture of the Hetzner Cloud Network Stack
TL;DR: To ensure robust and flexible connectivity for our users, Hetzner Cloud utilizes a host-centric network architecture. We transitioned from basic Linux bridges to a custom-orchestrated Open vSwitch (OvS) implementation, enabling advanced features like stateful firewalls and isolated private networks while maintaining full operational control.
This article serves as the opening installment of a series detailing the technical journey and current state of the networking layers powering Hetzner Cloud. We will explore the progression from early vServer setups to the modern Open vSwitch-based stack.
Note: While Load Balancers share similar logic, this discussion focuses specifically on VM hosts and Cloud servers. The overarching Cloud Control Plane is considered for this technical deep dive.
🛠️ Our Engineering Philosophy: "How We Roll"
At Hetzner, our architectural guiding principle is simple: Own the stack. We prefer building on modular blocks that we can operate, modify, and scale independently.
Rather than relying on "smart" physical network hardware—which often locks a provider into specific vendor ecosystems—we push the intelligence to the edge. In our model, the physical network is kept intentionally simple, providing stable IP connectivity, while the VM hosts handle the complex logic.
The Host-Centric Approach
By implementing higher-level functions on the host, we achieve:
- Flexibility: We aren't limited by hardware vendor feature sets.
- Isolation: The "blast radius" of a configuration error is typically limited to a single host.
- Separation of Concerns: Clear boundaries between physical infrastructure and virtualized network services.
⏳ A Technical Retrospective
The path to our current architecture involved several iterative leaps in technology to accommodate growth and feature demands.
| Era | Launch Date | Key Technologies | Networking Logic | Scale (Approx.) |
|---|---|---|---|---|
| Early vServers | 2011 | HDD, Linux Bridges | Static routes, dual-stack | instances |
| Hyper-Converged | 2015 | Ceph, BGP | 1:1 NAT (IPv4), Routed IPv6 | instances |
| Hetzner Cloud | 2018 | Direct Routing | Dropped NAT for direct IPv4 | Scaling rapidly |
| OvS Transition | 2019-2021 | Open vSwitch, VXLAN | Encapsulated private nets Public OvS Stateful Firewalls | Current State |
Key Milestones:
- 2011: Initial launch provided connectivity up to per host.
- 2015: Shifted to BGP (Border Gateway Protocol) for dynamic routing, which significantly improved IP mobility and allowed for seamless host maintenance.
- 2019: Introduced the
Open vSwitchdata plane to support private cloud networks via VXLAN encapsulation. - 2020/21: Migrated public traffic to OvS, enabling the launch of stateful firewalls using a combination of OvS flows and
netfilter.
🏗️ Anatomy of a VM Host
For a cloud server to function, the host must provide several critical networking "ingredients." We have moved away from centralized hardware appliances in favor of host-local services.
📋 Host Requirements Checklist
- Internet Reachability: Ensuring the VM is accessible globally.
- Private Networking: Isolated communication between specific servers/load balancers.
- Traffic Filtering: A robust, stateful firewall.
- Configuration: Automated IP assignment via DHCP.
Deep Dive into Components
1. Private Networks & VXLAN
To allow users to create isolated environments, we use Virtual eXtensible LAN (VXLAN).
- Each private network is assigned a unique VNI (Virtual Network Identifier).
- The host stack ensures strict isolation: traffic is only routed between members of the same VNI.
2. The Stateful Firewall
We implement firewalls as close to the VM as possible.
Why not centralize? Centralized firewalls would require complex network overlays to transport "clean" traffic and would create massive bottlenecks for connection tracking (conntrack).
By using Open vSwitch flows and netfilter on the host, we can precisely control:
- Ingress: Which ports/protocols can reach the server.
- Egress: Which traffic the server is permitted to send.
3. DHCP Services
Most cloud images are configured to use DHCP for their initial setup. The host provides the necessary DHCP server to hand out IPv4 addresses and routing tables for both public and private interfaces.
Summary of the Current Stack
The current state of the Hetzner Cloud network is a custom-orchestrated environment (via our internal tool, Flusskrebs) that manages Open vSwitch to provide a high-performance, flexible, and scalable networking experience.
Author Profile:
