Show HN: Uberblok – an interactive museum of ZFS internals
🏛️ Uberblok: An Interactive Gallery of ZFS Internals
Uberblok serves as a digital museum dedicated to the inner workings of ZFS, allowing users to explore the filesystem by reading its actual source code.
Storage systems have historically fought a constant war against unreliable hardware:
- Disks that report false data (silent corruption).
- Writes that are interrupted mid-stream.
- System crashes occurring during critical updates.
This interactive experience guides you through these challenges, one "room" at a time, mapping animations directly to the OpenZFS source (specifically version zfs-2.4.4, frozen at commit 71a9f9578616).
🖼️ The Hall: The Uberblock
The centerpiece of the museum is the struct uberblock, found in include/sys/uberblock_impl.h (lines 122–128).
struct uberblock {
uint64_t ub_magic; // Magic number: 0x00bab10c
uint64_t ub_txg; // Transaction group of the last sync
blkptr_t ub_rootbp; // Block pointer to the MOS (root of the tree)
// ... other fields ...
};
The Purpose of the Uberblock
A ZFS pool consists of millions of blocks scattered across multiple disks. To avoid scanning every single block after a crash, ZFS uses a single, small record to identify the current consistent state: the uberblock.
It contains two vital pieces of information:
ub_txg: The transaction group number.ub_rootbp: The pointer to the MOS (Meta Object Set), from which every other block in the pool can be reached.
The Ring Architecture
To ensure safety, ZFS doesn't overwrite the current uberblock. Instead, it uses a ring buffer system:
- Placement: Each disk has four labels (two at the start, two at the end).
- Storage: Each label contains a 128 KiB ring of uberblocks.
- Rotation: A new uberblock for transaction group is placed in a slot determined by:
(Note: If
multihost=on, one slot is reserved for MMP, reducing the available ring size by one).
Import Process: Upon importing a pool, ZFS scans every slot on every label and selects the valid uberblock with the highest ub_txg.
Capacity and Sectoring
The total space is 128 KiB. The number of available slots depends on the disk's sector size (ashift):
| Sector Size (ashift) | Slot Size | Total Slots |
|---|---|---|
| 512B (ashift 9) | 1 KiB | 128 |
| 1 KiB (ashift 10) | 1 KiB | 128 |
| 2 KiB (ashift 11) | 2 KiB | 64 |
| 4 KiB (ashift 12) | 4 KiB | 32 |
| 8 KiB+ (ashift 13+) | 8 KiB | 16 |
Interactive Tip: In the museum, you can adjust the sector size to see how subsequent
txgsoccupy different slots. You can also zoom () and rotate the 3D view.
🗺️ Map of the Visit
The museum is organized to follow the flow of data, from the root of the tree to the underlying hardware.
- 00 Hall: The uberblock and its protective ring.
- 01 History Wing: A sourced chronicle from the first lines of code to OpenZFS.
- 02 The Tree & Uberblock: Understanding block pointers and the "atomic" write that updates the tree.
- 03 CoW, Checksums & Self-Healing: How ZFS avoids
fsckand detects "lying" disks. - 04 Transaction Groups: The process of turning thousands of writes into a single commit (Open Quiescing Syncing).
- 05 The Vdevs: The device hierarchy (Mirrors, RAID-Z, dRAID, Log, Cache, Special).
- 06 RAID-Z: Variable-width stripes and the elimination of the "write hole."
- 07 Snapshots & Clones: Managing birth times and deadlists.
- 09 Metaslabs & Space Maps: Efficient free-space discovery via append-only logs.
- 11 Send / Receive: Treating datasets as streams for incremental backups.
- 12 Pipeline Transforms: The costs and benefits of Compression, Dedup, and Encryption.
- 13 Engine Room: Technical details on how the museum itself was constructed.
📜 A Legacy of Engineering
ZFS was built on a fundamental distrust of hardware. As Bill Moore put it:
"Never, ever trust the underlying hardware."
This museum is a tribute to the pioneers of this architecture:
- Jeff Bonwick: Lead developer.
- Bill Moore: Co-lead.
- Matt Ahrens: One of the two original authors.
The project began at Sun Microsystems in 2001. By the time it integrated into Solaris on October 31, 2005, the effort had spanned:
- 5 years of development.
- 56 patents.
- 92,000 lines of changes across 584 files.
Bonwick reflected on the magnitude of the achievement:
"I’m incredibly proud of the ZFS team, and deeply grateful: no one person can do something on this scale."
The museum also honors Sun's culture of transparency and academic contribution, citing milestones like the Slab Allocator (1994) and DTrace (2004), and the eventual release of ZFS under the CDDL. Today, this legacy is carried forward by OpenZFS (est. 2013), a community effort involving Linux, FreeBSD, macOS, and illumos, with over 700 contributors listed in its AUTHORS file.
🛠️ The Physical Layer
To ground the software in reality, the museum references the physical 3.5-inch hard drive:
- Top cover & Base casting
- Disc clamp & Platters
- Head stack (Arms and heads)
- Voice coil (and magnet)
- Spindle motor