I made a build profiler to understand Bun's compile times
Analyzing Bun's Compilation Speed with a Custom Build Profiler
Lalit Maganti developed buildprof, an open-source tracing utility designed to pinpoint exactly where time is spent during software compilation on Linux.
A real-time demonstration of
buildprof analyzing a clean build of ripgrep.
Why Profile Builds?
While some builds are slow simply because the codebase is massive, many bottlenecks are actually inevitable fixable. Common culprits include:
- Inefficient parallelism.
- Redundant computations (repeated work).
- Slow dependency downloads.
- Overly heavy compiler or linker calls.
buildprof brings these issues to light, allowing developers to decide which optimizations are actually worth the effort.
How to Use It
Integrating the tool is straightforward. You simply prefix your existing build command with buildprof --.
# Examples of buildprof in action:
buildprof -- make -j16
buildprof -- cargo build
buildprof -- ninja -C out/target
buildprof -- just
buildprof -- ./dev/custom-build-script.sh
The tool captures every process spawned by the command, including nested subprocesses, and maps them onto a linear timeline. In this view:
- X-axis: Time flows from left to right.
- Width: The length of the bar represents the duration.
- Hierarchy: Child processes are nested beneath their parents.
The Motivation: A Bun Mystery
The inspiration for this tool came from a specific claim made by Jarred Sumner (Chief Architect of the Bun runtime). He tweeted that Bun's transition to a Rust-based build was faster on Linux compared to the previous Zig build.
"Bun’s new Rust build was >5× faster on Linux than its old Zig build."
This claim felt counterintuitive to Lalit, as Zig projects typically compile faster than Rust projects of similar scale. This discrepancy created a "mystery" that needed solving.
The LTO Variable
One subtle but critical detail in the tweet was the optimization method:
- Zig Build: Used Full LTO (Link-Time Optimization).
- Rust Build: Used ThinLTO.
Understanding LTO
Normally, compilers optimize units in isolation. LTO allows optimization across these boundaries.
| LTO Type | Method | Performance Characteristic |
|---|---|---|
| Full LTO | Merges all units into one massive optimization task. | High optimization, but very slow/serial. |
| ThinLTO | Maintains more separation between units. | Faster, allows significant parallelism. |
Lalit suspected that the difference in LTO might be the primary driver of the speed increase, rather than the language change itself.
The Experiment
To test this, Lalit set up a 6-core, 12-thread Linux VM and wrote scripts to replay the CI builds for Bun 1.3.14 (Zig) and Bun 1.4.0 (Rust).
Investigation Checklist:
- Replicate CI build steps on a single machine.
- Preserve dependency chains.
- Compare execution times.
The results confirmed the massive gap:
| Build Era | Bun's Reported CI Median | Single-Machine Replay |
|---|---|---|
| Zig Era | ||
| Rust Era |
The Technical Approach: Builds as Process Trees
To understand why the Zig build was so slow, Lalit needed a trace. Since builds involve a fragmented ecosystem of tools, he realized that the only universal constant is the Operating System's process tree.
Regardless of whether you use cargo, ninja, or make, the build system eventually spawns a chain of processes.
Advantages of Process-Level Profiling
- Build-System Agnostic: It doesn't matter if it's
MakeorZig; if it spawns a process,buildprofsees it. - Captures Everything: It includes "invisible" work like
git clonefor dependencies or custom asset scripts. - Data Flow: By tracking file I/O, the tool can visualize which process produced the input for the next.
Findings: Pointing buildprof at Bun
Lalit applied buildprof to the Zig-era CI build. The results were immediate and striking.
The Discovery:
A single invocation of the ld.lld linker dominated the entire process. It ran in isolation at the end of the build for over 16 minutes, accounting for roughly of the total build time.
By inspecting the captured command line, the culprit was confirmed: Full LTO was enabled. While the process tree identified the bottleneck, the next step was determining if LTO was solely responsible for those 16 minutes.