Stress-Testing the Gleam Compiler via Fuzzing

Can we uncover compiler bugs by programmatically generating random code?

Published: Tuesday, August 25, 2026

Introduction

I keep a close eye on the Gleam issue tracker and its changelog. I have a deep appreciation for the project and the community driving it forward. However, whenever I encounter a bug involving code generation or discrepancies between the JavaScript and Erlang targets, a specific thought haunts me: Why isn't there a way to essentially "calculate" every possible Gleam program, execute them, and identify the failures?

I view this challenge like a game of chess. In chess, the number of possible board configurations is nearly infinite. In my analogy, the "board" consists of Gleam programs. I want an endless database of these programs to flush out those elusive edge cases that manual testing misses.

The LLM Experiment

My initial foray into this was using a Large Language Model (LLM). I fed it a history of Gleam issues and asked it to "think deeply" to synthesize new edge cases. It suggested various complex patterns:

• Intricate bit array combinations.
• Deeply nested anonymous functions.
• Complex use pattern nesting.

After spending roughly $20 in tokens, the result was a single discovered bug, which was promptly fixed: Issue #5613.

While it worked once, "LLM fuzzing" has significant drawbacks:

• High cost
• Non-deterministic results
• Feels like gambling with a slot machine

This led me to a path I had previously avoided due to the perceived workload: structure-aware fuzzing.

---

Understanding Structure-Aware Fuzzing

Software development is inherently difficult, and human intuition is limited. To bridge this gap, we use automation to hunt for bugs. The core theory is that if you provide enough random inputs, you will eventually hit a "corner case" that the original developer never anticipated.

Fuzzers generally fall into two categories:

For those interested in large-scale efforts, Google's OSS Fuzz is a prime example of continuous fuzzing for critical open-source software. Since we are dealing with a language rather than a network protocol, we need a stream of valid (or semi-valid) code—either as raw source or an Abstract Syntax Tree (AST)—rather than random bytes.

---

Why Gleam is Ideal for Fuzzing

Gleam possesses several characteristics that make it a "perfect storm" for this approach:

1. Differential Testing: We can compile the same program for both Erlang and JS and simply flag any divergence in behavior.
2. Conciseness: We can generate programs that exercise almost every language feature with very little code.
3. Type Safety: While Gleam's type system prevents most runtime crashes, it isn't infallible. The type system is perfect $\rightarrow$ Type inference bugs have occurred in the past, and different language features require tailored testing strategies.
4. Compositionality: Because it is functional and expression-oriented, building complex programs from smaller pieces is straightforward.
5. Rust Integration: The Gleam compiler is written in Rust, allowing us to leverage the powerful Rust fuzzing ecosystem. We can test internal compiler components without even needing to write a .gleam file to disk.

The Fuzzing Logic Flow

---

Resources & Methodology

To build this, I relied on several key resources. I highly recommend the work of Nick Fitzgerald, specifically his post on writing test case generators, which was my primary inspiration.

Note on Strategy: My fuzzer is generation-based (creating inputs from scratch) rather than mutation-based (modifying existing inputs).

According to this experiment, mutation-based fuzzing often finds more issues (especially for WASM). I plan to implement this in the future. For those wanting a deep dive into the theory, FuzzingBook is an excellent resource.

Source Code: You can find the implementation in this branch: daniellionel01/gleam (fuzzing branch).

---

Phase 1: The Parser

I made a strategic decision to use the public compiler API. While the API lacks strict stability guarantees, this approach:

• Ensures easier compatibility with future Gleam updates.
• Avoids hacking internal implementation details $\rightarrow$ prevents the creation of false positives/negatives.

Classifying Outputs

I developed a tool to categorize how the compiler handles various inputs.

Example Command: $ cargo run -p fuzzing-core --example classify

Results:

• pub fn main() { 1 } $\rightarrow$ Compiled (js: 39B, ts: 32B, erl: 238B)
• pub fn main() { let f = fn(x) { x + 1 }; f(41) } $\rightarrow$ Parse Error
• pub fn main() { 1 +. \ x \ } $\rightarrow$ Analysis Rejected (Javascript)
• pub fn main() { $\rightarrow$ Parse Error

To start, I used the fuzz crate to bombard the compiler with completely unstructured random inputs. The goal here wasn't to find logic bugs, but to see if I could trigger a full compiler crash (panic) rather than a graceful error message.

# Running the parser fuzzer for a short burst
$ cargo +nightly fuzz run parse_only --fuzz

If we define the set of all possible input strings as $\mathcal{S}$, we are searching for any $s \in \mathcal{S}$ such that the compiler enters an undefined state.