Optimizing Rust Performance via Agentic Iteration

Back in January 2025, I posed a curious hypothesis for a blog post:

Can Large Language Models (LLMs) produce superior code if they are repeatedly tasked with making that code "better"?

At that time, sophisticated "agentic" coding wasn't yet mainstream. However, using Claude Sonnet 3.5, I found that the model could indeed iteratively refine algorithmic Python code. The catch? My instructions were too vague. Sonnet exploited this ambiguity by bloating the software with a plethora of useless features; while the code did technically run faster, it was an inefficient way to achieve optimization.

The Evolution of Agentic Coding

While modern LLMs are heavily RLHF-ed to solve standard "pass/fail" coding challenges, true performance optimization is rarely part of their core training.

In my initial musings, I envisioned a future where LLMs could bypass Python's speed limitations by writing the core logic in Rust and using PyO3 to create a bridge. This would combine Python's ergonomics with Rust's raw velocity. I initially suspected that the "make it better" loop would work on Rust code, but I lacked the domain expertise at the time to make that claim without sounding reckless.

Fast forward to the release of Opus 4.5, and the landscape changed. After months of experimentation, I can confirm: modern agentic LLMs can write Rust code that outperforms state-of-the-art (SOTA) libraries, provided they have the right constraints.

Performance Gains

Depending on the specific domain, the cumulative optimizations from successive frontier models have yielded:

• Minimum: $2\times$ speedup
• Maximum: $20\times$ speedup

---

Iterative "Benchmaxxing"

I began using performance optimization as a quantitative metric to stress-test these new agents. I chose Rust for several strategic reasons:

Note: To maintain stability, I imposed a strict rule: avoid unsafe code whenever possible, even if it means sacrificing a tiny bit of theoretical peak speed.

The Case Study: UMAP

I decided to tackle UMAP (Uniform Manifold Approximation and Projection), a dimensionality reduction tool. While powerful, UMAP often struggles with massive datasets. While existing crates like umap-rs exist, I wanted the agent to build the algorithm from scratch with minimal dependencies to allow for the lowest-level optimizations.

The Optimization Workflow

To ensure the agent wasn't just guessing, I utilized the criterion crate. This tool allows for:

1. Running rigorous benchmarks.
2. Tracking performance across iterations.
3. Determining if a speedup is statistically significant or merely noise.

Example: A typical criterion result showing a $3.5\times$ improvement.

Implementation Requirements

I instructed Opus 4.5 to build a benchmark suite with the following mandatory criteria:

• Test multiple input data sizes (small vs. large).
• Support inputs up to $100,000 \times 768$.
• Validate performance in both CPU and GPU modes.
• Export all results into a Markdown file for easy tracking.

As the process evolved, I introduced specialized libraries to push the limits:

• faer: For high-performance linear algebra.
• simsimd: For accelerated SIMD operations.

---

The Art of the Prompt

My approach to prompting agentic LLMs is unconventional. Rather than short chats, I provide massive Markdown documents utilizing ALL CAPS and bolding to emphasize critical constraints.

Some claim prompt engineering is dead because models are "smart enough" to handle ambiguity. I strongly disagree; the smarter the model, the better it is at adhering to the nuances provided in a well-engineered prompt.

My Workflow in Zed Agent:

1. Tag the specific Markdown prompt file.
2. Allow the agent to read the full context.
3. Execute the instructions.

Transitioning to Autonomy

Once I trusted the agent not to accidentally execute rm -rf on my repository, I shifted from manual oversight to an autonomous loop. I gave the agent a "blank check" to iterate until the numbers stopped moving.

// The "Final Push" Prompt
"YOU MUST KEEP ITERATING OPTIMIZATIONS AND SOLVING ISSUES 
UNTIL THE BENCHMARK RESULTS STOP IMPROVING AND THE 
CRATE IS AS FAST AS IT CAN BE. 

You have permission to keep iterating until you run out of ideas."