Scaling Performance: Achieving a 232x Speedup in QR Kernels via Auto-Research

Originally posted on sankalp's blog on July 8, 2026

---

📌 Introduction

Recently, GPU Mode, in partnership with Core Automation, organized a competition centered around the concept of "auto-research." The objective was to develop a highly optimized implementation of batched square compact-Householder QR factorization (more commonly known as QR decomposition).

Out of 183 competitors, I secured 12th place, ultimately producing a kernel that was $\approx 232\times$ faster than the provided baseline.

"Some might dismiss this as mere 'loop engineering', and frankly, I'm okay with that."

In this post, I'll detail my methodology, the hurdles I encountered, and the insights I gained. Note: You don't need a PhD in linear algebra to follow along; I've prioritized the research process over the raw mathematics.

For those interested in the specifics, you can find the Problem Link and Leaderboard here. This was part of the Linear Algebra Kernels in the Age of Research series.

---

🛠️ Problem Breakdown

The Technical Goal

We were tasked with processing a batch of square $\text{FP32}$ CUDA matrices $A$ with dimensions $\text{batch} \times n \times n$. The goal was to produce a compact Householder representation identical to the output of torch.geqrf(A).

The required outputs were:

1. An $H$ matrix: The upper triangle contains $R$, and the lower triangle stores the Householder vectors.
2. A $\tau$ vector: Containing the reflector coefficients.

Verification Process

The checker validated the results by reconstructing $Q$ using torch.linalg.householder_product(H, tau) and extracting $R = \text{triu}(H)$. The following conditions had to be met:

• $A \approx QR$
• $Q^T Q \approx I$
• $Q^T A \approx R$

Constraints and Benchmarks

The leaderboard ranked submissions based on the geometric mean of runtimes across various matrix sizes and conditioning scenarios.

While internal computations could utilize lower precision (e.g., FP16, FP8, or NVFP4), the final factors had to pass $\text{FP32}$ accuracy checks.

A Concrete Example ($3 \times 3$)

Consider the matrix $A$: $A = \begin{bmatrix} 12 & -51 & 4 \\ 6 & 167 & -68 \\ -4 & 24 & -41 \end{bmatrix} = \begin{bmatrix} 6/7 & -69/175 & -58/175 \\ 3/7 & 158/175 & 6/175 \\ -2/7 & 6/35 & -33/35 \end{bmatrix} \begin{bmatrix} 14 & 21 & -14 \\ 0 & 175 & -70 \\ 0 & 0 & 35 \end{bmatrix} = QR$

In this scenario, $Q$ is orthogonal (unit-length, perpendicular columns) and $R$ is upper triangular (zeros below the diagonal). The compact Householder method allows us to map the first column $(12, 6, -4)$ to $(-14, 0, 0)$ in a single reflection.

---

🤖 Why This is "Auto-Research-able"

The contest environment was specifically designed to be agent-friendly. By providing the popcorn CLI, participants could automate the cycle of testing, benchmarking, and submitting.

The organizers allowed nearly unlimited submissions, provided they were spaced out to avoid crashing the Modal credit pool or clogging the queues.

---

🎓 Learning to Ask Better Questions

I entered this contest as an underdog. While I've spent a year learning GPU optimization (Triton and some CUDA), I am not a professional in the field. To put this in perspective, the person just above me on the leaderboard was a principal engineer at NVIDIA.

Aura farming aside, my strategy relied on converting unknown unknowns into known unknowns. The more domain knowledge I acquired, the more effectively I could prompt LLMs.

My Learning Roadmap

• Understand the basics of QR Decomposition.
• Compare Gram-Schmidt vs. Householder reflections.
• Research the Blocked Householder algorithm.
• Study the trailing WY-update architecture.

Through iterations with Claude and various educational videos, I realized that the Blocked Householder approach was the only viable path to high performance.

Real-World Application

This isn't just a contest exercise. Matrix decompositions are critical for modern LLM training optimizers that use preconditioning, such as:

• Shampoo-style optimizers: Which utilize these decompositions for scaling.
• Muon (used by Kimi): Which treats weight updates as matrices rather than flattened vectors.

---

🚀 The Path to Breakthroughs

The process involved a constant battle against local maxima. I had to introduce idea diversity to ensure the agent didn't just optimize a mediocre approach.

The "Codex-maxxing" Workflow

I spent hours letting the system iterate. At one point, the process looked like this:

By generating a "beam" of different candidate implementations, I could explore multiple optimization paths simultaneously.

---

🏁 Conclusion & Reflections

What I could have improved:

• Better initial theoretical grounding: Spending more time on the math before coding might have reduced early wasted iterations.
• More aggressive precision experimentation: I could have pushed the boundaries of NVFP4 further.

Ultimately, this experience proved that while domain expertise is a massive advantage, a robust auto-research loop combined with strategic LLM prompting can allow an underdog to compete with the best in the field.

References & Acknowledgements:

• GPU Mode & Core Automation for the contest.
• The open-source community for Triton/CUDA resources.