Baking a Model: A Metaphor for LLM Training

By Kent Beck | August 14, 2026

I recall my high school days, walking to the bus while intently studying a Motorola 6800 instruction set manual. At the time, the actual content—the timing tables, instruction encodings, and boolean expressions—was largely a mystery to me. However, I was completely captivated by the mechanics of the system.

I felt that if I could just grasp how this complex machine functioned, I would possess the power of control.

While I don't claim to have mastered every detail yet, that same fascination drives me today. I am interested in two distinct areas:

1. The inner workings of the models themselves.
2. The industrial machinery used to construct those models.

This post focuses on the latter: the process of model construction.

---

The Culinary Analogy

Creating a model is remarkably similar to baking. You start with raw ingredients and transform them into something entirely different. On their own, the raw components aren't particularly appetizing, but the final result is delicious.

Furthermore, baking is highly sensitive to $\Delta \text{initial conditions}$. A tiny adjustment at the start of the process can lead to a massive divergence in the final outcome.

I've recently been exploring cold proofing—the act of letting yeast develop slowly in the fridge overnight. This mirrored my growing understanding of how AI models are built.

Defining the "Model"

A model is just a computer you can talk to. (I'll start simple and add complexity as we go).

In reality, what we call a "model" is split into two distinct architectural halves:

• The User Interface (UI): This handles the "boring" stuff—authentication, input/output formatting, and sequencing. This is built using conventional programming (even if the definition of "conventional" has shifted lately).
• The Model Proper: This is the core engine, built using radically different, non-conventional techniques.

Unlike traditional software engineering, where you write a sequence of statements to produce a result, an AI model is the product of training. While both involve modifying an artifact to change future behavior, the methods are worlds apart.

---

The Training Pipeline

You cannot simply guess the correct numerical weights of a model in one shot. Instead, you find "approximately correct" numbers and refine them.

Interestingly, there isn't really a single step called "training." Instead, "training" is the sum of several phases: $\text{Training} = \text{Pre-training} + \text{Mid-training} + \text{Post-training}$

1. Pre-training: The Cold Proofing

Pre-training is a massive, high-stakes batch process. The team establishes the initial conditions—the raw data and a blank model—and then cycles that data through the system a "gajillion" times.

• The Risk: It is a colossal bet. We are talking about \text{Cost} \approx \100\text{M}+$ and months of development time.
• The Monitoring: Engineers must constantly watch for signs that the process has "gone into the weeds," requiring a restart or a tweak.

2. Post-training: The Shaping and Cooking

If pre-training is the cold proofing, post-training is the actual baking. This is where "model engineers" (often called researchers) identify where the raw model fails and apply specific tweaks.

They create small fragments of data and code—the "surviving experiments"—to refine the model's behavior. Once enough supplements are added, the model can finally handle a request like: "Give me 5 unusual focaccia toppings."

Comparison Summary

---

Process Visualization

The "Training" Checklist

• Establish initial data parameters
• Run massive pre-training cycles
• Identify raw model weaknesses
• Apply post-training supplements
• Integrate with conventional UI

---

Final Thoughts

While the baking analogy is helpful, it doesn't perfectly capture the reversible and collaborative nature of post-training. In a future post, I plan to dive into the various roles involved in this process and how their incentives, cultures, and rhythms diverge (e.g., short-term features vs. long-term future).

📢 Advisory Services

I am currently booking a limited number of custom talks and advisory engagements. I help you determine if your organization bends or breaks under pressure by:

1. Interviewing your staff.
2. Measuring actual software flows.
3. Providing the unvarnished truth and a roadmap for action.

---

Community Discussion

Jon Verrier noted a funny linguistic quirk in the process:

"We have ‘pre-training’ and ‘post-training’, but no step called ‘training’. We can meet for a pre-training chat, then move directly to post training coffee, and skip working out entirely."

def model_lifecycle():
    status = "raw"
    if pre_training():
        status = "precursor"
    if post_training():
        status = "delicious"
    return status