The Illusion of the Prompt

Prompt Engineering is a real skill. Prompts aren't real.

Currently, a vast number of people are driving themselves to the brink of madness by obsessively tweaking text strings to coax better behavior out of Large Language Models (LLMs). We are focusing on the wrong thing. Instead of polishing sentences, we should be losing our minds building interlocking optimization and evaluation pipelines.

To win the battle for AI reliability, we must stop obsessing over the "magic words" and instead deploy more AI-driven infrastructure to manage the models.

"The textual nature of prompts leads us to take the intentional stance towards systems which aren’t conscious, and thus miss the essential nature of their non-meaning." — Coda

---

The Engineer's Descent

I've spent roughly 25 years in the engineering trenches. Lately, I've had the "privilege" of struggling to make AI agents function reliably in production environments.

To be clear, I am talking about agents—systems designed to execute specific tasks on a user's behalf—rather than simple chatbots where the output is subjective and the "success" is just a vibe.

Agents vs. Chatbots

Building an agentic experience that isn't embarrassing requires a level of rigor that some developers simply ignore. While some are happy to ship a "subjective advice machine" and let the user wander blindly into a metaphorical bear-infested forest, my own sense of professional shame demands something better.

The Thrill of the Chaos

Despite the stress, this is the most exhilarating work of my career.

• At 22, writing Visual Basic was a rush.
• In 2007, working at a Brooklyn startup made me feel like a "golden god."
• Now, I am breaking my brain by using agents to build agents that evaluate other agents.

The line between high-level engineering and total psychological breakdown has never been thinner. The problem is that the discourse is filled with people who don't know what they're doing, while the experts remain quiet.

The "Haunted" Nature of LLMs

We know academically that LLMs cannot consistently follow instructions or tell the truth. However, in small-scale tests, they trick us into believing they are reliable. This illusion shatters the moment you scale to real users.

The Structured Output Struggle

I prefer interacting with LLMs via structured output (like JSON). While usually effective, the models still fail in bizarre ways. For example, you might request a title field of $\le 80$ characters. Most of the time, it works. Then, suddenly, the model will:

1. Ignore the limit.
2. Flood the field with a novel-length series of "notes to self" about how to format JSON.
3. Crash your parser.

{
  "title": "My Great Article",
  "note_to_self": "Remember to keep the title short. Wait, am I doing it right? I should check the JSON schema again. I will now repeat this 500 times..."
}

In one recent instance, the "fix" for this was simply renaming the field from title to heading. This is completely deranged, and because the fix is nonsensical, I expect the model will find a new way to break it soon.

Measuring the Madness

Because these systems are "haunted," you cannot rely on a few successful tests. You have to measure behavior at scale using the concept of pass^k.

$\text{Reliability} = \text{pass}^k$

This means running the same prompt $k$ times to see how often it actually succeeds.

Why "Prompt Ownership" is a Myth

Many companies try to solve brand voice or safety issues by having a "Voice Team" write the prompts. This is a fundamental mistake.

• The Context Problem: Adding a new prompt isn't like adding a line of code; it's throwing the agent into a new "contextual universe."
• The Interference Problem: Existing instructions interact with new ones in unpredictable ways.
• The Drift Problem: Models change for opaque reasons, meaning a prompt that worked yesterday might fail tomorrow.

If you believe that a domain expert can simply "write the right prompt" to scale a system, you are ignoring the essential instability of the technology.