The Normalization of Inexplicable Failures

Ramblings about technology | Sunday, September 27, 2026

In a recent episode of President Curtis, the titular character attempts to open a door twice, failing both times. The reason for the failure is physical: first, a corpse is blocking the way; second, a massive hoard of gold (valued at roughly a billion dollars) is in the way.

In both scenarios, the character's reaction to the malfunction is simply to mutter, "stupid thing sucks."

This is a perfectly reasonable model of how doors operate. (It is not!) I found these scenes absolutely hysterical—though perhaps that's just a sign that my own brain "sucks."

Jev: Scaling the Production of "Sucky" Doors

There has been significant chatter regarding Jev, an AI model from TypeSafe AI. Jev is designed to provide typed values accompanied by probability estimates. From what I can gather, the selling points are:

• It is fast.
• It is inexpensive.
• You can iterate on it rapidly.
• Seriously, it is fast and cheap.

I'll admit, I struggle to predict which technologies will actually achieve mass adoption. However, using Jev isn't like the old days of getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem. You still have to do the heavy lifting.

To actually verify if Jev is functioning correctly, you need:

• A robust set of evals.
• A ground-truth pipeline.

The irony is that if you've already built those two things, you are basically ready to fine-tune your own custom model. Instead, developers are feeding opaque queries into Jev and receiving opaque answers.

Charitably, this allows a company to check the "AI-powered" box and ship the product by Friday. When the downstream logic inevitably collapses, they can simply shrug and claim, "Well, AI makes mistakes."

Where did the error budgets go?

The Illusion of Confidence

A common rebuttal to my skepticism is: "But Jev provides confidence scores!"

The question is: What do you actually do with those scores? To utilize confidence scores meaningfully, you require two things:

1. An understanding of the calibration of those scores.
2. A mathematical model for the costs associated with uncertainty.

Jev's marketing focuses on benchmark scores, not calibration. Their "cookbook" suggests using confidence scores to traverse a classification tree, but that doesn't prove the scores are accurate. Usually, the implementation boils down to: "If the score is high enough, accept it. Now, what's for lunch?"

The documentation suggests arbitrary thresholds:

The docs include a disclaimer that the "correct" values depend on your specific domain. In reality, people use these scores as "cargo cults" or as a convenient excuse for why an API call failed.

The Erosion of Accountability

When a website button fails, I have a mental model for the failure: Someone deployed sloppy code $\rightarrow$ JavaScript syntax error on a specific path $\rightarrow$ HTTP 500.

Even if I can't see the logs, I assume there is a human whose job it is to figure out why that endpoint is returning a 500. For many, however, the experience is just "stupid thing sucks." Software already feels erratic; we're just increasing the frequency of that frustration.

My fear isn't that LLM-driven development will cause more failures—that's a given when using new tools. My fear is that "sometimes it just sucks" will become the standard conclusion of every technical investigation.

This is tragic because LLMs could actually fix this. We could use them to write the automated QA workflows that we currently ignore due to lack of time. The very evals needed to justify (or replace) Jev are only a few prompts away.

The current tragedy of software engineering is that we are building systems where neither the creator nor the user cares to check if there is a body blocking the door.

---

Footnotes & Asides:

1. This reminds me of the phrase: "Sometimes you get the elevator, sometimes you get the shaft." Again, a reasonable model of elevators.
2. I understand lock-in effects, but I'm baffled by the standardization of products that can't even deliver messages reliably. I've seen messages on all tiers (free to enterprise) vanish for weeks.
3. When Bill Gates failed to download Movie Maker, the world agreed it was a problem that needed solving—just not necessarily his problem. We should strive for that level of accountability for everyone, not just billionaires.