Small Models Have Arrived
The Era of Small Models Has Arrived
By Calvin French-Owen | August 26, 2026
Recently, I have been spending a significant amount of time experimenting with gpt-5.6-luna. The performance is striking: I consistently see speeds of tokens per second (tps) as it breezes through my personal knowledge base, emails, and codebase.
What is most impressive is the cost-efficiency. Even when conducting complex research or scanning through thousands of emails, the API expenses remain negligible—often just a few dozen cents.
Visualizing the shift in the Pareto frontier (Courtesy of artificialanalysis.ai)
With the introduction of GLM 5.3, we have a new high-efficiency option on the Pareto frontier. Admittingly, when I am deep in coding, I still gravitate toward the "heavy hitters" like Fable 5 or 5.6 Sol. Because of this preference for power, it is easy to overlook just how much progress "small" models have made.
The Consumer AI Paradox
Some investors have pointed out a strange trend: the lack of a boom in consumer-facing AI companies. To understand why, we have to look at the traditional legacy playbook for building massive consumer apps:
This path created giants like Google, Facebook, and Snapchat. However, AI introduces a new variable: per-request inference costs. This dramatically raises the capital requirement for entry.
Case Study: The Personalized News Feed
I often run a "pet eval" where I attempt to build a daily news site tailored specifically to me. The logic looks like this:
research @calvinfo on the internetgenerate micro-site with today's top stories
The cost difference between model generations is stark:
| Model Class | Estimated Cost per Run | Value Proposition |
|---|---|---|
| Sonnet-class | \approx \1.00$ | High quality, but expensive |
| Luna-class | \approx \0.10$ | "Good enough" and affordable |
If you are charging a subscription fee similar to The Economist or The Wall Street Journal, your margins are thin with the former, but viable with the latter.
The "IQ 180" vs. The "Responsive" Worker
While hiking recently, my Segment co-founder, Peter, and I discussed the nature of productivity across his various ventures (including Charm Industrial and Revoy). Peter categorizes professional output into two distinct buckets:
Bucket 1: The IQ 180 Work This is the "mad scientist" territory. It involves a genius-level mind inventing a radical solution to a problem that no one else had even conceptualized.
Bucket 2: The Responsive Work This is the act of pushing the ball forward. It is about being ultra-responsive and managing dozens of different fronts simultaneously to maintain momentum.
Peter noted that while his companies would be totally fine dead-in-the-water without the IQ 180 minds to solve deep technical hurdles, of the actual work falls into Bucket 2.
The Demand Split
I believe we will see a diverging demand for AI capabilities:
- Frontier Models: Demand will compound for fields requiring novel discovery (e.g., ).
- Small Models: Demand will explode for "fast/cheap/good-enough" utility.
In the real world, most of the "human tokens" in a company are spent on the latter. When dealing with vendors, customers, or colleagues, you generally don't need a breakthrough; you need someone who is responsive and gets the job done.
The Path Forward
To make these efficient models a reality for the enterprise, we still have several hurdles to clear:
- Develop new orchestration harnesses
- Solve prompt injection and safety concerns
- Implement granular roles and permissions
If you are currently experimenting with ways to make small models more useful in a business context, please reach out!
Footnotes
- Amazon and Netflix are the notable exceptions to the ad-marketplace playbook.
- Peter is likely being modest about his own contributions here.