Battle of the Physical AI: GPT-5.6 vs. Claude Fable 5

Published: July 20, 2026
Contributors: Chris Rackauckas & Anshul Singhvi
Source: JuliaHub Research Innovation

---

Executive Summary: The Benchmarks

JuliaHub recently conducted a rigorous evaluation of the latest frontier models integrated into our agent. We tested these models against five distinct modeling and simulation challenges, ranging in complexity. The goal was to determine which model best handles the nuances of Physical AI.

Performance Metrics

The following table summarizes the weighted scores, financial costs, and temporal efficiency of each model:

Key Takeaway: While Claude Fable 5 achieved the highest accuracy, the GPT-5.6 family (specifically terra and sol) offers significantly lower costs and faster turnaround times.

---

The Evaluation Framework: "The Container"

To ensure a fair fight, we utilized a "container" approach. A container is a controlled environment that bundles together:

1. A specific LLM (the model).
2. The Dyad harness (the agent framework).
3. A pinned configuration (fixed settings).
4. A sealed problem (the challenge and its ground truth).

The Workflow Pipeline

The process follows a strict linear progression from problem ingestion to final grading.

Detailed Step-by-Step

• Ingestion: The agent reads the sealed workspace.
• Derivation: The model determines the underlying physics.
• Implementation: The agent writes the code to a model.dyad file.
• Execution: The model is compiled and simulated.
• Verification: The agent checks its own work.
• Grading: A separate grader simulates both the agent's result and the ground truth to calculate the trajectory match.

Example Grading Logic: The final score is determined by the proximity of the submitted trajectory to the ground truth: $\text{Grade} = \int_{t_0}^{t_f} | \text{Truth}(t) - \text{Submitted}(t) | dt \rightarrow \text{Score: } 0.96$

---

Why Physical AI is Different

In standard software engineering, if the code compiles and passes the tests, it is generally considered "correct." However, Physical AI is far more treacherous.

• The Physics Gap: A simulation of a charged particle or an aircraft can run perfectly compile and execute without errors, yet still be physically impossible.
• The Agentic Loop: Agentic AI can actually exacerbate this. Because agents rely on feedback loops, they often write their own tests. If the agent's understanding of the physics is wrong, it will write tests that confirm that same wrong understanding.

"In engineering, the critical question isn't 'does it run?', but 'does this match the real world?'"

While this loop works for web development (where behavior is contained), it fails in physical regimes where simplifications don't hold.

---

Ensuring Unbiased Results

We recognize there is a significant trust deficit regarding AI benchmarks. Most model providers publish self-reported scores, but since they have every incentive to inflate these numbers, they cannot be taken at face value.

JuliaHub's mission is to provide the best possible experience for Dyad users. Since we support multiple backends, we are agnostic toward the provider; we simply want to recommend the model that actually works.

Controlled Variables

To isolate the model's performance, we pinned the following parameters across all trials:

• Harness: Dyad AI Agent
• Reasoning Effort: xhigh
• Context Window: $1 \times 10^6$ tokens
• Token Budget: $128\text{k}$

The only variable changed was the model itself:

- OpenAI GPT-5.6 (Terra, Sol, Luna)
- Anthropic Claude Fable 5

---

Visual Documentation

Below are the schematics and contributors involved in this internal sealed evaluation.

The Research Team: