Ornith-1.0: Advancing Agentic Coding via Self-Improving Open-Source Models

🌺 Aloha! Ornith-1.0 represents a new frontier in open-source AI, specifically engineered for agentic coding through a unique self-improvement mechanism.

🚀 Core Highlights

Ornith-1.0 isn't just a static model; it is a family of models designed to push the boundaries of how AI handles complex software engineering tasks.

1. State-of-the-Art (SOTA) Performance

Built upon the foundations of Gemma 4 and Qwen 3.5, Ornith-1.0 is available in several configurations to balance power and efficiency:

Dense Models: 9B and 31B.
Mixture-of-Experts (MoE): 35B and 397B.

These models currently lead the open-source pack in their respective size classes across critical benchmarks, including NL2Repo, SWE-Bench, OpenClaw, and Terminal-Bench 2.1.

2. The Self-Improving Engine

Unlike traditional fine-tuning, Ornith-1.0 utilizes a specialized Reinforcement Learning (RL) framework. This system doesn't just optimize the final answer; it optimizes the process.

By jointly refining the scaffold (the logic driving the agent) and the rollouts (the actual code execution), the model discovers superior search trajectories for solving bugs and implementing features.

3. Open Accessibility

The project is released under the MIT License. This means it is ~~restricted by region~~ globally accessible and free for all to use.

📊 Benchmark Evaluations

Each Ornith-1.0 variant was tested against size-appropriate baselines using identical decoding configurations.

Ornith-1.0-9B Performance

Benchmark	Ornith-1.0-9B	Qwen3.5-9B	Qwen3.5-35B	Gemma4-12B	Gemma4-31B
Terminal-Bench 2.1 (Terminus-2)	43.1	21.3	41.4	21	42.1
Terminal-Bench 2.1 (Claude Code)	40.6	18.9	38.9	-	-
SWE-bench Verified	69.4	53.2	70	44.2	52
SWE-bench Pro	42.9	31.3	44.6	27.6	35.7
SWE-bench Multilingual	52	39.7	60.3	32.5	51.7
NL2Repo	27.2	16.2	20.5	10.3	15.5
Claw-eval Avg	63.1	53.2	65.4	32.5	48.5
SWE Atlas - QnA	17.9	9.2	13.2	-	-
SWE Atlas - RF	16.6	4.3	10.2	-	-
SWE Atlas - TW	15.3	4.4	9.8	-	-

Ornith-1.0-35B Performance

Benchmark	Ornith-1.0-35B	Qwen3.5-35B	Qwen3.6-35B	Gemma4-31B	Qwen3.5-397B
Terminal-Bench 2.1 (Terminus-2)	64.2	41.4	52.5	42.1	53.5
Terminal-Bench 2.1 (Claude Code)	62.8	38.9	49.2	-	48.6
SWE-bench Verified	75.6	70	73.4	52	76.4
SWE-bench Pro	50.4	44.6	49.5	35.7	51.6
SWE-bench Multilingual	69.3	60.3	67.2	51.7	69.3
NL2Repo	34.6	20.5	29.4	15.5	36.8
Claw-eval Avg	69.8	65.4	68.7	48.5	70.7
SWE Atlas - QnA	37.1	13.2	15.5	-	20.4
SWE Atlas - RF	29.7	10.2	11.4	-	18.4
SWE Atlas - TW	27.8	9.8	13.3	-	18.5

Ornith-1.0-397B Performance

Benchmark	Ornith-1.0-397B	Qwen3.5-397B	Qwen3.7-Max	GLM-5.2-744B	Minimax-M3-428B	DeepSeek-V4-Pro	Claude Opus 4.7	Claude Opus 4.8
Term-Bench (Terminus-2)	77.5	53.5	73.5	81.0	64	64	70.3	85
Term-Bench (Claude Code)	78.2	48.6	69.8	82.7	-	66.5	69.7	78.9
SWE-bench Verified	82.4	76.4	80.4	-	-	80.6	80.8	87.6
SWE-bench Pro	62.2	51.6	60.6	62.1	59	55.4	64.3	69.2
SWE-bench Multilingual	78.9	69.3	78.3	-	-	76.2	-	-
NL2Repo	48.2	36.8	47.2	48.9	42.1	-	-	69.7
Claw-eval Avg	77.1	70.7	65.2	-	-	75.8	78.2	-
SWE Atlas - QnA	41.2	20.4	-	-	37.9	27.2	40.3	48.8
SWE Atlas - RF	42.6	18.4	-	-	-	-	48.6	46.7
SWE Atlas - TW	39.1	18.5	-	-	30.8	-	38.5	-

📝 Evaluation Notes

Terminal-Bench 2.1 (Terminus-2): Used Harbor/Terminus-2; parser=json, $\text{temp}=1.0$ , $\text{top\_p}=1.0$ , 128K context. Averaged over 5 runs with 32 CPU cores/48GB RAM (4h timeout).
Terminal-Bench 2.1 (Claude Code): Used Claude Code 2.1.126; $\text{temp}=1.0$ , $\text{top\_p}=1.0$ , max tokens 131,072.
SWE-bench (Verified/Pro/Multilingual): OpenHands harness; $\text{temp}=1.0$ , $\text{top\_p}=0.95$ , 256K context.
SWE Atlas: mini-SWE-agent harness; $\text{temp}=1.0$ , $\text{top\_p}=0.95$ , 128K context.
NL2Repo: $\text{temp}=1.0$ , $\text{top\_p}=1.0$ , 400K context, 48K output.
ClawEval: Real-user task distribution; $\text{temp}=0.6$ , 256K context.

🛠️ Quickstart & Deployment

Important: Reasoning Capabilities

Ornith-1.0 is a reasoning model. This means the model's output begins with a thought process before providing the final answer:

<think>
The user wants to refactor the API endpoint. 
I should first check the current implementation in /src/api/routes.ts...
</think>
Here is the refactored code: [Code Block]

To handle this, the serving recipes include a reasoning_content field for the chain-of-thought and an OpenAI-style tool_calls parser.

Serving Requirements

To run Ornith-1.0, ensure your environment meets these minimum version requirements:

Transformers $\ge 5.8.1$
vLLM $\ge 0.19.1$
SGLang $\ge 0.5.9$

Recommended Sampling:

$\text{Temperature} = 0.6$
$\text{Top\_p} = 0.95$
$\text{Top\_k} = 20$
(Note: Use $\text{temp}=1.0$ to match benchmark results)

Model Variants & Formats

All models support a 256K (262,144-token) context window.

Checkpoint	Architecture	Format	Best Use Case
`Ornith-1.0-9B`	Dense (~9B)	`bf16`	Single-GPU serving / Fine-tuning
`Ornith-1.0-9B-GGUF`	Dense (~9B)	`GGUF`	Local use via Ollama / llama.cpp
`Ornith-1.0-35B`	MoE (35B)	`bf16`	Full-precision multi-GPU serving
`Ornith-1.0-35B-FP8`	MoE (35B)	`FP8`	Reduced VRAM on FP8 GPUs
`Ornith-1.0-35B-GGUF`	MoE (35B)	`GGUF`	Quantized local inference