Beating GPT-5.6 Sol on Retrieval with 100x Cheaper Open Models

How Castform + Neon Outperform Frontier Models in Price and Efficiency

A specialized 4B open-source model, post-trained using Castform, has demonstrated the ability to retrieve search results with the same precision as GPT-5.6 Sol, while reducing costs by a factor of $100\times$.

---

"Most teams' best training data is just sitting in their databases. The problem is that turning raw data into something usable is hard, and letting agents read, search, and mutate data cheaply at scale requires advanced infra. Pointing Castform at Neon skips both." — Ying Hang Seah, cofounder of Castform

To build a high-performing agent, two critical pillars must be addressed:

1. Context: The ability to locate the exact data needed.
2. Reasoning: The ability to process that data effectively.

Neon (via Lakebase Postgres and Search extensions) handles the first; Castform solves the second.

---

The Evolution of Agentic Search

Around 2022, the industry focused heavily on embedding-only search. Every major database provider integrated these capabilities, and pgvector became Neon's most popular extension.

Initially, engineers built RAG (Retrieval-Augmented Generation) pipelines based on simple embedding similarity. However, the paradigm has shifted:

• Old Way: One-shot search $\rightarrow$ LLM Answer.
• New Way: Agentic Retrieval. Models now plan, search, and iterate in a loop to decompose complex problems.

The Cost of Intelligence

While powerful, multi-turn search using frontier models is prohibitively expensive and slow.

Metric	GPT-5.6 Sol (Frontier)	Post-Trained 4B (Open)
Latency	$\approx 10\text{s}$	Significantly Lower
Cost per Request	\approx \0.03$	\approx \0.0003$
Efficiency	Baseline	$100\times$ Cheaper

---

Introducing Castform: RL Post-Training Simplified

Out-of-the-box, open-source models often lag behind closed APIs. However, for specific tasks like search, they can match or beat frontier models if they are post-trained.

Castform allows developers to perform Reinforcement Learning (RL) post-training without needing to be experts in GPU internals or ML infrastructure. The goal is to make post-training as intuitive as prompt engineering.

The Castform + Neon Pipeline

1. Corpus Storage: Raw documents are hosted in Postgres on Neon.
2. Synthetic Data Generation: Castform uses lakebase_text and lakebase_vector to create training tasks.
3. RL Training: The model learns via rollouts where every search tool call hits Lakebase Search.
4. Production Inference: The final model utilizes the same search tools in the wild.

---

Leveraging Your Existing Data

Effective RL post-training requires three components:

• A Task: (e.g., answering a specific user query).
• An Environment: (e.g., a database to search).
• A Reward Function: (e.g., a way to score the answer).

Most enterprises possess the raw knowledge (wikis, support articles, operational DBs, product records) but lack the labeled datasets needed for training. Castform bridges this gap by turning an existing corpus into synthetic training tasks.

Example: From Raw Data to Training Set

• Raw Document: "Trains booked through Navan will be paid by GitLab travel card. Train rides must be standard cabin class with 14 day booking lead time."
• Inferred Ground Truth: "Train rides must be standard cabin class with a 14 day booking lead time."
• Synthetic Question: "When booking a rail trip in Navan, what are the rules for how early I need to reserve it and which seating level I'm expected to choose?"

---

Technical Implementation

Once the dataset is ready, you define the tools and the reward function.

The Tooling

The agent uses a hybrid search (combining BM25 and Vector search) merged via Reciprocal Rank Fusion (RRF).

def run_tool(tool, tool_args):
    # Single tool: hybrid search over Lakebase
    if tool == "search":
        query = tool_args["query"]
        bm25 = neon.lakebase_vector(query, k) 
        return rrf_merge(bm25, vector, k)

The Reward Function

The reward function guides the model to optimize for retrieval accuracy, proper citation, and final correctness.

def reward(trace, ground_truth):
    # Grade a trace against the ground-truth answer
    # Logic: Did it find the right chunk? Did it cite it? Is the answer right?
    return retrieval + citation + correctness

---

Observability and Infrastructure

Castform provides deep visibility into the RL process. Developers can monitor the Average Reward climb over time and inspect individual prompts to prevent "reward hacking" or identify broken tools.

Why Neon is Essential

RL training is computationally "bursty." Thousands of parallel rollouts can trigger dozens of simultaneous search calls. Neon's dynamic compute scaling ensures the infrastructure handles these spikes without crashing or lagging.