Beating GPT-5.6 Sol on retrieval with 100x cheaper open models

The cost problem in agentic search

On August 5, 2026, Neon published a joint write-up with Castform describing how a 4B open-source model, post-trained with reinforcement learning, can match a frontier proprietary model on retrieval tasks while costing roughly 100 times less per request.

The article traces the shift in how agents find information. Around 2022 the industry invested in embedding search — pgvector became Neon's most downloaded extension — and engineers hand-built RAG pipelines. By 2025, retrieval had become agentic: models plan and search in a loop instead of issuing a single query. Each loop iteration is another frontier-model call, and a typical multi-turn request with gpt-5.6-sol takes more than 10 seconds and costs about $0.03 end to end.

Teaching an open model to retrieve

Small open-weights models are about 100x cheaper, but out of the box they lag behind closed API models. RL post-training bridges that gap. Castform's goal is to let developers post-train models without handling machine learning and GPU internals — “as approachable as prompt engineering,” cofounder Ying Hang Seah writes, noting that most teams' best training data already sits in their databases.

The pipeline runs entirely on Neon: raw documents live in Postgres, synthetic training tasks are written with the lakebase_text and lakebase_vector extensions, every rollout's search tool call hits Lakebase Search, and production inference uses the same search tool call. The reward function grades three things — whether the model retrieved the right source, cited the right chunk, and landed on the correct answer.

Infrastructure built for bursty rollouts

Training produces a highly bursty workload: thousands of parallel rollouts, each making dozens of search calls. Neon's dynamic compute scaling absorbs those peaks without Castform provisioning for maximum capacity around the clock. The post argues the same infrastructure matters more as agents begin modifying data: branching can give each rollout an isolated database state, while time-travel queries make it possible to reconstruct what an agent encountered.

The result the authors highlight: on a specific task like search, a post-trained open model can match and beat frontier models at orders-of-magnitude lower cost — agentic retrieval no longer has to run on the most expensive model available.

📖 Source