Pi's minimalism is its advantage: a four-tool coding harness tops Databricks benchmark at lower cost

Pi, the coding harness from Earendil, is minimal on purpose: four built-in tools, and system and tool definitions that together come in below 1,000 tokens. In a post, the company argues that minimalism is not a limitation but the source of Pi's cost and performance advantage.

Databricks: the harness changes cost, not just quality

Databricks published a benchmark of coding agents on its multi-million line codebase, built from tasks its own engineers regularly perform rather than public benchmarks that have become oversaturated. The company reported that the harness a model is called from dramatically impacts cost and quality, adding that in many cases simple harnesses like Pi performed best on its workloads.

With Opus 4.8 at xhigh thinking effort, Pi recorded the highest overall pass rate while costing significantly less per task than Claude Code and Codex. Databricks also found that the same model at the same thinking effort could differ by more than 2x in cost per task depending on the harness while quality stayed the same, and measured Pi sending about 3x less context per turn, finishing tasks in fewer runs.

Shopify built Autoresearch as an extension

Shopify Engineering's David Cortés built pi-autoresearch, an autonomous optimization loop, directly as a Pi extension — by asking Pi to create it. The extension runs experiments to find what works and what causes regressions, discarding regressions and continuing to improve as long as the target is measurable. Shopify reported results including unit tests running 300 times faster, React component mounting 20% faster, shorter build times across projects and improvements to pnpm performance.

Earendil notes that none of these tools ship with Pi. The harness stays small, and users add what they need — extensibility instead of bloat, in the company's framing.

Why minimal wins now

About a year ago, native harnesses could claim a structural advantage because models were built around them. Earendil says that argument has weakened as frontier models became competent at terminal-style coding environments; Anthropic cutting Claude Code's system prompt by 80% is cited as a sign of the same trend. The question, it argues, is less about how native a harness is and more about how it handles context without redundancy and offers clean primitives.

The company also sees Pi's context discipline as an asset for local models, which usually have smaller context windows and long prefill times; keeping a stable prompt prefix avoids minute-long reprocessing. Earendil's conclusion: a harness that is cheaper, minimal and more performant at once.