Parsing JSON Objects Without Intermediate ASTs

Most JSON libraries follow the same two-step recipe: parse raw bytes into an intermediate tree that mirrors the document, then walk that tree to build the data types an application actually uses. A blog post on arthi-chaud.github.io asks what happens if that intermediate tree is removed altogether, and proposes an approach built on partially-initialised values.

Why the intermediate representation costs

The author describes the usual design as a pipeline: a parser turns a ByteString into a JSON AST, and a separate function such as fromJSON turns that AST into domain data, for example an Album record. The split is convenient — parsing stays independent of validation, and the conversion code can often be generated automatically with Template Haskell or Generics — but it is not free. The AST is an extra object in memory, at least as large as the final value, and building it costs computation that is discarded as soon as the domain object exists.

Partially-initialised objects and a bit set

The alternative threads a partially-initialised domain object through the parser itself, so fields are written as soon as they are read and no AST is ever built. In Haskell this leans on laziness: every field starts as undefined, which means the fields of the target type must not be strict. Safety is recovered with a bit set — a Word64 initialised to maxBound. Setting a field clears the bit at its position in the type definition, and when parsing ends a remaining value of zero proves the object is fully initialised. Otherwise the parser fails and can list the missing fields in its error message, while fields whose type is Maybe can simply be set to Nothing and accepted.

Benchmarks and caveats

The proof of concept was built with the Haskell flatparse library, with Template Haskell generating specialised parsers at compile time. In Criterion micro-benchmarks on an Intel machine with two Xeon Gold 6244 CPUs and 32 GB of RAM, aeson, Haskell's main JSON library, measured 1.051 μs for a book and 2.901 μs for an author with a list of books. The same library using an intermediate AST took 922.7 ns and 2.813 μs; without the AST it fell to 314.2 ns and 1.027 μs — roughly three times faster. The author notes the parser is not fully JSON-compliant and that such micro-benchmarks may not reflect real applications.

What the experiment shows

The post is explicit that it is not an argument against intermediate representations, which decouple parsing from validation and keep code maintainable. Its point is the runtime cost of that decoupling, and that merging the two steps is a workable, if less elegant, trade-off when speed matters. The idea is not new either: Rust's serde framework uses staged programming to move that layer to compile time, and strict languages can start from empty values instead of undefined.