Benchmark compares AI build costs and performance of JavaScript frameworks

A benchmark compared 12 JavaScript frameworks and meta-frameworks by asking an AI agent to build the same small Pokemon app with each one. It measured median cost, time, tokens, lines of code, and Lighthouse performance and accessibility across multiple runs. Costs ranged from $2.67 for Hono plus htmx to $4.97 for Angular, while most frameworks scored 100 on Lighthouse accessibility.

Build cost and effort

Median build costs ranged from $2.67 for Hono plus htmx to $4.97 for Angular. SvelteKit cost $4.28, Gracile plus Lit $4.65, and Angular $4.97, making them more expensive than the others. The benchmark says SvelteKit agents read the SvelteKit v3 migration guide in every run, Gracile agents often cloned its repository or read compiled source, and Angular's higher cost tracked its larger code size. Median run times ranged from 8.2 minutes for Astro to 17.2 minutes for Angular, while token use ranged from 3.85 million for Nuxt to 9.76 million for Angular.

Performance and accessibility

Lighthouse accessibility was 100 for every framework. Performance was 97 to 100, with Vanilla at 97 and Next.js and TanStack Start at 99. JavaScript bytes sent ranged from 0 KiB for Astro to 142 KiB for Next.js. Total Blocking Time ranged from 0 ms for several frameworks to 103 ms for Angular. Largest Contentful Paint ranged from 810 ms for Astro to 2090 ms for Next.js, and the benchmark notes that the metric broadly rewards multi-page apps and punishes single-page apps.

Code size and maintainability

Lines of non-blank source code ranged from 1370 for Hono plus htmx to 2009 for Angular. The post quotes Rich Harris: "Frameworks are not tools for organizing your code; they are tools for organizing your mind." It also argues that code size affects future token costs when editing. Hono plus htmx delivered roughly the same app with about 70 percent of Angular's code and faster Lighthouse results.

Caveats and method

The app was a relatively content-based Pokemon site with little interactivity, and Lighthouse and Core Web Vitals are biased against single-page apps. The test used the official MCP or skills where available, did not mandate SSR or CSR, and avoided Tailwind and Bootstrap. The author ran multiple iterations because LLM output and Lighthouse results are stochastic, and Angular failed twice, so its figures use three runs instead of five. The benchmark and code are available on GitHub.

Sources: nolanlawson.com