Frontier AI on Your Own Hardware: dlab's Open Source Week

In a class of 150 students, Tim Dettmers asked a question he says he had been afraid to ask: who is afraid of not finding a job after graduating? About eighty percent raised their hands. In a September 21 blog post he argues the fear rests on a wrong assumption — that research's future belongs to whoever owns the most GPUs.

The unit of research is no longer the paper

Agent-based workflows have made individual projects fast, Dettmers writes: work that took a year of engineering now takes weeks, sometimes days. When every project becomes cheap, scattered papers stop counting as good research. "The unit of research is the ecosystem," he writes; his lab spent months building components that build on each other, his lab is now releasing two open-source projects and four papers as one package: dlab Open Source Week.

Frontier models on ordinary hardware

An agent was pointed at the Mac and Metal implementations of the lab's inference framework then left to optimize the kernels alone. The result was quantized inference of a Qwen 3.6 35B-A3B model at 450 tokens per second at 1.5 bits per weight — about a tenth of the memory a half-precision model needs.

Larger models fit: Qwen 3.8 Flash Next, at 125 billion parameters, runs on a single 24 GB GPU; with an AMD Strix, an NVIDIA DGX Spark or a 128 GB MacBook, the framework runs DeepSeek V4.1, a 550-billion-parameter model. Context handling is automatic and inference stays fast at long contexts.

Autonomous research, run locally

The same components were combined into an autonomous research system that runs entirely locally, with no internet access. Dettmers says it beats deep research systems from frontier labs, along with Sakana AI's system and Google's ScientistOne. In a bioinformatics experiment the agent had to find a cheap problem in an unsettled field; it returned three candidates, and in about two hours it set a new lower bound for heuristic methods, tested the best such method in the literature and found issues in the data sources used for evaluation — though not the state of the art.

Long sessions are kept alive by CliffCompaction, an auto-compaction technique Dettmers says is considerably more powerful than the auto-compaction in Claude Code or Codex. Sessions run for millions of tokens, some past a hundred million, while overall cost falls by about fifty percent; one partner measured a 45% cut in its AI budget. On KernelBench it reaches state of the art, ahead of AlphaEvolve-style approaches and hierarchical memory systems.

An academia renaissance

Dettmers argues that the space of problems that are cheap to attack and valuable to solve is vast and uncontested, because well-resourced players compete on scale instead. He is preparing a four-week course at CMU and a full course next semester, aiming to put the material on YouTube.