Cerebras Explains How It Deploys Trillion-Parameter AI Models on Wafer-Scale Chips

Cerebras has published an explanation of how it deploys trillion-parameter frontier models on its wafer-scale chips. A single Cerebras WSE-3T wafer has 44 GB of on-chip memory, while trillion-parameter models need terabytes for their weights alone. The company says it splits a large model across multiple WSE-3T wafers to run multi-trillion parameter models at 1000-2000+ tokens per second.

Kimi K3 as an example

Cerebras uses the open source Kimi K3 model as an illustrative example. Kimi K3 has about 3 trillion parameters, with 69 linear-attention layers and 24 dense layers. Cerebras says Kimi K3 scores 88.3% on Terminal-Bench 2.1 versus 88.8% for GPT-5.6 Sol, and 91.2% on BrowseComp versus 90.4% for GPT-5.6 Sol. Cerebras says these results show frontier-level coding and agentic capabilities, with the model outperforming leading proprietary models on some benchmarks.

For the example, Cerebras assumes a 256k average request size, 4-bit weight quantization and an 8-bit KV cache. It calculates the size of each model layer, then divides the model across 93 layers. The result is about 16 GB per layer.

Mapping layers to wafers

The 44 GB capacity of each WSE-3T wafer determines how many layers fit on a wafer, and the answer depends on layer type. Dense layers need extra room for a KV cache because each dense full-attention layer stores keys and values for every token in every active request. Cerebras therefore places one dense layer per wafer and leaves the remaining SRAM for its KV cache.

Linear-attention layers do not need extra room for a KV cache. Instead of storing keys and values for every token, a linear-attention layer keeps a small, fixed-size state per request regardless of context length. That leaves room for two linear layers per wafer, at 2 x 16 GB = 32 GB. With 69 linear layers and 24 dense layers, the mapping produces 35 wafers for linear layers plus 24 wafers for dense layers, for a total of 59 wafers.

Pipeline execution and concurrency

Cerebras executes models in a pipeline. The compiler maps model layers across regions of wafers, and the runtime maps the path for tokens to flow between wafers. Only activations need to pass between wafers, and this can be done via RDMA over Ethernet at roughly 2 microseconds per hop. Because each wafer has 44 GB of on-chip SRAM and each layer is about 16 GB, entire layers can typically fit directly on the chip. In most cases, model weights never have to move, and layers do not need to be split across wafers.

Each wafer acts as one stage in an assembly line. For example, Wafer 1 runs layers 1-2 and sends activations to Wafer 2, which runs layers 3-4 and passes results onward until the last wafer produces the next token. To serve many requests, Cerebras keeps the pipeline full with different requests. Each stage can work on a different request at the same time, and each stage can process a micro-batch rather than a single request. Smaller micro-batches make each request faster, while larger micro-batches increase total throughput.

The amount of on-chip memory allocated to KV cache sets the maximum number of concurrent requests. Every request in flight needs its own KV cache on each dense-layer wafer. Cerebras notes that concurrency and batch size do not alone show inference efficiency; a more meaningful metric is total token throughput per unit of power, measured at the same output generation speed.

Sources: cerebras.ai