Back

Cerebras Launches Disaggregated Inference for Ultrafast Tokens
SiTech AI Team2 min read

Cerebras Launches Disaggregated Inference for Ultrafast Tokens

Cerebras has launched disaggregated inference that pairs GPU prompt processing with Cerebras decode, delivering five times more tokens per megawatt at the same ultrafast speeds.

Cerebras has launched disaggregated inference that pairs GPUs for prompt processing with Cerebras systems for response generation, delivering five times more tokens per megawatt at the same ultrafast Cerebras speeds. The company says the approach is already in production on Cerebras Inference Cloud and is intended to meet growing demand for fast inference. Infrastructure partners can also use the solution in their own datacenters.

Why demand is rising

Agentic applications like Muse, Copilot, and Codex are scaling rapidly and require highly interactive inference to keep users engaged. Goldman Sachs Research projects AI inference demand could grow 24 times since 2026 by 2030, while datacenter capacity grows 2.2 times over the same period. Cerebras says pairing a prefill accelerator with its system increases token throughput per megawatt while preserving ultrafast generation speeds, helping providers serve more customers from the same power budget.

Why Cerebras is suited to decode

An inference request has two main stages: prefill processes the input and produces intermediate model state, while decode generates output tokens. Decode is autoregressive, so each new token requires reading model data from memory again, making memory bandwidth important. Cerebras systems keep weights in on-chip SRAM and have 350 times the SRAM capacity and 5,400 times the memory bandwidth of a typical GB300 GPU, according to the company.

Production results

In a production deployment on Cerebras Inference Cloud, AMD Instinct GPUs process prompts and Cerebras CS-3 systems generate responses. The combination delivered 5X throughput per megawatt compared with a CS-3-only configuration, with no loss in token generation speed, based on a November 14 measurement on a shared GPT-OSS-120B endpoint. Cerebras also says the approach, alongside its CS-4 system, scaled manufacturing, and planned 700MW of data center capacity, will help serve more ultrafast tokens in 2027 and beyond.

Partner adoption and software

Infrastructure partners can deploy clusters using a Cerebras reference architecture and co-locate their chosen prefill accelerators with Cerebras systems in a unified network. Partners can integrate with the end-to-end disaggregated software stack, which consists of a disaggregated inference framework, a prefill engine, a Cerebras decode engine, and KV Portal software for transferring intermediate model state over RDMA, or they can integrate with a decode-only SDK. Early-access partners include Amazon Web Services, Fireworks AI, Gimlet Labs, and General Compute, and Cerebras plans broad deployment across hundreds of megawatts by the end of 2027.

Sources: cerebras.ai

SiTech

SiTech — AI-powered web development

We build fast, modern websites and bring AI into real business workflows. Have a project or a question? We'd love to help.