Qwen3.8-27B on One RTX 3090 vs Two: +20% Decode, +14% Cold Prefill, 3x on Cached Prompts

Qwen3.8-27B, the dense 27.8B model released on August 14, fits on a single 24 GB card in W4A16 quantization. Arsen Apostolov measured what a second RTX 3090 buys under tensor parallelism.

The setup

Both servers run vLLM with prefix caching on, from the same image and Qwen3.8-27B-W4A16-AutoRound-fast weights. One card holds 22.6 GB of VRAM and a 131,072-token context; the pair splits the weights across 21.7 GB per card and doubles the context to 262,144 tokens. The machine carries three RTX 3090s, dual Xeon E5-2660 v4, 60 GB of RAM and Ubuntu 26.04.

A 150-line script sends requests at four prompt sizes (600 to 9,300 tokens), three runs each, median taken, max_tokens=256 and temperature=0. Every cold request carries a unique first line, so no cached prefix can match.

Decode: 15–30% from the second card

One card sustains 123–154 tokens per second across the ladder, the pair 146–193. The gain averaged +22% in this run and +13% an hour earlier, because single-stream decode moves by roughly 10% between runs an hour apart. The author's deliberately unrounded figure is 15–30%, expect the lower end.

Prefill: cold is the cards, warm is the cache

Cold prefill is compute-bound: one card pushes 1,100–1,220 tokens per second, the pair 1,230–1,360. Time to first token climbs from 0.53 s to 8.5 s on one card as the prompt grows from 600 to 9,300 tokens, and from 0.50 s to 6.9 s on two — a 6% gap at 600 tokens, 23% at 9,300, 14% on average.

Warm requests are a different story: repeated prompts return in 0.5–0.75 s on the pair against 0.5–1.7 s on one card, a 2–3x difference that belongs to the prefix cache, not the cards. The tell is arithmetic — 8,600 tokens "prefilled" in 0.56 s is 15,000 tokens per second, ten times what two 3090s deliver on a 27B model.

What it costs

With both containers serving, the machine draws 620 W at the wall. A million generated tokens at 150 tokens per second take about 6,667 seconds, or 1.15 kWh — 0.34 BGN by day or 0.21 BGN at night on the Bulgarian dual-rate tariff, about $0.20 or $0.12. DeepInfra lists Qwen3.8-27B at $3.00 per million output tokens.

Apostolov does not argue that makes the API a bad deal — its price includes datacenter power, cooling and idle capacity. The narrower point is that on hardware already running, each token served is nearly pure margin. He also flags the test's limit: single-request only, continuous batching untested.