Qwenputer Embeds a RISC-V CPU in Qwen3.5 Weights

Qwenputer is a stock Qwen 3.5 4B model with a RISC-V CPU compiled into its weights. The CPU uses the model's Gated DeltaNet attention layers as exact RAM and runs arithmetic, C, and MicroPython code during forward passes without external tools or runtimes. It is not a plugin, tool, or sandbox.

How Gated DeltaNet becomes RAM

Three of four attention layers in Qwen3.5 use Gated DeltaNet, a linear RNN. Each head keeps a state matrix, and the delta rule updates it. By setting the gate to 1, write strength beta to 1, and using one-hot key encoding, writes overwrite one row and queries retrieve an exact value. This makes the GDN layers work as RAM, with state size that does not grow with conversation length. Schlag et al. (2021) showed linear attention with the delta rule can act as associative memory; Qwenputer makes it exact and addressable inside a production model.

CPU inside the model

The CPU is a 32-bit RISC-V core with RV32IM, the Zba extension, and a small bignum unit. It takes 11 layers in front of Qwen's 32 layers, passes official riscv-tests, matches qemu, and ran 18 million instructions free-running without error. It can also run C and MicroPython, though somewhat slowly, and implements the full RV32IM instruction set. Qwen 3.5 4B uses it without post-training, internally rather than through external tool calls: Qwen sends instructions in a tool-call-like format and reads responses. Each instruction becomes a few generated tokens named ADDR, VAL, and STEP, while RvChat loads firmware into GDN state and keeps trace tokens out of Qwen's layers. Qwen's original weights remain unchanged, with computer weights added in extra dimensions and layers.

Accuracy and normal text

On DeepMind arithmetic sets, Qwenputer scored 97.5% on div_big, 97.5% on mul_big, 100% on mul_div_multiple_longer, and 97.5% on add_or_sub_big. With a Python tool, Qwen scored 95%, 100%, 97.5%, and 97.5% on those sets, while base Qwen scored 72.5%, 55%, 90%, and 87.5%. On NUPA long-integer arithmetic, Qwenputer improved from 48.6% to 100%, matching a Python tool. About 230,000 replayed output positions matched a reference machine with zero mismatches. Normal text stayed within noise of base Qwen, with KL 6e-4 against a noise floor of 5e-4.

Experimental filesystem and limits

The experimental filesystem exposes internal put, append, get, list, and delete operations. File names hash into one of 1,024 slots in a DeltaNet directory, text is stored in other heads with eight tokens per row chained together, and collisions check the next slot. It holds up to 512 files, about 53,000 tokens of text, and saved state can carry files across conversations. Qwen gets changed facts right 90% of the time, versus 30% when pasting old chats, and is on par with Python memory tools at 83.3% versus 78.9%. Limits include an 18.8 GB checkpoint, need for the RvChat driver, and speed of about 3,200 CPU tokens per second on an L4. The model runs on a single 24 GB GPU, with code on GitHub and models on Hugging Face and ModelScope.

Sources: divagr.com