Nestri's virtio-nvgpu brings near-native NVIDIA GPU access to KVM guests

Nestri, the team behind the open-source cloud-gaming stack nesbox, has published virtio-nvgpu, an experimental virtio device that gives a Linux guest near-native access to an NVIDIA GPU under KVM. Instead of translating graphics API calls, it forwards NVIDIA kernel driver ioctls between guest and host at the driver ABI level, so the guest runs NVIDIA's own user-mode drivers unmodified, and the same Vulkan and NVENC libraries talk to the same card.

The target workload is headless streaming: a compositor inside the VM renders, composites and encodes frames on the GPU, and only compressed video leaves. There is no monitor, and the host keeps the card.

Forward the driver, not the API

A guest kernel module registers /dev/nvidiactl, /dev/nvidia0...N and /dev/nvidia-uvm. Every ioctl is serialized onto a virtqueue, translated by an ABI-aware component in the VMM and issued against the host's device files, while mmap calls map a shared memory window. A second queue running the other way tells the guest when the GPU is ready, so it can sleep instead of spinning.

Because the guest's user-mode driver builds GPU command buffers locally, individual draw calls never cross the boundary: a full sweep drew 813,691 frames while the backend served 13,792 messages, about one crossing per 59 frames.

The numbers

Benchmarks on an RTX 3060 with driver 595.99.02, host Ubuntu 26.04 on a Ryzen 7 9850X3D, compare a guest with 2 vCPUs and Linux 7.2 against the same machine's bare metal. Above roughly 2 ms a frame the difference stays within 2%: a 39 ms frame came out 0.4% faster in the guest, a 2 ms frame 1.7% slower. Very light frames lose more (7.1% at 0.5 ms), because a wake costs about 0.02 ms whatever the frame size.

CPU cost matches bare metal: 0.37 seconds against 0.40 for the same 12 seconds of unpaced rendering near 100 fps. Four guests share one card evenly, 103.7 fps together against 102.9 for a single guest, and all four encode H.264 at once at a steady 60 Hz.

What it does not have

The authors are explicit about the security surface: there is no IOMMU boundary between guest GPU work and the host, and the card stays in the host driver's domain, so the host NVIDIA driver is part of the trusted computing base. Ioctls the ABI profile does not describe are refused rather than forwarded, which the README calls attack-surface reduction, not hardware isolation. For mutually untrusted tenants, VFIO passthrough or vGPU remains the stronger answer.

The sandboxed per-guest helper, called isolate, is designed but not written; CUDA is forwarded but untested beyond enumeration; MIG and SR-IOV are out of scope. Driver support is explicit, with three ABI profiles shipped (535.129.03, 580.178.04, 595.71.05). The layout follows gVisor's nvproxy and chromeos/virtio-media.