NVIDIA Confidential Computing Keeps 96% of AI Inference Throughput on Blackwell

NVIDIA has published a technical breakdown of how confidential computing (CC) affects large language model inference and how TensorRT LLM was adapted so encrypted execution on Blackwell GPUs keeps most of its speed. It appeared on the NVIDIA Technical Blog on September 22, 2026.

Encrypted memory changes runtime assumptions

Confidential computing runs workloads inside memory-encrypted virtual machines with confidential GPUs and encrypted NVLink. Secure execution changes the runtime's assumptions about memory movement, timing, scheduling and multi-GPU communication, and those changes cost performance when the framework does not adapt.

To expose the overhead, the team chose a long input context, extended generation and low concurrency: DeepSeek-R1-0528-NVFP4 on the TensorRT LLM PyTorch backend, 32K input and 1K output tokens, 1 to 16 concurrent requests, TP=8 and an FP8 KV cache.

96% of throughput retained

Only the CC state changed: model, hardware, software versions, sequence lengths, parallelism and concurrency stayed constant on one NVIDIA DGX B200 system with eight B200 GPUs, an Intel TDX platform, CUDA 13.2, NCCL 2.30 and TensorRT LLM 1.3.0rc22.

Across concurrency 1 to 16, CC on retained 96.1-98.2% of CC-off output-token throughput, with mean time per output token within 1.2% to 4.3% of the baseline.

Three places the framework had to adapt

Host-to-device transfers pass through a software encrypted bounce buffer, because the GPU cannot directly access protected CVM memory. Pinned memory therefore loses its asynchronous advantage and some copies block the calling thread; TensorRT LLM picks memory types with CC in mind and moves repeated token readback to an asynchronous worker (PR #11573).

The kernel autotuner, which compares tactics with CUDA events, saw an unstable timing signal under CC and now measures with the GPU %globaltimer (PR #11657). NVLink SHARP multicast is unavailable in B200 CC configurations, so frameworks should detect NVLS availability and pick algorithms that fit the message size and topology.

Benchmark your own workload

NVIDIA's guidance is to treat security configuration and inference optimization as one deployment problem: enable confidential computing, attest the environment, and benchmark CC-on against CC-off with the workload you intend to serve.