AWS scales MoE reinforcement learning on EKS with EFA and DeepEP for 40% more throughput

AWS has published a reference architecture for large-scale reinforcement learning (RL) post-training of Mixture-of-Experts (MoE) models on Amazon EKS with EFA and DeepEP. An AWS Machine Learning Blog post of September 25, 2026 reports a 40 percent gain in aggregate rollout throughput from DeepEP over EFA.

A rollout loop limited by communication

Sparser MoE architectures cut inference costs but make training limited by communication rather than compute. The main source of that overhead is Expert Parallelism (EP), which adds dynamic all-to-all token routing across devices on top of Tensor, Data and Pipeline Parallelism.

An asynchronous RL job runs two workloads at once: rollout generation is distributed inference maximizing aggregate throughput, while policy training needs tightly coupled workers in lockstep, where a straggler can stall the job or trigger NCCL timeouts. Past one instance, traffic moves from in-node NVLink to slower inter-node links.

How DeepEP moves tokens over EFA

DeepEP replaces generic all-to-all collectives with two kernels: dispatch routes tokens to remote experts and combine gathers results, using NVLink inside a node and libfabric between nodes. AWS contributed upstream changes migrating DeepEP's primitives to libfabric, giving DeepEP v2 native EFA support; NCCL 2.31 adds the latest EFA optimizations. On P5 and P6 instances EFA with NVIDIA GPUDirect RDMA moves data directly between GPU memory buffers, bypassing the CPU.

What the benchmark shows, and what it takes to run

Across 48 P5en instances, 16 for training and 32 for inference, running a super-sparse MoE model, DeepEP over EFA raised rollout throughput by 40 percent and scaled to roughly a thousand accelerators, AWS says. The baseline was a Slime-based stack (CUDA 12.9, PyTorch 2.9.1, NCCL 2.27, SGLang 0.5.9, Slime 0.2.4) without EFA-accelerated expert parallelism; the improved one was DeepEP over EFA (CUDA 13.0, PyTorch 2.12, NCCL 2.31, EFA 1.49, DeepEP 2.0, SGLang 0.5.17, Miles 0.1.0).

The post lists Amazon EKS 1.31 or later, EFA installer 1.49 with the AWS OFI NCCL plugin and instances such as p5.48xlarge or p6-b200.48xlarge; communicating nodes must share an Availability Zone. Rollout generation suits Amazon EC2 Spot Instances, since interrupted workers requeue unfinished tasks while policy training keeps stable capacity. Jobs run through TorchX, with checkpoints in Amazon S3.