Cisco and Nvidia's USFP Optic Targets 12.8 Tbps in a Single Pluggable

Cisco has unveiled an ultra dense small form factor pluggable, or USFP, developed with Nvidia that promises up to eight times the bandwidth of today's leading edge optics in a single port. Announced at the Open Compute Project global summit this week, the module targets the dense fiber cabling connecting thousands of AI accelerators, NICs and switches in modern datacenters.

USFP Packs 12.8 Tbps Into One Pluggable

USFP modules support up to 32 lanes of 400 Gbps in a form factor roughly the size of today's OSFP pluggables. Each module carries 12.8 Tbps of Ethernet bandwidth, as much as a high end enterprise switch a few years ago. Today's most sophisticated pluggables top out at around 1.6 Tbps using eight 200 Gbps links.

The density could simplify AI datacenter cabling, where each GPU is commonly paired with an 800 Gbps and soon 1.6 Tbps NIC. One USFP pluggable could consolidate eight GPUs' cabling into a single fiber bundle. On the switch side, a future 204.8 Tbps switch would need only 16 USFP ports to support 128 endpoints at 1.6 Tbps.

Power Draw and Reliability Challenges

The design involves trade offs. A high end pluggable today draws about 20 watts, while the USFP spec allows up to 100 watts per module. Four times the links means four times as many electrical contacts that must seat correctly, so Cisco says USFP will use a two phase insertion mechanism: the module is placed into the cage first, then a second mechanism makes the electrical connection.

Reliability also matters. A failed laser or digital signal processor could disrupt more than one GPU and potentially affect entire systems. Cisco argues the blast radius stays smaller than with co packaged or near package optics, where one failure could drop a full rack of accelerators off the network.

Nvidia's role is notable because its recent Spectrum X and Quantum X switches use co packaged optics, which nets roughly 30 percent power savings by eliminating switch side pluggables. Its NICs still use pluggables, partly because they give integrators flexibility for smaller deployments that can rely on top of rack switches and direct attach copper cabling.

Cisco's initial USFP spec calls for 16 400 Gbps links per pluggable, about 6.4 Tbps. That aligns with Nvidia's push toward 600 kW racks with 144 GPUs, where its Vera Rubin Ultra platform promises 115.2 TB/s of scale out bandwidth per rack, or roughly 3.2 to 6.4 Tbps per GPU package depending on how bandwidth is counted. USFP will be an open standard for any vendor to adopt.

Microsoft Uses Cisco Silicon One Across Datacenters

Cisco also revealed that Microsoft is using its Silicon One P200 routing chips to distribute workloads across multiple datacenters, an approach vendors call scale across networking. Microsoft was among the first hyperscalers to embrace the model with its Fairwater clusters in Wisconsin and Georgia, using it to work around local power constraints.

Cisco touts the P200 as the first 51.2 Tbps system deployed in production by leading hyperscalers. It competes with Nvidia's Spectrum XGS and Broadcom's Jericho 4 based products.

Cisco Brings Live Protect to SONiC

Cisco also announced support for its Live Protect platform in SONiC, the vendor agnostic network operating system Microsoft originally developed and later open sourced. Cisco is one of the project's largest contributors. Rather than pushing its own software stack, Cisco says it extends the OS to better exploit ASIC features and integrate its services. Live Protect, meant to simplify security patching, is expected next year.

Sources: The Register