Amazon opens TorchNeuron beta to put PyTorch code on Trainium chips

A lower-friction path to Trainium

Amazon has opened TorchNeuron as a public beta after a private beta that began late last year. The software sits between PyTorch and Amazon’s Trainium chips, letting developers run existing PyTorch code after a basic configuration change instead of rewriting it for chip-specific interfaces.

Colin Brace, vice president of technology software at Annapurna Labs, described the goal as “If you know PyTorch, then you know Trainium.” He said adapting PyTorch code to other hardware can take months when it depends on Nvidia’s CUDA platform, and the new tool is aimed at removing that barrier.

Inside the beta

More than 30 companies and internal Amazon teams used TorchNeuron during the private beta. Deepgram, a San Francisco speech AI startup, said benchmarks showed Trainium processing about 60% more audio per dollar than its existing GPU setup, and Brace said the company now runs Trainium in production.

Amazon also said TorchNeuron will be released as open source on Tuesday, alongside the PyTorch Conference in San Jose, Calif. To cover a wide range of models, Amazon tests TorchNeuron nightly against more than 100,000 models from Hugging Face.

Inside Amazon, the Prime Video team used TorchNeuron to build Prime 360, an AI system that reconstructs NBA plays in 3D for replays from angles traditional cameras cannot capture. The feature runs on Trainium and will appear on select Prime Video broadcasts this NBA season.

Demand and market pressure

Trainium’s largest users include AI labs with close financial ties to Amazon. Anthropic uses the chips to train and run Claude models, and OpenAI agreed in February to use 2 gigawatts of Trainium capacity as part of a deal in which Amazon is investing $50 billion in OpenAI.

Andy Jassy wrote in April that Amazon’s chip business, including Trainium and Graviton, had an annual revenue run rate of more than $20 billion, and he estimated about $50 billion if the chips were sold by a standalone chip business. Amazon traces the unit to its 2015 acquisition of Israeli chipmaker Annapurna Labs for a reported $350 million.

Sources: GeekWire · About Amazon · PyTorch Blog