AWS Launches TorchNeuron to Move PyTorch Apps Onto Trainium Chips

Amazon Web Services has released a new developer tool called TorchNeuron that allows AI applications built with PyTorch to run on Amazon's custom Trainium AI chips by changing a single line of configuration, with no code rewrite required. The release represents Amazon's most direct challenge yet to Nvidia's position as the default hardware choice for AI apps.

How TorchNeuron Works

TorchNeuron is adapted from PyTorch, the free, open-source toolkit developers use to build, train, and run AI models. Amazon says about 85% of AI apps are written with PyTorch. Because the tool is based on PyTorch itself, developers who already know the framework can use it immediately. PyTorch was originally developed by Facebook and has been maintained by the Linux Foundation since 2022.

Nvidia has spent years ensuring its CUDA software infrastructure, which allows apps to run on Nvidia GPUs and other hardware, seamlessly supports PyTorch apps, embedding its hardware into the framework essentially by default. Amazon is not the first company to offer an easy porting path, since PyTorch users can also move their apps to AMD GPUs or Apple chips with a simple configuration change. However, AWS argues that Trainium offers something those options do not: access to Amazon's entire cloud.

The Cloud Argument

Colin Brace, VP of Technology Software at AWS Annapurna Labs, the unit that makes Amazon's custom chips, said startups building on AWS avoid betting on a single-purpose compute provider and scrambling to build everything else around it. He pointed to neoclouds tied to Nvidia hardware, such as Coreweave, Nebius, and Firmus, as alternatives that lack the infrastructure to go from prototype to production.

AWS claims that because the same Trainium chip handles both training and inference and is tightly integrated with cloud infrastructure, startups save money even if AWS does not offer the cheapest per-token price. Brace cited Deepgram, an AI voice app provider, which he said moved to Trainium using native PyTorch support and is now processing 70% more audio per dollar at production-grade accuracy.

Pushing PyTorch Away From Nvidia

Amazon says it is contributing changes upstream to the main PyTorch project, working with the PyTorch community to remove code that ties the framework to specific chips. The stated goal is to make PyTorch more hardware-agnostic for the entire developer community, not just Trainium users. If Amazon succeeds in deeply uncoupling PyTorch from Nvidia, the impact on the GPU giant could extend well beyond competition from other chip makers. As Brace put it, the real story is removing the lock-in that kept developers from having a real choice at all.

Sources: TechCrunch · GeekWire · PyTorch Blog