TorchNeuron: running HF models on AWS Trainium is easier than ever

Hugging Face says its libraries now work natively with TorchNeuron, a new PyTorch backend for AWS Trainium chips that entered public beta on October 20, 2026 and installs through the torch-neuronx pip package. The plugin exposes Neuron as a regular PyTorch device, with eager execution, torch.compile and native distributed tools such as FSDP and tensor parallelism.

Until now, Hugging Face users on Trainium and Inferentia relied on Optimum Neuron, built on a PyTorch XLA-based stack that the authors say required lazy tensors, static shapes and recompilation at every graph break, along with custom modeling code and trainers. They now recommend using Transformers, Diffusers, Accelerate, PEFT and TRL directly instead.

The company reports that 89% of the 100 most-used text-generation architectures work out of the box, up from 46% with Optimum Neuron v0.4.5. For training, the team ran a standard TRL fine-tuning job on Qwen3-8B across 8 Trainium2 chips, or 32 NeuronCores and half a trn2.48xlarge instance, producing an ordinary checkpoint that can load on other hardware. Some limits remain: static shapes are still advised to avoid recompilation, and asynchronous weight loading is unsupported.

In Diffusers, FLUX.1-dev, FLUX.2, Qwen-Image, Qwen-Image-2.1 and H3 support tensor parallelism. Custom Neuron kernels can also be shared on the Hub, with one vision transformer kernel claimed to deliver up to a 2.4x speedup. The change matters because it could let developers move existing PyTorch code to Amazon's chips with far less hardware-specific work.