Meta Deploys Google TPUs for AI Workloads as Amazon Releases TorchNeuron

The market for artificial intelligence hardware is facing a new shift as major cloud providers move to bypass Nvidia's software lock-in. Google and Amazon have released updates that enable their respective custom chips to run AI workloads directly through the PyTorch framework, a development that has led Meta to deploy Google's tensor processing units for production use.

Google announced that a new PyTorch-native version of the SGLang inference engine is now running in production at Meta. This update allows Meta's AI infrastructure to operate on Google's TPUs without requiring a migration to Google's JAX framework. "SGLang-TorchTPU is running in production at Meta today," said Zongwei Zhou, Meta's PyTorch-TPU lead, noting the work involved collaboration between Meta, Google, and the SGLang team. The engine provides Meta with a mature, PyTorch-native serving stack.

The release of this capability follows a period of shifting relationships regarding chip usage. Meta previously canceled contracts for using TPUs between 2020 and 2023. According to reports, discussions regarding Google Cloud TPU rentals for 2026 and potential purchases for 2027 were held last year, and Meta later entered into a multi-year rental agreement. Jana van Greunen, director of PyTorch engineering at Meta, stated that Meta "cannot be constrained by a single hardware architecture" and that the new software enables the deployment of PyTorch workloads to Google TPUs.

The push for PyTorch compatibility is not limited to Google. Amazon has open-sourced TorchNeuron, a native PyTorch backend designed for its Trainium2 and Trainium3 chips. This release occurred at the PyTorch Conference, one day after Google's announcement. Both companies appear to be positioning their hardware as viable alternatives to Nvidia by removing software barriers that previously limited their use with popular AI frameworks.

Performance metrics provided by Google and its partners suggest that TPUs can compete with Nvidia hardware for specific tasks. In a test conducted by Inferact, a startup founded by the creators of vLLM, a tuned vLLM setup on Google's seventh-generation Ironwood TPU exceeded 700 tokens per second for a single user serving the 2.8-trillion-parameter Kimi K3 model. The test utilized 16 Ironwood chips with speculative decoding. For comparison, vLLM's published setup on 16 Nvidia GB200s reached 452 tokens per second.

Further testing indicates that for certain models, such as Kimi K3 and Qwen 3.8 27B, the TPU kernels deliver throughput between 1.4 to 2 times the GB200 baseline at small batch sizes without speculative decoding. These figures arrive as chip manufacturers compete for dominance in the market for AI inference infrastructure.

Looking ahead, the SGLang-TorchTPU engine will expand its support to include models such as DeepSeek V4.1 Flash and Reflection Beam. The developers are working toward providing day zero support for new open models by the first quarter of 2027. Google, Meta, and RadixArk plan to discuss these developments further at the SGLang Summit 2026, scheduled for Nov. 12 to 13 in San Francisco.