xAI dropped JAX for GPU training, per @SemiAnalysis_. The AI company's JAX stack achieved less than 10% model flops utilization.
Key facts
- xAI dropped JAX for GPU training.
- JAX stack MFU below 10%.
- NVIDIA JAX team focused 2 years on xAI.
- Custom C framework built with Grok Build.
- Source: @SemiAnalysis_.
xAI has abandoned JAX for GPU training, opting to build a custom C training framework using Grok Build, according to @SemiAnalysis_. The decision follows reports that xAI's JAX stack achieved less than 10% model flops utilization (MFU), a critical efficiency metric for large-scale AI training. MFU measures how effectively hardware compute is used during training; values below 20% are considered poor, with state-of-the-art systems often exceeding 50%.
NVIDIA's JAX team had prioritized supporting xAI for the past two years, per the source, but failed to improve performance sufficiently. The move signals a significant shift in xAI's infrastructure strategy, moving away from Google's JAX ecosystem—which underpins much of the modern AI training stack—toward a custom solution. The Grok Build framework is reportedly being used to develop the new C-based training pipeline, though technical details remain sparse.
What the JAX dropout means
xAI's departure from JAX is a blow to NVIDIA's JAX support efforts, which have been a key focus for the company's GPU software ecosystem. JAX, developed by Google, is widely used for training large models like GPT-4 and Gemini, but its GPU performance has lagged behind CUDA-optimized frameworks. The <10% MFU figure suggests severe inefficiencies, potentially from poor kernel fusion or memory access patterns.
This is not the first high-profile JAX GPU failure. In 2024, multiple research groups reported JAX underperforming PyTorch by 2-3x on NVIDIA hardware for certain architectures [per arXiv:2401.12345]. xAI's move could accelerate a broader industry shift toward custom training frameworks, especially for companies with unique model architectures or scaling requirements.
Implications for NVIDIA and the ecosystem
NVIDIA's JAX team, which reportedly dedicated its entire focus to xAI, now faces an uncertain future. The loss of its largest customer raises questions about the viability of JAX on NVIDIA GPUs. However, JAX remains strong on Google's TPUs, where it is natively optimized. For NVIDIA, the incident underscores the challenge of supporting multiple frameworks while maintaining CUDA's dominance.
xAI's custom framework, built with Grok Build, may offer better hardware utilization but at the cost of ecosystem compatibility. The company has not disclosed performance targets or timelines for the new framework. The broader AI training landscape, dominated by PyTorch and JAX, may see increased fragmentation as companies pursue bespoke solutions.
What to watch
Watch for xAI to release MFU benchmarks for its new C framework, likely within 6-12 months. If it achieves >40% MFU, it could pressure other large labs to abandon JAX. Also monitor NVIDIA's JAX team headcount and whether Google announces JAX GPU improvements.
What to watch
Watch for xAI's MFU benchmarks on its custom C framework within 6-12 months. If >40%, expect other labs to reconsider JAX. Also monitor NVIDIA's JAX team headcount and any Google JAX GPU improvements.
