Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter open-weight MoE model, dropped on Hugging Face. The release, announced via X post by @rohanpaul_ai, signals a new scale for open-weight AI.
Key facts
- 2.4T total parameters, 95B active per token
- 25x sparsity ratio, 4% activated per forward pass
- Released on Hugging Face, announced via X post
- 41% larger than Qwen2.5-1.7T-A70B
- Requires ~4.8 TB VRAM in FP16 for loading
Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter open-weight mixture-of-experts (MoE) model, was released on Hugging Face, as announced via an X post by @rohanpaul_ai. According to @rohanpaul_ai, the model features 95 billion active parameters per token, yielding a sparsity ratio of roughly 25x — meaning only 4% of the total parameters are activated for each forward pass.
This is not the first large MoE from Qwen, but it is the largest open-weight model to date. Qwen previously released Qwen2.5-1.7T-A70B, a 1.7-trillion-parameter model with 70B active parameters. The jump to 2.4T total parameters with 95B active represents a 41% increase in total parameters and a 36% increase in active parameters, positioning Qwen3.8-2.4T-A95B as a frontier-scale model available for download.
The open-weight nature of this release is significant. While closed models like GPT-4 and Claude 3.5 Opus are estimated to have trillions of parameters, they are not publicly available. Qwen3.8-2.4T-A95B breaks that barrier, allowing researchers and developers to inspect, fine-tune, and deploy a model of this scale on their own infrastructure, provided they have the compute resources — which are substantial.
What the sparse architecture means
The MoE architecture with 95B active parameters per token is a deliberate trade-off. It allows the model to have the knowledge capacity of a 2.4T-parameter dense model while keeping inference cost closer to a 95B-parameter model. This is the same design philosophy as Mixtral-8x7B and DeepSeek-V3, but at a larger scale. The 25x sparsity ratio is on par with DeepSeek-V3's 37x sparsity, but Qwen's active parameter count is higher, which could translate to better per-token performance.
However, the practical deployment challenges are non-trivial. A 2.4T-parameter model in FP16 requires roughly 4.8 TB of VRAM just to load the weights. This necessitates multi-node inference setups, typically involving dozens of GPUs. The Hugging Face model card — if it exists — would detail the exact inference requirements, but the source post does not provide them. The company did not disclose the figure for training compute, but extrapolating from Qwen2.5-1.7T's reported training cost of around $20 million, this model likely cost tens of millions to train.
Why this matters beyond the parameter count
The unique angle here is not just the size — it's the timing. Qwen has been releasing open-weight models at a rapid clip, with Qwen2.5-1.7T released in late 2025 and now Qwen3.8-2.4T in early 2026. This cadence suggests a deliberate strategy to dominate the open-weight frontier, forcing competitors like Llama and Mistral to respond. The release also comes amid growing scrutiny of open-weight models' safety, but Qwen has not shown signs of slowing down.
For enterprises, this model could be a viable alternative to closed APIs for high-throughput, knowledge-intensive tasks. The 95B active parameter count means inference costs are manageable at scale, potentially undercutting API pricing for large batch workloads. But the infrastructure barrier remains high, and most organizations will need to rely on cloud providers or specialized inference companies to deploy it.
One caveat: the source is a single X post with a link to Hugging Face, and no official Qwen blog post or documentation has been cited. The model card, if it exists, would provide benchmark scores, training details, and licensing terms. Without that, the performance claims are unverified. Per the arXiv preprint on Qwen2.5, the previous generation scored 84.1 on HumanEval and 77.4 on MMLU, but we have no such numbers for this release. The absence of official benchmarks is a gap that needs filling.
The competitive landscape
Qwen3.8-2.4T-A95B enters a market where open-weight models are increasingly competitive. DeepSeek-V3, released in late 2025, has 671B total parameters with 37B active, and scored 88.5 on HumanEval. Llama 4, if it follows Meta's pattern, is expected to be in the 1T-parameter range but has not yet been released. Qwen's 2.4T total parameter count leapfrogs DeepSeek's scale, but the active parameter count of 95B is also higher, suggesting a more capable model per token.
But the real competition may be from closed models. GPT-5 and Claude 4 are reportedly in the trillions of parameters, and their API pricing is dropping. For developers who need guaranteed performance and don't want to manage infrastructure, closed APIs remain the simpler choice. Open-weight models like Qwen3.8-2.4T-A95B are for those who need customization, data privacy, or cost control at scale.
The release also raises questions about the sustainability of open-weight AI. Training a 2.4T-parameter model requires massive compute, and Qwen's parent company, Alibaba, has deep pockets. But smaller labs may not be able to keep up, leading to a consolidation of open-weight development among a few large players. This could reduce the diversity of open models over time, even as individual models grow larger.
In summary, Qwen3.8-2.4T-A95B is a landmark release for open-weight AI, but the lack of official documentation means the community must wait for benchmarks and technical details. The model's true impact will depend on its performance on standard benchmarks and its accessibility to researchers.
What to watch
Watch for an official Qwen blog post or arXiv paper detailing benchmark scores (e.g., MMLU, HumanEval) and training compute. Also monitor Hugging Face model card updates for licensing terms and inference benchmarks. If Qwen releases a smaller, more deployable variant, that could signal broader enterprise adoption.
