moe
30 articles about moe in AI news
FreeToken Runs 284B MoE Locally on a Gaming Desktop
FreeToken claims 284B MoE serving on a gaming desktop via unified PC inference. No benchmarks disclosed; feasibility depends on MoE sparsity and offload strategy.
Qwen3.8-2.4T-A95B Open-Weight MoE Hits Hugging Face
Qwen3.8-2.4T-A95B, a 2.4T-parameter open-weight MoE with 95B active parameters, was released on Hugging Face, marking a new scale for open-weight models.
Cursor Open-Sources MoE Megakernel for NVL72s
Cursor open-sourced Mixture-of-Kittens, an MoE megakernel for NVL72s, targeting inference efficiency. No benchmarks disclosed, but the move signals Cursor's infrastructure ambitions.
NVIDIA's Molt: 9.2K-Line RL Framework Scales to 1T-Parameter MoE Models
NVIDIA released Molt, a 9.2K-line PyTorch RL framework scaling to 1T-parameter MoE models via vLLM, targeting agentic tasks with fully-async rollout.
China's AI ecosystem standardizes on MoE with wide expert parallelism
China's AI ecosystem standardizes on MoE with wide expert parallelism to survive on weaker NPUs. Hardware makers now design 'supernode' systems.
NVIDIA Drops 30B Nemotron Audex Audio Model with MoE
NVIDIA released Nemotron Audex 30B-A3B, a 30B-parameter MoE audio model unifying ASR, understanding, and TTS with 3B active parameters.
Tencent Hunyuan Hy3: 295B MoE Hits 90% Agent Task Resolution
Tencent launched Hunyuan Hy3, a 295B MoE model with 21B active parameters, claiming 90% agent task resolution, surpassing DeepSeek V4 Pro and Qwen 3.7 Max.
MoEngage Buys Aampe for Tens of Millions, Bets AI Agents Replace Campaigns
MoEngage acquired Aampe for tens of millions to embed per-customer AI agents, targeting migrations from Salesforce and Adobe Marketing Cloud.
Chinese Lab's Free MoE Model Matches GPT-5.5 on Agentic Coding
A Chinese lab released an Apache-2.0 open-weights MoE model matching GPT-5.5 on agentic coding. This free model challenges proprietary AI's lead with sparse MoE architecture.
dMoE Cuts Active Experts from 69.5 to 14.6, Retains 99.11% Performance
dMoE reduces active experts from 69.5 to 14.6 in diffusion LLMs, retaining 99.11% performance while cutting memory 80% and speeding inference 1.66×.
JetBrains Open-Sources Mellum2: 12B MoE at 2.5B Active Params
JetBrains open-sourced Mellum2, a 12B MoE model with 2.5B active params, trained from scratch for code and reasoning.
ByteDance Lance 3B MoE Beats 7B Models on Multimodal Benchmarks
ByteDance released Lance, a 3B multimodal MoE model that beats 7B+ models on benchmarks through multi-task synergy and specialized pathways.
New MoE Framework Tames User Interest Shifts in Long-Sequence Recommendations
Researchers propose MoS, a model-agnostic MoE approach that handles long user sequences by detecting session hopping – where user interests shift across sessions. The theme-aware routing mechanism filters irrelevant sessions, while multi-scale fusion captures global and local patterns. Results show SOTA on benchmarks with fewer FLOPs than alternatives.
Fine-Tuning OpenAI's GPT-OSS 20B: A Practitioner's Guide to LoRA on MoE Models
A technical guide details the practical challenges and solutions for fine-tuning OpenAI's 20-billion parameter GPT-OSS model using LoRA. This is crucial for efficiently adapting large, complex MoE models to specific business domains.
Qwen 3.5 397B-A17B MoE Model Runs on M3 Mac at 5.7 TPS with 5.5GB Active Memory via SSD Streaming
Developer Dan reportedly runs the 209GB Qwen 3.5 397B-A17B MoE model on an M3 Mac at ~5.7 tokens per second using only 5.5GB of active memory by quantizing and streaming weights from SSD.
ELDR: Expert-Locality Decode Routing Cuts MoE TPOT by 13.9%
ELDR uses prefill expert signatures to route decode requests, cutting median TPOT by 5.9–13.9% in vLLM at scale.
Qwen3.6-27B: How to Run a 17GB Local Model That Beats 397B MoE on Coding Tasks
Qwen3.6-27B delivers flagship-level coding performance in a 55.6GB model that can be quantized to 16.8GB, making high-quality local coding assistance accessible.
NVIDIA Nemotron 3 Super: 120B Hybrid Mamba-Transformer MoE with 1M Context
NVIDIA has released Nemotron 3 Super, a 120B parameter open hybrid Mamba-Transformer Mixture of Experts model with 12B active parameters and 1M token context length. The company claims it delivers up to 7.5x higher throughput than similar open models.
Alibaba Qwen3.6-35B-A3B: 3B-Active Sparse MoE Hits 73.4% on SWE-Bench
Alibaba released Qwen3.6-35B-A3B, a sparse mixture-of-experts model with 35B total but only 3B active parameters. It shows significant gains over its predecessor, scoring 73.4% on SWE-bench Verified and beating Claude 3.5 Sonnet on several vision tasks.
Cursor AI Claims 1.84x Faster MoE Inference on NVIDIA Blackwell GPUs
Cursor AI announced a rebuilt inference engine for Mixture-of-Experts models on NVIDIA's new Blackwell GPUs, resulting in a claimed 1.84x speedup and improved output accuracy.
Stanford Releases Free LLM & Transformer Cheatsheets Covering LoRA, RAG, MoE
Stanford University has released a free, open-source collection of cheatsheets covering core LLM concepts from self-attention to RAG and LoRA. This provides a consolidated technical reference for engineers and researchers.
Google Releases Gemma 4 Family Under Apache 2.0, Featuring 2B to 31B Models with MoE and Multimodal Capabilities
Google has released the Gemma 4 family of open-weight models, derived from Gemini 3 technology. The four models, ranging from 2B to 31B parameters and including a Mixture-of-Experts variant, are available under a permissive Apache 2.0 license and feature multimodal processing.
Kimi 2.5's 1T Parameter MoE Model Runs on 96GB Mac Hardware via SSD Streaming
Developers have demonstrated that Kimi 2.5's 1 trillion parameter Mixture-of-Experts model can run on Mac hardware with just 96GB RAM by streaming expert weights from SSD, with only 32B parameters active per token.
Step-3.5-Flash: 196B Open-Source MoE Model Activates Only 11B Parameters, Outperforms Kimi K2.5 and Claude Opus 4.5 on Key Benchmarks
Shanghai-based StepFun's Step-3.5-Flash, a 196B parameter sparse mixture-of-experts model that activates only 11B parameters per token, achieves top scores on AIME 2025 (97.3) and LiveCodeBench-V6 (86.4) while costing 18.9x less to run than Kimi K2.5.
NVIDIA Releases Nemotron-Cascade 2: A 30B MoE Model with 3B Active Parameters
NVIDIA has open-sourced Nemotron-Cascade 2, a 30B parameter Mixture-of-Experts model that activates only 3B parameters per token. It claims 'gold medal performance' on IMO and IOI 2025 benchmarks.
The Hidden Cost of Mixture-of-Experts: New Research Reveals Why MoE Models Struggle at Inference
A groundbreaking paper introduces the 'qs inequality,' revealing how Mixture-of-Experts architectures suffer a 'double penalty' during inference that can make them 4.5x slower than dense models. The research shows training efficiency doesn't translate to inference performance, especially with long contexts.
Beyond Homogenization: How Expert Divergence Learning Unlocks MoE's True Potential
Researchers have developed Expert Divergence Learning, a novel pre-training strategy that combats expert homogenization in Mixture-of-Experts language models. By encouraging functional specialization through domain-aware routing, the method improves performance across benchmarks with minimal computational overhead.
Google Lyria 3 Pro Music AI Demoed: Generates '1990s Boy Band' Version of Rilke Poetry
A researcher gained early access to Google's Lyria 3 Pro music generation AI, demonstrating its ability to transform Rainer Maria Rilke's 'First Elegy' into a 1990s boy band track. The demo highlights rapid stylistic remixing capabilities not yet publicly available.
Moonshot AI Releases 1.56T-Parameter Kimi K3, Requires 2x B200 Nodes
Moonshot AI released Kimi K3, a 1.56T parameter MoE model at 1561 GB, requiring 2x B200 nodes. No benchmarks disclosed.
Alibaba Releases RynnBrain 1.1 Embodied AI Models at 2B-122B Scales
Alibaba released RynnBrain 1.1 on Hugging Face with 2B, 9B, and 122B-A10B MoE models for robot manipulation, but disclosed no benchmarks.