NVIDIA's Groq 3 LPX hit 3,400 output tokens/s on Gemma 4 31B, per a tweet from @kimmonismus. The dedicated token-generation accelerator is now in full production on the Vera Rubin platform.
Key facts
- 3,400 output tokens/s on Gemma 4 31B
- 100,000-token context in benchmarking
- 4x faster responsiveness claimed vs nearest alternative
- Nebius first cloud to deploy via Token Factory
- Full production on Vera Rubin platform
NVIDIA is positioning Groq 3 LPX as a latency weapon for agentic workloads, not just a raw-throughput play. The 3,400 tokens-per-second figure on Gemma 4 31B with a 100,000-token context came from Artificial Analysis benchmarking According to @kimmonismus, and NVIDIA claims it's the fastest recorded result for that model. The company also asserts 4x faster responsiveness than the nearest alternative platform for agents and latency-sensitive workloads — a claim that, if it holds, would make Groq 3 LPX a serious contender for real-time AI applications where perceived speed matters more than raw batch throughput.
Why the 4x responsiveness claim matters
Latency is the differentiator here. While many accelerators push high aggregate throughput, the 4x responsiveness figure targets the round-trip time that agents experience — the gap between sending a prompt and receiving the first token. That's the metric that determines whether an AI agent feels snappy or sluggish in interactive settings. NVIDIA hasn't disclosed the full benchmark methodology behind the 4x claim, so independent verification via Artificial Analysis or similar suites will be the test.
Deployment and early access
Nebius will be the first AI cloud to deploy Groq 3 LPX through its Token Factory, followed by Groq itself [per the source]. This sequencing gives Nebius a first-mover advantage in offering the accelerator to its cloud customers, potentially ahead of AWS, Azure, or Google Cloud. The "Token Factory" branding suggests a focus on high-volume token generation — a natural fit for LLM inference at scale, but also for applications like real-time translation, code completion, and interactive agents.
The claim of "intelligence too fast to meter" in the source tweet is marketing hyperbole, but the underlying numbers are concrete. If Groq 3 LPX sustains 3,400 tokens/s in production environments, it would set a new bar for single-model inference performance on a widely used open-weight model like Gemma 4 31B.
Key Takeaways
- NVIDIA's Groq 3 LPX claims 3,400 tokens/s on Gemma 4 31B, with Nebius first to deploy.
- The 4x responsiveness claim targets agent latency.
What to watch
Watch for independent verification of the 3,400 tokens/s and 4x responsiveness claims on Artificial Analysis within the next quarter. Also track Nebius's Token Factory launch date and whether other major clouds (AWS, Azure) announce Groq 3 LPX availability, which would signal broader adoption beyond the initial deployment.
