tokens
30 articles about tokens in AI news
NVIDIA Groq 3 LPX Hits 3,400 Tokens/s on Gemma 4 31B
NVIDIA's Groq 3 LPX claims 3,400 tokens/s on Gemma 4 31B, with Nebius first to deploy. The 4x responsiveness claim targets agent latency.
MARGINAL: The Open-Source Governor That Stops Claude Code From Wasting Tokens
MARGINAL is an open-source runtime governor that stops Claude Code from wasting tokens by observing repeated actions and intervening only with proof. Contribute via GitHub issues or try the demo to optimize your agent's efficiency.
OpenAI's Ultrafast Mode Hits 750 Tokens/s on GPT-5.6 Sol
OpenAI launched Ultrafast mode for GPT-5.6 Sol at 14x speed and 750 tokens/s, powered by Cerebras. Preview limited to select customers, targeting latency-sensitive enterprise workflows.
OpenAI Cuts GPT-5.6 Luna Price 80% to $0.20/M Tokens
OpenAI cut GPT-5.6 Luna prices 80% to $0.20/M input tokens, citing Sol-optimized kernels that cut serving costs 20%. Luna now undercuts Gemini Flash-Lite and Claude Haiku.
CLI vs MCP: Hands-On Experiments Show 250 Tokens vs Over 2,000 for the Same Task
An AI developer ran three experiments comparing CLI and MCP for AI agents. CLI used 250 tokens for file ops and Git; MCP used over 2,000 due to tool schema overhead, but MCP won for web fetching (250 tokens vs 2,000+ for CLI).
Claude Opus 5 Is Now in Claude Code: How to Use Fast Mode and Save 50% on Tokens
Claude Opus 5 is now in Claude Code with Fast Mode (2.5x speed) at Opus 4.8 pricing. Run `claude code --model opus-5` to start saving 50% on tokens immediately.
SWE-Pruner Pro Saves 39% Tokens by Reading LLM Hidden States
SWE-Pruner Pro saves up to 39% tokens on coder LLMs by reading keep-or-prune signals from hidden states, maintaining task quality without external heuristics.
Google’s Frozen v2 chip: 6–10× tokens/W for Gemini, 2028 target
Google is developing Frozen v2, a chip freezing Gemini architecture into silicon for 6–10× tokens/W, deployment as early as 2028, driven by compute shortage.
Why MCP Can't Save Your Input Tokens (And What Actually Works in Claude Code)
MCP tools can't reduce your Claude Code input tokens since they run after context entry. Use a pre-send hotkey or CLI pipe to clean prompts before they enter the token meter.
Open-weight models now run 29% of gateway tokens, up from 11% in April
Open-weight models now handle 29% of gateway tokens, up from 11% in April. The 18-point jump signals accelerating enterprise adoption of open architectures like Llama 3 and Mistral.
The Caveman Skill for Claude Code Saves 8.5% Tokens
Caveman skill for Claude Code saves 8.5% tokens, not 65%. Safe to use with no quality loss. Install via SkillsBench.
Lovable spent $85K on tokens to learn agentic coding at scale
Lovable spent $85K on tokens for agentic coding. Debugging costs dominate, challenging enterprise adoption.
FreeLLMAPI Aggregates 1.7B Free Tokens/Month Across 11 Providers
FreeLLMAPI aggregates 11 free LLM providers into one endpoint, offering 1.7B tokens/month with automatic fallover. Reduces friction for side projects but faces provider tolerance risks.
BioMatrix: A single decoder reads proteins, molecules, language on 304B tokens
BioMatrix, a decoder-only biological foundation model, achieves SOTA on 77 of 80 tasks after training on 304B tokens of sequences, structures, and language.
MCP Tool Overload Eats 1.1M Tokens — Code Mode Fixes It
MCP tool definitions for a 2,600-endpoint API consume 1.1M tokens, breaking agent context. Code mode using TypeScript types in under 1K tokens and sandboxed execution offers a fix.
Thinking Tokens Drive Hidden Inference Costs in Agentic Pipelines
Thinking tokens from OpenAI, Anthropic, and Google models are priced at output rates, silently inflating costs 5x–10x in agentic pipelines. Google's 80% price cut threat exposes a structural asymmetry between startups and tech giants.
Nadella: AI's New Unit Is 'Tokens per Dollar per Watt'
Satya Nadella defined AI's supply-side economics as 'Tokens per Dollar per Watt', urging infrastructure focus for companies, industries, and countries.
Google Open-Sources DiffusionGemma, 26B Model Hits 1K Tokens/Sec on H100
Google open-sourced DiffusionGemma, a 26B-parameter diffusion text model hitting 1,000 tokens/sec on H100 — 4x faster than autoregressive models, but with lower quality.
MiniMax M3 Sparse Attention: 15.6x Decoding Speedup at 1M Tokens
MiniMax M3 sparse attention achieves 9.7x prefilling and 15.6x decoding speedup at 1M tokens, reversing M2's full-attention stance.
Cerebras Hits 981 Tokens/sec on 1T-Parameter Kimi K2.6, Claims 6.7× GPU Cloud Speedup
Cerebras reported 981 tokens/sec on the 1T-parameter Kimi K2.6 model, a 6.7× speedup over the next GPU cloud, validated by an independent third party.
Median Coding Agent Hits 96k Input Tokens, Rewriting Inference Economics
SemiAnalysis found median coding agent uses 96k input tokens from 432k requests, shifting inference cost focus from output to context.
Glean benchmark: Off-the-shelf MCP costs 30% more tokens than indexed context
Glean benchmark: off-the-shelf MCP in Claude Cowork loses 2.5x more tasks and uses 30% more tokens than indexed context.
CLAUDE.md Wastes 7K+ Tokens Per Turn; Skills Cut to 50
A 1,000-line CLAUDE.md burns 7,000-10,000 tokens per turn on instructions the model already knows. Skills using progressive disclosure cut that to ~50 tokens.
ByteDance GenLIP: ViT Predicts Language Tokens Directly with 8B Samples
ByteDance's GenLIP trains ViTs to predict language tokens directly with a single autoregressive objective, outperforming baselines on 8B samples.
Talkie: Vintage LLM Trained on 260B Pre-1931 English Tokens
Talkie is a new 'vintage language model' trained on 260 billion tokens of historical English text from before 1931, developed by a team including Alec Radford, co-author of the original GPT paper. It offers a unique linguistic artifact for NLP research.
Doby Cuts Claude Code Navigation Tokens by 95% with Spec-First Workflow
A spec-first fix workflow that slashes navigation tokens 95% and enforces plan docs as source of truth before code changes.
OpenAI Engineer Processed 210B Tokens, Sparking AI Efficiency Debate
An OpenAI engineer processed 210 billion tokens in one week, equivalent to 33 Wikipedia-sized datasets. This extreme usage spotlights a growing trend where high AI consumption by engineers leads to a 10x cost increase and a high volume of discarded code.
Claude Mythos Preview Priced at $25/$125 Per Million Tokens
Anthropic's Claude Mythos model is available in private preview at $25 per million input tokens and $125 per million output tokens. This positions it as a premium but competitively priced option in the high-performance LLM market.
Google's Gemma 4B Model Runs on Nintendo Switch at 1.5 Tokens/Second
A developer successfully ran Google's 4-billion parameter Gemma language model on a Nintendo Switch, achieving 1.5 tokens/second inference. This demonstrates the increasing feasibility of running small LLMs on consumer-grade edge hardware.
Gemma 4 26B A4B Hits 45.7 tokens/sec Decode Speed on MacBook Air via MLX Community
A community benchmark shows the Gemma 4 26B A4B model running at 45.7 tokens/sec decode speed on a MacBook Air using the MLX framework. This highlights rapid progress in efficient local deployment of mid-size language models on consumer Apple Silicon.