llm
30 articles about llm in AI news
Massive Activations Found in Hybrid Linear Attention LLMs
New paper finds massive activations in hybrid linear attention LLMs, forming pre-attention spikes and inter-spike plateaus. Code and checkpoints released on Hugging Face; implications for quantization and deployment.
Meta Paper: LLM Judge Accuracy vs. Golden Data Is Hollow
Meta paper argues LLM judge accuracy vs golden data is insufficient, pushing for reliability metrics like consistency and calibration. Omar Sanseviero flagged it on X.
Spotify Engineers: LLM A/B Tests Recover Only 39% of Human Treatment Effects
Spotify Engineering tested LLM-based A/B testing on the Upworthy dataset, finding raw predictions recover only 39% of human treatment effects. The bias is systematic, and calibration works only under unverifiable assumptions.
Treat Your LLM as an Untrusted Client: Hardening GitLab MCP for Enterprise Use
Secure GitLab MCP by treating the LLM as untrusted: add fail-closed Gitleaks scanning, per-project read limits, and expose only 8 essential tools to prevent data exfiltration.
LLM Gateway Moves That Cut Multi-Provider AI Bills 40–85%
Towards AI details an LLM gateway routing layer that cuts multi-provider AI costs by 40–85%, with pricing from $0.10 to $30 per million tokens. It matters for retail teams managing escalating AI spend.
Relay-OPD: On-Policy Distillation Fixes Prefix Failure in LLMs
Relay-OPD introduces on-policy distillation where a teacher briefly takes over student LLM generation to fix prefix failure, reducing error compounding without full teacher compute.
Xiaomi MiMo-V2.5 Tops Global LLM Token Usage at 31.2T/Month
Xiaomi MiMo-V2.5 leads global LLM token usage on OpenRouter at 31.2T/month, as Chinese models capture 63.5% share vs US 35.5%.
First Movers in Agentic Commerce May Build Lasting Advantages as LLMs
Beet.TV reports that first movers in agentic commerce, where AI agents autonomously shop, gain lasting advantages as LLMs improve memory. This matters because early adoption creates data moats that compound over time, potentially reshaping competition in retail.
LLM Waterfall Pattern: 429 Failover Beats Retries & Circuit Breakers
The LLM waterfall pattern cascades requests across providers on 429 errors, outperforming retries and circuit breakers for zero-downtime AI inference.
SWE-Pruner Pro Saves 39% Tokens by Reading LLM Hidden States
SWE-Pruner Pro saves up to 39% tokens on coder LLMs by reading keep-or-prune signals from hidden states, maintaining task quality without external heuristics.
NUS CIMERA Chip Cuts LLM Memory Wall with Compute-in-Interconnect
NUS researchers propose CIMERA, an LLM inference accelerator integrating compute-in-interconnect and memory to mitigate the memory wall, detailed in arXiv:2607.13649 (July 2026).
100+ Papers Surveyed: LLMs' Metacognition Gap
A systematic survey of 100+ papers reveals gaps in LLM metacognition, including 10-30% miscalibration in top models like GPT-4 and Claude 3.
7 AI Agent Cost Optimization Strategies That Cut LLM Bills by Up to 90%
The source outlines seven cost optimization strategies for AI agents, including prompt compression and model routing, that can reduce LLM bills by up to 90%. This matters for retail and luxury brands deploying AI at scale where inference costs can become prohibitive.
LLMs Learn to Switch Reasoning Effort at Inference Time
@rasbt explains how LLMs switch reasoning effort using inference-time methods and training, potentially cutting token usage by 30–50% on simple queries.
GPT-Red: OpenAI's LLM Super-Hacker Finds 84% of Attacks, Humans 13%
OpenAI's GPT-Red LLM finds 84% of attacks vs 13% for humans, hardening GPT-5.6 Sol. Automated red-teaming shifts safety paradigm.
Airbnb Cuts LLM Eval From Weeks to a Day With Deterministic Caching
Airbnb cut LLM eval from weeks to a day with deterministic caching and micro adapters. The approach trains bug-fix patches in under an hour per GPU.
Graphify Turns Codebases Into Queryable Graphs Without Vectors or LLMs
Graphify maps codebases into local knowledge graphs using tree-sitter AST parsing, no LLMs or vector stores. At 82k+ stars, it supports 40+ languages and 20+ AI assistants.
NVIDIA TwoTower: 2.4x Faster LLM Decoding, 98.7% Quality
NVIDIA TwoTower clones a pretrained LLM into a frozen context tower and trainable denoiser tower, achieving 2.42x faster generation with 98.7% quality on a 30B MoE model.
ZML releases free LLM inference server supporting Nvidia
ZML released LLMD, a free inference server for LLMs supporting Nvidia, AMD, Google TPU, Apple Metal, and Intel Arc, aiming to reduce AI costs and break vendor lock-in.
LLMForge: 7 Models Score 0.89 on CAD Benchmark; VLMs Fix Cylinders
LLMForge scores 0.89 on 97-design CAD benchmark. VLM critic achieves 100% watertight meshes but cylinders remain a failure mode.
LLM agents fail nonlinearly as tasks lengthen, 27-paper synthesis finds
27-paper synthesis finds LLM agent failures compound nonlinearly with task length. Six failure clusters identified across 19 benchmarks.
Alibaba's MIPI fixes LLM training-inference mismatch with direct RL
Alibaba's MIPI directly optimizes inference policy, fixing the mismatch in LLM post-training via the MIPU framework.
BayesBench: LLMs Match Bayesian Posteriors But Fail Downstream Prediction
BayesBench tests 7 LLMs on multi-turn Bayesian reasoning. Scaling improves latent inference but not prediction, exposing a critical gap for agentic deployment.
LLMs Spontaneously Develop Human-Like Brain Regions for Language, Math
LLMs spontaneously develop human-like brain regions for language, math, physics, and social reasoning, per @LiorOnAI. Two optimization processes converged on the same solution.
3 MCP Gateway Security Gaps LiteLLM's Audit Found (And How to Fix Them in
LiteLLM's audit revealed 3 MCP gateway gaps: fail-open resolver, unpinned servers, opt-in least-privilege. Fix them in Claude Code with version pinning and allowed_tools.
FreeLLMAPI Aggregates 1.7B Free Tokens/Month Across 11 Providers
FreeLLMAPI aggregates 11 free LLM providers into one endpoint, offering 1.7B tokens/month with automatic fallover. Reduces friction for side projects but faces provider tolerance risks.
LLMs Default to Zod Schemas, Breaking MCPFusion Security Contracts
LLMs default to raw Zod schemas, bypassing MCPFusion's defineModel() and risking data leaks. The Developer Prover enforces MVA architecture via rejection.
OpenAI, Broadcom Unveil Jalapeño ASIC for LLM Inference
OpenAI and Broadcom unveiled Jalapeño, a custom ASIC for LLM inference, targeting volume deployment by late 2026. No performance metrics were disclosed.
Miami Startup Claims 12M-Token LLM Inference at $8 vs. $2,600 on Claude
Miami startup claims 12M-token LLM inference for $8 vs. $2,600 on Claude Opus 4.6. No paper or benchmarks released yet.
Zalando Introduces MLLM-Based Evaluation for Product Retrieval
Zalando presents a multimodal LLM-based evaluation for product retrieval, aiming to enhance search relevance in e-commerce. This matters as it could set a new standard for assessing AI in retail search.