long context
30 articles about long context in AI news
New Research Diagnoses LLMs' Struggle with Multiple Knowledge Updates in Context
A new arXiv paper reveals a persistent bias in LLMs when facts are updated multiple times within a long context. Models increasingly favor the earliest version, failing to track the latest state—a critical flaw for dynamic knowledge tasks.
DeepSeek V4-Pro: 1.6T parameters, open weights, undercuts rivals 10x
DeepSeek unveiled V4-Pro and V4-Flash, its largest open-weight models with up to 1.6 trillion parameters and a 1M-token context window. The new hybrid attention architecture cuts compute for long contexts by 73–90%, enabling prices far below OpenAI, Google, and Anthropic.
The Hidden Cost of Mixture-of-Experts: New Research Reveals Why MoE Models Struggle at Inference
A groundbreaking paper introduces the 'qs inequality,' revealing how Mixture-of-Experts architectures suffer a 'double penalty' during inference that can make them 4.5x slower than dense models. The research shows training efficiency doesn't translate to inference performance, especially with long contexts.
MIT's RLM Handles 10M+ Tokens, Outperforms RAG on Long-Context Benchmarks
MIT researchers introduced Recursive Language Models (RLMs), which treat long documents as an external environment and use code to search, slice, and filter data, achieving 58.00 on a hard long-context benchmark versus 0.04 for standard models.
λ-RLM: 8B Parameter Model Using Typed λ-Calculus Beats 405B Performance on Long-Context Tasks
Researchers developed λ-RLM, an 8B parameter model that outperforms 405B models on long-context tasks by replacing recursive code with typed λ-calculus combinators. This approach guarantees termination and reduces latency by up to 4.1x.
Sakana AI's Doc-to-LoRA: A Hypernetwork Breakthrough for Efficient Long-Context Processing
Sakana AI introduces Doc-to-LoRA, a lightweight hypernetwork that meta-learns to compress long documents into efficient LoRA adapters, dramatically reducing the computational costs of processing lengthy text. This innovation addresses the quadratic attention bottleneck that makes long-context AI models expensive and slow.
Meta's QTT Method Fixes Long-Context LLM 'Buried Facts' Problem, Boosts Retrieval Accuracy
Meta researchers identified a failure mode where LLMs with 128K+ context windows miss information buried in the middle of documents. Their Query-only Test-Time Training (QTT) method adapts models at inference, significantly improving retrieval accuracy.
Anthropic Surpasses Google in Extended Context AI, Redefining Long-Form Reasoning
Anthropic's Claude has reportedly outperformed Google's models in maintaining attention and reasoning across extended contexts, marking a significant shift in the AI landscape where context length has become a critical competitive frontier.
Beyond the Token Limit: How Claude Opus 4.6's Architectural Breakthrough Enables True Long-Context Reasoning
Anthropic's Claude Opus 4.6 represents a fundamental shift in large language model architecture, moving beyond simple token expansion to create genuinely autonomous reasoning systems. The breakthrough enables practical use of million-token contexts through novel memory management and hierarchical processing.
How One Developer Achieved a 46:1 Context Cache Ratio to Manage 39 Projects
The key takeaway is that maximizing Claude Code's prompt cache through long, context-dense sessions is the most effective way to scale individual productivity across multiple projects.
How Claude Code's 3-Tier Compaction System Saves You Money and Keeps Context
Learn how Claude Code's intelligent, three-tiered compaction system works to manage long conversations efficiently, preserving key context while optimizing for token usage and cost.
How to Run Claude Code 24/7 Without Burning Your Context Window
Implement a hard 50K token session cap and a three-tier memory system (daily notes, MEMORY.md, PARA knowledge graph) to prevent context bloat and memory decay in long-running Claude Code agents.
The Cognitive Divergence: AI Context Windows Expand as Human Attention Declines, Creating a Delegation Feedback Loop
A new arXiv paper documents the exponential growth of AI context windows (512 tokens in 2017 to 2M in 2026) alongside a measured decline in human sustained-attention capacity. It introduces the 'Delegation Feedback Loop' hypothesis, where easier AI delegation may further erode human cognitive practice. This is a foundational study on human-AI interaction dynamics.
Qwen 3.6 Plus Preview Launches on OpenRouter with Free 1M Token Context, Disrupting API Pricing
Alibaba's Qwen team has released a preview of Qwen 3.6 Plus on OpenRouter with a 1 million token context window, charging $0 for both input and output tokens. This directly undercuts paid long-context offerings from Anthropic and OpenAI.
MemoryCD: New Benchmark Tests LLM Agents on Real-World, Lifelong User Memory for Personalization
Researchers introduce MemoryCD, the first large-scale benchmark for evaluating LLM agents' long-context memory using real Amazon user data across 12 domains. It reveals current methods are far from satisfactory for lifelong personalization.
Context Graph for Agentic Coding: A New Abstraction for LLM-Powered Development
A new "context graph" abstraction is emerging for AI coding agents, designed to manage project state and memory across sessions. It aims to solve the persistent context problem in long-running development tasks.
Memory Sparse Attention (MSA) Enables 100M Token Context Windows with Minimal Performance Loss
Memory Sparse Attention (MSA) is a proposed architecture that allows AI models to store and reason over massive long-term memory directly within their attention mechanism, eliminating the need for external retrieval systems. The approach reportedly enables context windows of up to 100 million tokens with minimal performance degradation.
Anthropic's Pricing Revolution: Million-Token Context Now Standard for Claude AI
Anthropic has eliminated the 5x surcharge for million-token contexts in Claude 3 Opus and Claude 3.5 Sonnet, making long-context AI dramatically more affordable. This pricing overhaul removes barriers for developers analyzing large documents, codebases, and datasets.
Claude Code's 1M Context Window Is Now GA — And It's Priced Like Regular Context
Claude Opus 4.6 and Sonnet 4.6 now support 1M tokens with no long-context premium, making massive codebase analysis cheaper than competitors.
VSPrefill: The Vertical-Slash Breakthrough That Makes 128K Contexts Practical
Researchers have developed VSPrefill, a novel sparse attention mechanism that dramatically accelerates long-context processing in LLMs. Using lightweight indexing of vertical columns and slash diagonals, it achieves 4.95x speedup while maintaining 98.35% accuracy at 128k context lengths.
Microsoft Paper Probes Long-Horizon Agent Generalization Gap
Microsoft Research paper on long-horizon agent generalization identifies failure modes and proposes improvements for extended tasks.
Stop Losing Agent Context: Implement Session Memory Files in Your Claude
A simple pattern using structured markdown files to persist session state across context windows, preventing Claude Code agents from redoing work or making inconsistent decisions.
Codex 'Chronicle' Research Preview Adds Memory for Daily Developer Context
A research preview of 'Chronicle' for Codex has been released. It enables the AI coding assistant to accumulate memories from a developer's daily workflow to improve context.
Researchers Achieve Ultra-Long-Horizon Agentic Science with Cohesive AI Agents
A research team has developed AI agents capable of executing and maintaining coherent, long-horizon scientific research workflows. This addresses a core challenge in creating autonomous systems for complex discovery.
NVIDIA Nemotron 3 Super: 120B Hybrid Mamba-Transformer MoE with 1M Context
NVIDIA has released Nemotron 3 Super, a 120B parameter open hybrid Mamba-Transformer Mixture of Experts model with 12B active parameters and 1M token context length. The company claims it delivers up to 7.5x higher throughput than similar open models.
Gemini CLI Launches Subagents with Isolated Context & Custom Instructions
The Gemini CLI tool has launched a 'Subagents' feature, allowing users to run multiple specialized AI agents concurrently, each with its own isolated context and system prompt. This enables more complex, modular workflows by preventing instruction bleed between tasks.
Is Sliding Window All You Need? An Open Framework for Long-Sequence
A new arXiv paper provides a complete, open-source framework for training long-sequence recommender systems using sliding windows. It demonstrates up to +6.34% recall gains on retail data and introduces a novel embedding layer for large vocabularies, making the technique practical for academic and industrial research.
HORIZON Benchmark Diagnoses Long-Horizon Failures in GPT-5 and Claude Agents
A new benchmark called HORIZON systematically analyzes where and why LLM agents like GPT-5 and Claude fail on long-horizon tasks. The study collected over 3100 agent trajectories and provides a scalable method for failure attribution, offering practical guidance for building more reliable agents.
ContextSim: A New LLM Framework for Context-Aware Recommender System Simulation
A new arXiv preprint introduces ContextSim, a framework that uses LLM agents to simulate users interacting with recommender systems within realistic daily scenarios (time, location, needs). Experiments show it generates more human-aligned interactions and that RS parameters optimized with it yield improved real-world engagement.
MemPalace Hits 96.6% on LongMemEval, Beats Paid AI Memory Tools
MemPalace, an open-source AI memory system built by actress Milla Jovovich and developer Ben Sigman, achieved 96.6% on the LongMemEval benchmark—the highest local-only score ever recorded—using a memory palace architecture that stores all conversations verbatim.