scaling
30 articles about scaling in AI news
Scaling Laws Differ for Native Multimodal VLMs
A systematic study reveals distinct scaling laws for native multimodal pre-training, showing vision-language models require different compute-optimal size/token ratios than language-only models.
Cerebras CS4 Stays on 5nm as SRAM Scaling Flattens
Cerebras CS4 stays on 5nm due to SRAM scaling flattening, per @SemiAnalysis_. 3nm offers no density gain, so the chip prioritizes yield and cost.
Huawei's τ Scaling Law Redefines Transistor Race Without EUV
Huawei's τ Scaling Law at IEEE ISCAS replaces geometric transistor scaling with time-based optimization, targeting 1.4nm density by 2031 without EUV, challenging US export controls.
LoopCTR: A New 'Loop Scaling' Paradigm for Efficient
A new research paper introduces LoopCTR, a method for scaling Transformer-based CTR models by recursively reusing shared layers during training. This 'train-multi-loop, infer-zero-loop' approach achieves state-of-the-art performance with lower deployment costs, directly addressing a core industrial constraint in recommendation systems.
Lloyds Banking Group Details 'Atlas' ML Platform for Scaling AI in a
A technical blog post details how Lloyds Banking Group rebuilt its internal Machine Learning platform, Atlas, on a cloud-native architecture to overcome scaling limits and meet stringent regulatory requirements. This is a blueprint for operationalizing AI in high-stakes, governed industries.
Scaling Law Plateau Not Universal: More Tokens Boost Reasoning AI Performance
Empirical evidence indicates the 'second scaling law'—performance gains from increased computation—does not fully plateau for many reasoning tasks. Benchmark results may be artificially limited by token budgets, not model capability.
UniMixer: A Unified Architecture for Scaling Laws in Recommendation Systems
A new arXiv paper introduces UniMixer, a unified scaling architecture for recommender systems. It bridges attention-based, TokenMixer-based, and factorization-machine-based methods into a single theoretical framework, aiming to improve parameter efficiency and scaling return on investment (ROI).
UniScale: A Co-Design Framework for Data and Model Scaling in E-commerce Search Ranking
Researchers propose UniScale, a framework that jointly optimizes data collection and model architecture for search ranking, moving beyond just scaling model parameters. It addresses diminishing returns from parameter scaling alone by creating a synergistic system for high-quality data and specialized modeling. This approach, validated on a large-scale e-commerce platform, shows significant gains in key business metrics.
Roman Yampolskiy: 'AGI is a Question of Cost, Not Time' as Scaling Laws Hold
AI safety researcher Roman Yampolskiy argues that achieving AGI is now a matter of computational and financial resources, not theoretical possibility, citing the continued validity of scaling laws and early signs of recursive self-improvement.
Qwen's Tiny Titan: How a 2B Parameter Multimodal Model Challenges AI Scaling Assumptions
Alibaba's Qwen team has released Qwen2-VL-2B, a surprisingly capable 2-billion parameter multimodal model with native 262K context length, extensible to 1M tokens. This compact model challenges assumptions about AI scaling while offering practical long-context capabilities for resource-constrained environments.
Beyond Better Models: The Compute Scaling Revolution Driving AI's Next Leap
New analysis reveals that scaling compute infrastructure may deliver 10× annual efficiency gains in AI development, surpassing algorithmic improvements alone. The real leverage comes from combining innovative ideas with massive computational resources.
Agent Harness Scaling: EFC Predicts Success at R2 0.99 vs 0.42
New research introduces Effective Feedback Compute (EFC), which predicts agent success at R2 0.99 vs 0.42 for raw tokens. Reallocating compute by EFC lifts success 3x at the same budget.
OpenAI Readies General-Purpose LLM With Test-Time Compute Scaling
OpenAI is releasing a general-purpose LLM that improves with test-time compute, per an internal message. The model shows math gains without specialized training.
Cerebras IPO Challenges GPU Scaling Orthodoxy
Cerebras filed for IPO on April 21, betting wafer-scale chips can disrupt Nvidia's GPU cluster model for AI workloads.
Stateless Memory for Enterprise AI Agents: Scaling Without State
The paper replaces stateful agent memory with immutable decision logs using event-sourcing, allowing thousands of concurrent agent instances to scale horizontally without state bottlenecks.
VISTA: A Novel Two-Stage Framework for Scaling Sequential Recommenders to Lifelong User Histories
Researchers propose VISTA, a two-stage modeling framework that decomposes target attention to scale sequential recommendation to a million-item user history while keeping inference costs fixed. It has been deployed on a platform serving billions.
OpenReward Launches: A Minimalist Service for Scaling RL Environment Serving
OpenReward, a new product from Ross Taylor, launches as a focused service for serving reinforcement learning environments at scale. It aims to solve infrastructure bottlenecks for RL training pipelines.
NVIDIA's Nemotron-Terminal: A Systematic Pipeline for Scaling Terminal-Based AI Agents
NVIDIA researchers introduce Nemotron-Terminal, a comprehensive data engineering pipeline designed to scale terminal-based large language model agents. The system bridges the gap between raw terminal data and high-quality training datasets, addressing key challenges in agent reliability and generalization.
Robotics' Scaling Breakthrough: How SONIC's 42M-Parameter Model Achieves Perfect Real-World Transfer
Researchers have demonstrated that robotics can scale like language models, with SONIC training a 42M-parameter model on 100M human motion frames. The system achieved 100% success transferring to real robots without fine-tuning, marking a paradigm shift in robotic learning.
Enterprise AI Goes Mainstream: How Major Corporations Are Scaling Operations with Intelligent Voice Systems
Major corporations including FedEx, Marriott, and Volkswagen are deploying advanced AI voice systems to handle millions of customer interactions, enabling instant scalability during peak demand periods without traditional hiring constraints.
Dyna-2 World-Action Model Trained on 1M Hours Video
Dyna-2, trained on 1M+ hours of egocentric video, jointly predicts future video and actions. Claims new scaling laws but no benchmarks or technical details released.
Sam Altman: AI Token Usage Growing Exponentially
Sam Altman claims AI token usage grows exponentially, referencing a 6.5-year baseline. The remark signals compounding inference demand that pressures infrastructure scaling.
Intology's Locus beats human-tuned Qwen3-1.7B in auto post-training
Intology's Locus beat human-tuned Qwen3-1.7B (51.6% vs 49.4%) on PostTrainBench by scaling compute 64x, showing AI research agents need longer timescales.
Epoch AI: Parallelization limits could delay intelligence explosion
Epoch AI argues parallelization limits could delay an intelligence explosion by years, as scaling beyond 10^28 FLOP faces diminishing returns from hardware constraints.
NVIDIA's Molt: 9.2K-Line RL Framework Scales to 1T-Parameter MoE Models
NVIDIA released Molt, a 9.2K-line PyTorch RL framework scaling to 1T-parameter MoE models via vLLM, targeting agentic tasks with fully-async rollout.
Google Chooses Intel EMIB-T for 9th-Gen TPUs, Breaking TSMC's CoWoS Monopoly
Google picks Intel EMIB-T for 9th-gen TPU, breaking TSMC CoWoS monopoly. Move signals architectural bet on power integrity and reticle-free scaling.
PadCaptioner: 3B video caption model beats 7B rivals with parallel decoding
PadCaptioner, a 3B model, beats 7B rivals in dense video captioning via lossless parallel autoregressive decoding, challenging scaling orthodoxy.
Production Deployment Patterns for AI Agent Systems: From Prototype to Scale
The article presents CI/CD, monitoring, rollback, and scaling patterns for AI agent production deployments from a SaaS practitioner. It emphasizes treating multi-agent workflows as atomic units, using OpenTelemetry tracing, and implementing circuit breakers for resilience.
Reverse-engineering Nvidia's cuda-checkpoint reveals 70x cold-start speedup path
Reverse-engineering Nvidia's cuda-checkpoint reveals PCIe bandwidth underutilization. The tool enables up to 70x faster cold starts for GPU servers, critical for AI inference scaling.
PhotoQuilt Makes Training-Free Photomosaics at 14K Resolution
PhotoQuilt generates training-free photomosaics at any resolution, bootstrapping a global layout at low res then upscaling tiles via FLUX, scaling past 14K without quadratic attention cost.