memory bandwidth
30 articles about memory bandwidth in AI news
China's DFSX SuperNode Doubles GB200 Memory Bandwidth on 14nm
China's DFSX SuperNode claims 2x GB200 memory bandwidth using 14nm vertical towers, but lacks benchmarks or ship dates, warranting skepticism.
Dongfang Suanxin Claims 14nm HBM-Free Chip Beats H200 Bandwidth
China's Dongfang Suanxin claims a 14nm HBM-free AI chip beats Nvidia H200 memory bandwidth, challenging US export controls.
M4 Max Mac Studio Tops GB10 in Local AI Decode Throughput
M4 Max Mac Studio beats GB10 and Strix Halo in local AI decode throughput but memory bandwidth caps large model performance. Tom's Hardware tested llama.cpp across three platforms.
Nvidia Vera Rubin Shifts AI Strategy Beyond Raw GPU Speed
Nvidia's Vera Rubin architecture pivots from raw GPU FLOPS to system-level AI infrastructure, targeting memory bandwidth and interconnect bottlenecks that constrain large-scale model training.
Huawei Ascend SuperPOD Decode Throughput Estimated 1.3-1.7x Behind GB300
Huawei Ascend SuperPOD decode throughput estimated 1.3-1.7x behind GB300, narrower than 4x training gap, due to memory bandwidth and sharding.
NVIDIA Vera CPU Benchmarks: 1.55x Faster Than Intel Xeon in Phoronix Tests
NVIDIA Vera CPU benchmarks show 1.55x performance over Intel Xeon 6980P and 10% over AMD EPYC 9575F, with 1.2 TB/s memory bandwidth.
Supermicro's 7,000-lb Double-Wide Rack Targets B300 Memory Squeeze
Supermicro's 7,000-pound double-wide rack targets B300 memory squeeze with denser GPU packing and liquid cooling, announced at Computex 2026.
Apple M7 Ultra Chip Reportedly Supports 1.5TB Unified Memory
Apple's M7 Ultra chip reportedly supports 1.5TB unified memory, doubling the M3 Ultra and matching eight Nvidia B200 GPUs, but DRAM supply constraints threaten pricing.
Micron Backs Anthropic Series H With Multi-Year Memory Supply Deal
Micron invests in Anthropic's Series H and inks multi-year memory supply deal for HBM, DRAM, and SSDs. Critics flag circular arrangement as bubble risk.
Roundhill Memory ETF (DRAM) Surges 90% in 36 Days, Fastest ETF Ever
Roundhill Memory ETF surged 90% since April 2, hitting $6.5B assets in 36 days—fastest ETF ever—driven by AI demand for DRAM.
Cisco Reveals Scale-Across GPU Networking Needs 14x DCI Bandwidth
Cisco's chief architect detailed the massive bandwidth requirements for connecting AI clusters via 'scale-across' GPU networking, which needs 14x the capacity of traditional data center interconnects. This shift is creating a multi-billion dollar market for 800G coherent pluggables and deep-buffered switches.
Nvidia to Ship 1.19 Exabytes of HBM in 2026, Apple iPhone Memory 2x Larger
An analysis projects Nvidia will ship ~1.19 exabytes of HBM memory in 2026 for AI infrastructure, while Apple will ship ~2.4 exabytes of LPDDR5 for iPhones, putting AI's massive hardware scale in consumer market perspective.
Google's TurboQuant AI Research Report Sparks Sell-Off in Micron, Samsung, and SK Hynix Memory Stocks
Google's TurboQuant research blog publication triggered immediate market reaction, with shares of major memory manufacturers dropping 2-4% as investors anticipate AI-driven efficiency gains reducing future memory demand.
Google's TurboQuant Cuts LLM KV Cache Memory by 6x, Enables 3-Bit Storage Without Accuracy Loss
Google released TurboQuant, a novel two-stage quantization algorithm that compresses the KV cache in long-context LLMs. It reduces memory by 6x, achieves 3-bit storage with no accuracy drop, and speeds up attention scoring by up to 8x on H100 GPUs.
Qwen 3.5 397B-A17B MoE Model Runs on M3 Mac at 5.7 TPS with 5.5GB Active Memory via SSD Streaming
Developer Dan reportedly runs the 209GB Qwen 3.5 397B-A17B MoE model on an M3 Mac at ~5.7 tokens per second using only 5.5GB of active memory by quantizing and streaming weights from SSD.
Memory Market Squeeze Threatens iPhone Price Hikes as AI Demands Strain Supply
A global RAM shortage and price increases could force Apple to raise iPhone prices by up to $250, according to industry analysis. The tech giant is reportedly unwilling to absorb the cost, passing it directly to consumers amid surging memory demands from AI applications.
AI Agents Get a Memory Upgrade: New Framework Treats Multi-Agent Memory as Computer Architecture
A new paper proposes treating multi-agent memory systems as a computer architecture problem, introducing a three-layer hierarchy and identifying critical protocol gaps. This approach could significantly improve reasoning, skills, and tool usage in collaborative AI systems.
AI's Insatiable Appetite: Nvidia's Rubin Chip Demands 288GB Memory, Sparking Global Shortage Crisis
Nvidia's upcoming Rubin AI chip requires 288GB of RAM—800% more than top desktop computers—creating unprecedented memory demand. Massive purchases by OpenAI and Alphabet have depleted supply, driving DDR4 prices up 2352% and causing a global memory chip shortage.
AI Gold Rush Strains Apple Hardware: High-Memory Macs Sell Out as Local AI Agents Go Mainstream
A surge in demand for local AI development has created severe inventory shortages for high-memory Apple hardware. Mac Studio orders with 128GB or 512GB RAM face 6+ week delays as consumers buy up every available unit to run powerful AI agents like OpenClaw.
Memory Supply Squeeze Hits Non-AI Sectors as DRAM Prices Double
DRAM prices surged 93-98% QoQ in Q1 2026 as AI data centers consume fab capacity, nine industry groups warned the Trump administration on June 3, threatening supply for automotive, telecom, and medical devices.
China's 14nm AI Chip Hits 520 TFLOPS Via Architecture, Not Shrink
China's 14nm AI chip claims 520 TFLOPS and 6.4TB/s bandwidth via software-defined and 3D near-memory architecture, bypassing advanced node restrictions.
Dell Ships First Nvidia Vera Rubin NVL72 Rack to CoreWeave
Dell delivered the first Nvidia Vera Rubin NVL72 rack to CoreWeave. Each rack packs 72 Rubin GPUs, 36 Vera CPUs, 3.6 exaFLOPS FP4 inference, 75 TB memory, and 260 TB/s NVLink bandwidth.
AI Data Center HBM Shortage Intensifies as Samsung, SK Hynix, and Micron Struggle with Supply
AI data centers are aggressively stockpiling high-bandwidth memory (HBM), creating a supply crunch. Only three manufacturers—Samsung, SK Hynix, and Micron—can produce this critical component for AI servers.
Nvidia, SK Group Announce $500B AI Infrastructure Partnership
Nvidia and SK Group announced a $500B partnership for HBM4 memory supply and a 2 GW AI data center in South Korea, locking in SK Hynix as Nvidia's primary memory supplier through 2030.
Silicon Photonics Hits 300-mm Wafer Scale for AI Interconnects
Silicon photonics moves to 300-mm wafers for AI interconnects, cutting cost per Gbps by ~30% and addressing bandwidth bottlenecks in 100,000+ GPU clusters.
POSTECH 10+ Layer Chip Stack Hits 4× Density of 12-Hi HBM
POSTECH developed 10+ layer chip stacking with 4× HBM density, targeting AI inference memory bottlenecks.
Reverse-engineering Nvidia's cuda-checkpoint reveals 70x cold-start speedup path
Reverse-engineering Nvidia's cuda-checkpoint reveals PCIe bandwidth underutilization. The tool enables up to 70x faster cold starts for GPU servers, critical for AI inference scaling.
Nvidia RTX Pro 6000 Hits $13,250, Up 55% in a Year
Nvidia raised RTX Pro 6000 Blackwell to $13,250, up 55% in a year. Memory shortage and AI demand drive prices.
dMoE Cuts Active Experts from 69.5 to 14.6, Retains 99.11% Performance
dMoE reduces active experts from 69.5 to 14.6 in diffusion LLMs, retaining 99.11% performance while cutting memory 80% and speeding inference 1.66×.
Google's Virgo Network Links 134,000 TPU v8 Chips with 47 Pbps Fabric
Google unveiled its Virgo networking stack for TPU v8, capable of linking 134,000 chips in a single fabric with 47 petabits/sec of bi-sectional bandwidth. This represents a massive scale-up in interconnect technology for large-scale AI model training.