Cloud providers are interconnecting multiple datacenters for AI mega clusters of hundreds of thousands to millions of accelerators, a new paradigm called 'scale-across'. According to @SemiAnalysis_, this shift was forced by challenges in running chip interconnect at unprecedented scale.
Key facts
- Scale-across interconnects multiple datacenters for AI clusters.
- Clusters now reach hundreds of thousands to millions of accelerators.
- Novel challenges forced cloud providers to adopt this approach.
- Distinct from traditional scale-up or scale-out architectures.
The rapid growth of AI training and inference clusters—now reaching hundreds of thousands to millions of accelerators—has pushed cloud providers past the physical limits of a single datacenter. To continue scaling compute, they are now deploying chip interconnects that span multiple datacenters, a technique the analyst firm SemiAnalysis dubs 'scale-across.'
The approach reflects a structural shift: while previous generations scaled within a single facility (scale-up) or across racks (scale-out), scale-across involves linking entire datacenters into a unified compute fabric. This requires novel networking hardware, cabling, and scheduling software to maintain low-latency communication across potentially hundreds of kilometers.
SemiAnalysis notes that this trend is already visible in the largest AI infrastructure projects, including those from hyperscalers like Microsoft, Google, and Amazon. The scale-across architecture is distinct from traditional wide-area networking (WAN) because it aims to preserve the tight coupling needed for distributed training, where synchronization overhead can cripple throughput.
Key challenges include maintaining consistent bandwidth and latency across datacenter boundaries, managing power and cooling at multi-datacenter scale, and ensuring reliability when a single training run may span thousands of accelerators spread across sites. The move to scale-across also has implications for network equipment vendors, who must design switches and optical interconnects optimized for this new use case.
While the tweet does not specify which providers have deployed scale-across or the exact technologies used, the concept aligns with known industry trends: Microsoft's multi-region training clusters, Google's TPU pod expansions, and Amazon's Elastic Fabric Adapter (EFA) upgrades. The shift is likely to accelerate as clusters approach the million-accelerator threshold.
What to watch
Watch for hyperscaler announcements of multi-datacenter training runs, such as Microsoft's next-generation MAIA accelerator deployment or Google's TPU v5p cluster expansions. Also monitor networking vendor earnings calls for mentions of 'scale-across' or 'datacenter interconnect for AI' as a growth driver.
