GPU cluster
Hundreds to tens of thousands of GPUs interconnected for large-scale AI training. Hyperscaler scale: Meta 24k H100, xAI 100k H100 Colossus, Microsoft+OpenAI multi-billion-dollar build. NVLink + InfiniBand fabric.
Topology: 8 GPUs per "node" via NVLink (900 GB/s between GPUs), nodes together via InfiniBand (400 Gbps NDR per port). 1000 H100 = ~$30M hardware + $5-10M/year electricity. Classic training: 16k GPUs × 30 days for a frontier LLM run. Bottleneck shift 2024: electricity + datacenter construction more important than chip supply. Modular build: Lambda Labs, CoreWeave, Crusoe Energy provide "GPU-as-a-service" to AI labs. xAI's Memphis cluster (2024-2025): 100k → 200k → 1M GPU roadmap, controversial due to grid impact.