Grace Blackwell needs a new scheduler: 27% more usable GPUs from the same racks
NVL72 puts all 72 GPUs in a rack on one NVLink fabric, but only rack-local jobs get the speedup. How SkyPilot keeps multi-node training and inference on NVLink with topology-aware placement, in-place recovery from GPU failures, and automatic defragmentation.
NVL72 systems offer 72 GPUs in a rack with NVLink for higher bandwidth, but workload placement is key for performance.
Using NVLink over InfiniBand improves performance by 43% in distributed training.
NVL72 allows GPU communication across nodes with a unified fabric.
Workload placement affects performance, as InfiniBand can still bottleneck multi-rack jobs.
Summarised automatically by AI from the original article by Hacker News. AI can make mistakes, so check the original for details.
Not all high-speed interconnects are created equal, and using the wrong one can be quite costly: running the same distributed training job over InfiniBand rather than NVLink can degrade performance by 43%.
Historically, NVLink was limited to communication within a single node, meaning that multi-node AI workloads were forced to communicate over InfiniBand or a cloud equivalent (EFA on AWS, GPUDirect-TCPX on GCP). NVIDIA’s NVL72 systems remove this limitation by connecting all 72 GPUs in a rack with a unified NVLink fabric, so GPUs on different nodes can exchange data with an order of magnitude more bandwidth.
However, maximizing the benefits of NVL72 systems is highly dependent on workload placement: a multi-node job spread across racks is still bottlenecked by InfiniBand. Preserving rack-locality gets harder as hardware failures take nodes offline and jobs finishing at different times scatter free capacity across racks.
Read the full story on Hacker NewsThat's the opening of the story. The full piece is published by Hacker News.
LYNX SHARE — lightweight open-source DAW in C++20/JUCE with the FL Studio workflow: Channel Rack, Piano Roll, Playlist, Mixer and .flp import - Ka1-S/LYNX-SHARE
The US power sector could release 42 percent more planet-heating carbon dioxide pollution by 2030 than it did in 2025 as it builds out more fossil fuel infrastructure to meet…
