Amazon SageMaker HyperPod now supports automatic Slurm topology management

Amazon Web Services has enhanced its SageMaker HyperPod service with automatic Slurm topology management, a feature that optimizes network configurations for distributed machine learning training workloads. The new capability automatically selects and maintains optimal network topology configurations based on GPU instance types within clusters, eliminating the need for manual topology file updates or Slurm reconfiguration. The system dynamically adapts as clusters scale up, scale down, or replace nodes, ensuring job placement remains optimized throughout the cluster lifecycle. The service supports different topology models depending on instance characteristics, including tree topology for hierarchical interconnect instances like ml.p5.48xlarge and ml.p5e.48xlarge, and block topology for uniform high-bandwidth connectivity instances such as ml.p6e-gb200.NVL72. For mixed instance type clusters, HyperPod automatically selects compatible topology configurations that work across all nodes. The feature leverages the fact that network topology directly impacts distributed training performance by ensuring GPU-to-GPU communication is faster and NCCL collective operations are more efficient when jobs run on topologically close nodes. The topology-aware scheduling feature is enabled by default on new SageMaker HyperPod Slurm clusters with supported GPU instance types and requires no additional configuration. The enhancement is available across all AWS regions where SageMaker HyperPod is currently supported.