Amazon SageMaker HyperPod Slurm clusters now support specifying minimum capacity requirements with continuous provisioning

Amazon Web Services has enhanced its SageMaker HyperPod service with minimum capacity requirements (MinCount) for Slurm orchestration clusters that use continuous provisioning. The new feature allows organizations to specify the minimum number of instances that must be successfully provisioned before an instance group transitions to InService status, addressing a key limitation where distributed training workloads could receive insufficient partial capacity to operate effectively. The enhancement specifically targets distributed training frameworks like PyTorch FSDP, Megatron-LM, and NVIDIA NeMo, which typically require a fixed number of participating nodes to function properly. Previously, HyperPod's continuous provisioning would make clusters available as soon as any capacity became available, potentially leaving training jobs without enough resources to start efficiently. With MinCount, the instance group remains in Creating or Updating status until the specified threshold is met, after which HyperPod continues provisioning additional instances until the full target count is reached. The feature includes a three-hour timeout mechanism that automatically rolls back the instance group to its last known good state if the minimum capacity requirements cannot be satisfied. Organizations can configure MinCount through the CreateCluster or UpdateCluster API requests, providing greater control over cluster availability and helping teams meet service level agreements or cost-efficiency targets before committing to large-scale training runs.