Amazon SageMaker HyperPod now supports data capture for inference workloads

Amazon Web Services has added data capture capabilities to SageMaker HyperPod, allowing organizations to systematically record inference request and response payloads from their machine learning models. The feature addresses a critical operational gap for enterprises deploying generative AI and ML models, enabling them to monitor model performance, meet compliance requirements, debug production issues, and build datasets for model fine-tuning without building custom logging infrastructure. The data capture functionality offers flexible deployment options, allowing customers to record inference traffic at the SageMaker endpoint, load balancer, or model pod level depending on their visibility needs. AWS designed the feature to operate asynchronously and never block inference requests, ensuring production availability remains intact. Captured data is automatically delivered to Amazon S3 buckets with configurable sampling rates and encryption using customer-managed AWS KMS keys, giving organizations control over cost and data protection. The capability is now available across all AWS regions that support SageMaker HyperPod clusters using the EKS orchestrator. Organizations can enable data capture through the HyperPod Inference Operator or SageMaker JumpStart when deploying their models, providing immediate operational visibility that was previously only available through expensive custom solutions.