AWS re:Invent 2023 - Train and tune state-of-the-art ML models on Amazon SageMaker (AIM335)

Gal Oshri introduces himself and colleagues, presenting on training and tuning state-of-the-art ML models on AWS SageMaker. The audience is involved with a count of how many are currently training ML models, especially on larger scales. The talk outlines the agenda, including challenges in large-scale model training, demonstrations, and research use cases.

Machine learning is showcased as a versatile tool across various applications like recommendations and autonomous driving, highlighting recent advances in image generation from ML models. Gal discusses how algorithmic improvements, particularly the transformer architecture, and increased availability of data and compute power have contributed to the enhancement of model outputs over recent years.

Challenges in training large-scale ML models are detailed, including the need for efficient hardware, proper orchestration, dataset management, scaling infrastructure, and cost management. Gal emphasizes the importance of optimizing both financial costs and team resources, advocating for effective use of SageMaker in addressing these challenges.

Amazon SageMaker is presented as a solution to the aforementioned challenges. A high-level overview of SageMaker's operation is provided, detailing how it streamlines the training process through a structured API that manages compute resources, health checks, and data integration, ensuring efficient model training and cost-effectiveness.

SageMaker’s capabilities for loading data, using built-in algorithms, and providing distributed training options were discussed. Features like logging, checkpoint synchronization, and instance management further highlight how SageMaker can facilitate efficient training processes while ensuring resiliency against failures during training jobs.

Training model efficiency and performance tracking using SageMaker tools like the profiler were discussed. The profiler helps optimize GPU usage and performance issues, which is critical for minimizing the overall training costs and times through better hardware utilization.

Emily introduces the concept of smart sifting—a new feature in SageMaker aimed at improving training efficiency by refining data during training, thus reducing overall training time and costs up to 35% without negatively impacting model accuracy. She presents how to implement this feature simply using SageMaker's framework.

The introduction of Amazon SageMaker HyperPod is announced for managing large-scale training instances while allowing easier access and control over the training environment, providing managed benefits without sacrificing performance with features such as resilience and reduced setup time.

The discussion shifts to fine-tuning models, emphasizing how it can be executed on proprietary models without exposing sensitive data. New security enhancements in SageMaker facilitate this process while maintaining the model’s confidentiality, showcasing an efficient end-to-end workflow.