Using Amazon SageMaker Pipelines, organizations can automate their machine learning (ML) workloads and distribute them over many AWS accounts and AWS Regions as part of their Machine Learning Operations (MLOps) strategy.

However, monitoring SageMaker Pipelines can become complex when they are distributed across many AWS environments. Developers and operations engineers must manually switch between multiple accounts and Regions to inspect SageMaker Pipeline executions, resulting in operational overhead.

Amazon SageMaker Studio provides monitoring for SageMaker Pipelines within a single account and Region. Organizations can use services like Amazon CloudWatch, AWS Lambda, Amazon DynamoDB, and Amazon EventBridge to build dashboards that track SageMaker Pipelines executions across multiple AWS environments, tailored to their observability requirements.

In this post, we present a solution designed to centralize the monitoring of SageMaker Pipelines across AWS accounts and Regions using Amazon CloudWatch custom dashboards. The accompanying GitHub repository provides a customizable AWS Cloud Development Kit (AWS CDK) example of the required infrastructure.

The solution is designed to provide detailed, near-real time visibility from a single interface into SageMaker Pipelines executions running in many Regions and many accounts to help streamline daily operations. In the next section, we examine in detail the architecture of the solution.