Run on AWS

A simple guide to create an AWS stack to run your ZenML pipelines

This page aims to quickly set up a minimal production stack on AWS. With just a few simple steps, you will set up an IAM role with specifically-scoped permissions that ZenML can use to authenticate with the relevant AWS resources.

1) Set up credentials and local environment

To follow this guide, you need:

An active AWS account with necessary permissions for AWS S3, SageMaker, ECR, and ECS.

ZenML installed

AWS CLI installed and configured with your AWS credentials. You can follow the instructions here.

Once ready, navigate to the AWS console:

Choose an AWS region In the AWS console, choose the region where you want to deploy your ZenML stack resources. Make note of the region name (e.g., us-east-1, eu-west-2, etc.) as you will need it in subsequent steps.

Create an IAM role

For this, you'll need to find out your AWS account ID. You can find this by running:

aws sts get-caller-identity --query Account --output text

This will output your AWS account ID. Make a note of this as you will need it in the next steps. (If you're doing anything more esoteric with your AWS account and IAM roles, this might not work for you. The account ID here that we're trying to get is the root account ID that you use to log in to the AWS console.)

Then create a file named assume-role-policy.json with the following content:

"Version": "2012-10-17",

"Statement": [

"Effect": "Allow",

"Principal": {

"AWS": "arn:aws:iam::<YOUR_ACCOUNT_ID>:root",

"Service": "sagemaker.amazonaws.com"

},

"Action": "sts:AssumeRole"

Make sure to replace the placeholder <YOUR_ACCOUNT_ID> with your actual AWS account ID that we found earlier.

Now create a new IAM role that ZenML will use to access AWS resources. We'll use zenml-role as a role name in this example, but you can feel free to choose something else if you prefer. Run the following command to create the role: