ter an S3 Artifact Store and add it to your stack:zenml integration install s3 -y

The only configuration parameter mandatory for registering an S3 Artifact Store is the root path URI, which needs to point to an S3 bucket and take the form s3://bucket-name. Please read the documentation relevant to the S3 service that you are using on how to create an S3 bucket. For example, the AWS S3 documentation is available here.

With the URI to your S3 bucket known, registering an S3 Artifact Store and using it in a stack can be done as follows:

# Register the S3 artifact-store

zenml artifact-store register s3_store -f s3 --path=s3://bucket-name

# Register and set a stack with the new artifact store

zenml stack register custom_stack -a s3_store ... --set

Depending on your use case, however, you may also need to provide additional configuration parameters pertaining to authentication or pass advanced configuration parameters to match your S3-compatible service or deployment scenario.

Infrastructure Deployment

An S3 Artifact Store can be deployed directly from the ZenML CLI:

zenml artifact-store deploy s3-artifact-store --flavor=s3 --provider=aws ...

You can pass other configurations specific to the stack components as key-value arguments. If you don't provide a name, a random one is generated for you. For more information about how to work use the CLI for this, please refer to the dedicated documentation section.

Authentication Methods

Integrating and using an S3-compatible Artifact Store in your pipelines is not possible without employing some form of authentication. If you're looking for a quick way to get started locally, you can use the Implicit Authentication method. However, the recommended way to authenticate to the AWS cloud platform is through an AWS Service Connector. This is particularly useful if you are configuring ZenML stacks that combine the S3 Artifact Store with other remote stack components also running in AWS.