Run on GCP

A simple guide to quickly set up a minimal stack on GCP.

The GCP integration currently only works for Python versions <3.11. The ZenML team is aware of this dependency clash/issue and is working on a fix. For now, please use Python <3.11 together with the GCP integration.

This page aims to quickly set up a minimal production stack on GCP. With just a few simple steps you will set up a service account with specifically-scoped permissions that ZenML can use to authenticate with the relevant GCP resources.

While this guide focuses on Google Cloud, we are seeking contributors to create a similar guide for other cloud providers. If you are interested, please create a pull request over on GitHub.

1) Choose a GCP project

In the Google Cloud console, on the project selector page, select or create a Google Cloud project. Make sure a billing account is attached to this project to allow the use of some APIs.

This is how you would do it from the CLI if this is preferred.

gcloud projects create <PROJECT_ID> --billing-project=<BILLING_PROJECT>

If you don't plan to keep the resources that you create in this procedure, create a new project. After you finish these steps, you can delete the project, thereby removing all resources associated with the project.

2) Enable GCloud APIs

The following APIs will need to be enabled within your chosen GCP project.

Cloud Functions API # For the vertex orchestrator

Cloud Run Admin API # For the vertex orchestrator

Cloud Build API # For the container registry

Artifact Registry API # For the container registry

Cloud Logging API # Generally needed

3) Create a dedicated service account

The service account should have these following roles.

AI Platform Service Agent

Storage Object Admin

These roles give permissions for full CRUD on storage objects and full permissions for compute within VertexAI.

4) Create a JSON Key for your service account