hestrator register skypilot_orchestrator -f vm_awszenml orchestrator connect skypilot_orchestrator --connector cloud_connector

The next step is to register an AWS container registry. Similar to the orchestrator, we will use our connector as we are setting up the container registry:

zenml container-registry register cloud_container_registry -f aws --uri=<ACCOUNT_ID>.dkr.ecr.<REGION>.amazonaws.com

zenml container-registry connect cloud_container_registry --connector cloud_connector

With the components registered, everything is set up for the next steps.

For more information, you can always check the dedicated Skypilot orchestrator guide.

In order to launch a pipeline on GCP with the SkyPilot orchestrator, the first thing that you need to do is to install the GCP and Skypilot integrations:

zenml integration install gcp skypilot_gcp -y

Before we start registering any components, there is another step that we have to execute. As we explained in the previous section, components such as orchestrators and container registries often require you to set up the right permissions. In ZenML, this process is simplified with the use of Service Connectors. For this example, we need to use the Service Account authentication feature of our GCP service connector:

zenml service-connector register cloud_connector --type gcp --auth-method service-account --service_account_json=@<PATH_TO_SERVICE_ACCOUNT_JSON> --project_id=<PROJECT_ID> --generate_temporary_tokens=False

Once the service connector is set up, we can register a Skypilot orchestrator:

zenml orchestrator register skypilot_orchestrator -f vm_gcp

zenml orchestrator connect skypilot_orchestrator --connect cloud_connector

The next step is to register a GCP container registry. Similar to the orchestrator, we will use our connector as we are setting up the container registry:

zenml container-registry register cloud_container_registry -f gcp --uri=gcr.io/<PROJECT_ID>

zenml container-registry connect cloud_container_registry --connector cloud_connector