oud_container_registry --connector cloud_connectorWith the components registered, everything is set up for the next steps.

For more information, you can always check the dedicated Skypilot orchestrator guide.

In order to launch a pipeline on Azure with the SkyPilot orchestrator, the first thing that you need to do is to install the Azure and SkyPilot integrations:

zenml integration install azure skypilot_azure -y

Before we start registering any components, there is another step that we have to execute. As we explained in the previous section, components such as orchestrators and container registries often require you to set up the right permissions. In ZenML, this process is simplified with the use of Service Connectors. For this example, we will need to use the Service Principal authentication feature of our Azure service connector:

zenml service-connector register cloud_connector --type azure --auth-method service-principal --tenant_id=<TENANT_ID> --client_id=<CLIENT_ID> --client_secret=<CLIENT_SECRET>

Once the service connector is set up, we can register a Skypilot orchestrator:

zenml orchestrator register skypilot_orchestrator -f vm_azure

zenml orchestrator connect skypilot_orchestrator --connect cloud_connector

The next step is to register an Azure container registry. Similar to the orchestrator, we will use our connector as we are setting up the container registry.

zenml container-registry register cloud_container_registry -f azure --uri=<REGISTRY_NAME>.azurecr.io

zenml container-registry connect cloud_container_registry --connector cloud_connector

With the components registered, everything is set up for the next steps.

For more information, you can always check the dedicated Skypilot orchestrator guide.

Having trouble with setting up infrastructure? Try reading the stack deployment section of the docs to gain more insight. If that still doesn't work, join the ZenML community and ask!

Running a pipeline on a cloud stack