idle clusters, preventing unnecessary cloud costs.You can configure the SkyPilot VM Orchestrator to use a specific VM type, and resources for each step of your pipeline can be configured individually. Read more about how to configure step-specific resources here.

The SkyPilot VM Orchestrator does not currently support the ability to schedule pipelines runs

All ZenML pipeline runs are executed using Docker containers within the VMs provisioned by the orchestrator. For that reason, you may need to configure your pipeline settings with docker_run_args=["--gpus=all"] to enable GPU support in the Docker container.

How to deploy it

You don't need to do anything special to deploy the SkyPilot VM Orchestrator. As the SkyPilot integration itself takes care of provisioning VMs, you can simply use the orchestrator as you would any other ZenML orchestrator. However, you will need to ensure that you have the appropriate permissions to provision VMs on your cloud provider of choice and to configure your SkyPilot orchestrator accordingly using the service connectors feature.

The SkyPilot VM Orchestrator currently only supports the AWS, GCP, and Azure cloud platforms.

How to use it

To use the SkyPilot VM Orchestrator, you need:

One of the SkyPilot integrations installed. You can install the SkyPilot integration for your cloud provider of choice using the following command:Copy  # For AWS
  pip install "zenml[connectors-aws]"
  zenml integration install aws skypilot_aws

# for GCP
  pip install "zenml[connectors-gcp]"
  zenml integration install gcp skypilot_gcp # for GCP

# for Azure
  pip install "zenml[connectors-azure]"
  zenml integration install azure skypilot_azure # for Azure

Docker installed and running.

A remote artifact store as part of your stack.

A remote container registry as part of your stack.

A remote ZenML deployment.

The appropriate permissions to provision VMs on your cloud provider of choice.

A service connector configured to authenticate with your cloud provider of choice.