. If not set, the cluster will not be autostopped.down: Tear down the cluster after all jobs finish (successfully or abnormally). If idle_minutes_to_autostop is also set, the cluster will be torn down after the specified idle time. Note that if errors occur during provisioning/data syncing/setting up, the cluster will not be torn down for debugging purposes.

stream_logs: If True, show the logs in the terminal as they are generated while the cluster is running.

docker_run_args: Additional arguments to pass to the docker run command. For example, ['--gpus=all'] to use all GPUs available on the VM.

The following code snippets show how to configure the orchestrator settings for each cloud provider:

Code Example:

from zenml.integrations.skypilot_aws.flavors.skypilot_orchestrator_aws_vm_flavor import SkypilotAWSOrchestratorSettings

skypilot_settings = SkypilotAWSOrchestratorSettings(

cpus="2",

memory="16",

accelerators="V100:2",

accelerator_args={"tpu_vm": True, "runtime_version": "tpu-vm-base"},

use_spot=True,

spot_recovery="recovery_strategy",

region="us-west-1",

zone="us-west1-a",

image_id="ami-1234567890abcdef0",

disk_size=100,

disk_tier="high",

cluster_name="my_cluster",

retry_until_up=True,

idle_minutes_to_autostop=60,

down=True,

stream_logs=True

docker_run_args=["--gpus=all"]

@pipeline(

settings={

"orchestrator.vm_aws": skypilot_settings

Code Example:

from zenml.integrations.skypilot_gcp.flavors.skypilot_orchestrator_gcp_vm_flavor import SkypilotGCPOrchestratorSettings

skypilot_settings = SkypilotGCPOrchestratorSettings(

cpus="2",

memory="16",

accelerators="V100:2",

accelerator_args={"tpu_vm": True, "runtime_version": "tpu-vm-base"},

use_spot=True,

spot_recovery="recovery_strategy",

region="us-west1",

zone="us-west1-a",

image_id="ubuntu-pro-2004-focal-v20231101",

disk_size=100,

disk_tier="high",

cluster_name="my_cluster",

retry_until_up=True,

idle_minutes_to_autostop=60,

down=True,

stream_logs=True

@pipeline(

settings={

"orchestrator.vm_gcp": skypilot_settings