e pipeline will not start correctly.

How it worksThe HyperAI orchestrator works with Docker Compose, which can be used to construct machine learning pipelines. Under the hood, it creates a Docker Compose file which it then deploys and executes on the configured HyperAI instance. For each ZenML pipeline step, it creates a service in this file. It uses the service_completed_successfully condition to ensure that pipeline steps will only run if their connected upstream steps have successfully finished.

If configured for it, the HyperAI orchestrator will connect the HyperAI instance to the stack's container registry to ensure a smooth transfer of Docker images.

Scheduled pipelines

Scheduled pipelines are supported by the HyperAI orchestrator. Currently, the HyperAI orchestrator supports the following inputs to Schedule:

Cron expressions via cron_expression. When pipeline runs are scheduled, they are added as a crontab entry on the HyperAI instance. Use this when you want pipelines to run in intervals. Using cron expressions assumes that crontab is available on your instance and that its daemon is running.

Scheduled runs via run_once_start_time. When pipeline runs are scheduled this way, they are added as an at entry on the HyperAI instance. Use this when you want pipelines to run just once and at a specified time. This assumes that at is available on your instance.

How to deploy it

To use the HyperAI orchestrator, you must configure a HyperAI Service Connector in ZenML and link it to the HyperAI orchestrator component. The service connector contains credentials with which ZenML connects to the HyperAI instance.

Additionally, the HyperAI orchestrator must be used in a stack that contains a container registry and an image builder.

How to use it

To use the HyperAI orchestrator, we must configure a HyperAI Service Connector first using one of its supported authentication methods. For example, for authentication with an RSA-based key, create the service connector as follows: