_settings})

def my_pipeline() -> None:

my_step()# Or configure the pipelines options

my_pipeline = my_pipeline.with_options(

settings={"docker": docker_settings}

Configuring them on a step gives you more fine-grained control and enables you to build separate specialized Docker images for different steps of your pipelines:

docker_settings = DockerSettings()

# Either add it to the decorator

@step(settings={"docker": docker_settings})

def my_step() -> None:

pass

# Or configure the step options

my_step = my_step.with_options(

settings={"docker": docker_settings}

Using a YAML configuration file as described here:

settings:

docker:

...

steps:

step_name:

settings:

docker:

...

Check out this page for more information on the hierarchy and precedence of the various ways in which you can supply the settings.

Using a custom parent image

By default, ZenML performs all the steps described above on top of the official ZenML image for the Python and ZenML version in the active Python environment. To have more control over the entire environment used to execute your pipelines, you can either specify a custom pre-built parent image or a Dockerfile that ZenML uses to build a parent image for you.

If you're going to use a custom parent image (either pre-built or by specifying a Dockerfile), you need to make sure that it has Python, pip, and ZenML installed for it to work. If you need a starting point, you can take a look at the Dockerfile that ZenML uses here.

Using a pre-built parent image

To use a static parent image (e.g., with internal dependencies installed) that doesn't need to be rebuilt on every pipeline run, specify it in the Docker settings for your pipeline:

docker_settings = DockerSettings(parent_image="my_registry.io/image_name:tag")

@pipeline(settings={"docker": docker_settings})

def my_pipeline(...):

...

To use this image directly to run your steps without including any code or installing any requirements on top of it, skip the Docker builds by specifying it in the Docker settings: