Fetching pipelines

Inspecting a finished pipeline run and its outputs.

Once a pipeline run has been completed, we can access the corresponding information in code, which enables the following:

Loading artifacts like models or datasets saved by previous runs

Accessing metadata or configurations of previous runs

Programmatically inspecting the lineage of pipeline runs and their artifacts

The hierarchy of pipelines, runs, steps, and artifacts is as follows:

As you can see from the diagram, there are many layers of 1-to-N relationships.

Let us investigate how to traverse this hierarchy level by level:

Pipelines

Get a pipeline via the client

After you have run a pipeline at least once, you can also fetch the pipeline via the Client.get_pipeline() method.

from zenml.client import Client

pipeline_model = Client().get_pipeline("first_pipeline")

Check out the ZenML Client Documentation for more information on the Client class and its purpose.

Discover and list all pipelines

If you're not sure which pipeline you need to fetch, you can find a list of all registered pipelines in the ZenML dashboard, or list them programmatically either via the Client or the CLI.

You can use the Client.list_pipelines() method to get a list of all pipelines registered in ZenML:

from zenml.client import Client

pipelines = Client().list_pipelines()

Alternatively, you can also list pipelines with the following CLI command:

zenml pipeline list

Runs

Each pipeline can be executed many times, resulting in several Runs.

Get all runs of a pipeline

You can get a list of all runs of a pipeline using the runs property of the pipeline:

runs = pipeline_model.runs

The result will be a list of the most recent runs of this pipeline, ordered from newest to oldest.

Alternatively, you can also use the pipeline_model.get_runs() method which allows you to specify detailed parameters for filtering or pagination. See the ZenML SDK Docs for more information.

Get the last run of a pipeline