Linking model binaries/data to a Model

Artifacts generated during pipeline runs can be linked to models in ZenML. This connecting of artifacts provides lineage tracking and transparency into what data and models are used during training, evaluation, and inference.

There are a few ways to link artifacts:

Configuring the Model at a pipeline level

The easiest way is to configure the model parameter on the @pipeline decorator or @step decorator:

from zenml import Model, pipeline

model = Model(

name="my_model",

version="1.0.0"

@pipeline(model=model)

def my_pipeline():

...

This will automatically link all artifacts from this pipeline run to the specified model configuration.

Controlling artifact types and linkage

A ZenML model supports linking three types of artifacts:

Data artifacts: These are the default artifacts. If nothing is specified, all artifacts are grouped under this category.

Model artifacts: If there is a physical model artifact like a .pkl file or a model neural network weights file, it should be grouped in this category.

Deployment artifacts: These artifacts are to do with artifacts related to the endpoints and deployments of the models.

You can also explicitly specify the linkage on a per-artifact basis by passing a special configuration to the Annotated output:

from zenml import step, ArtifactConfig

from typing import Tuple

from typing_extensions import Annotated

import pandas as pd

@step

def svc_trainer(

X_train: pd.DataFrame,

y_train: pd.Series,

gamma: float = 0.001,

) -> Tuple[

# This third argument marks this as a Model Artifact

Annotated[ClassifierMixin, ArtifactConfig("trained_model", is_model_artifact=True)],

# This third argument marks this as a Data Artifact

Annotated[str, ArtifactConfig("deployment_uri", is_deployment_artifact=True)],

]:

...