 configuration documentation for more information.Custom materializers

Configuring a step/pipeline to use a custom materializer

Defining which step uses what materializer

ZenML automatically detects if your materializer is imported in your source code and registers them for the corresponding data type (defined in ASSOCIATED_TYPES). Therefore, just having a custom materializer definition in your code is enough to enable the respective data type to be used in your pipelines.

However, it is best practice to explicitly define which materializer to use for a specific step and not rely on the ASSOCIATED_TYPES to make that connection:

class MyObj:

...

class MyMaterializer(BaseMaterializer):

"""Materializer to read data to and from MyObj."""

ASSOCIATED_TYPES = (MyObj)

ASSOCIATED_ARTIFACT_TYPE = ArtifactType.DATA

# Read below to learn how to implement this materializer

# You can define it at the decorator level

@step(output_materializers=MyMaterializer)

def my_first_step() -> MyObj:

return 1

# No need to explicitly specify materializer here:

# it is coupled with Artifact Version generated by

# `my_first_step` already.

def my_second_step(a: MyObj):

print(a)

# or you can use the `configure()` method of the step. E.g.:

my_first_step.configure(output_materializers=MyMaterializer)

When there are multiple outputs, a dictionary of type {<OUTPUT_NAME>: <MATERIALIZER_CLASS>} can be supplied to the decorator or the .configure(...) method:

class MyObj1:

...

class MyObj2:

...

class MyMaterializer1(BaseMaterializer):

"""Materializer to read data to and from MyObj1."""

ASSOCIATED_TYPES = (MyObj1)

ASSOCIATED_ARTIFACT_TYPE = ArtifactType.DATA

class MyMaterializer2(BaseMaterializer):

"""Materializer to read data to and from MyObj2."""

ASSOCIATED_TYPES = (MyObj2)

ASSOCIATED_ARTIFACT_TYPE = ArtifactType.DATA

# This is where we connect the objects to the materializer

@step(output_materializers={"1": MyMaterializer1, "2": MyMaterializer2})