s within a unique directory in the artifact store:Materializers are designed to be extensible and customizable, allowing you to define your own serialization and deserialization logic for specific data types or storage systems. By default, ZenML provides built-in materializers for common data types and uses cloudpickle to pickle objects where there is no default materializer. If you want direct control over how objects are serialized, you can easily create custom materializers by extending the BaseMaterializer class and implementing the required methods for your specific use case. Read more about materializers here.

ZenML provides a built-in CloudpickleMaterializer that can handle any object by saving it with cloudpickle. However, this is not production-ready because the resulting artifacts cannot be loaded when running with a different Python version. In such cases, you should consider building a custom Materializer to save your objects in a more robust and efficient format.

Moreover, using the CloudpickleMaterializer could allow users to upload of any kind of object. This could be exploited to upload a malicious file, which could execute arbitrary code on the vulnerable system.

When a pipeline runs, ZenML uses the appropriate materializers to save and load artifacts using the ZenML fileio system (built to work across multiple artifact stores). This not only simplifies the process of working with different data formats and storage systems but also enables artifact caching and lineage tracking. You can see an example of a default materializer (the numpy materializer) in action here.

PreviousHandle Data/Artifacts

NextReturn multiple outputs from a step

Last updated 19 days ago