gmax(prediction.numpy())

return classes[maxindex]The custom predict function should get the model and the input data as arguments and return the model predictions. ZenML will automatically take care of loading the model into memory and starting the seldon-core-microservice that will be responsible for serving the model and running the predict function.

After defining your custom predict function in code, you can use the seldon_custom_model_deployer_step to automatically build your function into a Docker image and deploy it as a model server by setting the predict_function argument to the path of your custom_predict function:

from zenml.integrations.seldon.steps import seldon_custom_model_deployer_step

from zenml.integrations.seldon.services import SeldonDeploymentConfig

from zenml import pipeline

@pipeline

def seldon_deployment_pipeline():

model = ...

seldon_custom_model_deployer_step(

model=model,

predict_function="<PATH.TO.custom_predict>",  # TODO: path to custom code

service_config=SeldonDeploymentConfig(

model_name="<MODEL_NAME>",  # TODO: name of the deployed model

replicas=1,

implementation="custom",

resources=SeldonResourceRequirements(

limits={"cpu": "200m", "memory": "250Mi"}

),

serviceAccountName="kubernetes-service-account",

),

Advanced Custom Code Deployment with Seldon Core Integration

Before creating your custom model class, you should take a look at the custom Python model section of the Seldon Core documentation.

The built-in Seldon Core custom deployment step is a good starting point for deploying your custom models. However, if you want to deploy more than the trained model, you can create your own custom class and a custom step to achieve this.

See the ZenML custom Seldon model class as a reference.

PreviousMLflow

NextBentoML

Last updated 15 days ago