running for the Inference Endpoint. Defaults to 0.max_replica: (Optional) The maximum number of replicas (instances) to scale to for the Inference Endpoint. Defaults to 1.

revision: (Optional) The specific model revision to deploy on the Inference Endpoint for the Hugging Face repository .

task: Select a supported Machine Learning Task. (e.g. "text-classification", "text-generation")

custom_image: (Optional) A custom Docker image to use for the Inference Endpoint.

namespace: The namespace where the Inference Endpoint will be created. The same namespace can be passed used while registering the Hugging Face model deployer.

endpoint_type: (Optional) The type of the Inference Endpoint, which can be "protected", "public" (default) or "private".

For more information and a full list of configurable attributes of the Hugging Face Model Deployer, check out the SDK Docs and Hugging Face endpoint code.

Run inference on a provisioned inference endpoint

The following code example shows how to run inference against a provisioned inference endpoint:

from typing import Annotated

from zenml import step, pipeline

from zenml.integrations.huggingface.model_deployers import HuggingFaceModelDeployer

from zenml.integrations.huggingface.services import HuggingFaceDeploymentService

# Load a prediction service deployed in another pipeline

@step(enable_cache=False)

def prediction_service_loader(

pipeline_name: str,

pipeline_step_name: str,

running: bool = True,

model_name: str = "default",

) -> HuggingFaceDeploymentService:

"""Get the prediction service started by the deployment pipeline.

Args:

pipeline_name: name of the pipeline that deployed the MLflow prediction

server

step_name: the name of the step that deployed the MLflow prediction

server

running: when this flag is set, the step only returns a running service

model_name: the name of the model that is deployed

"""

# get the Hugging Face model deployer stack component

model_deployer = HuggingFaceModelDeployer.get_active_model_deployer()