Hugging Face

Deploying models to Huggingface Inference Endpoints with Hugging Face :hugging_face:.

Hugging Face Inference Endpoints provides a secure production solution to easily deploy any transformers, sentence-transformers, and diffusers models on a dedicated and autoscaling infrastructure managed by Hugging Face. An Inference Endpoint is built from a model from the Hub.

This service provides dedicated and autoscaling infrastructure managed by Hugging Face, allowing you to deploy models without dealing with containers and GPUs.

When to use it?

You should use Hugging Face Model Deployer:

if you want to deploy Transformers, Sentence-Transformers, or Diffusion models on dedicated and secure infrastructure.

if you prefer a fully-managed production solution for inference without the need to handle containers and GPUs.

if your goal is to turn your models into production-ready APIs with minimal infrastructure or MLOps involvement

Cost-effectiveness is crucial, and you want to pay only for the raw compute resources you use.

Enterprise security is a priority, and you need to deploy models into secure offline endpoints accessible only via a direct connection to your Virtual Private Cloud (VPCs).

If you are looking for a more easy way to deploy your models locally, you can use the MLflow Model Deployer flavor.

How to deploy it?

The Hugging Face Model Deployer flavor is provided by the Hugging Face ZenML integration, so you need to install it on your local machine to be able to deploy your models. You can do this by running the following command:

zenml integration install huggingface -y

To register the Hugging Face model deployer with ZenML you need to run the following command:

zenml model-deployer register <MODEL_DEPLOYER_NAME> --flavor=huggingface --token=<YOUR_HF_TOKEN> --namespace=<YOUR_HF_NAMESPACE>

Here,

token parameter is the Hugging Face authentication token. It can be managed through Hugging Face settings.