BentoML

Deploying your models locally with BentoML.

BentoML is an open-source framework for machine learning model serving. it can be used to deploy models locally, in a cloud environment, or in a Kubernetes environment.

The BentoML Model Deployer is one of the available flavors of the Model Deployer stack component. Provided with the BentoML integration it can be used to deploy and manage BentoML models or Bento on a local running HTTP server.

The BentoML Model Deployer can be used to deploy models for local development and production use cases. While the integration mainly works in a local environment where pipelines are run, the used Bento can be exported and containerized, and deployed in a remote environment. Within the BentoML ecosystem, Yatai and bentoctl are the tools responsible for deploying the Bentos into the Kubernetes cluster and Cloud Platforms. Full support for these advanced tools is in progress and will be available soon.

When to use it?

You should use the BentoML Model Deployer to:

Standardize the way you deploy your models to production within your organization.

if you are looking to deploy your models in a simple way, while you are still able to transform your model into a production-ready solution when that time comes.

If you are looking to deploy your models with other Kubernetes-based solutions, you can take a look at one of the other Model Deployer Flavors available in ZenML.

BentoML also allows you to deploy your models in a more complex production-grade setting. Bentoctl is one of the tools that can help you get there. Bentoctl takes your built Bento from a ZenML pipeline and deploys it with bentoctl into a cloud environment such as AWS Lambda, AWS SageMaker, Google Cloud Functions, Google Cloud AI Platform, or Azure Functions. Read more about this in the From Local to Cloud with bentoctl section.