 to interact with the remote model server include:deploy_model - Deploys a model to the serving environment and returns a Service object that represents the deployed model server. find_model_server - Finds and returns a list of Service objects that represent model servers that have been deployed to the serving environment, the services are stored in the DB and can be used as a reference to know what and where the model is deployed. stop_model_server - Stops a model server that is currently running in the serving environment. start_model_server - Starts a model server that has been stopped in the serving environment. delete_model_server - Deletes a model server from the serving environment and from the DB.

ZenML uses the Service object to represent a model server that has been deployed to a serving environment. The Service object is saved in the DB and can be used as a reference to know what and where the model is deployed. The Service object consists of 2 main attributes, the config and the status. The config attribute holds all the deployment configuration attributes required to create a new deployment, while the status attribute holds the operational status of the deployment, such as the last error message, the prediction URL, and the deployment status.

from zenml.integrations.huggingface.model_deployers import HuggingFaceModelDeployer

model_deployer = HuggingFaceModelDeployer.get_active_model_deployer()

services = model_deployer.find_model_server(

pipeline_name="LLM_pipeline",

pipeline_step_name="huggingface_model_deployer_step",

model_name="LLAMA-7B",

if services:

if services[0].is_running:

print(

f"Model server {services[0].config['model_name']} is running at {services[0].status['prediction_url']}"

else:

print(f"Model server {services[0].config['model_name']} is not running")

model_deployer.start_model_server(services[0])

else:

print("No model server found")

service = model_deployer.deploy_model(

pipeline_name="LLM_pipeline",

pipeline_step_name="huggingface_model_deployer_step",