Can't deploy my custom endpoint with my Concrete ML package
Hello everyone at HF!I am trying to use Custom Inference Handler, to be able to use our Concrete ML (GitHub - zama-ai/concrete-ml: Concrete ML: Privacy Preserving ML framework built on top of Concrete, with bindings to traditional ML frameworks.) models directly in HF.I have followed theCreate custom Inference Handlertutorial. I have created zama-fhe/cml_logreg_test/ (it’s a private repo). In their, I have arequirements.txtfile:$ cat requirements.txt 

concrete-ml
numpyThere, I have also myhandler.pyfile:$ head handler.py 

from typing import Dict, List, Any
import numpy as np
from concrete.ml.deployment import FHEModelServer


class EndpointHandler:
    def __init__(self, path):

        # For server
        self.fhemodel_server = FHEModelServer(path)I am able to make it work locally. Now, I try to deploy, a bit like at the end of your tuto. Now, my issue is that, when I try to create the endpoint, it seems it can’t findconcretepackage, while this latter is in therequirements.txtfile. I am surprised because in your tuto, you use anholidayspackage that you also have in yourrequirements.txtso I would say this kind of setup is working.The error I see in my console is like:96w4z 2023-12-29T16:54:49.633Z   File "/app/huggingface_inference_toolkit/utils.py", line 188, in check_and_register_custom_pipeline_from_directory
96w4z 2023-12-29T16:54:49.633Z     custom_pipeline = check_and_register_custom_pipeline_from_directory(model_dir)
96w4z 2023-12-29T16:54:49.633Z   File "/repository/handler.py", line 3, in <module>
96w4z 2023-12-29T16:54:49.633Z     inference_handler = get_inference_handler_either_custom_or_default_handler(HF_MODEL_DIR, task=HF_TASK)
96w4z 2023-12-29T16:54:49.633Z   File "/app/webservice_starlette.py", line 57, in some_startup_task
96w4z 2023-12-29T16:54:49.633Z     from concrete.ml.deployment import FHEModelServer
96w4z 2023-12-29T16:54:49.633Z   File "<frozen importlib._bootstrap_external>", line 850, in exec_module
96w4z 2023-12-29T16:54:49.633Z ModuleNotFoundError: No module named 'concrete'
96w4z 2023-12-29T16:54:49.633Z   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
96w4z 2023-12-29T16:54:49.633Z     spec.loader.exec_module(handler)
96w4z 2023-12-29T16:54:49.633Z   File "/app/huggingface_inference_toolkit/handler.py", line 41, in get_inference_handler_either_custom_or_default_handler
96w4z 2023-12-29T16:54:49.633Z     await handler()
96w4z 2023-12-29T16:54:49.633Z   File "/opt/conda/lib/python3.9/site-packages/starlette/routing.py", line 682, in startup
96w4z 2023-12-29T16:54:52.165Z     from concrete.ml.deployment import FHEModelServer
96w4z 2023-12-29T16:54:52.165Z   File "<frozen importlib._bootstrap_external>", line 850, in exec_module
96w4z 2023-12-29T16:54:52.165Z 2023-12-29 16:54:52,165 | INFO | Found custom pipeline at /repository/handler.py
96w4z 2023-12-29T16:54:52.165Z 
96w4z 2023-12-29T16:54:52.165Z     await handler()
96w4z 2023-12-29T16:54:52.165Z     await self._router.startup()
96w4z 2023-12-29T16:54:52.165Z   File "/opt/conda/lib/python3.9/site-packages/starlette/routing.py", line 705, in lifespan
96w4z 2023-12-29T16:54:52.165Z Application startup failed. Exiting.
96w4z 2023-12-29T16:54:52.165Z ModuleNotFoundError: No module named 'concrete'where the important part isModuleNotFoundError: No module named ‘concrete’.Could you help me a bit, please? I can video chat with some of you if needed, to show it to you. Thanks a lot for what you’re doing, HF is awesome, and it’s one of the reasons I want to deploy here!
2023-12-29T17:30:33Z
Could we see the log of therequirements.txtinstallation?
2023-12-29T17:46:03Z
OK, we’ve found the solution. It was about the fact that the revision for the endpoint was fixed, ie not the last revision. You have to press “get last revision”. It’s maybe not natural to beginners.Anyway, sorry for the noise, keep the good work guys.
2023-12-29T17:53:51Z
However, I would say that the “get latest revision” is not very reliable. For some reason:I resumeI click on “get latest revision”I see the new revisionI click on resumeI go to “logs”I come back to “settings”and then, when I am back here, the latest revision has been replaced by an old one.Is that that we have to click on “Update endpoints”? I feel the user might not see that, it’s not obvious, why not directly using the new SHA1?