Creating inference endpoint with custom handler - is this how it should work?
I don’t seem to quite understand how to deploy a model with a custom handler as an inference endpoint. Specifically, I’m trying to use this model (which alread has a custom handler):philschmid/stable-diffusion-v1-4-endpoints · Hugging FaceI went through these steps trying to follow the tutorial:Initialize Git LFS:git lfs installClone the desired repo:git clone https://huggingface.co/philschmid/stable-diffusion-v1-4-endpointsInstall huggingface-hub python package (in a venv):python -m venv venv+source venv/bin/activate+pip install huggingface-hubLogin to Huggingface:huggingface-cli login→ Choose Yes when asked to add token as git credentialRun this command (don’t really understand what it does):git config --global credential.helper storeAs far as I understand, I don’t need to change anything in the code, since it already has a custom handler. I now need to push the whole thing to a repo that I created before. It’s currently empty: shttps://huggingface.co/eykarim/stable-diffusion-v1I move up one folder into/stable-diffusion-v1-4-endpointsI delete the existing.gitso I can push the whole code to my own repo:sudo rm -r .gitI initialize a new git:git init+git remote add origin master+git remote set-url origin https://huggingface.co/eykarim/stable-diffusion-v1Add all files to commit to my new remote repo:git add *+git commit -m "my first commit"+git push origin masterAt this point I’m getting a git errorerror failed to push some refs to https://huggingface.co/eykarim/stable-diffusion-v1As far as I understand this should be the right way. Once I have code with a handler.py in my repo, I should be able to find it in the inference endpoints search mask, right? (see screenshot)Bildschirmfoto_2022-11-21_10-42-47981×620 54.2 KBIs this the right way to do it? And if so: Maybe someone could help me fix the Git error? That would be great!
2022-11-24T07:45:22Z
Anybody? If someone has a link/resource that could help me solve this by myself, I’d be more than happy!
2022-11-24T09:12:31Z
cc@philschmidbtw, might be better to post this in the “inference endpoints” section, rather than “beginners”
2022-11-24T12:55:53Z
Hello@eykarim,You can check out this tutorial:Custom Inference with Hugging Face Inference Endpointsor the documentation on how to create custom handlers:Create custom Inference HandlerThose include e2e example with uploading the model.FYI we also have a dedicated inference endpoints section in the forum.
2022-11-27T07:31:14Z
Hi Phil! Thanks for the tutorial. I worked through it a second time yesterday and now have a model deployed:eykarim/stable-diffusion-v1 · Hugging FaceAs I wrote in my original post, I was assuming that once I have a model deployed with a custom handler (marked as “endpoints-template”) I should be able to deploy it with an Inference Endpoint. However, the model does not show up in the search bar, nor do many of the other models. I don’t understand what requirements a model needs to fullfil in order to be deployed as an Inference Endpoint. Why do some models with an endpoint handler show up in the search bar and others don’t?Bildschirmfoto_2022-11-26_20-17-13914×422 36.5 KB
2022-11-27T09:44:58Z
I responded in your other thread:Why is my model not showing up in the search bar for Inference Endpoints? - #2 by philschmid