Inference Endpoint: `'NoneType' object is not callable`. No other context to go on [Pyannote speaker diarization]
I’m trying to create a pyannoteInference Endpoint with a custom inference handler.I’m using HuggingFace’s example by@philschmidvirtually as-is:philschmid/pyannote-speaker-diarization-endpoint · Hugging FaceI’ve run into 2 problems:I created a project from scratch, heavily borrowing from@philschmid’s example, but addingpyannote-audio~=2.1directly torequirements.txt(in his example he used a pyannote clone due to a HuggingFace requirement conflict that has since been fixed upstream).I can run it locally just fine, and deploy it to HuggingFace just fine.However when running on the cloud against any input I get the following error:ERROR | 'NoneType' object is not callable.There is no other context whatsoever to go on.This is the first line logged after the 3 logs apparently associated with the endpoint initialization:bm7hp 2022-11-04T23:13:20.279Z 2022-11-04 23:13:20,279 | INFO | Initializing model from directory:/repository
bm7hp 2022-11-04T23:13:20.279Z 2022-11-04 23:13:20,279 | INFO | Found custom pipeline at /repository/handler.py
bm7hp 2022-11-04T23:13:21.392Z 2022-11-04 23:13:21,392 | INFO | Model initialized successfully
bm7hp 2022-11-04T23:16:08.468Z 2022-11-04 23:16:08,468 | ERROR | 'NoneType' object is not callableI tried adding logging in theEndpointHandler__init__and__call__, but these aren’t showing up. The error must be happening even prior to the__init__getting called.Just in case, I tried directly cloning@philschmidrepo above.I get an error on deploy, complaining that files on the pyannote/speaker-diarization repo cannot be accessed because they don’t exist or because I’m unauthorized. (I don’t have exact error available right now but can reproduce if needed).I encountered similar errors while implementing (1), solved by approving pyannote’s terms of service on their repos. Not sure why these errs recur here.Any pointers would be much appreciated.
2022-11-07T08:46:56Z
Hello@ataiiiThere is some hick-hup withpyannote.audioandhuggingface_hubYou can see more here:[BUG] parameter 'segmentation_onset' does not exist · Issue #1100 · pyannote/pyannote-audio · GitHubThis might be the reason for your issues. Can you share your repository?
2022-11-07T08:50:27Z
I checked why the example is not longer working an it turns out thatpyannotehas added agateto the repository:pyannote/speaker-diarization · Hugging Face.I ll try to find time to adapt my example. In the meantime you could accept the gate and put the model into the same repository as thehandler.pyinstead of loading it directly.
2022-11-08T22:48:09Z
Hey@philschmidthanks for looking into this.I updated the pipeline fetch to use my auth token, and now it seems to work!Pipeline.from_pretrained("pyannote/speaker-diarization@2.1", use_auth_token="XXX")Honestly I’m not sure why, because fetching the pipeline didn’t seem like the issue I was having in (1).Perhaps some dependency was updated?Anyways, it’s working now on my repo!
2022-11-09T08:55:35Z
Hello@ataiii,I fixed yesterday myphilschmid/pyannote-speaker-diarization-endpoint · Hugging Faceexample
2022-11-13T01:06:26Z
Hi@philschmidI’m still getting an error on runtime (a new ones).huggingface_diarization_result:  {"error":"CUDA error: CUBLAS_STATUS_INVALID_VALUE when calling `cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc)`"}Getting this error both on a direct clone of your repo and on my own similar repo.I deployed an instance off of my repo a few days ago as well (as I said above). That instance is still running and has no issues.Redeploying same exact repo now consistently yields this error. Tried multiple cloud providers and machine types.Seems to be some sort of configuration issue on Huggingface’s end?