Gateway Problem
I bought huggingface pro. I am trying to use the model microsoft/Phi-3-mini-128k-instruct on HuggingFace’s Serverless API since yesterday through the following code:response = self.client.chat.completions.create(model=self.model_name,messages=[{“role”: “user”, “content”: prompt}],temperature=0,max_tokens=300)However, I keep having the same errors:and504 Server Error: Gateway Time-out for url:https://api-inference.huggingface.co/models/microsoft/Phi-3-mini-128k-instruct/v1/chat/completionsI tried other models such as meta-llama/Llama-3.2-1B and I keep having the same problem. In the midst of the errors, there were like one or two sucessful requests. Is this normal?
2025-01-05T19:07:10Z
I am getting the same thing.
2025-01-07T18:55:46Z
Hi@JoseLuisNevesThe model microsoft/Phi-3-mini-128k-instruct  is not loaded on the serverless API, but you can use this model withInference Endpoints. Inference Endpoints allows you to easilydeploy your modelson dedicated, fully-managed infrastructure, and will give you the flexibility to quickly create endpoints on CPU or GPU resources. It’sbilledby compute uptime vs character usage.