API Limits on Free Inference API
The documentation is rather vague on the limits of the Free Inference API and similarly vague what subscribing to a ‘Pro’ account would change in the limits.Could somebody comment in their experience what the limits of the Inference API are? In particularDoes moving to Pro change the limit for the model size which can be used? (Free has a limit of 10GB)Are there any hourly / monthly character (or token?) limits for queries or responses?Is there any rate limiting (request per minute)?Does Pro change anything regarding the time until a model is loaded / unloaded?Bonus question:Is there a way to use quantized models with Free Inference API?Thanks!
2023-12-06T14:28:46Z
I am interested on this too.
2024-03-03T10:26:40Z
I’m interested on this topic too. Thanks to bring it up.
2024-04-24T19:41:49Z
I’m interested on this topic too
2024-05-28T13:40:54Z
Still no clear information about this. Any HF members could clarify, please?
2024-05-30T10:04:41Z
Did anyone figure this out?
2024-06-03T09:27:32Z
Also running into api limit rates with the PRO subscription. Would be helpful if there were stats on how to navigate the limits.
2024-07-28T08:33:31Z
Hi@GuusBouwensNL, can you tell what api rate limit are you able to achieve with the PRO plan?After how many requests are you hitting the rate limit.
2024-08-06T04:24:52Z
Interested on this too
2024-08-07T11:06:32Z
Also arrived here with the same interest.
2024-08-22T09:59:43Z
I’m also interested in the answer to these questions!
2024-08-22T10:01:17Z
Ok… found it. It’s flexible…image797×140 14.5 KB
2024-09-15T02:40:53Z
Still not clear.
2024-09-15T03:58:04Z
It is still vague and fluid at all for Pro account users.In Pro status, the restrictions are relaxed. I think it’s being done, so it’s being done. That’s all I can say.Plus, unlike in 2023, even the smaller models don’t work as well now.The Free Serverless Inference API and widgets for all but the very best models is virtually obsolete now.Is there a way to use quantized models with Free Inference API?So far, this is still not possible. Now that quantization is becoming more and more commonplace and there is particularly no need not to do it, HF will have to deal with it eventually…