On Demand GPU model hosting?
Hi,I need to deploy a few small models for demo’s. The demo’s will only be run for a few hours off and on over the course of a month or two. I don’t want to pay for huge amounts of hosting, but I absolutely need to have on-demand full usage of GPU and RAM so they behave correctly during the demos. Any suggestions?Thanks,Winton
2024-02-20T22:13:58Z
Serverless GPUs from inferless? Are your demos for running inference?
2024-02-22T10:34:21Z
Hi,Some examples include:RunpodLambda LabsBanana.devVast.aiThese provide the cheapest prices for both on-demand GPUs and spot instances.If you want an easier solution to deploy models, thenInference Endpointsmight be worth a look. It includes a scale-to-zero option which means that you don’t need to pay if there are no requests.