Zero-Shot Text Classification on a low-end CPU-only machine?
I want to do zero-shot text classification either this model [1] (711 MB) or something similar:Want to achieve high throughput in classification requests per second.The classification will run on low-end hardware: some Hetzner [2] machine without GPU (Hetzner is great, reliable and cheap they just do not have GPU machines), something like this:CCX13: Dedicated vCPU, 2 VCPU, 8 GB RAMCX32:  Shared vCPU, 4 VCPU, 8 GB RAMNow there are multiple options for deploying and serving LLMs:lmdeploytext-generation-inferenceTensorRT-LLMvllmThere are more and more new frameworks for this. I am a bit lost.Would you suggest the best option for deploying the above-listed model (No-GPU hardware)?[1]MoritzLaurer/roberta-large-zeroshot-v2.0-c · Hugging Face[2]https://www.hetzner.com/cloud/P.S. I know about this post:way to make inference Zero Shot pipeline faster?BeginnersHi 
Can you guys give me tips how to make Zero Shot pipeline inference faster? 
My current approach right now is reducing the model size/parameter 
(trying to train “base model” instead of "large model) 
Is there another approach?These recomendations - they’re great. But, uh, it’s been four years
2024-10-08T10:04:48Z
@joeddav@nielsrAny good updates in the last 4 years on zero-shot text classification, mainly in low-end environments without GPUs?
2024-10-08T12:21:43Z
You could check if Yake would be enough for your case, at least its resource demands are negligible compared to newer methods:GitHub - LIAAD/yake: Single-document unsupervised keyword extraction
2024-10-09T09:09:03Z
thanks@juhoinkinen!as far as I understand: YAKE works for paragraph-long or document-long text.and this zero-shot classification model in original text: it works well even with few words/sentence. This is what I need: sentence classification.