Run models on a desktop computer?
Hello,I’ve been using some huggingface models in notebooks on SageMaker, and I wonder if it’s possible to run these models (fromHF.co) directly on my own PC? I’m mainly interested in Named Entity Recognition models at this point.I assume it’d be slower than using SageMaker, but how much slower? Like… infeasibly slow?I’m a software engineer and longtime Linux user, but fairly new to AI/ML.Also, I browsed through the docs here a little bit, but didn’t see a basic “Getting Started” type of page – does that exist?Thanks for any advice.
2023-02-05T05:23:20Z
hello@antcodes,Yes, you can run all models from the hub locally.Maybe you can start by here:InstallationSetting up a local python environment, and installing the required packages.For example, if you run this code, frombase-NERIt will download the model to your local cache.You can read more about the pipelinesherefrom transformers import AutoTokenizer, AutoModelForTokenClassification
from transformers import pipeline

tokenizer = AutoTokenizer.from_pretrained("dslim/bert-base-NER")
model = AutoModelForTokenClassification.from_pretrained("dslim/bert-base-NER")

nlp = pipeline("ner", model=model, tokenizer=tokenizer)
example = "My name is Wolfgang and I live in Berlin"

ner_results = nlp(example)
print(ner_results)
2023-02-05T06:02:17Z
Thanks so much for the quick reply. That’s really helpful! I’m going to get started setting up a python virtual environment.
2023-08-29T11:29:58Z
I tried installing “HuggingFaceH4/starchat-beta” but it did not work, the above code worked but for Starchat model I am getting multiple memory issues. I have 1 GPU 15GB with 64GB RM Ubuntu.
2023-10-29T22:18:43Z
Hi - I’ve tried doing it this way and with git and huggingface-cli on Windows all to no effect. The model I’m trying to download, facebook/nllb-moe-54b, is quite large and I encouter errors every time I try to download. Is there a way to ensure that this is done correctly?
2023-12-05T03:42:14Z
is there any suggestion to simply assess local GPU performance againt the target model that run viapipeline?I believe the most choice of GPU for individual developers are Nvidia RTX serials, just want to make sure the GPU can run the model before downloading from hf.
2024-01-25T14:52:53Z
Hello sir. My illusion diffusion site not working abd shows share in community. Please solve this problem
2024-06-16T12:16:26Z
I have a question, Can we run spaces locally? For example chatting with a hugging face space model directly from Visual Studio Code like GPT-4.5 turbo.