Issues with loading models in 8bit in Colab
Hello. I am using Google Colab. I run this code!pip install sentencepiece -q!pip install accelerate -q!pip install bitsandbytes -qfrom transformers import AutoTokenizer, AutoModelForCausalLMtk = AutoTokenizer.from_pretrained(“microsoft/Orca-2-13b”)model =AutoModelForCausalLM.from_pretrained(“microsoft/Orca-2-13b”, load_in_8bit=True)but I always get errorImportError: Usingload_in_8bit=Truerequires Accelerate:pip install accelerateand the latest version of bitsandbytespip install -i https://test.pypi.org/simple/ bitsandbytesor pip install bitsandbytes` .They are already installed. Why this happens?