Error trying to use instruct_pipeline in order to avoid trust_remote_code
Note: superbeginner here.Working with Databricks’ dolly-v2-3b.Installed git 2.40.0.windows.1 and downloaded DollyInstalled CUDA 12.1.1_531.14Installed python 3.11.3 and installed the necessary using the following commands:py.exe -m pip install numpypy.exe -m pip install accelerate>=0.12.0 transformers[torch]==4.25.1py.exe -m pip install numpy --pre torch --force-reinstall --index-urlhttps://download.pytorch.org/whl/nightly/cu117--userNow, when I run the following commands, everything works ok and Dolly will produce responses.import torchfrom transformers import pipelinegenerate_text = pipeline(model=“databricks/dolly-v2-3b”, torch_dtype=torch.bfloat16, trust_remote_code=True, device_map=“auto”)However, I want to not use the trust_remote_code. So I downloaded the instruct_pipeline.py and added it to the C:\Users\xxxxxx\AppData\Local\Programs\Python\Python311\Lib. But when I try to do:import torchfrom instruct_pipeline import InstructionTextGenerationPipelineI get an error as per the screenshot:dolly screen2009×700 121 KBNot sure how to proceed?I’m running everything on windows 11. I’ve also noticed my GPU (1660 super, driver v531.41) is not used at all (0% gpu at task manager), shouldn’t utilization jump to 100% when I run a prompt?
2023-04-27T06:30:28Z
The latest version of CUDA which pytorch supports is version 11.8 which you can find installation information for hereStart Locally | PyTorchYou can confirm that pytorch is using the installed CUDA software by including this in your script:print(“CUDA available:”, torch.cuda.is_available())print(“Number of GPUs:”, torch.cuda.device_count())print(“GPU name:”, torch.cuda.get_device_name(0))