Facing issues with AWQ inference engine while finetuning idefics 2 8b Awq model
Hi Folks,I tried to Finetune idefics2-8b-AWQ model but unfortunately i encountered this error “ImportError: /usr/local/lib/python3.10/dist-package/awq_inference_engine.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN3c104cuda9SetDeviceEi” so i tried installing awq inference engine but pypi doesnt have awq inference engine package so i tried downloading from a third party and done a setup but still i landed on that same error, can someone help me with this?Screenshot from 2024-06-15 01-20-101342×59 12.1 KB
2024-08-01T12:43:23Z
same issue
2024-08-01T12:44:07Z
pip install autoawq --upgrade did the trick for me