How to use GPTQ on falcon-7b
I was wondering that about how to quantize a model and was exploring in that regard.But I am unable to quantize the tiiuae/falcon-7b model on colab.colab.research.google.comGoogle Colaboratoryhere is my notebook and please ignore the rest it was taken from other notebooks.And would be really helpful if you can point to a guide to know about the parameters to use in this regard.