Is it possible to add this or any other Dynamic Quants model to ollama?
#2
by
Crandel
- opened
I have no python knowledge but want to try Dynamic Quants model locally using ollama. Is it possible?
I have no python knowledge but want to try Dynamic Quants model locally using ollama. Is it possible?
You can if you want but youll need to convert it to 16bit then convert to gguf, then to 4bit but then youd rather just use the GGUFs we aready uploaded
Crandel
changed discussion status to
closed