GGUF for Ollama
#3
by
SwimTreeWire
- opened
I would like to use this with ollama. How can i make the GGUF from this repo?
This is a new architecture and support hasn't been merged into Llama.cpp yet.
how can this be achieved? can i somehow make a GGUF myself and upload?
Nope, that means someone has to write the support for the model in the backend itself. You can probably sub to https://github.com/ggml-org/llama.cpp/issues/15748 to get updates.