Thinking prompt not included in template
#3
by
Doctor-Chad-PhD
- opened
Hi,
I was just wondering if the thinking prompt could be added to the jinja template in tokenizer_config.json?
On line 198 it's just the regular Qwen template, so when converting this model to quants or using it with any inference software other than the official repo, it fails to start reasoning.
I believe it's also missing from the added_tokens.json file.
The only way to invoke it right now is to manually add <|begin_of_thought|> to the end of a prompt.
Thank you.
Could you please provide a bit more detail on how you are running the model?
With this chat template (and without the thinking prompt) the model is still producing reasoning. I also checked the default 7B quant on Ollama and I wasn't running into any issues there.