how to make determinstic output?
I run the model multiple times with the same input but the outputs are different.
This is expected by default. Does anyone know how to produce deterministic output?
In addition to (temperature=0, top_p =1)
Try setting a seed value (eg: seed=0).
For purely deterministic, I always use Temp=0, Top_K=0, Repeat_Penalty=1, Min_P=1, Top_P=0.
For Seed number you can set it to whatever value you like, but I recommend using the maximum "Highly Composite Number" that your environment supports.
Link to OEIS article for Highly Composite Numbers. https://oeis.org/A002182
For KoboldCPP this number is "720720", and for LM Studio this number is "4488062423933088000". LM Studio does allow you to use exponents, so you can try "9.14876906094457e+256" but it might not give the best answer compared to the other number.