HOW TO determine the best threshold for predictions when making inference with a finetune model?
Hello, I finetune a model but the F-score is not quite good for certains classes. To avoid a lot of false positives I decide to set a threshhold for the probabilities and I would like to know how to determine, the best threshhold ?Should I use the mean, median , or just look at accuracy of the model on the test_data ?
2021-12-21T14:09:13Z
Hi,The best way to determine the threshold is to compute the true positive rate (TPR) and false positive rate (FPR) at different thresholds, and then plot the so-calledROC-curve. The ROC curve plots, for every threshold, the corresponding true positive rate and false positive rate.Then, selecting the point (i.e. threshold) that is most to the top left of the curve will yield the best balance among the two.Sklearn provides animplementationof this, however it’s for binary classification only. Note that there areextensionsfor multiclass classification.
2021-12-22T15:07:51Z
Thank you actually, I am doing multiclassification, I forget to mention.
2023-12-25T15:32:53Z
May I have any paper reference for this statement, as I am willing to include this statement in my paper
2023-12-25T20:50:30Z
Hi,This is just taught in most machine learning courses actuallysee for instanceClassification: ROC Curve and AUC  |  Machine Learning  |  Google for Developers.But feel free to cite me