title: Choosing correct threshold value for classification in logistic regression
body: <p>I am working on an ad's clicked or not on a website classification dataset(a pretty much balanced one). I need to know the correct probability threshold for classifying whether website visitors will click ad or not.</p>
  
  <p>Objective:- Since ads are expensive on the website, we want to make sure ads are displayed only to those who have good chance of converting and unnecessary clicks will only add heavily to our cost. </p>
  
  <p>So now I have to select threshold value in such a way that we have good number of conversions and have low ad clicks which don't convert.</p>
  
  <p>I ran logistic regression (1=ad clicked 0=not clicked) and attached image has data on <code>accuracy</code>, <code>recall</code>, <code>precision</code>, <code>auc score</code>, <code>f1_score</code> at threshold values 0.4,0.45, 0.5, 0.55, 0.6.</p>
  
  <p>As per my knowledge that threshold value should be selected where f1_score is maximum. However, since we want to minimize ad clicks which don't convert we want to have high precision i.e. as low false positives as possible.</p>
  
  <p>For this dataset the f1 score is max at threshold=0.45, however precision is higher at threshold=0.55 &amp; 0.6. What threshold value should I select given our mentioned objective?</p>
  
  <p>Tried looking for answers on determining correct threshold value but found none very satisfying. searched stackoverflow and web.  </p>
  
  <pre><code>THRESHOLD=0.4 #0.4 to 0.6 incremented by 0.5
  
  predictions=np.where(classifier.predict_proba(X_test)[:,1] &gt; THRESHOLD,1,0)
  pd.DataFrame(data=[accuracy_score(Y_test, predictions), recall_score(Y_test, predictions),
                     precision_score(Y_test, predictions), roc_auc_score(Y_test, predictions), f1_score(Y_test,predictions)], 
               index=["accuracy", "recall", "precision", "roc_auc_score",'f1_score'])
  </code></pre>
  
  <p>Correct threshold value</p>