Out of a hundred trials, how many errors does your speaker verifier
  make?
  Out of a hundred trials, how many errors does your speaker verifier make? For
the user this is an important, practical question, but researchers and vendors
typically sidestep it and supply instead the conditional error-rates that are
given by the ROC/DET curve. We posit that the user's question is answered by
the Bayes error-rate. We present a tutorial to show how to compute the
error-rate that results when making Bayes decisions with calibrated likelihood
ratios, supplied by the verifier, and an hypothesis prior, supplied by the
user. For perfect calibration, the Bayes error-rate is upper bounded by
min(EER,P,1-P), where EER is the equal-error-rate and P, 1-P are the prior
probabilities of the competing hypotheses. The EER represents the accuracy of
the verifier, while min(P,1-P) represents the hardness of the classification
problem. We further show how the Bayes error-rate can be computed also for
non-perfect calibration and how to generalize from error-rate to expected cost.
We offer some criticism of decisions made by direct score thresholding.
Finally, we demonstrate by analyzing error-rates of the recently published
DCA-PLDA speaker verifier.
