Label Inference Attacks from Log-loss Scores
  Log-loss (also known as cross-entropy loss) metric is ubiquitously used
across machine learning applications to assess the performance of
classification algorithms. In this paper, we investigate the problem of
inferring the labels of a dataset from single (or multiple) log-loss score(s),
without any other access to the dataset. Surprisingly, we show that for any
finite number of label classes, it is possible to accurately infer the labels
of the dataset from the reported log-loss score of a single carefully
constructed prediction vector if we allow arbitrary precision arithmetic.
Additionally, we present label inference algorithms (attacks) that succeed even
under addition of noise to the log-loss scores and under limited precision
arithmetic. All our algorithms rely on ideas from number theory and
combinatorics and require no model training. We run experimental simulations on
some real datasets to demonstrate the ease of running these attacks in
practice.
