Accurate, Data-Efficient Learning from Noisy, Choice-Based Labels for
  Inherent Risk Scoring
  Inherent risk scoring is an important function in anti-money laundering, used
for determining the riskiness of an individual during onboarding
$\textit{before}$ fraudulent transactions occur. It is, however, often fraught
with two challenges: (1) inconsistent notions of what constitutes as high or
low risk by experts and (2) the lack of labeled data. This paper explores a new
paradigm of data labeling and data collection to tackle these issues. The data
labeling is choice-based; the expert does not provide an absolute risk score
but merely chooses the most/least risky example out of a small choice set,
which reduces inconsistency because experts make only relative judgments of
risk. The data collection is synthetic; examples are crafted using optimal
experimental design methods, obviating the need for real data which is often
difficult to obtain due to regulatory concerns. We present the methodology of
an end-to-end inherent risk scoring algorithm that we built for a large
financial institution. The system was trained on a small set of synthetic data
(188 examples, 24 features) whose labels are obtained via the choice-based
paradigm using an efficient number of expert labelers. The system achieves 89%
accuracy on a test set of 52 examples, with an area under the ROC curve of 93%.
