Integrating Reward Maximization and Population Estimation: Sequential
  Decision-Making for Internal Revenue Service Audit Selection
  We introduce a new setting, optimize-and-estimate structured bandits. Here, a
policy must select a batch of arms, each characterized by its own context, that
would allow it to both maximize reward and maintain an accurate (ideally
unbiased) population estimate of the reward. This setting is inherent to many
public and private sector applications and often requires handling delayed
feedback, small data, and distribution shifts. We demonstrate its importance on
real data from the United States Internal Revenue Service (IRS). The IRS
performs yearly audits of the tax base. Two of its most important objectives
are to identify suspected misreporting and to estimate the "tax gap" - the
global difference between the amount paid and true amount owed. We cast these
two processes as a unified optimize-and-estimate structured bandit. We provide
a novel mechanism for unbiased population estimation that achieves rewards
comparable to baseline approaches. This approach has the potential to improve
audit efficacy, while maintaining policy-relevant estimates of the tax gap.
This has important social consequences given that the current tax gap is
estimated at nearly half a trillion dollars. We suggest that this problem
setting is fertile ground for further research and we highlight its interesting
challenges.
