Predicting Census Survey Response Rates via Interpretable Nonparametric
  Additive Models with Structured Interactions
  Accurate and interpretable prediction of survey response rates is important
from an operational standpoint. The US Census Bureau's well-known ROAM
application uses principled statistical models trained on the US Census
Planning Database data to identify hard-to-survey areas. An earlier
crowdsourcing competition revealed that an ensemble of regression trees led to
the best performance in predicting survey response rates; however, the
corresponding models could not be adopted for the intended application due to
limited interpretability. In this paper, we present new interpretable
statistical methods to predict, with high accuracy, response rates in surveys.
We study sparse nonparametric additive models with pairwise interactions via
$\ell_0$-regularization, as well as hierarchically structured variants that
provide enhanced interpretability. Despite strong methodological underpinnings,
such models can be computationally challenging -- we present new scalable
algorithms for learning these models. We also establish novel non-asymptotic
error bounds for the proposed estimators. Experiments based on the US Census
Planning Database demonstrate that our methods lead to high-quality predictive
models that permit actionable interpretability for different segments of the
population. Interestingly, our methods provide significant gains in
interpretability without losing in predictive performance to state-of-the-art
black-box machine learning methods based on gradient boosting and feedforward
neural networks. Our code implementation in python is available at
https://github.com/ShibalIbrahim/Additive-Models-with-Structured-Interactions.
