Differentiable NAS Framework and Application to Ads CTR Prediction
  Neural architecture search (NAS) methods aim to automatically find the
optimal deep neural network (DNN) architecture as measured by a given objective
function, typically some combination of task accuracy and inference efficiency.
For many areas, such as computer vision and natural language processing, this
is a critical, yet still time consuming process. New NAS methods have recently
made progress in improving the efficiency of this process. We implement an
extensible and modular framework for Differentiable Neural Architecture Search
(DNAS) to help solve this problem. We include an overview of the major
components of our codebase and how they interact, as well as a section on
implementing extensions to it (including a sample), in order to help users
adopt our framework for their applications across different categories of deep
learning models. To assess the capabilities of our methodology and
implementation, we apply DNAS to the problem of ads click-through rate (CTR)
prediction, arguably the highest-value and most worked on AI problem at
hyperscalers today. We develop and tailor novel search spaces to a Deep
Learning Recommendation Model (DLRM) backbone for CTR prediction, and report
state-of-the-art results on the Criteo Kaggle CTR prediction dataset.
