Regularization for Deep Learning: A Taxonomy
  Regularization is one of the crucial ingredients of deep learning, yet the
term regularization has various definitions, and regularization methods are
often studied separately from each other. In our work we present a systematic,
unifying taxonomy to categorize existing methods. We distinguish methods that
affect data, network architectures, error terms, regularization terms, and
optimization procedures. We do not provide all details about the listed
methods; instead, we present an overview of how the methods can be sorted into
meaningful categories and sub-categories. This helps revealing links and
fundamental similarities between them. Finally, we include practical
recommendations both for users and for developers of new regularization
methods.
