On the impact of selected modern deep-learning techniques to the
  performance and celerity of classification models in an experimental
  high-energy physics use case
  Beginning from a basic neural-network architecture, we test the potential
benefits offered by a range of advanced techniques for machine learning, in
particular deep learning, in the context of a typical classification problem
encountered in the domain of high-energy physics, using a well-studied dataset:
the 2014 Higgs ML Kaggle dataset. The advantages are evaluated in terms of both
performance metrics and the time required to train and apply the resulting
models. Techniques examined include domain-specific data-augmentation, learning
rate and momentum scheduling, (advanced) ensembling in both model-space and
weight-space, and alternative architectures and connection methods. Following
the investigation, we arrive at a model which achieves equal performance to the
winning solution of the original Kaggle challenge, whilst being significantly
quicker to train and apply, and being suitable for use with both GPU and CPU
hardware setups. These reductions in timing and hardware requirements
potentially allow the use of more powerful algorithms in HEP analyses, where
models must be retrained frequently, sometimes at short notice, by small groups
of researchers with limited hardware resources. Additionally, a new wrapper
library for PyTorch called LUMIN is presented, which incorporates all of the
techniques studied.
