Extracting more from boosted decision trees: A high energy physics case
  study
  Particle identification is one of the core tasks in the data analysis
pipeline at the Large Hadron Collider (LHC). Statistically, this entails the
identification of rare signal events buried in immense backgrounds that mimic
the properties of the former. In machine learning parlance, particle
identification represents a classification problem characterized by overlapping
and imbalanced classes. Boosted decision trees (BDTs) have had tremendous
success in the particle identification domain but more recently have been
overshadowed by deep learning (DNNs) approaches. This work proposes an
algorithm to extract more out of standard boosted decision trees by targeting
their main weakness, susceptibility to overfitting. This novel construction
harnesses the meta-learning techniques of boosting and bagging simultaneously
and performs remarkably well on the ATLAS Higgs (H) to tau-tau data set (ATLAS
et al., 2014) which was the subject of the 2014 Higgs ML Challenge
(Adam-Bourdarios et al., 2015). While the decay of Higgs to a pair of tau
leptons was established in 2018 (CMS collaboration et al., 2017) at the
4.9$\sigma$ significance based on the 2016 data taking period, the 2014 public
data set continues to serve as a benchmark data set to test the performance of
supervised classification schemes. We show that the score achieved by the
proposed algorithm is very close to the published winning score which leverages
an ensemble of deep neural networks (DNNs). Although this paper focuses on a
single application, it is expected that this simple and robust technique will
find wider applications in high energy physics.
