Optimization of Genomic Classifiers for Clinical Deployment: Evaluation
  of Bayesian Optimization to Select Predictive Models of Acute Infection and
  In-Hospital Mortality
  Acute infection, if not rapidly and accurately detected, can lead to sepsis,
organ failure and even death. Current detection of acute infection as well as
assessment of a patient's severity of illness are imperfect. Characterization
of a patient's immune response by quantifying expression levels of specific
genes from blood represents a potentially more timely and precise means of
accomplishing both tasks. Machine learning methods provide a platform to
leverage this 'host response' for development of deployment-ready
classification models. Prioritization of promising classifiers is dependent, in
part, on hyperparameter optimization for which a number of approaches including
grid search, random sampling and Bayesian optimization have been shown to be
effective. We compare HO approaches for the development of diagnostic
classifiers of acute infection and in-hospital mortality from gene expression
of 29 diagnostic markers. We take a deployment-centered approach to our
comprehensive analysis, accounting for heterogeneity in our multi-study patient
cohort with our choices of dataset partitioning and hyperparameter optimization
objective as well as assessing selected classifiers in external (as well as
internal) validation. We find that classifiers selected by Bayesian
optimization for in-hospital mortality can outperform those selected by grid
search or random sampling. However, in contrast to previous research: 1)
Bayesian optimization is not more efficient in selecting classifiers in all
instances compared to grid search or random sampling-based methods and 2) we
note marginal gains in classifier performance in only specific circumstances
when using a common variant of Bayesian optimization (i.e. automatic relevance
determination). Our analysis highlights the need for further practical,
deployment-centered benchmarking of HO approaches in the healthcare context.
