Comparing hundreds of machine learning classifiers and discrete choice
  models in predicting travel behavior: an empirical benchmark
  Researchers have compared machine learning (ML) classifiers and discrete
choice models (DCMs) in predicting travel behavior, but the generalizability of
the findings is limited by the specifics of data, contexts, and authors'
expertise. This study seeks to provide a generalizable empirical benchmark by
comparing hundreds of ML and DCM classifiers in a highly structured manner. The
experiments evaluate both prediction accuracy and computational cost by
spanning four hyper-dimensions, including 105 ML and DCM classifiers from 12
model families, 3 datasets, 3 sample sizes, and 3 outputs. This experimental
design leads to an immense number of 6,970 experiments, which are corroborated
with a meta dataset of 136 experiment points from 35 previous studies. This
study is hitherto the most comprehensive and almost exhaustive comparison of
the classifiers for travel behavioral prediction. We found that the ensemble
methods and deep neural networks achieve the highest predictive performance,
but at a relatively high computational cost. Random forests are the most
computationally efficient, balancing between prediction and computation. While
discrete choice models offer accuracy with only 3-4 percentage points lower
than the top ML classifiers, they have much longer computational time and
become computationally impossible with large sample size, high input
dimensions, or simulation-based estimation. The relative ranking of the ML and
DCM classifiers is highly stable, while the absolute values of the prediction
accuracy and computational time have large variations. Overall, this paper
suggests using deep neural networks, model ensembles, and random forests as
baseline models for future travel behavior prediction. For choice modeling, the
DCM community should switch more attention from fitting models to improving
computational efficiency, so that the DCMs can be widely adopted in the big
data context.
