The Perils of Learning Before Optimizing
  Formulating real-world optimization problems often begins with making
predictions from historical data (e.g., an optimizer that aims to recommend
fast routes relies upon travel-time predictions). Typically, learning the
prediction model used to generate the optimization problem and solving that
problem are performed in two separate stages. Recent work has showed how such
prediction models can be learned end-to-end by differentiating through the
optimization task. Such methods often yield empirical improvements, which are
typically attributed to end-to-end making better error tradeoffs than the
standard loss function used in a two-stage solution. We refine this explanation
and more precisely characterize when end-to-end can improve performance. When
prediction targets are stochastic, a two-stage solution must make an a priori
choice about which statistics of the target distribution to model-we consider
expectations over prediction targets-while an end-to-end solution can make this
choice adaptively. We show that the performance gap between a two-stage and
end-to-end approach is closely related to the price of correlation concept in
stochastic optimization and show the implications of some existing POC results
for the predict-then-optimize problem. We then consider a novel and
particularly practical setting, where multiple prediction targets are combined
to obtain each of the objective function's coefficients. We give explicit
constructions where (1) two-stage performs unboundedly worse than end-to-end;
and (2) two-stage is optimal. We use simulations to experimentally quantify
performance gaps and identify a wide range of real-world applications from the
literature whose objective functions rely on multiple prediction targets,
suggesting that end-to-end learning could yield significant improvements.
