Approximate Dynamic Programming with Neural Networks in Linear Discrete
  Action Spaces
  Real-world problems of operations research are typically high-dimensional and
combinatorial. Linear programs are generally used to formulate and efficiently
solve these large decision problems. However, in multi-period decision
problems, we must often compute expected downstream values corresponding to
current decisions. When applying stochastic methods to approximate these
values, linear programs become restrictive for designing value function
approximations (VFAs). In particular, the manual design of a polynomial VFA is
challenging.
  This paper presents an integrated approach for complex optimization problems,
focusing on applications in the domain of operations research. It develops a
hybrid solution method that combines linear programming and neural networks as
part of approximate dynamic programming.
  Our proposed solution method embeds neural network VFAs into linear decision
problems, combining the nonlinear expressive power of neural networks with the
efficiency of solving linear programs. As a proof of concept, we perform
numerical experiments on a transportation problem. The neural network VFAs
consistently outperform polynomial VFAs, with limited design and tuning effort.
