Reinforcement Learning for Mixed-Integer Problems Based on MPC
  Model Predictive Control has been recently proposed as policy approximation
for Reinforcement Learning, offering a path towards safe and explainable
Reinforcement Learning. This approach has been investigated for Q-learning and
actor-critic methods, both in the context of nominal Economic MPC and Robust
(N)MPC, showing very promising results. In that context, actor-critic methods
seem to be the most reliable approach. Many applications include a mixture of
continuous and integer inputs, for which the classical actor-critic methods
need to be adapted. In this paper, we present a policy approximation based on
mixed-integer MPC schemes, and propose a computationally inexpensive technique
to generate exploration in the mixed-integer input space that ensures a
satisfaction of the constraints. We then propose a simple compatible advantage
function approximation for the proposed policy, that allows one to build the
gradient of the mixed-integer MPC-based policy.
