SeaPearl: A Constraint Programming Solver guided by Reinforcement
  Learning
  The design of efficient and generic algorithms for solving combinatorial
optimization problems has been an active field of research for many years.
Standard exact solving approaches are based on a clever and complete
enumeration of the solution set. A critical and non-trivial design choice with
such methods is the branching strategy, directing how the search is performed.
The last decade has shown an increasing interest in the design of machine
learning-based heuristics to solve combinatorial optimization problems. The
goal is to leverage knowledge from historical data to solve similar new
instances of a problem. Used alone, such heuristics are only able to provide
approximate solutions efficiently, but cannot prove optimality nor bounds on
their solution. Recent works have shown that reinforcement learning can be
successfully used for driving the search phase of constraint programming (CP)
solvers. However, it has also been shown that this hybridization is challenging
to build, as standard CP frameworks do not natively include machine learning
mechanisms, leading to some sources of inefficiencies. This paper presents the
proof of concept for SeaPearl, a new CP solver implemented in Julia, that
supports machine learning routines in order to learn branching decisions using
reinforcement learning. Support for modeling the learning component is also
provided. We illustrate the modeling and solution performance of this new
solver on two problems. Although not yet competitive with industrial solvers,
SeaPearl aims to provide a flexible and open-source framework in order to
facilitate future research in the hybridization of constraint programming and
machine learning.
