Symbolic Regression using Mixed-Integer Nonlinear Optimization
  The Symbolic Regression (SR) problem, where the goal is to find a regression
function that does not have a pre-specified form but is any function that can
be composed of a list of operators, is a hard problem in machine learning, both
theoretically and computationally. Genetic programming based methods, that
heuristically search over a very large space of functions, are the most
commonly used methods to tackle SR problems. An alternative mathematical
programming approach, proposed in the last decade, is to express the optimal
symbolic expression as the solution of a system of nonlinear equations over
continuous and discrete variables that minimizes a certain objective, and to
solve this system via a global solver for mixed-integer nonlinear programming
problems. Algorithms based on the latter approach are often very slow. We
propose a hybrid algorithm that combines mixed-integer nonlinear optimization
with explicit enumeration and incorporates constraints from dimensional
analysis. We show that our algorithm is competitive, for some synthetic data
sets, with a state-of-the-art SR software and a recent physics-inspired method
called AI Feynman.
