SmartChoices: Hybridizing Programming and Machine Learning
  We present SmartChoices, an approach to making machine learning (ML) a first
class citizen in programming languages which we see as one way to lower the
entrance cost to applying ML to problems in new domains. There is a growing
divide in approaches to building systems: on the one hand, programming
leverages human experts to define a system while on the other hand behavior is
learned from data in machine learning. We propose to hybridize these two by
providing a 3-call API which we expose through an object called SmartChoice. We
describe the SmartChoices-interface, how it can be used in programming with
minimal code changes, and demonstrate that it is an easy to use but still
powerful tool by demonstrating improvements over not using ML at all on three
algorithmic problems: binary search, QuickSort, and caches. In these three
examples, we replace the commonly used heuristics with an ML model entirely
encapsulated within a SmartChoice and thus requiring minimal code changes. As
opposed to previous work applying ML to algorithmic problems, our proposed
approach does not require to drop existing implementations but seamlessly
integrates into the standard software development workflow and gives full
control to the software developer over how ML methods are applied. Our
implementation relies on standard Reinforcement Learning (RL) methods. To learn
faster, we use the heuristic function, which they are replacing, as an initial
function. We show how this initial function can be used to speed up and
stabilize learning while providing a safety net that prevents performance to
become substantially worse -- allowing for a safe deployment in critical
applications in real life.
