Stabilising viscous extensional flows using Reinforcement Learning
  The four-roll mill, wherein four identical cylinders undergo rotation of
identical magnitude but alternate signs, was originally proposed by GI Taylor
to create local extensional flows and study their ability to deform small
liquid drops. Since an extensional flow has an unstable eigendirection, a drop
located at the flow stagnation point will have a tendency to escape. This
unstable dynamics can however be stabilised using, e.g., a modulation of the
rotation rates of the cylinders. Here we use Reinforcement Learning, a branch
of Machine Learning devoted to the optimal selection of actions based on
cumulative rewards, in order to devise a stabilisation algorithm for the
four-roll mill flow. The flow is modelled as the linear superposition of four
two-dimensional rotlets and the drop is treated as a rigid spherical particle
smaller than all other length scales in the problem. Unlike previous attempts
to devise control, we take a probabilistic approach whereby speed adjustments
are drawn from a probability density function whose shape is improved over time
via a form of gradient ascent know as Actor-Critic method. With enough
training, our algorithm is able to precisely control the drop and keep it close
to the stagnation point for as long as needed. We explore the impact of the
physical and learning parameters on the effectiveness of the control and
demonstrate the robustness of the algorithm against thermal noise. We finally
show that Reinforcement Learning can provide a control algorithm effective for
all initial positions and that can be adapted to limit the magnitude of the
flow extension near the position of the drop.
