Single-step deep reinforcement learning for open-loop control of laminar
  and turbulent flows
  This research gauges the ability of deep reinforcement learning (DRL)
techniques to assist the optimization and control of fluid mechanical systems.
It combines a novel, "degenerate" version of the proximal policy optimization
(PPO) algorithm, that trains a neural network in optimizing the system only
once per learning episode, and an in-house stabilized finite elements
environment implementing the variational multiscale (VMS) method, that computes
the numerical reward fed to the neural network. Three prototypical examples of
separated flows in two dimensions are used as testbed for developing the
methodology, each of which adds a layer of complexity due either to the
unsteadiness of the flow solutions, or the sharpness of the objective function,
or the dimension of the control parameter space. Relevance is carefully
assessed by comparing systematically to reference data obtained by canonical
direct and adjoint methods. Beyond adding value to the shallow literature on
this subject, these findings establish the potential of single-step PPO for
reliable black-box optimization of computational fluid dynamics (CFD) systems,
which paves the way for future progress in optimal flow control using this new
class of methods.
