Control of a fly-mimicking flyer in complex flow using deep
  reinforcement learning
  An integrated framework of computational fluid-structural dynamics (CFD-CSD)
and deep reinforcement learning (deep-RL) is developed for control of a
fly-scale flexible-winged flyer in complex flow. Dynamics of the flyer in
complex flow is highly unsteady and nonlinear, which makes modeling the
dynamics challenging. Thus, conventional control methodologies, where the
dynamics is modeled, are insufficient for regulating such complicated dynamics.
Therefore, in the present study, the integrated framework, in which the whole
governing equations for fluid and structure are solved, is proposed to generate
a control policy for the flyer. For the deep-RL to successfully learn the
control policy, accurate and ample data of the dynamics are required. However,
satisfying both the quality and quantity of the data on the intricate dynamics
is extremely difficult since, in general, more accurate data are more costly.
In the present study, two strategies are proposed to deal with the dilemma. To
obtain accurate data, the CFD-CSD is adopted for precisely predicting the
dynamics. To gain ample data, a novel data reproduction method is devised,
where the obtained data are replicated for various situations while conserving
the dynamics. With those data, the framework learns the control policy in
various flow conditions and the learned policy is shown to have remarkable
performance in controlling the flyer in complex flow fields.
