Inverse Design of Grating Couplers Using the Policy Gradient Method from
  Reinforcement Learning
  We present a proof-of-concept technique for the inverse design of
electromagnetic devices motivated by the policy gradient method in
reinforcement learning, named PHORCED (PHotonic Optimization using REINFORCE
Criteria for Enhanced Design). This technique uses a probabilistic generative
neural network interfaced with an electromagnetic solver to assist in the
design of photonic devices, such as grating couplers. We show that PHORCED
obtains better performing grating coupler designs than local gradient-based
inverse design via the adjoint method, while potentially providing faster
convergence over competing state-of-the-art generative methods. As a further
example of the benefits of this method, we implement transfer learning with
PHORCED, demonstrating that a neural network trained to optimize 8$^\circ$
grating couplers can then be re-trained on grating couplers with alternate
scattering angles while requiring >10$\times$ fewer simulations than control
cases.
