The Differentiable Cross-Entropy Method
  We study the cross-entropy method (CEM) for the non-convex optimization of a
continuous and parameterized objective function and introduce a differentiable
variant that enables us to differentiate the output of CEM with respect to the
objective function's parameters. In the machine learning setting this brings
CEM inside of the end-to-end learning pipeline where this has otherwise been
impossible. We show applications in a synthetic energy-based structured
prediction task and in non-convex continuous control. In the control setting we
show how to embed optimal action sequences into a lower-dimensional space. DCEM
enables us to fine-tune CEM-based controllers with policy optimization.
