Post-Workshop Report on Science meets Engineering in Deep Learning,
  NeurIPS 2019, Vancouver
  Science meets Engineering in Deep Learning took place in Vancouver as part of
the Workshop section of NeurIPS 2019. As organizers of the workshop, we created
the following report in an attempt to isolate emerging topics and recurring
themes that have been presented throughout the event. Deep learning can still
be a complex mix of art and engineering despite its tremendous success in
recent years. The workshop aimed at gathering people across the board to
address seemingly contrasting challenges in the problems they are working on.
As part of the call for the workshop, particular attention has been given to
the interdependence of architecture, data, and optimization that gives rise to
an enormous landscape of design and performance intricacies that are not
well-understood. This year, our goal was to emphasize the following directions
in our community: (i) identify obstacles in the way to better models and
algorithms; (ii) identify the general trends from which we would like to build
scientific and potentially theoretical understanding; and (iii) the rigorous
design of scientific experiments and experimental protocols whose purpose is to
resolve and pinpoint the origin of mysteries while ensuring reproducibility and
robustness of conclusions. In the event, these topics emerged and were broadly
discussed, matching our expectations and paving the way for new studies in
these directions. While we acknowledge that the text is naturally biased as it
comes through our lens, here we present an attempt to do a fair job of
highlighting the outcome of the workshop.
