Data Engineering for Everyone
  Data engineering is one of the fastest-growing fields within machine learning
(ML). As ML becomes more common, the appetite for data grows more ravenous. But
ML requires more data than individual teams of data engineers can readily
produce, which presents a severe challenge to ML deployment at scale. Much like
the software-engineering revolution, where mass adoption of open-source
software replaced the closed, in-house development model for infrastructure
code, there is a growing need to enable rapid development and open contribution
to massive machine learning data sets. This article shows that open-source data
sets are the rocket fuel for research and innovation at even some of the
largest AI organizations. Our analysis of nearly 2000 research publications
from Facebook, Google and Microsoft over the past five years shows the
widespread use and adoption of open data sets. Open data sets that are easily
accessible to the public are vital to accelerating ML innovation for everyone.
But such open resources are scarce in the wild. So, what if we are able to
accelerate data-set creation via automatic data set generation tools?
