Understanding the World Through Action
  The recent history of machine learning research has taught us that machine
learning methods can be most effective when they are provided with very large,
high-capacity models, and trained on very large and diverse datasets. This has
spurred the community to search for ways to remove any bottlenecks to scale.
Often the foremost among such bottlenecks is the need for human effort,
including the effort of curating and labeling datasets. As a result,
considerable attention in recent years has been devoted to utilizing unlabeled
data, which can be collected in vast quantities. However, some of the most
widely used methods for training on such unlabeled data themselves require
human-designed objective functions that must correlate in some meaningful way
to downstream tasks. I will argue that a general, principled, and powerful
framework for utilizing unlabeled data can be derived from reinforcement
learning, using general purpose unsupervised or self-supervised reinforcement
learning objectives in concert with offline reinforcement learning methods that
can leverage large datasets. I will discuss how such a procedure is more
closely aligned with potential downstream tasks, and how it could build on
existing techniques that have been developed in recent years.
