The MineRL BASALT Competition on Learning from Human Feedback
  The last decade has seen a significant increase of interest in deep learning
research, with many public successes that have demonstrated its potential. As
such, these systems are now being incorporated into commercial products. With
this comes an additional challenge: how can we build AI systems that solve
tasks where there is not a crisp, well-defined specification? While multiple
solutions have been proposed, in this competition we focus on one in
particular: learning from human feedback. Rather than training AI systems using
a predefined reward function or using a labeled dataset with a predefined set
of categories, we instead train the AI system using a learning signal derived
from some form of human feedback, which can evolve over time as the
understanding of the task changes, or as the capabilities of the AI system
improve.
  The MineRL BASALT competition aims to spur forward research on this important
class of techniques. We design a suite of four tasks in Minecraft for which we
expect it will be hard to write down hardcoded reward functions. These tasks
are defined by a paragraph of natural language: for example, "create a
waterfall and take a scenic picture of it", with additional clarifying details.
Participants must train a separate agent for each task, using any method they
want. Agents are then evaluated by humans who have read the task description.
To help participants get started, we provide a dataset of human demonstrations
on each of the four tasks, as well as an imitation learning baseline that
leverages these demonstrations.
  Our hope is that this competition will improve our ability to build AI
systems that do what their designers intend them to do, even when the intent
cannot be easily formalized. Besides allowing AI to solve more tasks, this can
also enable more effective regulation of AI systems, as well as making progress
on the value alignment problem.
