Combining Learning from Human Feedback and Knowledge Engineering to
  Solve Hierarchical Tasks in Minecraft
  Real-world tasks of interest are generally poorly defined by human-readable
descriptions and have no pre-defined reward signals unless it is defined by a
human designer. Conversely, data-driven algorithms are often designed to solve
a specific, narrowly defined, task with performance metrics that drives the
agent's learning. In this work, we present the solution that won first place
and was awarded the most human-like agent in the 2021 NeurIPS Competition
MineRL BASALT Challenge: Learning from Human Feedback in Minecraft, which
challenged participants to use human data to solve four tasks defined only by a
natural language description and no reward function. Our approach uses the
available human demonstration data to train an imitation learning policy for
navigation and additional human feedback to train an image classifier. These
modules, combined with an estimated odometry map, become a powerful
state-machine designed to utilize human knowledge in a natural hierarchical
paradigm. We compare this hybrid intelligence approach to both end-to-end
machine learning and pure engineered solutions, which are then judged by human
evaluators. Codebase is available at
https://github.com/viniciusguigo/kairos_minerl_basalt.
