Beyond imitation: Zero-shot task transfer on robots by learning concepts
  as cognitive programs
  Humans can infer concepts from image pairs and apply those in the physical
world in a completely different setting, enabling tasks like IKEA assembly from
diagrams. If robots could represent and infer high-level concepts, it would
significantly improve their ability to understand our intent and to transfer
tasks between different environments. To that end, we introduce a computational
framework that replicates aspects of human concept learning. Concepts are
represented as programs on a novel computer architecture consisting of a visual
perception system, working memory, and action controller. The instruction set
of this "cognitive computer" has commands for parsing a visual scene, directing
gaze and attention, imagining new objects, manipulating the contents of a
visual working memory, and controlling arm movement. Inferring a concept
corresponds to inducing a program that can transform the input to the output.
Some concepts require the use of imagination and recursion. Previously learned
concepts simplify the learning of subsequent more elaborate concepts, and
create a hierarchy of abstractions. We demonstrate how a robot can use these
abstractions to interpret novel concepts presented to it as schematic images,
and then apply those concepts in dramatically different situations. By bringing
cognitive science ideas on mental imagery, perceptual symbols, embodied
cognition, and deictic mechanisms into the realm of machine learning, our work
brings us closer to the goal of building robots that have interpretable
representations and commonsense.
