Learning User Preferences via Reinforcement Learning with Spatial
  Interface Valuing
  Interactive Machine Learning is concerned with creating systems that operate
in environments alongside humans to achieve a task. A typical use is to extend
or amplify the capabilities of a human in cognitive or physical ways, requiring
the machine to adapt to the users' intentions and preferences. Often, this
takes the form of a human operator providing some type of feedback to the user,
which can be explicit feedback, implicit feedback, or a combination of both.
Explicit feedback, such as through a mouse click, carries a high cognitive
load. The focus of this study is to extend the current state of the art in
interactive machine learning by demonstrating that agents can learn a human
user's behavior and adapt to preferences with a reduced amount of explicit
human feedback in a mixed feedback setting. The learning agent perceives a
value of its own behavior from hand gestures given via a spatial interface.
This feedback mechanism is termed Spatial Interface Valuing. This method is
evaluated experimentally in a simulated environment for a grasping task using a
robotic arm with variable grip settings. Preliminary results indicate that
learning agents using spatial interface valuing can learn a value function
mapping spatial gestures to expected future rewards much more quickly as
compared to those same agents just receiving explicit feedback, demonstrating
that an agent perceiving feedback from a human user via a spatial interface can
serve as an effective complement to existing approaches.
