Learning Reward Functions from Scale Feedback
  Today's robots are increasingly interacting with people and need to
efficiently learn inexperienced user's preferences. A common framework is to
iteratively query the user about which of two presented robot trajectories they
prefer. While this minimizes the users effort, a strict choice does not yield
any information on how much one trajectory is preferred. We propose scale
feedback, where the user utilizes a slider to give more nuanced information. We
introduce a probabilistic model on how users would provide feedback and derive
a learning framework for the robot. We demonstrate the performance benefit of
slider feedback in simulations, and validate our approach in two user studies
suggesting that scale feedback enables more effective learning in practice.
