Learning to Play Cup-and-Ball with Noisy Camera Observations
  Playing the cup-and-ball game is an intriguing task for robotics research
since it abstracts important problem characteristics including system
nonlinearity, contact forces and precise positioning as terminal goal. In this
paper, we present a learning model based control strategy for the cup-and-ball
game, where a Universal Robots UR5e manipulator arm learns to catch a ball in
one of the cups on a Kendama. Our control problem is divided into two
sub-tasks, namely $(i)$ swinging the ball up in a constrained motion, and
$(ii)$ catching the free-falling ball. The swing-up trajectory is computed
offline, and applied in open-loop to the arm. Subsequently, a convex
optimization problem is solved online during the ball's free-fall to control
the manipulator and catch the ball. The controller utilizes noisy position
feedback of the ball from an Intel RealSense D435 depth camera. We propose a
novel iterative framework, where data is used to learn the support of the
camera noise distribution iteratively in order to update the control policy.
The probability of a catch with a fixed policy is computed empirically with a
user specified number of roll-outs. Our design guarantees that probability of
the catch increases in the limit, as the learned support nears the true support
of the camera noise distribution. High-fidelity Mujoco simulations and
preliminary experimental results support our theoretical analysis.
