KeyPose: Multi-View 3D Labeling and Keypoint Estimation for Transparent
  Objects
  Estimating the 3D pose of desktop objects is crucial for applications such as
robotic manipulation. Many existing approaches to this problem require a depth
map of the object for both training and prediction, which restricts them to
opaque, lambertian objects that produce good returns in an RGBD sensor. In this
paper we forgo using a depth sensor in favor of raw stereo input. We address
two problems: first, we establish an easy method for capturing and labeling 3D
keypoints on desktop objects with an RGB camera; and second, we develop a deep
neural network, called $KeyPose$, that learns to accurately predict object
poses using 3D keypoints, from stereo input, and works even for transparent
objects. To evaluate the performance of our method, we create a dataset of 15
clear objects in five classes, with 48K 3D-keypoint labeled images. We train
both instance and category models, and show generalization to new textures,
poses, and objects. KeyPose surpasses state-of-the-art performance in 3D pose
estimation on this dataset by factors of 1.5 to 3.5, even in cases where the
competing method is provided with ground-truth depth. Stereo input is essential
for this performance as it improves results compared to using monocular input
by a factor of 2. We will release a public version of the data capture and
labeling pipeline, the transparent object database, and the KeyPose models and
evaluation code. Project website: https://sites.google.com/corp/view/keypose.
