3D Hand Shape and Pose from Images in the Wild
  We present in this work the first end-to-end deep learning based method that
predicts both 3D hand shape and pose from RGB images in the wild. Our network
consists of the concatenation of a deep convolutional encoder, and a fixed
model-based decoder. Given an input image, and optionally 2D joint detections
obtained from an independent CNN, the encoder predicts a set of hand and view
parameters. The decoder has two components: A pre-computed articulated mesh
deformation hand model that generates a 3D mesh from the hand parameters, and a
re-projection module controlled by the view parameters that projects the
generated hand into the image domain. We show that using the shape and pose
prior knowledge encoded in the hand model within a deep learning framework
yields state-of-the-art performance in 3D pose prediction from images on
standard benchmarks, and produces geometrically valid and plausible 3D
reconstructions. Additionally, we show that training with weak supervision in
the form of 2D joint annotations on datasets of images in the wild, in
conjunction with full supervision in the form of 3D joint annotations on
limited available datasets allows for good generalization to 3D shape and pose
predictions on images in the wild.
