Interactive Annotation of 3D Object Geometry using 2D Scribbles
  Inferring detailed 3D geometry of the scene is crucial for robotics
applications, simulation, and 3D content creation. However, such information is
hard to obtain, and thus very few datasets support it. In this paper, we
propose an interactive framework for annotating 3D object geometry from both
point cloud data and RGB imagery. The key idea behind our approach is to
exploit strong priors that humans have about the 3D world in order to
interactively annotate complete 3D shapes. Our framework targets naive users
without artistic or graphics expertise. We introduce two simple-to-use
interaction modules. First, we make an automatic guess of the 3D shape and
allow the user to provide feedback about large errors by drawing scribbles in
desired 2D views. Next, we aim to correct minor errors, in which users drag and
drop mesh vertices, assisted by a neural interactive module implemented as a
Graph Convolutional Network. Experimentally, we show that only a few user
interactions are needed to produce good quality 3D shapes on popular benchmarks
such as ShapeNet, Pix3D and ScanNet. We implement our framework as a web
service and conduct a user study, where we show that user annotated data using
our method effectively facilitates real-world learning tasks. Web service:
http://www.cs.toronto.edu/~shenti11/scribble3d.
