The Neurally-Guided Shape Parser: Grammar-based Labeling of 3D Shape
  Regions with Approximate Inference
  We propose the Neurally-Guided Shape Parser (NGSP), a method that learns how
to assign fine-grained semantic labels to regions of a 3D shape. NGSP solves
this problem via MAP inference, modeling the posterior probability of a label
assignment conditioned on an input shape with a learned likelihood function. To
make this search tractable, NGSP employs a neural guide network that learns to
approximate the posterior. NGSP finds high-probability label assignments by
first sampling proposals with the guide network and then evaluating each
proposal under the full likelihood. We evaluate NGSP on the task of
fine-grained semantic segmentation of manufactured 3D shapes from PartNet,
where shapes have been decomposed into regions that correspond to part instance
over-segmentations. We find that NGSP delivers significant performance
improvements over comparison methods that (i) use regions to group per-point
predictions, (ii) use regions as a self-supervisory signal or (iii) assign
labels to regions under alternative formulations. Further, we show that NGSP
maintains strong performance even with limited labeled data or noisy input
shape regions. Finally, we demonstrate that NGSP can be directly applied to CAD
shapes found in online repositories and validate its effectiveness with a
perceptual study.
