Two-shot Spatially-varying BRDF and Shape Estimation
  Capturing the shape and spatially-varying appearance (SVBRDF) of an object
from images is a challenging task that has applications in both computer vision
and graphics. Traditional optimization-based approaches often need a large
number of images taken from multiple views in a controlled environment. Newer
deep learning-based approaches require only a few input images, but the
reconstruction quality is not on par with optimization techniques. We propose a
novel deep learning architecture with a stage-wise estimation of shape and
SVBRDF. The previous predictions guide each estimation, and a joint refinement
network later refines both SVBRDF and shape. We follow a practical mobile image
capture setting and use unaligned two-shot flash and no-flash images as input.
Both our two-shot image capture and network inference can run on mobile
hardware. We also create a large-scale synthetic training dataset with
domain-randomized geometry and realistic materials. Extensive experiments on
both synthetic and real-world datasets show that our network trained on a
synthetic dataset can generalize well to real-world images. Comparisons with
recent approaches demonstrate the superior performance of the proposed
approach.
