SurfaceNet+: An End-to-end 3D Neural Network for Very Sparse Multi-view
  Stereopsis
  Multi-view stereopsis (MVS) tries to recover the 3D model from 2D images. As
the observations become sparser, the significant 3D information loss makes the
MVS problem more challenging. Instead of only focusing on densely sampled
conditions, we investigate sparse-MVS with large baseline angles since the
sparser sensation is more practical and more cost-efficient. By investigating
various observation sparsities, we show that the classical depth-fusion
pipeline becomes powerless for the case with a larger baseline angle that
worsens the photo-consistency check. As another line of the solution, we
present SurfaceNet+, a volumetric method to handle the 'incompleteness' and the
'inaccuracy' problems induced by a very sparse MVS setup. Specifically, the
former problem is handled by a novel volume-wise view selection approach. It
owns superiority in selecting valid views while discarding invalid occluded
views by considering the geometric prior. Furthermore, the latter problem is
handled via a multi-scale strategy that consequently refines the recovered
geometry around the region with the repeating pattern. The experiments
demonstrate the tremendous performance gap between SurfaceNet+ and
state-of-the-art methods in terms of precision and recall. Under the extreme
sparse-MVS settings in two datasets, where existing methods can only return
very few points, SurfaceNet+ still works as well as in the dense MVS setting.
The benchmark and the implementation are publicly available at
https://github.com/mjiUST/SurfaceNet-plus.
