Towards Geometry Guided Neural Relighting with Flash Photography
  Previous image based relighting methods require capturing multiple images to
acquire high frequency lighting effect under different lighting conditions,
which needs nontrivial effort and may be unrealistic in certain practical use
scenarios. While such approaches rely entirely on cleverly sampling the color
images under different lighting conditions, little has been done to utilize
geometric information that crucially influences the high-frequency features in
the images, such as glossy highlight and cast shadow. We therefore propose a
framework for image relighting from a single flash photograph with its
corresponding depth map using deep learning. By incorporating the depth map,
our approach is able to extrapolate realistic high-frequency effects under
novel lighting via geometry guided image decomposition from the flashlight
image, and predict the cast shadow map from the shadow-encoding transformed
depth map. Moreover, the single-image based setup greatly simplifies the data
capture process. We experimentally validate the advantage of our geometry
guided approach over state-of-the-art image-based approaches in intrinsic image
decomposition and image relighting, and also demonstrate our performance on
real mobile phone photo examples.
