Single Image BRDF Parameter Estimation with a Conditional Adversarial
  Network
  Creating plausible surfaces is an essential component in achieving a high
degree of realism in rendering. To relieve artists, who create these surfaces
in a time-consuming, manual process, automated retrieval of the
spatially-varying Bidirectional Reflectance Distribution Function (SVBRDF) from
a single mobile phone image is desirable. By leveraging a deep neural network,
this casual capturing method can be achieved. The trained network can estimate
per pixel normal, base color, metallic and roughness parameters from the Disney
BRDF. The input image is taken with a mobile phone lit by the camera flash. The
network is trained to compensate for environment lighting and thus learned to
reduce artifacts introduced by other light sources. These losses contain a
multi-scale discriminator with an additional perceptual loss, a rendering loss
using a differentiable renderer, and a parameter loss. Besides the local
precision, this loss formulation generates material texture maps which are
globally more consistent. The network is set up as a generator network trained
in an adversarial fashion to ensure that only plausible maps are produced. The
estimated parameters not only reproduce the material faithfully in rendering
but capture the style of hand-authored materials due to the more global loss
terms compared to previous works without requiring additional post-processing.
Both the resolution and the quality is improved.
