The Impact of Hole Geometry on Relative Robustness of In-Painting
  Networks: An Empirical Study
  In-painting networks use existing pixels to generate appropriate pixels to
fill "holes" placed on parts of an image. A 2-D in-painting network's input
usually consists of (1) a three-channel 2-D image, and (2) an additional
channel for the "holes" to be in-painted in that image. In this paper, we study
the robustness of a given in-painting neural network against variations in hole
geometry distributions. We observe that the robustness of an in-painting
network is dependent on the probability distribution function (PDF) of the hole
geometry presented to it during its training even if the underlying image
dataset used (in training and testing) does not alter. We develop an
experimental methodology for testing and evaluating relative robustness of
in-painting networks against four different kinds of hole geometry PDFs. We
examine a number of hypothesis regarding (1) the natural bias of in-painting
networks to the hole distribution used for their training, (2) the underlying
dataset's ability to differentiate relative robustness as hole distributions
vary in a train-test (cross-comparison) grid, and (3) the impact of the
directional distribution of edges in the holes and in the image dataset. We
present results for L1, PSNR and SSIM quality metrics and develop a specific
measure of relative in-painting robustness to be used in cross-comparison grids
based on these quality metrics. (One can incorporate other quality metrics in
this relative measure.) The empirical work reported here is an initial step in
a broader and deeper investigation of "filling the blank" neural networks'
sensitivity, robustness and regularization with respect to hole "geometry"
PDFs, and it suggests further research in this domain.
