Level set learning with pseudo-reversible neural networks for nonlinear
  dimension reduction in function approximation
  Due to the curse of dimensionality and the limitation on training data,
approximating high-dimensional functions is a very challenging task even for
powerful deep neural networks. Inspired by the Nonlinear Level set Learning
(NLL) method that uses the reversible residual network (RevNet), in this paper
we propose a new method of Dimension Reduction via Learning Level Sets (DRiLLS)
for function approximation. Our method contains two major components: one is
the pseudo-reversible neural network (PRNN) module that effectively transforms
high-dimensional input variables to low-dimensional active variables, and the
other is the synthesized regression module for approximating function values
based on the transformed data in the low-dimensional space. The PRNN not only
relaxes the invertibility constraint of the nonlinear transformation present in
the NLL method due to the use of RevNet, but also adaptively weights the
influence of each sample and controls the sensitivity of the function to the
learned active variables. The synthesized regression uses Euclidean distance in
the input space to select neighboring samples, whose projections on the space
of active variables are used to perform local least-squares polynomial fitting.
This helps to resolve numerical oscillation issues present in traditional local
and global regressions. Extensive experimental results demonstrate that our
DRiLLS method outperforms both the NLL and Active Subspace methods, especially
when the target function possesses critical points in the interior of its input
domain.
