Probabilistic partition of unity networks: clustering based deep
  approximation
  Partition of unity networks (POU-Nets) have been shown capable of realizing
algebraic convergence rates for regression and solution of PDEs, but require
empirical tuning of training parameters. We enrich POU-Nets with a Gaussian
noise model to obtain a probabilistic generalization amenable to gradient-based
minimization of a maximum likelihood loss. The resulting architecture provides
spatial representations of both noiseless and noisy data as Gaussian mixtures
with closed form expressions for variance which provides an estimator of local
error. The training process yields remarkably sharp partitions of input space
based upon correlation of function values. This classification of training
points is amenable to a hierarchical refinement strategy that significantly
improves the localization of the regression, allowing for higher-order
polynomial approximation to be utilized. The framework scales more favorably to
large data sets as compared to Gaussian process regression and allows for
spatially varying uncertainty, leveraging the expressive power of deep neural
networks while bypassing expensive training associated with other probabilistic
deep learning methods. Compared to standard deep neural networks, the framework
demonstrates hp-convergence without the use of regularizers to tune the
localization of partitions. We provide benchmarks quantifying performance in
high/low-dimensions, demonstrating that convergence rates depend only on the
latent dimension of data within high-dimensional space. Finally, we introduce a
new open-source data set of PDE-based simulations of a semiconductor device and
perform unsupervised extraction of a physically interpretable reduced-order
basis.
