Learning Positional Embeddings for Coordinate-MLPs
  We propose a novel method to enhance the performance of coordinate-MLPs by
learning instance-specific positional embeddings. End-to-end optimization of
positional embedding parameters along with network weights leads to poor
generalization performance. Instead, we develop a generic framework to learn
the positional embedding based on the classic graph-Laplacian regularization,
which can implicitly balance the trade-off between memorization and
generalization. This framework is then used to propose a novel positional
embedding scheme, where the hyperparameters are learned per coordinate (i.e,
instance) to deliver optimal performance. We show that the proposed embedding
achieves better performance with higher stability compared to the
well-established random Fourier features (RFF). Further, we demonstrate that
the proposed embedding scheme yields stable gradients, enabling seamless
integration into deep architectures as intermediate layers.
