Computing on Functions Using Randomized Vector Representations
  Vector space models for symbolic processing that encode symbols by random
vectors have been proposed in cognitive science and connectionist communities
under the names Vector Symbolic Architecture (VSA), and, synonymously,
Hyperdimensional (HD) computing. In this paper, we generalize VSAs to function
spaces by mapping continuous-valued data into a vector space such that the
inner product between the representations of any two data points represents a
similarity kernel. By analogy to VSA, we call this new function encoding and
computing framework Vector Function Architecture (VFA). In VFAs, vectors can
represent individual data points as well as elements of a function space (a
reproducing kernel Hilbert space). The algebraic vector operations, inherited
from VSA, correspond to well-defined operations in function space. Furthermore,
we study a previously proposed method for encoding continuous data, fractional
power encoding (FPE), which uses exponentiation of a random base vector to
produce randomized representations of data points and fulfills the kernel
properties for inducing a VFA. We show that the distribution from which
elements of the base vector are sampled determines the shape of the FPE kernel,
which in turn induces a VFA for computing with band-limited functions. In
particular, VFAs provide an algebraic framework for implementing large-scale
kernel machines with random features, extending Rahimi and Recht, 2007.
Finally, we demonstrate several applications of VFA models to problems in image
recognition, density estimation and nonlinear regression. Our analyses and
results suggest that VFAs constitute a powerful new framework for representing
and manipulating functions in distributed neural systems, with myriad
applications in artificial intelligence.
