Learning shared neural manifolds from multi-subject FMRI data
  Functional magnetic resonance imaging (fMRI) is a notoriously noisy
measurement of brain activity because of the large variations between
individuals, signals marred by environmental differences during collection, and
spatiotemporal averaging required by the measurement resolution. In addition,
the data is extremely high dimensional, with the space of the activity
typically having much lower intrinsic dimension. In order to understand the
connection between stimuli of interest and brain activity, and analyze
differences and commonalities between subjects, it becomes important to learn a
meaningful embedding of the data that denoises, and reveals its intrinsic
structure. Specifically, we assume that while noise varies significantly
between individuals, true responses to stimuli will share common,
low-dimensional features between subjects which are jointly discoverable.
Similar approaches have been exploited previously but they have mainly used
linear methods such as PCA and shared response modeling (SRM). In contrast, we
propose a neural network called MRMD-AE (manifold-regularized multiple decoder,
autoencoder), that learns a common embedding from multiple subjects in an
experiment while retaining the ability to decode to individual raw fMRI
signals. We show that our learned common space represents an extensible
manifold (where new points not seen during training can be mapped), improves
the classification accuracy of stimulus features of unseen timepoints, as well
as improves cross-subject translation of fMRI signals. We believe this
framework can be used for many downstream applications such as guided
brain-computer interface (BCI) training in the future.
