Deep driven fMRI decoding of visual categories
  Deep neural networks have been developed drawing inspiration from the brain
visual pathway, implementing an end-to-end approach: from image data to video
object classes. However building an fMRI decoder with the typical structure of
Convolutional Neural Network (CNN), i.e. learning multiple level of
representations, seems impractical due to lack of brain data. As a possible
solution, this work presents the first hybrid fMRI and deep features decoding
approach: collected fMRI and deep learnt representations of video object
classes are linked together by means of Kernel Canonical Correlation Analysis.
In decoding, this allows exploiting the discriminatory power of CNN by relating
the fMRI representation to the last layer of CNN (fc7). We show the
effectiveness of embedding fMRI data onto a subspace related to deep features
in distinguishing semantic visual categories based solely on brain imaging
data.
