Long-range and hierarchical language predictions in brains and
  algorithms
  Deep learning has recently made remarkable progress in natural language
processing. Yet, the resulting algorithms remain far from competing with the
language abilities of the human brain. Predictive coding theory offers a
potential explanation to this discrepancy: while deep language algorithms are
optimized to predict adjacent words, the human brain would be tuned to make
long-range and hierarchical predictions. To test this hypothesis, we analyze
the fMRI brain signals of 304 subjects each listening to 70min of short
stories. After confirming that the activations of deep language algorithms
linearly map onto those of the brain, we show that enhancing these models with
long-range forecast representations improves their brain-mapping. The results
further reveal a hierarchy of predictions in the brain, whereby the
fronto-parietal cortices forecast more abstract and more distant
representations than the temporal cortices. Overall, this study strengthens
predictive coding theory and suggests a critical role of long-range and
hierarchical predictions in natural language processing.
