Natural Language Statistical Features of LSTM-generated Texts
  Long Short-Term Memory (LSTM) networks have recently shown remarkable
performance in several tasks dealing with natural language generation, such as
image captioning or poetry composition. Yet, only few works have analyzed text
generated by LSTMs in order to quantitatively evaluate to which extent such
artificial texts resemble those generated by humans. We compared the
statistical structure of LSTM-generated language to that of written natural
language, and to those produced by Markov models of various orders. In
particular, we characterized the statistical structure of language by assessing
word-frequency statistics, long-range correlations, and entropy measures. Our
main finding is that while both LSTM and Markov-generated texts can exhibit
features similar to real ones in their word-frequency statistics and entropy
measures, LSTM-texts are shown to reproduce long-range correlations at scales
comparable to those found in natural language. Moreover, for LSTM networks a
temperature-like parameter controlling the generation process shows an optimal
value---for which the produced texts are closest to real language---consistent
across all the different statistical features investigated.
