{'fraction_non_alphanumeric': 0.015276893698281349, 'fraction_numerical': 0.002546148949713558, 'mean_word_length': 5.364372469635628, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Temporal coding in neural networks

The 'rules' by which information is temporally encoded within biological and artificial neural networks remains poorly understood. To better understand the 'problem' we tested the temporal encoding and extrapolation (generalization) of artificial neural networks on the spiking patterns of 55 simultaneously recorded neurons. The results indicated that to optimize performance a 4-layer network works best both for a feedforward and a recurrent architecture. Further, the results indicate that both hidden layers should have roughly 2 n units and that the learning rate between the input and first hidden layer should be roughly an order of magnitude larger than the learning rates for subsequent layers. Interestingly, the number of hidden units in the final network is consistent with the known histology and function of the neural tissue from which the recordings were obtained. The results suggest that the fan-out architectures typically found in this neural tissue may be crucial to the ability of this organism to function well under novel conditions. And, these results indicate that temporal data may be encoded best in a highly distributed fashion across large numbers of hidden units where the weighted interconnections are largest, more deterministic, in the early layers and smallest, more statistical, in the final layers of the network. Further, the same architectural guidelines developed herein are shown to apply directly to other temporal problems where the information is encoded in a spatially distributed fashion.