{'fraction_non_alphanumeric': 0.017154389505549948, 'fraction_numerical': 0.0, 'mean_word_length': 5.048780487804878, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Organisation of Past States in Recurrent Neural Networks: Implicit Embedding

Non{relaxing recurrent neural networks (RNNs) generalize feedfor-ward neural networks (FFNNs) in a straightforward way. However, this change largely innuences the usage of the neural network, for example the training, and the capabilities of the network. Due to their inherent dynamics RNNs are often used for prediction and modeling of dynamical systems. The internal dynamics of the network can be used to represent information about the past whereas in FFNNs a vector of the relevant past has to be built beforehand and passed to the network as input. The aim of this paper is to highlight the dependence of the reconstruction on the model which is used for the prediction and to propose an alternative approach to embedding based on recurrent neural networks. We analyse the reconstruction by RNNs, which is done implicitly by exploiting the ability of the network to represent past states in its activations.