{'fraction_non_alphanumeric': 0.019143117593436645, 'fraction_numerical': 0.0018231540565177757, 'mean_word_length': 5.114206128133705, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The unified Reservoir Computing concept and its digital hardware implementations

Recurrent Neural Networks (RNNs) are able by construction to solve temporal problems, such as the control or modelling of non-linear processes. This is due to the recurrent connections, which allow information to be retained for a period of time inside the network. However, this recurrent topology also greatly increases the complexity of the dynamic behaviour of these networks. It is a well known fact that the presence of feedback inside a dynamic system makes it prone to unstable behaviour. This dynamical complexity is for a large part the reason why current learning rules for these types of networks suffer from slow training speeds and convergence problems [9]. Recently, three elegant solutions to this problem have been proposed independently under the terms Liquid State Machine [4], Echo State Machine [1] and Backpropagation Decorrelation (BPDC) [7]. Each of these techniques avoids the training problem of a RNN, while still being able to use its powerful temporal processing capabilities, by using the – randomly chosen – network as a reservoir whose weights are not changed during the training phase. Instead, the response of the reservoir to a certain input is observed from the outside by a conventional and simple classification or regression algorithm that is far easier to train. These approaches conveniently combine the shortterm memory property of the reservoir with the ease of training and fast convergence of a very simple linear readout function. The reservoir acts as a complex kernel-type filter that computes many random nonlinear combinations of the current and past inputs, with a finite memory. The memoryless readout function is then able to linearly combine this information to compute the actual output. The three methods cited above all take a different approach towards the idea of using a RNN as reservoir. The LSM offers different types of reservoirs, built from very simple nodes like threshold logic gates, to complex and biologically realistic Leaky Integrate & Fire neurons. Echo State Networks are built from sigmoidal neurons and are more directed towards practi-