{'fraction_non_alphanumeric': 0.028994845360824743, 'fraction_numerical': 0.004510309278350515, 'mean_word_length': 5.162698412698413, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A model of cerebellar automation of voluntary basal-ganglia control

Cerebellar learning can be roughly categorized as supervised learning, while reinforcement learning is used as a model of basal-ganglia function [1]. Basal-ganglia learn voluntary actions by trial and error [2], whereas cerebellum is among other things specialized in automation and fine tuning of motor control [3]. Cerebellum is often modeled as a feedback error learner, where the output signal of a non-adaptive feedback controller is used as a teaching signal for the adaptive feedforward controller [4]. Biologically this teaching signal can be thought of as a reflex. In this scheme, learning new tasks would require a new handcrafted reflex signal every time. Moreover, designing workable reflex signal becomes increasingly tedious with the growing task complexity. This can be circumvented by using a reinforcement learner to learn a coarse version of the required feedback controller from onedimensional reward signal [5]. Here we use a basal-ganglia-style actor-critic algorithm [6], instead of a hardwired reflex, in concert with the cerebellar predictor. Addition of the cerebellar model can speed up the learning in a typically slow reinforcement based algorithm. Actor-critic algorithms and cerebellar models have traditionally been studied separately (but see [5]). In a combined model, the role of cerebellum overlaps with the actor part of the reinforcement learning algorithm. Our goal is to learn, how the division of labor between the modules could be optimized.