{'fraction_non_alphanumeric': 0.03605015673981191, 'fraction_numerical': 0.002612330198537095, 'mean_word_length': 5.51360544217687, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Habituation and goal-directed arbitration mechanisms and failures under partial observability

We often need to make decisions under incomplete information (partial observability) and the brain manages to add the right minimal context to the decision-making. Partial observability may also be handled by other mechanisms than adding contextual experience / memory. We propose that parallel and sequential arbitration of Habituation (Model-Free, MF) and Goal-Directed (Model-Based, MB) behavior may be at play to deal with partial observability “on-the-fly”, and that MB may be of different types (going beyond the MF/MB dichotomy [4]). To illustrate this, we identify, describe and model with Reinforcement Learning (RL) a behavioral anomaly (an habituation failure) occurring during the so-called Hotel Elevators Rows (HER, for short) task: a prototypical partial observation situation that can be reduced to the well studied Two and One Sequence Choice Tasks. The following hypothesis are supported by RL simulation results: (1) a parallel (semi)model-based successor representation mechanism is operative while learning to habituate which detects model-based mismatches and serves as an habituation surveillance, (2) a retrospective inference is triggered to identify the source of the habituation failure (3) a model-free mechanism can trigger model-based mechanisms in states in which habituation failed. The “failures” in the title refer to: the habituation failures that need to be monitored and surveilled (1) and to the failures that we identified in prototypical state of the art Model-Based algorithms (like DynaQ) when facing partial observability. As other research on MF/MB arbitration shows, the identification of these new mechanisms could shine light into new treatments for addiction, compulsive behavior (like compulsive checking) and understand better accidents caused by habituation behaviors.