{'fraction_non_alphanumeric': 0.01567398119122257, 'fraction_numerical': 0.0, 'mean_word_length': 5.435483870967742, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Vision paper: reinforcement learning in smart spatio-temporal environments

Smart cities offer more and more real-time information provided by sensor networks and traffic cameras. This information can be very valuable for transportation planing. For instance, knowing which parking spots are currently available close to my destination is very valuable in order to reduce the travel time and thus, maximize the resource usage and minimize the traffic load. The future development of this information is usually uncertain. However, algorithms for routing applications should consider that new information will become available during travelling along the computed path. In order to exploit the provided information to a full extend, it is not sufficient to compute a static route or travel plan because the optimality of the plan might degrade as the state of the environment might consistently change. We argue that in order to plan transportation and understand observed trajectories in smart environments, it is necessary to compute action policies (i.e. the most promising action for all situations and in particular, the encountered situations) instead of static routes. Analogously, it makes sense to understand human behaviour based on the sequence of decisions in the encountered situations. To compute and analyse policies, the field of reinforcement learning already provides a rich set of tools. We describe existing approaches of reinforcement learning in spatial tasks and highlight which new challenges arise from developing reinforcement learning techniques for smart environments.