{'fraction_non_alphanumeric': 0.021425628347754428, 'fraction_numerical': 0.0057684384013185, 'mean_word_length': 5.322916666666667, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Learning to Plan in Complex Stochastic Domains

Probabilistic planning offers a powerful framework for general problem solving. Historically, probabilistic planning algorithms have contributed to a variety of critical application areas and technologies, including conservation biology [42], self-driving cars [48, 37], and space exploration [14, 6, 20]. However, optimal planning is known to be P-Complete with respect to the size of the state-action space [41], imposing a harsh constraint on the types of problems that may be solved in real time. One way of enabling probabilistic planning algorithms to efficiently solve more complex problems is to provide knowledge about the task of interest; this knowledge might be in the form of an abstracted representation that reduces the dimensionality of the problem, or a heuristic that biases exploration toward the optimal solution. Unfortunately, these forms of knowledge are highly specific to the particular problem instance, and often require significant reworking in order to transfer between slight variants of the same task. As a result, each new problem instance requires a newly engineered body of knowledge. Alternatively, we propose learning to plan, in which planners acquire useful domain knowledge about how to solve families of related problems from a small training set of tasks, eliminating the need for hand engineering knowledge. The critical insight is that problems that are too complex to solve efficiently often resemble much simpler problems for which optimal solutions may be computed. By extracting relevant characteristics of the simple problems’ solutions, we develop algorithms to solve the more complex problems by learning about the structure of optimal behavior in the training tasks. In particular, we introduce goal-based action priors [2], that guide planners according to which actions are likely to be useful under different conditions. The priors are informed during a training stage in which simple, tractable tasks are solved, and whose solutions inform the planner about optimal behavior in much more complex tasks from the same domain. We demonstrate that goal-based action priors dramatically reduce the time taken to find a near-optimal plan compared to baselines, and suggest that learning to plan is a compelling means of scaling planning algorithms to solve families of complex tasks without the need for hand engineered knowledge.