{'fraction_non_alphanumeric': 0.02223926380368098, 'fraction_numerical': 0.0023006134969325155, 'mean_word_length': 5.365853658536586, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Learning Hierarchical Plans by Reading Simple English Narratives

We describe an approach for learning a rich plan representation from a parallel corpus of commonsense narratives. Each narrative is an ordered natural language description of the steps required to accomplish common domestic tasks, including “get the mail” and “make a bed”, and there are tens to hundreds of differently written narratives for each task. With the goal of learning a single rich plan structure, we 1) convert each narrative from English statements into a sequence of logical predicates, 2) find a global alignment for the sequences, and 3) use the sequences to construct a single underlying plan representation that can be used in language understanding problems. Doing this requires being able to distinguish different ways to accomplish the same goal from missing information, and recognize and compactly represent recurring plan sub-sequences. We describe a simple algorithm that recursively finds graph cycles by applying rules to merge nodes to learn a sequential, parameterized composition (part-of) and abstraction (is-a) plan hierarchy. We hope that these plan representations will help us learn procedural knowledge from increasingly more sophisticated text, where the sub-goals for various actions are not stated.