{'fraction_non_alphanumeric': 0.014991896272285251, 'fraction_numerical': 0.0024311183144246355, 'mean_word_length': 5.446475195822455, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An Integrated Approach to Multilingual Hypertext Generation 1

Text planning combines the tasks of content determination and discourse planning in order to construct atext plan from a suitably represented generation goal. The sentence planning step involves such tasks as sentence aggregation, referring expression generation, and sometimes lexicalization, and aims at converting text plans into sentence plans. Linguistic realization, finally, converts the sentence plans into surface text through syntactic, morphological and orthographic1 processing. In this paper, we will describe an approach to natural language generation that collapses these three steps into a single process of text realization, converting a generation goal into a surface text without any intermediate representations. In order to achieve this, we adopt a generalized grammar formalism, where phrase structure and text structure can be described by the same kind of rules, and use pragmatic principles inspired by the Gricean maxims of quality and quantity to guide the application of these rules. We call the approach integrated, first because it integrates text planning, sentence planning and linguistic realization into a single process, but also because it permits the integration of different realization techniques, such as canned text, templates and grammar rules, in a flexible and efficient manner. It goes without saying that the integrated approach cannot hope to achieve the same level of sophistication as systems adopting specialized techniques for the different tasks involved in text planning, sentence planning, and linguistic realization. Nevertheless, we believe that it provides a useful alternative for applications which do not require the full power of state of the art methods. It is easy to implement, relatively efficient and robust, and provides more flexibility than simpler approaches relying only on canned text or template filling. The approach has been developed and tested in the context of the TREE project, 2 which deals with multi-lingual generation of employment advertisements on the World Wide Web. Since the TREE application will be used throughout the paper to illustrate our ideas, we will set the stage in section 2 by giving a brief introduction to the architecture of the TREE system. In section 3 we will introduce the generalized grammar formalism and in section 4 we will explain the pragmatic principles guiding the text realization process.