{'fraction_non_alphanumeric': 0.024849899933288858, 'fraction_numerical': 0.016511007338225483, 'mean_word_length': 4.997, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Dialogue Analysis for Diverse Situations

Research on human language must cope with diverse and abundant complexity. One common approach is to reduce the scope of study to single languages, single cultures, single situations and homogeneous sources of data. This succeeds, at the cost of becoming unaware of the broader complexities of the phenomena studied, and also at the cost of missing the broad applicability of narrow, successful results. In the case of studies of human dialogue, it has been common to restrict or ignore the variability and complexity of the subject, so that in multiple studies of “conversation,” for example, it is treated as if the subject matter were held in common. This paper explores diverse applications of a particular descriptive theory of dialogue structure. It reports application of the theory to an opportunistic diversity of dialogues in English, seeking to sketch the generality and limits of the theory. One use of such explorations is to guide work seeking to overcome the limits that are found. There have been studies of task oriented dialogue for decades, but little exploration of what other kinds of dialogue the results of such studies might cover. This paper finds that methods that work for task oriented dialogue sometimes work beyond that boundary as well. This paper reports an exploration of the nature of dialogue coherence. There are many papers that describe contributions to coherence without identifying the nature of coherence. Here we are concerned with the nature of coherence as a whole. There are many views of coherence in language, and of dialogue coherence in particular. To appreciate them, a distinction must be made between coherence and cohesion. Cohesion represents connectedness of text that arises from the use of particular linguistic devices such as pronouns, anaphoric reference, patterns of topic or focus and many more. In detail it is language dependent. The classic exploration of cohesion is. (Halliday and Hasan 1976) In contrast, coherence is a kind of impression that arises (or not) in a person who attempts to understand particular language use. In general it is not language dependent, in the sense that a translation of a coherent text is coherent, even if the cohesive devices of the source text are absent in the target language. Although not all of the views of coherence make this distinction, it is essential for comparisons. One view says that coherence is simply the collective effect of use of cohesive devices, a view that is more often assumed than stated. Another view sees coherence as arising from propositional consistency, plus relevance in Grice’s sense. (Goldberg 1983) Another group of views sees coherence as arising Proceedings of EDILOG 2002 110 from genre or common recurrent patterns of expression. (Farrell 1983) Using Grice’s Cooperative Principle and maxims, contributions to coherence are found in signaling creative violation of maxims (Mura 1983). Some scholars regard coherence as identified with continuity of topic, theme or focus. (Crow 1983) Others seek (or criticize) a more formulaic scheme, as for example well formedness of sequences of acts. (Jacobs and Jackson 1983) Others, notably Searle, seek for an extension of the concept of speech act to account for the structure of “conversation”; the extension is finally seen as implausible. (Searle 1991) The view being explored here rejects each of those views as insufficient. Coherence represents integrity of intentions that, in the processes of interpretation, are imputed to the text creator – here the dialogue participants. This integrity includes consistency, sequential continuity and apparent completeness of the imputed intentions and their representation in language. Coherence as a technical focus shapes the exploration in ways that differ from the effects of an interpretation focus, a generation focus or more generally a participant or agent focus. It is simply an alternate technical focus, expected to be informative in its own way. It requires an uncommonly broad exploration of natural dialogue data. The intentions exhibited in dialogue depend very strongly on the dialogue situation. In a hostage negotiation or a medical interview, intentions may arise that are never seen in tutoring transcripts or laboratory elicitations. Yet the notion of coherence seems equally applicable. So, for any particular view of coherence, it is worthwhile to examine situations in which it produces a credible account, other situations where it does not, and eventually how these collections differ. Situation here is an informal term. We do not yet have enough evidence to select a technical view of situation that helps in modeling coherence. The models and the relevant notion of situation can be expected to develop together. Dialogue Macrogame Theory There is an approach to intention-based dialogue annotation that is useful in this exploration. It is called Dialogue Macrogame Theory (DMT). (Mann 2002) Because it is widely unknown, it is sketched here. The terminology of DMT is intended to avoid confusion. There are various uses of “dialogue game” and closely related terms. (Mann, Carlisle et al. 1975; Levin and Moore 1977; Mann, Carlisle et al. 1977; Mann 1979; Carlson 1983; Mann 1988; Kowtow, Isard et al. 1993; Traum 1994; Carletta, Isard et al. 1996; Poesio and Traum 1998; Traum 1999; Stolcke, Reis et al. 2000; Traum 2000; Kreutel and Matheson 2001; Mann 2001a) DMT differs from these precedents enough that to simply call it “dialogue game theory” would be inappropriate. DMT is a framework in which analysts can describe dialogues in a systematic way. It does not explain why such analysis is possible, but rather it enables the identification of certain observable regularities and patterns in dialogue. The key analytic facility of DMT is the capability of analysts to understand particular kinds of human interactions. DMT has two major types of abstract constructs, Dialogue Macrogames and