{'fraction_non_alphanumeric': 0.024510138594934117, 'fraction_numerical': 0.0071004301222093264, 'mean_word_length': 5.093178036605657, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
DELOS: An Automatically Tagged Economic Corpus for Modern Greek

Text corpora resources have become an essential tool for Natural Language Processing tasks over the past years. A wide range of applications like information retrieval, ontology and terminology extraction require a sufficiently large corpus but of restricted domain. Manual tagging of such a corpus is very costly, making automatic annotation by a set of linguistic tools a very challenging idea. DELOS, described in this paper, is a Modern Greek corpus of economic domain consisting of 5 million word tokens, which is automatically tagged for morphology and shallow syntactic relations. The annotating tools described are embodied in an integrated system and their application to the corpus is performed using the GATE text engineering platform. The system output is a textual database marked up with the annotation tagset in plain text as well as in XML format. .


Introduction
In the last years, research in Computational Linguistics has been relying heavily on textual corpora. From lexical acquisition to machine translation, most applications require large corpora that reflect a wide range of domains and genre. Certain tasks, on the other hand, like information retrieval, ontology and terminology extraction necessitate domain-specific corpora.
Many approaches in natural language processing make use of knowledge-poor resources, even raw corpora, while the rest need annotated textual databases which provide the necessary linguistic (grammatical, syntactic, or semantic) information. Manual annotation of such corpora is very costly and time-consuming, while using a set of linguistic tools in order to automatically provide annotation tags for the corpus tokens appears to be a challenge. Many languages, including Modern Greek, are not sufficiently equipped with linguistic resources, thus significantly reducing the options and the ability of language engineers for research and development.
In this paper we describe DELOS, a Modern Greek corpus of economic-domain consisting of 5 million word tokens. DELOS was created at first in order to be utilized in a lexicographic application for economic terminology and it has been further restructured to constitute a standardized, uniformly annotated textual database, easy to use for tasks reliant on text processing. Annotation includes morphological and shallow syntactic information such as phrase chunking and elementary subject-object relations.
Regarding morphology, Modern Greek is language rich in inflections. There are eleven different part-ofspeech (pos) categories, six of which are declinable. Features characterizing nouns, articles and adjectives are three genders, four cases and two numbers. Verbs are characterized by their voice, mood, tense, number and person. Pronouns have different features depending on their type (personal, relative, interrogative etc.). As to sentence structure, Modern Greek is free in the order in which phrases are combined to form clauses. The subject and the object(s) of a verb can assume any position within the clause, either preceding or following the verb.
The rest of the paper is organized as follows: First the sources of the collection are presented and its coverage described. The linguistic tools used to automatically tag it are described in section 3. Statistical information concerning the corpus as well as a detailed explanation of the tagset are given in sections 4 and 5. The tools' accuracy and the evaluation of the linguistic process follow in section 6 and concluding remarks in the last section.

Makeup of DELOS
The motivation for creating DELOS was the DELOS project (Sintichakis et al., 2000), funded by the Greek Ministry of Development, which aimed at constructing a Greek-English dictionary of economic terminology. In order for the corpus to constitute a reliable source of information for lexicographic purposes, it had to possess balance and representativeness. Texts were sampled systematically to reflect tendencies in spoken and written language. The collection consists of texts taken from the financial newspaper EXPRESS, reports from the Foundation for Economic and Industrial Research, research papers from the Athens University of Economics and several reports from the Bank of Greece. The documents are of varying genre like press reportage, news, articles, interviews and scientific studies and cover all the basic areas of the economic domain, i.e. Microeconomics, Macroeconomics, International Economics, Finance, Business Administration, Economic History, Law and Economics, Public Economics etc.

Annotation Tools
As a first processing stage, numerous text filters were applied for the correction of typical typing errors and the conversion of character sets. The collection of texts was then linguistically annotated by a series of tools which operate on the GATE text engineering platform (Cunningham et al., 1996) and which are applied to the texts sequentially, each adding its own tagset to the corpus. Their output is in plain text as well as in XML format. A brief description of each of these modules follows next.

Tokenizer
The tokenizer detects the word tokens in the corpus, i.e. words, abbreviations, numbers, punctuation marks etc.
The tool is actually embodied within the sentence boundary detector.

Sentence Boundary Detection
Sentence boundaries were detected by a sentence splitter based on a variation of transformation-based errordriven learning (Stamatatos et al., 1999). Acronyms and abbreviations constitute a significant source of ambiguity regarding the detection of sentence boundaries. The splitter uses a small, manually annotated, training subcorpus in order to extract a set of disambiguation rules, which are then applied to every potential sentence boundary in the actual corpus in order to determine sentence boundaries. Neither lexicons of any sort, nor lists of abbreviations are utilized.

Morphological Analysis
The next tool is a morphological analyzer (Sgarbas et al., 1998) developed according to the two-level morphology model. Rules specifying the permissible alterations to the orthographic form of a morpheme are realized as finite state transducers. A lexicon representing the morphotactic description of the language consists of a set of morphemes and information about the order in which these can be concatenated. Currently it consists of approximately 60,000 entries of lemmata. For the declinable words that are included in the morphological lexicon, the lemma, gender, number, case, person, mood values of the word are available as well as more specific information depending on its pos category and type information for pronouns, conjunctions etc. For words not covered by the lexicon, morphological information may be guessed based on the word suffix.

Phrase Chunking
The boundaries of intrasentential, non-overlapping noun, verb, prepositional, adverbial phrases and conjunctions are detected by a multi-pass chunker (Stamatatos et al., 2000). Simple chunks are identified during the first parsing passes while later ones deal with more complicated situations (i.e. combining already detected small chunks into longer ones). The chunker exploits minimal linguistic resources: a keyword lexicon containing 450 keywords (i.e. closed-class words such as articles, prepositions etc.) and a suffix lexicon of 300 of the most common word suffixes in Modern Greek.

Headword detection
Using empirical rules, the headword of the noun and prepositional phrases is detected as a last step of the chunker. The headword of a phrase is the word which determines the basic grammatical properties of the phrase (case, number etc.).

Shallow Parsing
The parser, last in the tool chain, detects elementary shallow syntactic dependencies, i.e. subject-verb-object relationships using the linguistic information provided by the previous modules along with a set of simple empirical rules taken from the Modern Greek syntactic theory. POS category information, case, voice and number information as well as information about the relative distance of the candidate subject or object from the verb are essential. A distinction between direct and indirect objects is made. Candidate subjects and objects are limited to being heads of noun and prepositional phrases as opposed to entire clauses.

Corpus Statistics
A detailed statistical description of the corpus is shown in the following table. Due to its economic domain, DELOS is rich in numbers (monetary amounts and percentages), proper nouns (names of people and companies), company acronyms and abbreviations. It consists of a total of approximately 5 million word tokens which form around 140,000 sentences.

Morphological tagset
The entire morphological tagset appearing in DELOS is presented in this section. It was selected to be in accordance to the annotation scheme proposed in the PAROLE project (LE2 4017-10379) which was implemented manually for a small balanced corpus by the Institute for Language and Speech Processing (ILSP).
The tag format is word<tag*lemma> Word is the actual word as it appears in the corpus, lemma is its lemma, and tag is a set of characters providing morphological information for the word depending on the pos category of the word. The set of characters for every pos is shown in the following tables. Any invalid field or any field with unknown value has a dash (-) as a symbol.      Prepositional articles (στον, στην etc.) are tagged like articles except for the first character introducing their tag, which is S, for prepositions (see Table 11). Their lemma value is that of the article, i.e. ο. For example the tag of στον would be <STdmsa*ο>.

Syntactic tagset
Noun, verb, adverbial, prepositional phrases and conjunctions linking them are introduced by the tags NP, VP, ADP, PP and CON respectively. The phrase body is enclosed within square brackets. The * symbol at the beginning of a word denotes a headword in a noun or a prepositional phrase. Below follows a piece of morphologically and syntactically annotated text. In the XML output format, every constituent (phrasal or lexical token) is given a unique id number. Verb phrases have a subject, direct and indirect object field that takes the value of the noun or prepositional phrase id of the phrase the headword of which functions as a subject, direct and indirect object of the verb phrase's verb respectively.

Evaluation of Tagging
Starting with the evaluation of the sentence boundary detection process, the detector reached an accuracy of 98.5%. As accuracy we define the percentage of the number of positive (a punctuation mark considered wrongly to be the end of a sentence) and negative (a sentence end not detected) errors. The detector has been trained by manually tagging the sentence boundaries of approximately 15000 sentences of the corpus and tested on 1500 new corpus sentences. Considering that, as mentioned before, DELOS is rich in proper nouns, abbreviations, acronyms, numbers and punctuation marks, and therefore a baseline accuracy value (i.e. regarding every full stop, exclamation mark, quotation mark and set of dots as a sentence boundary) does not exceed 57-58%, the accuracy reached is quite satisfactory.
The morphological lexicon at this time covers most of the closed-class words appearing in the corpus and approximately 65% of the declinable words and it is being constantly enriched. More specifically, 39% of the adjective lemmata, 25% of noun lemmata and 31% of the verb lemmata and most of the closed-class words appearing in the corpus are currently unknown. For declinable words included in the lexicon, accuracy of their morphological features exceeds 98%. For those words not covered in the lexicon, pos tagging performance turns out to be as shown in the following  The recall and precision metrics for the morphological analyzer, the chunker and the shallow parser are defined as follows: Recall = the number of correctly predicted types divided by the total number of types appearing in the input text.
Precision = the number of correctly predicted types divided by the total number of types predicted.
A type in the above definitions can be a pos category, a phrase boundary or a shallow parsing relation. The above results have been obtained by testing 1000 occurrences of every pos category (either predicted or theoretical, depending on whether positive or negative examples are searched for) in the corpus.
Nouns and adjectives have similar and sometimes identical endings. Adjectives and adverbs, as well as articles and certain types of pronouns have very often the same orthographic form. Their distinction is not straightforward without context information, which explains their lower precision values.
The tag correctness in respect to the most important grammatical features, again for words not included in the lexicon, given, however, that their pos tag is predicted correctly, is shown in Table 13.

Feature
Accuracy (  For the chunker, testing reveals 89.5% recall and 94.5% precision, which is more than encouraging as Delos is a corpus rich in complicated syntactic structures. Foreign words, the endings of which are similar to endings of Modern Greek words, are also the cause of several chunking errors. Table 14 shows the performance of the chunker in more detail for every phrase type.

Phrase
Precision (  The problem of adverbs that have an identical orthographic form with adjectives accounts for the lower recall value in the detection of adverbial phrase boundaries. Lower precision in the noun phrase detection is attributed to the significant number of noun phrases which are not introduced by an article or a pronoun. Recall and precision for the subject-verb-object dependencies detection are approximately 70%. These results have been obtained by testing 500 verb occurrences (a total of approximately 200 different verbs) in the corpus. Certain errors in previous stages (like case and number tagging of the headword of noun phrases) are to a large extent responsible for decrease in shallow parsing performance.

Conclusion
The problem of several languages not being sufficiently equipped with adequate resources is significant for language engineers. Full manual construction of such resources is expensive and not always feasible. In this paper we have presented the creation of an economic corpus for Modern Greek as well as its automatic annotation process and detailed description. Annotation reaches the level of shallow parsing. DELOS can be used for any linguistic application like extraction of economic terminology or ontological information.