{'fraction_non_alphanumeric': 0.019572953736654804, 'fraction_numerical': 0.0017793594306049821, 'mean_word_length': 5.537790697674419, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
IULA's LSP Multilingual Corpus: compilation and processing

The Institut Universitari de Linguistica Aplicada (Institute for Applied Linguistics), known as IULA, is a public institution that belongs to the Universitat Pompeu Fabra (UPF) in Barcelona and is closely connected to the Translators and Interpreters Faculty of UPF. IULA is devoted to both postgraduate teaching and research in applied linguistics, covering basically the following fields: lexicology, lexicography and terminology; computational linguistics and linguistic engineering; and language variation. Although within each of these areas different, specific projects are being carried out, there is a single unifying project to which every researcher in IULA contributes to some extent, namely the creation of computer based resources for linguistic research in the fields covered by IULA’s main research areas. This common project consists in the creation of a language for specific purposes (LSP), multilingual corpus1. It is primarily intended to be useful for researchers in the areas of applied linguistics that are under focus in IULA and for the teaching activities in IULA and in the translators’ faculty. But we also foresee collaboration with other research centres, both public and private: IULA is already heavily involved in a joint project on computational resources for Catalan with other research institutions in Catalunya. The creation of the LSP, multilingual corpus implies three basic steps: the compilation of the corpus, its processing, and its exploration. In section 1 we discuss the main decisions taken on the compilation of the corpus and the way it is internally organised. In section 2, we present the current state of development and integration of corpus processing tools. The integration of processing tools follows naturally the compilation of the corpus itself and we develop them only in case we have not found already existing tools which are available and easily adapted and integrated to our environment. The various subsections in section 2 cover the different processing levels that we foresee for Catalan, Spanish and English: structural tagging, text handling, morphological analysis, morphological disambiguation and syntactic analysis.