{'fraction_non_alphanumeric': 0.04002864303616183, 'fraction_numerical': 0.0188327962764053, 'mean_word_length': 4.853310980720872, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 8, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Open Korean Corpora: A Practical Report

Korean is often referred to as a low-resource language in the research community. While this claim is partially true, it is also because the availability of resources is inadequately advertised and curated. This work curates and reviews a list of Korean corpora, first describing institution-level resource development, then further iterate through a list of current open datasets for different types of tasks. We then propose a direction on how open-source dataset construction and releases should be done for less-resourced languages to promote research.


Introduction
The Korean language is less explored in terms of corpus and computational linguistics, but its prevalence is often underrated. It regards about 80 million language users and is recently adopted in multilingual research as it is bound to CJK (Chinese, Japanese, and Korean), also handling a distinguished writing system.
However, compared to the industrial need, the interest in Korean natural language processing (NLP) has not been developed much in international viewpoints, which recurrently hinders the related publication and further academic extension. Besides, in the recent NLP, where the benchmark practice is a trend, such systems lack at this point, deterring abroad and even native researchers who start Korean NLP from finding directions. Park et al. (2016) has shown a decent survey, but it seems that the techniques are mainly on the NLP pipeline. Also, albeit some curations on Korean NLP 1 and datasets 2 , we considered that little more organization is required, and better if internationally available. Our attempts are expected to mitigate the challenges that the researchers who handle Korean from a multi-or cross-lingual viewpoint may face.
In this paper, we scrutinize the struggles of government, institutes, industry, and individuals to construct public Korean NLP resources. First, we state how the institutional organizations have tackled the issue by making up the accessible resources, and point out the limitation thereof regarding international availability and license, to finally introduce and curate the fully public datasets along with the proposed criteria. Through this, we want to find out the current state of Korean corpora across the NLP tasks and whether they are freely or conditionally available. Our survey is to be curated and updated in the public repository 3 .

Accessible Resources
With the increase in popularity of machine learningdriven methods in NLP, constructing a novel dataset and releasing it to the public can be considered the cornerstone of advancing research of a given language. While we believe many useful datasets exist behind industry walls, this is not particularly useful for advancing open research. Fortunately, there are organizations that construct and distribute cleaned, pre-processed datasets which are occasionally accompanied by a task and the annotation. In the context of Korean, there are numerous efforts in this field driven by governmentaffiliated organizations.

Datasets from public institutions
National Institute of Korean Language (NIKL) is an institution that establishes the norm for Korean linguistics 4 . However, at the same time, it usually undergoes the massive dataset construction from the view of computational linguistics, to apt to the new wave of language artificial intelligence (AI). Widely known ones include Korean word dictionaries 5 and Sejong Corpus (Kim, 2006). The dictionary contains fundamental and new lexicons that make up Korean (along with the content), and the Sejong Corpus is a large-scale labeled NLP pipeline corpus for the tasks such as constituency and dependency parsing, mainly provided in .json format. Besides, recently, labeled corpora of about 300 million word size is released 6 , covering inter-sentence tasks such as similarity and entailment.

Electronics and Telecommunications Research
Institute (ETRI) has been collecting, refining, and tagging language processing and speech learning data over a long period of time 7 . Aside from NIKL, which mainly focuses on classical NLP pipelines, ETRI has also built a database for semantic analysis and question answering (QA), which are the outcome of a project Exo-brain 8 . The project includes syntax-semantic ones such as part of speech (POS) tagging and semantic role labeling (SRL), simultaneously providing construction guidelines for the corpora.
AI HUB is a platform organized by National Information Society Agency (NIA) in which a largescale dataset are integrated 9 . The datasets are built for various tasks at the government level, to promote the development of the AI industry. Provided resources are labeled or parallel corpora in reallife domains. Here, the domains are law, patent, common sense, open dialog, machine reading comprehension, and machine translation. Also, about 1,200 hours of speech corpus is provided to be used in spoken language modeling 10 . Recently, some new datasets have been distributed on wellness and emotional dialog, so that many people can have trials for social good and public AI. Also, open dictionary NIAdic 11 is freely available, provided by K-ICT Big Data Center. 5 The search portal is provided in https://stdict. korean.go.kr/main/main.do while the full word and content list are available at https://github.com/ korean-word-game/db 6 https://corpus.korean.go.kr/ 7 https://www.etri.re.kr/intro.html 8 http://exobrain.kr/pages/ko/result/ outputs.jsp 9 http://www.aihub.or.kr/ 10 https://www.aihub.or.kr/aidata/105 11 https://kbig.kr/portal/kbig/ knowledge/files/bigdata_report.page? bltnNo=10000000016451

Accessibility
The above datasets guarantee high quality, along with well-defined guidelines and the well-educated workers. However, their usage is often unfortunately confined to domestic researchers for procedural issues. Researchers abroad can indeed access the data, but they may face difficulty filling out and submitting the particular application form, instead of the barrier-free downloading system. Also, in most cases, modification and redistribution are restricted, making them uncompetitive in view of quality enhancement (Han et al., 2017).
Here, we want to introduce datasets that can be utilized as an alternative to the limitedly accessible Korean NLP resources. Instead of scrutinizing all available corpora, we are going to curate them under specific criteria.

Open Datasets
All the datasets to be introduced from now on are fully open access. This means that the dataset is downloadable with a single click or cloning, or at least one can acquire the dataset with simple signing. We set three checklists for the status of the corpus, namely documentation, usage, and redistribution. The first one is on how fine-grained the corpus description is.
• Does the corpus have any documentation on the usage? (doc) • Does the corpus have a related article? 12 (art) • Does the corpus have a internationally available publication? (inter) Next, we check whether the dataset is commercially available, academic use only, or unknown (com, acad, unk). For the last one, We also investigate if redistribution is available with or without modification, if neither, or unknown (rd, rd/modx, no, unk). These attributes are noted along with each corpus title.  (Choi et al., 1994). KMOU NER [art, acad, rd] is a named entity recognition (NER) dataset built by Korean Marine and Ocean University 18 . The named entities are tagged for about 24K utterances according to name, time, and number. The data source are Exo-brain (by ETRI) and their own data combined, while the redistribution is available only for the latter.
AIR×NAVER NER/SRL [doc, acad, no] adopted the NER 19 and SRL 20 data constructed by Changwon National University for the purpose of a public competition 21 , and is annotated according to CoNLL format (Tjong Kim Sang and De Meulder, 2003). Corpus size is about 90K and 35K each.  (Conneau et al., 2018). Just as in the original dataset, the pairs are labelled with entailment, contradiction, or neutral. About 940K examples are provided for training, and 2,490 and 5,010 respectively for dev and test. For KorSTS, the scoring was done from 0 to 5 to elaborate rather than the binary label that determines paraphrase. Following the scheme of NLI, 5,749 training data were machine translated using the STS-B dataset (Cer et al., 2017) as a source, while 1,500 dev set and 1,379 test set pairs are human translated.

Entailment and sentence similarity
ParaKQC [inter, com, rd] Cho et al. (2020) 24 originally consists of 10,000 questions and commands, and each instance is labeled with 4 topics (mail, smart agent, scheduling, and weather) and 4 speech acts (wh-question, alternative question, prohibition, and requirement). The sentence set can be extended to about 540K sentence pairs that determine sentence similarity and paraphrase.

Sentence classification and QA
NSMC [doc, com, rd] is a review sentiment corpus 25 of size 200K, which consists of Naver movie comments automatically labeled according to the methodology of Maas et al. (2011). It adopts pos/neg binary labels, and it has been widely used as a benchmark for pretrained language models.

BEEP! [inter, com, rd] Moon et al. (2020) is a
hand-labeled, crowd-sourced dataset of about 9.4K Naver entertainment news comments with hate speech and social bias 26 . Bias and hate attribute consists of 3 labels, namely gender/others/none and hate/offensive/none, respectively. 3i4K [inter, com, rd] Cho et al. (2018) aims an utterance-level speech act classification of the Korean language 27 . The volume reaches 61K, handlabeled with 7 classes, namely fragment, statement, question, command, rhetorical question/command, and intonation-dependent utterances.
KorQuAD 1.0, 2.0 [inter, com, rd/mod-x] provides human-generated QA corpus and leaderboard for Korean 28 . KorQuAD 1.0 (Lim et al., 2019) benchmarks SQuAD 1.0 (Rajpurkar et al., 2016) and consists of total 70K questions. KorQuAD 2.0 of size 100K aims at machine reading comprehension for structured HTML natural questions, which was created referring to the scheme of Google Natural Questions (Kwiatkowski et al., 2019).

Parallel corpora
Sci-news-sum-kr [doc, acad, rd] contains about 50 Korean news summarizations generated by two Korean natives 29 . Since the size is not large, it is recommended to be used as a dev set. KAIST Translation Evaluation Set [doc, acad, no] is an evaluation set of size about 3,000 for enko MT 31 , augmented with index, original sentence, translation, related articles, and text source.
Transliteration Dataset [doc, com, rd] is not an official data repository 33 , but en-ko transliteration is collected from public dictionaries such as NIKL or Wiktionary 34 . A total of about 35K en (word) -ko (pronunciation) pairs are included.  in en-ko 35 , and consists of 7,186 words excerpted from the loanword dictionary 36 .

Korean in multilingual corpora
Multilingual G2P Conversion [inter, com, rd] Gorman et al. (2020) is a shared task of SIGMOR-PHON 2020 37 , which aims to transform grapheme sequence into a phoneme sequence. The dataset was created with WikiPron 38 , and has been built for 10 languages including Korean (3,600 pairs for train, and 450 for dev/test each).

Speech corpora
Speech datasets are usually massive, that a downloading via a single click is not necessarily guaranteed. Thus, we listed some of them as open even if they require some application form. (2018)

Summary
In total, we surveyed 32 corpora, namely 18 Korean text corpora, 9 multilingual corpora, and 5 speech corpora. They are composed of 7 datasets on parsing and tagging, 7 datasets on entailment, paraphrasing, and summarization, 8 datasets on (spoken language) classification, QA, and dialog, 5 datasets on machine translation/transliteraion, and 5 datasets on speech (pre-)processing 47 . We provide the full specification in Table 1  • The canonical, current version of this paper will be regularly published as a revision, e.g., on arxiv.org, based on a community-open version of this paper. • The resources will also have a corresponding registry, following the same metadata protocol for usability in different types of research, as we used in this protocol. • Each new contribution to the resource list will have a corresponding entry in the acknowledgments section.
We will make the registry machine parseable, so that other curated sites such as nlpprogress. org, can utilize the registry to automate updates. The project will be maintained as an open-source project, under a permissive license. A living document is a new territory for the field of academia, but we strongly believe that given the rapid progress of NLP research, this is an experiment worth attempting; and hope that a successful effort can inspire other languages to follow the same approach. Our approach is to be described in the public repository, guaranteeing the accessibility for domestic and abroad researchers. Also, a large portion of the data are expected to be more easily accessible via Koco 48 and Korpora 49 , the recently constructed dataset wrappers for Korean NLP.

Conclusion
In this paper, we investigated the Korean NLP datasets constructed and released as public resources. Our curation suggests a variety of open corpora that are freely available. This information will not only be helpful for the Korean researchers who want to start NLP, but also for the abroad ones who are interested in Korean NLP. Nonetheless, we think that Korean open corpora are still less disclosed or not yet sufficient. It is notable that the Korean government is currently supplying substantial funds to build a database. To guide this well, appropriate management and documentation should be guaranteed, so that the construction is meaningful and the outcome is internationally available. 48 https://github.com/inmoonlight/koco 49 https://github.com/ko-nlp/Korpora