{'fraction_non_alphanumeric': 0.031113464848404607, 'fraction_numerical': 0.019594862968356944, 'mean_word_length': 5.201970443349754, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Evaluating Suitability of SNOMED CT in Structured Searches for COVID-19 Studies

Studies investigating the suitability of SNOMED CT in COVID-19 datasets are still scarce. The purpose of this study was to evaluate the suitability of SNOMED CT for structured searches of COVID-19 studies, using the German Corona Consensus Dataset (GECCO) as example. Suitability of the international standard SNOMED CT was measured with the scoring system ISO/TS 21564, and intercoder reliability of two independent mapping specialists was evaluated. The resulting analysis showed that the majority of data items had either a complete or partial equivalent in SNOMED CT (complete equivalent: 141 items; partial equivalent: 63 items; no equivalent: 1 item). Intercoder reliability was moderate, possibly due to non-establishment of mapping rules and high percentage (74%) of different but similar concepts among the 86 non-equal chosen concepts. The study shows that SNOMED CT can be utilized for COVID-19 cohort browsing. However, further studies investigating mapping rules and further international terminologies are necessary.


Introduction
The ongoing COVID-19 pandemic, caused by the severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2), has triggered large numbers of health studies. Therefore, the NFDI4Health Task Force COVID-19 initiative, a German research project, was established to centralize clinical, epidemiological and public health studies on COVID-19 in Germany [1]. Interlinked complementary web-based platforms [2] [3] provide study related information, exploration, visualization tools and storage of semantically annotated items. The NFDI4Health Task Force COVID-19 identified surveys and datasets for inclusion. To provide FAIR data -i.e., data that are findable, accessible, interoperable and reusable -data elements need to be interoperable [4]. This requires the use of established, standardized vocabularies, terminologies or ontologies such as SNOMED CT.
As an example dataset to evaluate usage of SNOMED CT within the NFDI4Health Task Force COVID-19, we used the German Corona Consensus Dataset (GECCO), an interoperable pioneer dataset providing data items for standardized COVID-19 research [5]. GECCO was developed within the German National Research Network of University Medicine on COVID-19 ("Netzwerk Universitätsmedizin") and specifies information to be collected in COVID-19 studies. The GECCO core data set contains 81 elements combined with 281 response options, classified into 13 categories ( Figure 1). In order to enhance semantic and syntactic interoperability of research data on SARS-CoV-2, GECCO uses Fast Healthcare Interoperability Resources (FHIR) to define machine-readable data formats for interoperable data exchange. In addition, GECCO dataset elements are mapped to international terminologies (SNOMED CT, LOINC, UCUM, ICD-10-GM and ATC) to ensure semantic interoperability. We performed a mapping of GECCO using SNOMED CT. SNOMED CT concepts enable stable mechanisms capturing information and supporting data integration. However, the understanding of terms of a terminology may differ, implementing challenges to both developers and users.
This study evaluates intercoder reliability and suitability of mapping SNOMED CT to GECCO. Results are, for example, of relevance for the development of standardized filtering and browsing options in COVID-19 study search portals, such as the ones developed within the NFDI4Health Task Force COVID-19.

Mapping procedure
Within the GECCO dataset, we identified 205 data items (response options representing simple yes/no/unknown responses were considered as straightforward and excluded from analysis). Mapping was conducted independently by two coders (CV, SK), both medical doctors with broad experience in the area of medical terminologies. No mapping rules were established prior to the mapping process. Mapping was conducted with the SNOMED CT browser (Release: International Edition 2020-07-31).
Concepts behind each of the 205 data items were analyzed briefly and mapped manually to SNOMED CT. Both coders (CV, SK) mapped each data element to SNOMED CT concepts and then decided which concept was suitable for each individual data element. A joint mapping version was established based on consensual validation by both coders. Mapping results of the joint mapping version were rated and classified using ISO/TS 21564:2019 "Health Informatics -Terminology resource map quality measures (MapQual)" to measure the degree of equivalence between the dataset and target concept of SNOMED CT (Table 1) [6].

Intercoder Reliability
Mapping results between the two coders were compared among each other and with the original SNOMED CT mapping of the identified GECCO data items. Percentage share of SNOMED CT concepts to the GECCO dataset equally chosen by the coders and nominal Krippendorff's α was used as a measure of intercoder agreement. Different mapping choices were analyzed and divided into similar and non-similar concepts among the two coders.

ISO/TS 21564 Equivalence Assessment Score
The joint mapping version of the GECCO dataset with SNOMED CT achieved a high average ISO/TS 21564 equivalence assessment score of 0.52 (Table 1). Overall, 69% of data items had equivalent meaning in the chosen SNOMED CT concept. All but one data item had a match in the used SNOMED CT system. ISO equivalence assessment by data category is presented in Figure 1. Fifteen (7%) data items had multiple target concepts (e.g., terms describing a conjunction of several diseases), and 189 (92%) data items had a unique respective SNOMED CT concept.

Intercoder Reliability
Results on intercoder reliability are shown in Table 2. In the original GECCO data set, 61 (30%) of the items were not mapped to SNOMED CT resulting in 144 items which were compared to the two coders (Table 2). Analyzing the 92 (45%) non-equal mapping concepts of the two coders, we found that 64 (74%) concepts were different but related or similar concepts from SNOMED CT (Table3).

Discussion and Conclusion
In this article, we showed that SNOMED CT concepts capture typical data elements relevant to COVID-19 studies, exemplified by the GECCO dataset, very well. The mapping results reached a high ISO/TS 21564 equivalence assessment score. Sass et al. standardized Germany's Electronic Disease Management Program for bronchial asthma using standards such as LOINC, UCUM as well as SNOMED CT and found comparable high mapping results [7]. SNOMED CT could therefore be used to enable detailed structured searches, for example in study portals like the one developed by the NFDI4Health Task Force COVID-19 and beyond. However, not all data elements could be perfectly matched to SNOMED CT, and for improved accuracy, a hybrid approach using several terminologies might be preferable. Intercoder reliability between the two coders in our study was moderate, however, the majority of non-equal matches were identified as similar concepts. An elevated count of discrepancies may be due to the fact that no rules were determined prior to the mapping as improvement of intercoder results after establishment of mapping rules has been previously reported [8].
In the future, implementation of mapping rules is necessary to improve mapping quality and intercoder reliability. We plan to further facilitate browsing of COVID-19 research information within the NFDI4Health Task Force COVID-19 project and will continue to evaluate the possibility of a faceted search of study information using SNOMED CT.