{'fraction_non_alphanumeric': 0.037368222891566265, 'fraction_numerical': 0.012707078313253012, 'mean_word_length': 5.347072879330944, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Pseudonymization of PHI Items in German Clinical Reports

. We describe the adaptation of a non-clinical pseudonymization system, originally developed for a German email corpus, for clinical use. This tool replaces previously identified Protected Health Information (PHI) items as carriers of privacy-sensitive information (original names for people, organizations, places, etc.) with semantic type-conformant, yet, fictitious surrogates. We evaluate the generated substitutes for grammatical correctness, semantic and medical plausibility and find particularly low numbers of error instances (less than 1%) on all of these dimensions.


Introduction
One of the main reasons for the impressive advancements in nearly all branches of natural language processing (NLP) is the abundant mass of data accessible for training and operating NLP engines. While their petabyte dimension seems out of reach for clinical applications for the time being, massive regulatory obstacles to increasing volumes of clinical raw data and distributing them within the NLP community are in place. Such legal constraints securing individual data privacy are imposed on all sorts of personalized medical documents (almost) all over the world (cf. the General Data Protection Rule (2016/679) for the EU [1] or the Health Insurance Portability and Accountability Act (HIPAA) for the USA) [2]. HIPAA, for instance, enumerates 18 Protected Health Information (PHI) categories that need to be obfuscated when patient data leave the secured walls of any local clinical site. Human efforts to break this de-identification bottleneck are laborious, costly and error-prone. Current automatic approaches to de-identification achieve a detection rate for PHI items between 94% up to 99%, yet seem unable to close the remaining gap completely [3,4].
In order to deal with such unrecognized PHI items in an ingenious way, the deidentification problem can be rephrased as a pseudonymization task. Pseudonymization (an approach originating from the seminal work of Sweeney [5]) replaces text stretches which contain confidential, i.e., privacy-sensitive, information by semantic type-conformant, natural, yet at the same time fictitious (and hence non-confidential) surrogates (e.g., the invented name "Jessica Johnson" consistently replaces the original patient name "Jennifer White" in the entire document). As an outstanding advantage, this kind of camouflage is particularly robust against unwarranted effects of de-identification errors, since potential leaks due to an unrecognized PHI item (say, "Jennifer White" remains unchanged as "Jennifer White") are not at all obvious to spot and thus hard to decipher in terms of re-identification of concrete individuals. As a consequence, privacy information might not be leaked despite incomplete de-identification, an assumption referred to as "Hiding in Plain Sight" (HiPS) [6]. The majority of work on medical pseudonymization, no surprise, has been conducted for the English language, with data coming from well-known repositories allowing access via Data Use Agreements (clinical notes from MIMIC II [7], Physionet and i2b2 [8,9]), while several non-sharable datasets have been created as well [10,11,12]. As far as non-English clinical language is concerned, French [12], Danish [13], Swedish [14] and Dutch [15] EHRs have also been dealt with.
Deeper evaluation efforts related to the re-identification risk after pseudonymization began with two studies reporting encouraging evidence (on smallscale data sets though, incorporating less than 100 patient records) that experienced physicians were unable to re-identify patients they had been treating from pseudonymized record data [10,12]. In another small-scale experiment testing the purported 'naturalness' of substitutions, evaluators, by and large, were unable to recognize pseudonymized documents (only 3.5% of these documents were correctly identified whereas in 1.5% of the cases they categorized non-pseudonymized documents as pseudonymized) [14]. Recent experiments on larger data scales study much more aggressive attack scenarios, both in machine-supported [3] and human expert-centered [4] re-identification settings. Both studies conclude that such massive attacks can attenuate, but not eliminate, the protective effect of pseudonymization. Furthermore, enormous man-and compute-power are needed on the attackers' side to organize and run such operations.

Methods
Data. We used two German-language clinical corpora, namely the Jena part of the national reference corpus 3000PA [16] which contains 1,106 real discharge summaries sampled from the Jena University Hospital's information system (approved by the local ethics committee (4639-12/15)), and JSYNCC [17], a complementary corpus made of German medical textbook documents mimicking clinical reports for educational purposes (chosen to boost the volume of experimental data and genre diversity). 3000PA was annotated for HIPAA PHI categories with an instance-based inter-annotator agreement (IAA) of F1=0.96 [16]. JSYNCC was also annotated for these HIPAA categories, with an IAA of F1=0.79. For this study, we had to adjust the HIPAA category system, since, e.g., in the original schema, a generic Name category was introduced whereas in our refined type system (see description below), we further divide this category into Female Name and Male Name, as well as Given and Family Name. The same applies to location identifications, e.g., streets and cities, which are generally summarized as Locations by HIPAA. Basic corpus statistics of the original data, the original annotated HIPAA PHI entities (PHIHIPAA), and the evaluation subset are summarized in Table 1. For the clinical application, we extended this type hierarchy by two subtypes. The toplevel Organization type was assigned the subtype Medical Unit which is technically divided into a common and an identifier part: identifying location and person names of institutions and station or room numbers are subject to obfuscation whereas its common part is kept as-is in the surrogation step (the specific disease characteristics of a patient treated at, say, a "Department of Dermatology" cannot be naturally preserved when department names are arbitrarily exchanged). The second extension relates to the Person type, with an additional subtype, Physical Attributes, which subsumes a person's Age in years, Height in (centi)meters, and Weight in (kilo)grams.
Pseudonymization System. After transforming PHI-sensitive text mentions as defined by the above entity type system, pseudonymization requires generating a surrogate for each PHI instance by transforming the original text mention into a type-conformant artificial mention substitute. For this task, we extended a rule-based surrogate generation system designed for non-medical purposes (see [18] for details). Following clinical conventions for obfuscations, we implemented cut-offs at Age > 89 and shifts of Height and Weight by some constant increment while preserving the Body Mass Index (BMI) as a non-identifying attribute.

Results
The PHI columns in Table 1 display the number of text mentions of standard HIPAA categories (PHIHIPAA), those resulting from our refined type system (PHIRefined), and the surrogates produced on the basis of the latter (PHISurrogates). From the 3000PA and JSYNCC corpus, we selected 100 and 200 documents, respectively, for evaluation. Our evaluators were all medical students and native speakers of German. Overall, the results are more than promising. We encountered no morpho-syntactic error at all (which is not a trivial result because of the rich morphological constraints German language has to obey). Only the 3000PA corpus produced five low-level language-bound semantic errors due to spelling alternatives for German umlauts (e.g., ö oe). We found 23 domain knowledge-bound medical errors in 3000PA and three in JSYNCC. Some of them were due to implausible date conversions, only two errors were due to false annotations (e.g., a typist's name was annotated as medical staff name). Some odd results, not necessarily false, also popped up. For instance, typical Arabic or Asian names were replaced with typical German substitutes, yet a migration background of that patient was linked with the anamnesis.

Discussion
Porting the pseudonymization system originally developed for email documents to the clinical domain turned out to be a comparatively straightforward task, once clinicspecific adaptations of the type system had been made. Furthermore, standard HIPAA categories had to be refined at a finer level of granularity to allow for natural and informative surrogates (e.g., preserving (fe)male names). With an overall error rate of way below 1%, the clinical pseudonymization system we describe here is robust enough to be deployed on a larger, routine scale. While naturalness of substitutions seems preserved, we have not been dealing with the re-identification risk of our approach up until now (see the brief discussion in the final paragraph of Section 1).

Conclusion
We here presented the first pseudonymization system for German clinical documents.
With only a few adaptation steps (updates of the type hierarchy by clinically relevant PHI categories and changes at the code level to cope with these extensions) the original pseudonymization system primarily built for an email corpus [19] can be easily reused in the clinical domain. The evaluation results show that error rates for machine-generated surrogates are negligible. Due to its modular design and transparent implementation (cf. [18]), the pseudonymization procedure can easily be ported to other (European) languages as well (but note that even within the English-speaking language community such a transfer is not so straightforward as it seems because, e.g., different date schemes or zip code patterns have to be dealt with [11]). Integrating the perspectives from two entirely different application domains (clinical reports vs. emails) revealed interesting insights. In particular, it turned out that finer levels of granularity than those defined by HIPAA-style categories were needed and seemingly unrelated categories (e.g., physical attributes of a patient) could be subsumed under unifying types. Hence, this work also contributes to a better understanding of what constitutes person-identifying information under digital privacy considerations. The PHI annotations for JSYNCC, corresponding type-conformant pseudonyms and programming code for the pseudonymization system are available under https://doi.org/10.5281/zenodo.4584505