{'fraction_non_alphanumeric': 0.027777777777777776, 'fraction_numerical': 0.009259259259259259, 'mean_word_length': 5.039106145251397, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Semi-Supervised Learning of Bayesian Language Models with Pitman-Yor Priors

The recent years have seen the emergence of multiple stochastic language and grammar models, that make use of Pitman-Yor processes as Bayesian priors. Thus far, those models proved very effective for NLP tasks that involve unsupervised inference. The aim of this project is to investigate on semi-supervised learning methods and to test their applicability and effectiveness on this class of language models. The original methods have to be adapted as the usual semi-supervised inference with ExpectationMaximization (EM) is not applicable. The alternative is to retreat to Gibbs sampling. The two major contributions are novel variants of the Stable Mixing method (Corduneanu and Jaakkola, 2002) and of Active Learning (Cohn et al., 1996). Unlike their originals, the new methods are applicable to Gibbs-based inference methods. For an exemplary word separation task on the Bernstein Ratner corpus, the new Stable Mixing variant improves the word F0-score by 9% in comparison to the base-line approach.