{'fraction_non_alphanumeric': 0.012345679012345678, 'fraction_numerical': 0.0, 'mean_word_length': 5.485714285714286, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Distributed Algorithms for Large Scale Learning and Inference in Graphical Models

Over the past few years we have witnessed an increasing popularity in the use of graphical models for applications in computational biology, computer vision and natural language processing. Despite the large body of work, most existing learning and inference algorithms can neither cope with large scale problems which require huge amounts of memory and computation nor can they effectively parallelize structured learning with a small number of training instances. In this paper, we propose a family of model-parallel learning and inference algorithms which exploit distributed computation and memory storage to handle large scale graphical models as well as a small number of training instances. Our algorithms combine primal-dual methods and dual decomposition to preserve the same theoretical guarantees (and for certain parameter choices the same solution) as if they were run on a single machine. We demonstrate the capabilities of our algorithms in a wide variety of scenarios, showing the advantages and disadvantages of model-parallelization.