{'fraction_non_alphanumeric': 0.017435897435897435, 'fraction_numerical': 0.0010256410256410256, 'mean_word_length': 5.379084967320262, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A decomposition approach to imbalanced classification

An important characteristic of many modern systems is the availability of large amounts of event data, collected through various sensors. Certain events occur very rarely among these, but may be critical to a successfully functioning system. Examples of these include faulty products, credit card frauds, among others. In this paper, we propose a framework for solving this problem, of detecting rare events, when modeled as a supervised learning task. Specifically, we consider an imbalanced 2-class classification problem. We overcome the challenge of class imbalance by decomposing the original learning task into many simpler learning tasks. A useful feature of the proposed algorithm is that the decision rule is simple enough to infer the importance of individual covariates in rare event detection. We present performance results on some public datasets to demonstrate the effectiveness of the proposed algorithm.