{'fraction_non_alphanumeric': 0.030612244897959183, 'fraction_numerical': 0.0, 'mean_word_length': 5.384615384615385, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Abnormality detection using deep neural networks with robust quasi-norm autoencoding and semi-supervised learning

Abnormality detection in biomedical images is a one-class classification problem, where methods learn a statistical model to characterize the inlier class using training data solely from the inlier class. Typical methods (i) need well-curated training data and (ii) have formulations that are unable to utilize expert feedback through (a small amount of) labeled outliers. In contrast, we propose a novel deep neural network framework that (i) is robust to corruption and outliers in the training data, which are inevitable in real-world deployment, and (ii) can leverage expert feedback through high-quality labeled data. We introduce an autoencoder formulation that (i) gives robustness through a non-convex loss and a heavy-tailed distribution model on the residuals and (ii) enables semi-supervised learning with labeled outliers. Results on three large medical datasets show that our method outperforms the state of the art in abnormality-detection accuracy.