{'fraction_non_alphanumeric': 0.031522468142186455, 'fraction_numerical': 0.0, 'mean_word_length': 5.2426778242677825, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Sparse Neural Additive Model: Interpretable Deep Learning with Feature Selection via Group Sparsity

Interpretable machine learning has demonstrated im-pressive performance while preserving explainability. In particular, neural additive models (NAM) oﬀer the interpretability to the black-box deep learning and achieve state-of-the-art accuracy among the large family of generalized additive models. In order to empower NAM with feature selection and improve the generalization, we propose the sparse neural additive models (SNAM) that employ the group sparsity regularization (e.g. Group LASSO), where each feature is learned by a sub-network whose trainable parameters are clustered as a group. We study the theoretical properties for SNAM with novel techniques to tackle the non-parametric truth, thus extending from clas-sical sparse linear models such as the LASSO, which only works on the parametric truth. Speciﬁcally, we show that SNAM with subgradient and proximal gradient descents provably converges to zero training loss as t → ∞ , and that the estimation error of SNAM vanishes asymptotically as n → ∞ . We also prove that SNAM, similar to LASSO, can have exact support recovery, i.e. perfect feature selection, with appropriate regularization. Moreover, we show that the SNAM can generalize well and preserve the ‘identiﬁability’, recovering each feature’s eﬀect. We validate our theories via extensive experiments and further testify to the good accuracy and eﬃciency of SNAM.