{'fraction_non_alphanumeric': 0.03210382513661202, 'fraction_numerical': 0.00273224043715847, 'mean_word_length': 5.540178571428571, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Gaze-Guided Class Activation Mapping: Leverage Human Visual Attention for Network Attention in Chest X-rays Classification

The attention mechanism in artificial neural networks is conceptually interlinked with human visual attention, and studies have shown that either artificial or human attention can facilitate computer vision tasks. However, little research has investigated the visual interpretation of neural network’s attention with the help of human visual attention. This paper describes a novel eye gaze-guided class activation mapping (GG-CAM) method that augments deep convolutional neural network (CNN) attention with human attention for improved chest X-ray (CXR) pathology classification and CNN’s visual interpretability. A loss function designed for attention supervision and an improved multi-task learning scheme are developed to achieve the primary classification task and the auxiliary attention supervision task. The GG-CAM is a generic and light-weight method, with only 3 additional trainable parameters, and can be applied to many classification CNNs. GG-CAM-modified CNNs do not require human attention as an input when fully trained. Experimental results show that GG-CAM modified ResNet50 and EfficientNetV2 (S) outperform state-of-the-art CNNs for CXR pathology classification, and demonstrate better visual interpretability on CNN’s classification decisions. Code used for this paper is available at https://github.com/hz-zhu/GG-CAM.