Explainable multiple abnormality classification of chest CT volumes with
  deep learning
  Understanding model predictions is critical in healthcare, to facilitate
rapid verification of model correctness and to guard against use of models that
exploit confounding variables. We introduce the challenging new task of
explainable multiple abnormality classification in volumetric medical images,
in which a model must indicate the regions used to predict each abnormality. To
solve this task, we propose a multiple instance learning convolutional neural
network, AxialNet, that allows identification of top slices for each
abnormality. Next we incorporate HiResCAM, an attention mechanism, to identify
sub-slice regions. We prove that for AxialNet, HiResCAM explanations are
guaranteed to reflect the locations the model used, unlike Grad-CAM which
sometimes highlights irrelevant locations. Armed with a model that produces
faithful explanations, we then aim to improve the model's learning through a
novel mask loss that leverages HiResCAM and 3D allowed regions to encourage the
model to predict abnormalities based only on the organs in which those
abnormalities appear. The 3D allowed regions are obtained automatically through
a new approach, PARTITION, that combines location information extracted from
radiology reports with organ segmentation maps obtained through morphological
image processing. Overall, we propose the first model for explainable
multi-abnormality prediction in volumetric medical images, and then use the
mask loss to achieve a 33% improvement in organ localization of multiple
abnormalities in the RAD-ChestCT data set of 36,316 scans, representing the
state of the art. This work advances the clinical applicability of multiple
abnormality modeling in chest CT volumes.
