Synergistic Learning of Lung Lobe Segmentation and Hierarchical
  Multi-Instance Classification for Automated Severity Assessment of COVID-19
  in CT Images
  Understanding chest CT imaging of the coronavirus disease 2019 (COVID-19)
will help detect infections early and assess the disease progression.
Especially, automated severity assessment of COVID-19 in CT images plays an
essential role in identifying cases that are in great need of intensive
clinical care. However, it is often challenging to accurately assess the
severity of this disease in CT images, due to variable infection regions in the
lungs, similar imaging biomarkers, and large inter-case variations. To this
end, we propose a synergistic learning framework for automated severity
assessment of COVID-19 in 3D CT images, by jointly performing lung lobe
segmentation and multi-instance classification. Considering that only a few
infection regions in a CT image are related to the severity assessment, we
first represent each input image by a bag that contains a set of 2D image
patches (with each cropped from a specific slice). A multi-task multi-instance
deep network (called M$^2$UNet) is then developed to assess the severity of
COVID-19 patients and also segment the lung lobe simultaneously. Our M$^2$UNet
consists of a patch-level encoder, a segmentation sub-network for lung lobe
segmentation, and a classification sub-network for severity assessment (with a
unique hierarchical multi-instance learning strategy). Here, the context
information provided by segmentation can be implicitly employed to improve the
performance of severity assessment. Extensive experiments were performed on a
real COVID-19 CT image dataset consisting of 666 chest CT images, with results
suggesting the effectiveness of our proposed method compared to several
state-of-the-art methods.
