Towards dynamic multi-modal phenotyping using chest radiographs and
  physiological data
  The healthcare domain is characterized by heterogeneous data modalities, such
as imaging and physiological data. In practice, the variety of medical data
assists clinicians in decision-making. However, most of the current
state-of-the-art deep learning models solely rely upon carefully curated data
of a single modality. In this paper, we propose a dynamic training approach to
learn modality-specific data representations and to integrate auxiliary
features, instead of solely relying on a single modality. Our preliminary
experiments results for a patient phenotyping task using physiological data in
MIMIC-IV & chest radiographs in the MIMIC- CXR dataset show that our proposed
approach achieves the highest area under the receiver operating characteristic
curve (AUROC) (0.764 AUROC) compared to the performance of the benchmark method
in previous work, which only used physiological data (0.740 AUROC). For a set
of five recurring or chronic diseases with periodic acute episodes, including
cardiac dysrhythmia, conduction disorders, and congestive heart failure, the
AUROC improves from 0.747 to 0.798. This illustrates the benefit of leveraging
the chest imaging modality in the phenotyping task and highlights the potential
of multi-modal learning in medical applications.
