Multimodal Machine Learning in Precision Health
  As machine learning and artificial intelligence are more frequently being
leveraged to tackle problems in the health sector, there has been increased
interest in utilizing them in clinical decision-support. This has historically
been the case in single modal data such as electronic health record data.
Attempts to improve prediction and resemble the multimodal nature of clinical
expert decision-making this has been met in the computational field of machine
learning by a fusion of disparate data. This review was conducted to summarize
this field and identify topics ripe for future research. We conducted this
review in accordance with the PRISMA (Preferred Reporting Items for Systematic
reviews and Meta-Analyses) extension for Scoping Reviews to characterize
multi-modal data fusion in health. We used a combination of content analysis
and literature searches to establish search strings and databases of PubMed,
Google Scholar, and IEEEXplore from 2011 to 2021. A final set of 125 articles
were included in the analysis. The most common health areas utilizing
multi-modal methods were neurology and oncology. However, there exist a wide
breadth of current applications. The most common form of information fusion was
early fusion. Notably, there was an improvement in predictive performance
performing heterogeneous data fusion. Lacking from the papers were clear
clinical deployment strategies and pursuit of FDA-approved tools. These
findings provide a map of the current literature on multimodal data fusion as
applied to health diagnosis/prognosis problems. Multi-modal machine learning,
while more robust in its estimations over unimodal methods, has drawbacks in
its scalability and the time-consuming nature of information concatenation.
