Maximum Likelihood Estimation for Multimodal Learning with Missing
  Modality
  Multimodal learning has achieved great successes in many scenarios. Compared
with unimodal learning, it can effectively combine the information from
different modalities to improve the performance of learning tasks. In reality,
the multimodal data may have missing modalities due to various reasons, such as
sensor failure and data transmission error. In previous works, the information
of the modality-missing data has not been well exploited. To address this
problem, we propose an efficient approach based on maximum likelihood
estimation to incorporate the knowledge in the modality-missing data.
Specifically, we design a likelihood function to characterize the conditional
distribution of the modality-complete data and the modality-missing data, which
is theoretically optimal. Moreover, we develop a generalized form of the
softmax function to effectively implement maximum likelihood estimation in an
end-to-end manner. Such training strategy guarantees the computability of our
algorithm capably. Finally, we conduct a series of experiments on real-world
multimodal datasets. Our results demonstrate the effectiveness of the proposed
approach, even when 95% of the training data has missing modality.
