Mixture-of-experts VAEs can disregard variation in surjective multimodal
  data
  Machine learning systems are often deployed in domains that entail data from
multiple modalities, for example, phenotypic and genotypic characteristics
describe patients in healthcare. Previous works have developed multimodal
variational autoencoders (VAEs) that generate several modalities. We consider
subjective data, where single datapoints from one modality (such as class
labels) describe multiple datapoints from another modality (such as images). We
theoretically and empirically demonstrate that multimodal VAEs with a mixture
of experts posterior can struggle to capture variability in such surjective
data.
