Training Multimodal Systems for Classification with Multiple Objectives
  We learn about the world from a diverse range of sensory information.
Automated systems lack this ability as investigation has centred on processing
information presented in a single form. Adapting architectures to learn from
multiple modalities creates the potential to learn rich representations of the
world - but current multimodal systems only deliver marginal improvements on
unimodal approaches. Neural networks learn sampling noise during training with
the result that performance on unseen data is degraded. This research
introduces a second objective over the multimodal fusion process learned with
variational inference. Regularisation methods are implemented in the inner
training loop to control variance and the modular structure stabilises
performance as additional neurons are added to layers. This framework is
evaluated on a multilabel classification task with textual and visual inputs to
demonstrate the potential for multiple objectives and probabilistic methods to
lower variance and improve generalisation.
