Generalized Twin Gaussian Processes using Sharma-Mittal Divergence
  There has been a growing interest in mutual information measures due to their
wide range of applications in Machine Learning and Computer Vision. In this
paper, we present a generalized structured regression framework based on
Shama-Mittal divergence, a relative entropy measure, which is introduced to the
Machine Learning community in this work. Sharma-Mittal (SM) divergence is a
generalized mutual information measure for the widely used R\'enyi, Tsallis,
Bhattacharyya, and Kullback-Leibler (KL) relative entropies. Specifically, we
study Sharma-Mittal divergence as a cost function in the context of the Twin
Gaussian Processes (TGP)~\citep{Bo:2010}, which generalizes over the
KL-divergence without computational penalty. We show interesting properties of
Sharma-Mittal TGP (SMTGP) through a theoretical analysis, which covers missing
insights in the traditional TGP formulation. However, we generalize this theory
based on SM-divergence instead of KL-divergence which is a special case.
Experimentally, we evaluated the proposed SMTGP framework on several datasets.
The results show that SMTGP reaches better predictions than KL-based TGP, since
it offers a bigger class of models through its parameters that we learn from
the data.
