Tackling Ordinal Regression Problem for Heterogeneous Data: Sparse and
  Deep Multi-Task Learning Approaches
  Many real-world datasets are labeled with natural orders, i.e., ordinal
labels. Ordinal regression is a method to predict ordinal labels that finds a
wide range of applications in data-rich domains, such as natural, health and
social sciences. Most existing ordinal regression approaches work well for
independent and identically distributed (IID) instances via formulating a
single ordinal regression task. However, for heterogeneous non-IID instances
with well-defined local geometric structures, e.g., subpopulation groups,
multi-task learning (MTL) provides a promising framework to encode task
(subgroup) relatedness, bridge data from all tasks, and simultaneously learn
multiple related tasks in efforts to improve generalization performance. Even
though MTL methods have been extensively studied, there is barely existing work
investigating MTL for heterogeneous data with ordinal labels. We tackle this
important problem via sparse and deep multi-task approaches. Specifically, we
develop a regularized multi-task ordinal regression (MTOR) model for smaller
datasets and a deep neural networks based MTOR model for large-scale datasets.
We evaluate the performance using three real-world healthcare datasets with
applications to multi-stage disease progression diagnosis. Our experiments
indicate that the proposed MTOR models markedly improve the prediction
performance comparing with single-task ordinal regression models.
