Discriminative, Generative and Self-Supervised Approaches for
  Target-Agnostic Learning
  Supervised learning, characterized by both discriminative and generative
learning, seeks to predict the values of single (or sometimes multiple)
predefined target attributes based on a predefined set of predictor attributes.
For applications where the information available and predictions to be made may
vary from instance to instance, we propose the task of target-agnostic learning
where arbitrary disjoint sets of attributes can be used for each of predictors
and targets for each to-be-predicted instance. For this task, we survey a wide
range of techniques available for handling missing values, self-supervised
training and pseudo-likelihood training, and adapt them to a suite of
algorithms that are suitable for the task. We conduct extensive experiments on
this suite of algorithms on a large collection of categorical, continuous and
discretized datasets, and report their performance in terms of both
classification and regression errors. We also report the training and
prediction time of these algorithms when handling large-scale datasets. Both
generative and self-supervised learning models are shown to perform well at the
task, although their characteristics towards the different types of data are
quite different. Nevertheless, our derived theorem for the pseudo-likelihood
theory also shows that they are related for inferring a joint distribution
model based on the pseudo-likelihood training.
