Learning Rank Functionals: An Empirical Study
  Ranking is a key aspect of many applications, such as information retrieval,
question answering, ad placement and recommender systems. Learning to rank has
the goal of estimating a ranking model automatically from training data. In
practical settings, the task often reduces to estimating a rank functional of
an object with respect to a query. In this paper, we investigate key issues in
designing an effective learning to rank algorithm. These include data
representation, the choice of rank functionals, the design of the loss function
so that it is correlated with the rank metrics used in evaluation. For the loss
function, we study three techniques: approximating the rank metric by a smooth
function, decomposition of the loss into a weighted sum of element-wise losses
and into a weighted sum of pairwise losses. We then present derivations of
piecewise losses using the theory of high-order Markov chains and Markov random
fields. In experiments, we evaluate these design aspects on two tasks: answer
ranking in a Social Question Answering site, and Web Information Retrieval.
