LoL: A Comparative Regularization Loss over Query Reformulation Losses
  for Pseudo-Relevance Feedback
  Pseudo-relevance feedback (PRF) has proven to be an effective query
reformulation technique to improve retrieval accuracy. It aims to alleviate the
mismatch of linguistic expressions between a query and its potential relevant
documents. Existing PRF methods independently treat revised queries originating
from the same query but using different numbers of feedback documents,
resulting in severe query drift. Without comparing the effects of two different
revisions from the same query, a PRF model may incorrectly focus on the
additional irrelevant information increased in the more feedback, and thus
reformulate a query that is less effective than the revision using the less
feedback. Ideally, if a PRF model can distinguish between irrelevant and
relevant information in the feedback, the more feedback documents there are,
the better the revised query will be. To bridge this gap, we propose the
Loss-over-Loss (LoL) framework to compare the reformulation losses between
different revisions of the same query during training. Concretely, we revise an
original query multiple times in parallel using different amounts of feedback
and compute their reformulation losses. Then, we introduce an additional
regularization loss on these reformulation losses to penalize revisions that
use more feedback but gain larger losses. With such comparative regularization,
the PRF model is expected to learn to suppress the extra increased irrelevant
information by comparing the effects of different revised queries. Further, we
present a differentiable query reformulation method to implement this
framework. This method revises queries in the vector space and directly
optimizes the retrieval performance of query vectors, applicable for both
sparse and dense retrieval models. Empirical evaluation demonstrates the
effectiveness and robustness of our method for two typical sparse and dense
retrieval models.
