From Optimizing Engagement to Measuring Value
  Most recommendation engines today are based on predicting user engagement,
e.g. predicting whether a user will click on an item or not. However, there is
potentially a large gap between engagement signals and a desired notion of
"value" that is worth optimizing for. We use the framework of measurement
theory to (a) confront the designer with a normative question about what the
designer values, (b) provide a general latent variable model approach that can
be used to operationalize the target construct and directly optimize for it,
and (c) guide the designer in evaluating and revising their operationalization.
We implement our approach on the Twitter platform on millions of users. In line
with established approaches to assessing the validity of measurements, we
perform a qualitative evaluation of how well our model captures a desired
notion of "value".
