Uncovering Latent Biases in Text: Method and Application to Peer Review
  Quantifying systematic disparities in numerical quantities such as employment
rates and wages between population subgroups provides compelling evidence for
the existence of societal biases. However, biases in the text written for
members of different subgroups (such as in recommendation letters for male and
non-male candidates), though widely reported anecdotally, remain challenging to
quantify. In this work, we introduce a novel framework to quantify bias in text
caused by the visibility of subgroup membership indicators. We develop a
nonparametric estimation and inference procedure to estimate this bias. We then
formalize an identification strategy to causally link the estimated bias to the
visibility of subgroup membership indicators, provided observations from time
periods both before and after an identity-hiding policy change. We identify an
application wherein "ground truth" bias can be inferred to evaluate our
framework, instead of relying on synthetic or secondary data. Specifically, we
apply our framework to quantify biases in the text of peer reviews from a
reputed machine learning conference before and after the conference adopted a
double-blind reviewing policy. We show evidence of biases in the review ratings
that serves as "ground truth", and show that our proposed framework accurately
detects these biases from the review text without having access to the review
ratings.
