AI and Holistic Review: Informing Human Reading in College Admissions
  College admissions in the United States is carried out by a human-centered
method of evaluation known as holistic review, which typically involves reading
original narrative essays submitted by each applicant. The legitimacy and
fairness of holistic review, which gives human readers significant discretion
over determining each applicant's fitness for admission, has been repeatedly
challenged in courtrooms and the public sphere. Using a unique corpus of
283,676 application essays submitted to a large, selective, state university
system between 2015 and 2016, we assess the extent to which applicant
demographic characteristics can be inferred from application essays. We find a
relatively interpretable classifier (logistic regression) was able to predict
gender and household income with high levels of accuracy. Findings suggest that
data auditing might be useful in informing holistic review, and perhaps other
evaluative systems, by checking potential bias in human or computational
readings.
