When do Words Matter? Understanding the Impact of Lexical Choice on
  Audience Perception using Individual Treatment Effect Estimation
  Studies across many disciplines have shown that lexical choice can affect
audience perception. For example, how users describe themselves in a social
media profile can affect their perceived socio-economic status. However, we
lack general methods for estimating the causal effect of lexical choice on the
perception of a specific sentence. While randomized controlled trials may
provide good estimates, they do not scale to the potentially millions of
comparisons necessary to consider all lexical choices. Instead, in this paper,
we first offer two classes of methods to estimate the effect on perception of
changing one word to another in a given sentence. The first class of algorithms
builds upon quasi-experimental designs to estimate individual treatment effects
from observational data. The second class treats treatment effect estimation as
a classification problem. We conduct experiments with three data sources (Yelp,
Twitter, and Airbnb), finding that the algorithmic estimates align well with
those produced by randomized-control trials. Additionally, we find that it is
possible to transfer treatment effect classifiers across domains and still
maintain high accuracy.
