Uncovering the Source of Machine Bias
  We develop a structural econometric model to capture the decision dynamics of
human evaluators on an online micro-lending platform, and estimate the model
parameters using a real-world dataset. We find two types of biases in gender,
preference-based bias and belief-based bias, are present in human evaluators'
decisions. Both types of biases are in favor of female applicants. Through
counterfactual simulations, we quantify the effect of gender bias on loan
granting outcomes and the welfare of the company and the borrowers. Our results
imply that both the existence of the preference-based bias and that of the
belief-based bias reduce the company's profits. When the preference-based bias
is removed, the company earns more profits. When the belief-based bias is
removed, the company's profits also increase. Both increases result from
raising the approval probability for borrowers, especially male borrowers, who
eventually pay back loans. For borrowers, the elimination of either bias
decreases the gender gap of the true positive rates in the credit risk
evaluation. We also train machine learning algorithms on both the real-world
data and the data from the counterfactual simulations. We compare the decisions
made by those algorithms to see how evaluators' biases are inherited by the
algorithms and reflected in machine-based decisions. We find that machine
learning algorithms can mitigate both the preference-based bias and the
belief-based bias.
