Finding Regions of Heterogeneity in Decision-Making via Expected
  Conditional Covariance
  Individuals often make different decisions when faced with the same context,
due to personal preferences and background. For instance, judges may vary in
their leniency towards certain drug-related offenses, and doctors may vary in
their preference for how to start treatment for certain types of patients. With
these examples in mind, we present an algorithm for identifying types of
contexts (e.g., types of cases or patients) with high inter-decision-maker
disagreement. We formalize this as a causal inference problem, seeking a region
where the assignment of decision-maker has a large causal effect on the
decision. Our algorithm finds such a region by maximizing an empirical
objective, and we give a generalization bound for its performance. In a
semi-synthetic experiment, we show that our algorithm recovers the correct
region of heterogeneity accurately compared to baselines. Finally, we apply our
algorithm to real-world healthcare datasets, recovering variation that aligns
with existing clinical knowledge.
