Counterfactual Inference of Second Opinions
  Automated decision support systems that are able to infer second opinions
from experts can potentially facilitate a more efficient allocation of
resources; they can help decide when and from whom to seek a second opinion. In
this paper, we look at the design of this type of support systems from the
perspective of counterfactual inference. We focus on a multiclass
classification setting and first show that, if experts make predictions on
their own, the underlying causal mechanism generating their predictions needs
to satisfy a desirable set invariant property. Further, we show that, for any
causal mechanism satisfying this property, there exists an equivalent mechanism
where the predictions by each expert are generated by independent
sub-mechanisms governed by a common noise. This motivates the design of a set
invariant Gumbel-Max structural causal model where the structure of the noise
governing the sub-mechanisms underpinning the model depends on an intuitive
notion of similarity between experts which can be estimated from data.
Experiments on both synthetic and real data show that our model can be used to
infer second opinions more accurately than its non-causal counterpart.
