To Trust, or Not to Trust? A Study of Human Bias in Automated Video
  Interview Assessments
  Supervised systems require human labels for training. But, are humans
themselves always impartial during the annotation process? We examine this
question in the context of automated assessment of human behavioral tasks.
Specifically, we investigate whether human ratings themselves can be trusted at
their face value when scoring video-based structured interviews, and whether
such ratings can impact machine learning models that use them as training data.
We present preliminary empirical evidence that indicates there might be biases
in such annotations, most of which are visual in nature.
