HumanAL: Calibrating Human Matching Beyond a Single Task
  This work offers a novel view on the use of human input as labels,
acknowledging that humans may err. We build a behavioral profile for human
annotators which is used as a feature representation of the provided input. We
show that by utilizing black-box machine learning, we can take into account
human behavior and calibrate their input to improve the labeling quality. To
support our claims and provide a proof-of-concept, we experiment with three
different matching tasks, namely, schema matching, entity matching and text
matching. Our empirical evaluation suggests that the method can improve the
quality of gathered labels in multiple settings including cross-domain (across
different matching tasks).
