Zero-shot Sequence Labeling: Transferring Knowledge from Sentences to
  Tokens
  Can attention- or gradient-based visualization techniques be used to infer
token-level labels for binary sequence tagging problems, using networks trained
only on sentence-level labels? We construct a neural network architecture based
on soft attention, train it as a binary sentence classifier and evaluate
against token-level annotation on four different datasets. Inferring token
labels from a network provides a method for quantitatively evaluating what the
model is learning, along with generating useful feedback in assistance systems.
Our results indicate that attention-based methods are able to predict
token-level labels more accurately, compared to gradient-based methods,
sometimes even rivaling the supervised oracle network.
