Learning Neural Sequence-to-Sequence Models from Weak Feedback with
  Bipolar Ramp Loss
  In many machine learning scenarios, supervision by gold labels is not
available and consequently neural models cannot be trained directly by maximum
likelihood estimation (MLE). In a weak supervision scenario, metric-augmented
objectives can be employed to assign feedback to model outputs, which can be
used to extract a supervision signal for training. We present several
objectives for two separate weakly supervised tasks, machine translation and
semantic parsing. We show that objectives should actively discourage negative
outputs in addition to promoting a surrogate gold structure. This notion of
bipolarity is naturally present in ramp loss objectives, which we adapt to
neural models. We show that bipolar ramp loss objectives outperform other
non-bipolar ramp loss objectives and minimum risk training (MRT) on both weakly
supervised tasks, as well as on a supervised machine translation task.
Additionally, we introduce a novel token-level ramp loss objective, which is
able to outperform even the best sequence-level ramp loss on both weakly
supervised tasks.
