Reliable Weakly Supervised Learning: Maximize Gain and Maintain Safeness
  Weakly supervised data are widespread and have attracted much attention.
However, since label quality is often difficult to guarantee, sometimes the use
of weakly supervised data will lead to unsatisfactory performance, i.e.,
performance degradation or poor performance gains. Moreover, it is usually not
feasible to manually increase the label quality, which results in weakly
supervised learning being somewhat difficult to rely on. In view of this
crucial issue, this paper proposes a simple and novel weakly supervised
learning framework. We guide the optimization of label quality through a small
amount of validation data, and to ensure the safeness of performance while
maximizing performance gain. As validation set is a good approximation for
describing generalization risk, it can effectively avoid the unsatisfactory
performance caused by incorrect data distribution assumptions. We formalize
this underlying consideration into a novel Bi-Level optimization and give an
effective solution. Extensive experimental results verify that the new
framework achieves impressive performance on weakly supervised learning with a
small amount of validation data.
