From Random to Supervised: A Novel Dropout Mechanism Integrated with
  Global Information
  Dropout is used to avoid overfitting by randomly dropping units from the
neural networks during training. Inspired by dropout, this paper presents
GI-Dropout, a novel dropout method integrating with global information to
improve neural networks for text classification. Unlike the traditional dropout
method in which the units are dropped randomly according to the same
probability, we aim to use explicit instructions based on global information of
the dataset to guide the training process. With GI-Dropout, the model is
supposed to pay more attention to inapparent features or patterns. Experiments
demonstrate the effectiveness of the dropout with global information on seven
text classification tasks, including sentiment analysis and topic
classification.
