Learning-based Hybrid Local Search for the Hard-label Textual Attack
  Deep neural networks are vulnerable to adversarial examples in Natural
Language Processing. However, existing textual adversarial attacks usually
utilize the gradient or prediction confidence to generate adversarial examples,
making it hard to be deployed in real-world applications. To this end, we
consider a rarely investigated but more rigorous setting, namely hard-label
attack, in which the attacker could only access the prediction label. In
particular, we find that the changes on prediction label caused by word
substitutions on the adversarial example could precisely reflect the importance
of different words. Based on this observation, we propose a novel hard-label
attack, called Learning-based Hybrid Local Search (LHLS) algorithm, which
effectively estimates word importance with the prediction label from the attack
history and integrate such information into hybrid local search algorithm to
optimize the adversarial perturbation. Extensive evaluations for text
classification and textual entailment using various datasets and models show
that our LHLS significantly outperforms existing hard-label attacks regarding
the attack performance as well as adversary quality.
