CorrAttack: Black-box Adversarial Attack with Structured Search
  We present a new method for score-based adversarial attack, where the
attacker queries the loss-oracle of the target model. Our method employs a
parameterized search space with a structure that captures the relationship of
the gradient of the loss function. We show that searching over the structured
space can be approximated by a time-varying contextual bandits problem, where
the attacker takes feature of the associated arm to make modifications of the
input, and receives an immediate reward as the reduction of the loss function.
The time-varying contextual bandits problem can then be solved by a Bayesian
optimization procedure, which can take advantage of the features of the
structured action space. The experiments on ImageNet and the Google Cloud
Vision API demonstrate that the proposed method achieves the state of the art
success rates and query efficiencies for both undefended and defended models.
