Security Theater: On the Vulnerability of Classifiers to Exploratory
  Attacks
  The increasing scale and sophistication of cyberattacks has led to the
adoption of machine learning based classification techniques, at the core of
cybersecurity systems. These techniques promise scale and accuracy, which
traditional rule or signature based methods cannot. However, classifiers
operating in adversarial domains are vulnerable to evasion attacks by an
adversary, who is capable of learning the behavior of the system by employing
intelligently crafted probes. Classification accuracy in such domains provides
a false sense of security, as detection can easily be evaded by carefully
perturbing the input samples. In this paper, a generic data driven framework is
presented, to analyze the vulnerability of classification systems to black box
probing based attacks. The framework uses an exploration exploitation based
strategy, to understand an adversary's point of view of the attack defense
cycle. The adversary assumes a black box model of the defender's classifier and
can launch indiscriminate attacks on it, without information of the defender's
model type, training data or the domain of application. Experimental evaluation
on 10 real world datasets demonstrates that even models having high perceived
accuracy (>90%), by a defender, can be effectively circumvented with a high
evasion rate (>95%, on average). The detailed attack algorithms, adversarial
model and empirical evaluation, serve.
