FENCE: Feasible Evasion Attacks on Neural Networks in Constrained
  Environments
  As advances in Deep Neural Networks (DNNs) demonstrate unprecedented levels
of performance in many critical applications, their vulnerability to attacks is
still an open question. We consider evasion attacks at testing time against
Deep Learning in constrained environments, in which dependencies between
features need to be satisfied. These situations may arise naturally in tabular
data or may be the result of feature engineering in specific application
domains, such as threat detection in cyber security. We propose a general
iterative gradient-based framework called FENCE for crafting evasion attacks
that take into consideration the specifics of constrained domains and
application requirements. We apply it against Feed-Forward Neural Networks
trained for two cyber security applications: network traffic botnet
classification and malicious domain classification, to generate feasible
adversarial examples. We extensively evaluate the success rate and performance
of our attacks, compare their improvement over several baselines, and analyze
factors that impact the attack success rate, including the optimization
objective and the data imbalance. We show that with minimal effort (e.g.,
generating 12 additional network connections), an attacker can change the
model's prediction from the Malicious class to Benign and evade the classifier.
We show that models trained on datasets with higher imbalance are more
vulnerable to our FENCE attacks. Finally, we demonstrate the potential of
performing adversarial training in constrained domains to increase the model
resilience against these evasion attacks.
