Machine Learning (In) Security: A Stream of Problems
  Machine Learning (ML) has been widely applied to cybersecurity, and is
currently considered state-of-the-art for solving many of the field's open
issues. However, it is very difficult to evaluate how good the produced
solutions are, since the challenges faced in security may not appear in other
areas (at least not in the same way). One of these challenges is the concept
drift, that actually creates an arms race between attackers and defenders,
given that any attacker may create novel, different threats as time goes by (to
overcome defense solutions) and this "evolution" is not always considered in
many works. Due to this type of issue, it is fundamental to know how to
correctly build and evaluate a ML-based security solution. In this work, we
list, detail, and discuss some of the challenges of applying ML to
cybersecurity, including concept drift, concept evolution, delayed labels, and
adversarial machine learning. We also show how existing solutions fail and, in
some cases, we propose possible solutions to fix them.
