Learning detectors of malicious web requests for intrusion detection in
  network traffic
  This paper proposes a generic classification system designed to detect
security threats based on the behavior of malware samples. The system relies on
statistical features computed from proxy log fields to train detectors using a
database of malware samples. The behavior detectors serve as basic reusable
building blocks of the multi-level detection architecture. The detectors
identify malicious communication exploiting encrypted URL strings and domains
generated by a Domain Generation Algorithm (DGA) which are frequently used in
Command and Control (C&C), phishing, and click fraud. Surprisingly, very
precise detectors can be built given only a limited amount of information
extracted from a single proxy log. This way, the computational requirements of
the detectors are kept low which allows for deployment on a wide range of
security devices and without depending on traffic context such as DNS logs,
Whois records, webpage content, etc. Results on several weeks of live traffic
from 100+ companies having 350k+ hosts show correct detection with a precision
exceeding 95% of malicious flows, 95% of malicious URLs and 90% of infected
hosts. In addition, a comparison with a signature and rule-based solution shows
that our system is able to detect significant amount of new threats.
