Anomaly Detection for Network Connection Logs
  We leverage a streaming architecture based on ELK, Spark and Hadoop in order
to collect, store, and analyse database connection logs in near real-time. The
proposed system investigates outliers using unsupervised learning; widely
adopted clustering and classification algorithms for log data, highlighting the
subtle variances in each model by visualisation of outliers. Arriving at a
novel solution to evaluate untagged, unfiltered connection logs, we propose an
approach that can be extrapolated to a generalised system of analysing
connection logs across a large infrastructure comprising thousands of
individual nodes and generating hundreds of lines in logs per second.
