K-Metamodes: frequency- and ensemble-based distributed k-modes
  clustering for security analytics
  Nowadays processing of Big Security Data, such as log messages, is commonly
used for intrusion detection purposed. Its heterogeneous nature, as well as
combination of numerical and categorical attributes does not allow to apply the
existing data mining methods directly on the data without feature
preprocessing. Therefore, a rather computationally expensive conversion of
categorical attributes into vector space should be utilised for analysis of
such data. However, a well-known k-modes algorithm allows to cluster the
categorical data directly and avoid conversion into the vector space. The
existing implementations of k-modes for Big Data processing are ensemble-based
and utilise two-step clustering, where data subsets are first clustered
independently, whereas the resulting cluster modes are clustered again in order
to calculate metamodes valid for all data subsets. In this paper, the novel
frequency-based distance function is proposed for the second step of
ensemble-based k-modes clustering. Besides this, the existing feature
discretisation method from the previous work is utilised in order to adapt
k-modes for processing of mixed data sets. The resulting k-metamodes algorithm
was tested on two public security data sets and reached higher effectiveness in
comparison with the previous work.
