Scaling associative classification for very large datasets
  Supervised learning algorithms are nowadays successfully scaling up to
datasets that are very large in volume, leveraging the potential of in-memory
cluster-computing Big Data frameworks. Still, massive datasets with a number of
large-domain categorical features are a difficult challenge for any classifier.
Most off-the-shelf solutions cannot cope with this problem. In this work we
introduce DAC, a Distributed Associative Classifier. DAC exploits ensemble
learning to distribute the training of an associative classifier among parallel
workers and improve the final quality of the model. Furthermore, it adopts
several novel techniques to reach high scalability without sacrificing quality,
among which a preventive pruning of classification rules in the extraction
phase based on Gini impurity. We ran experiments on Apache Spark, on a real
large-scale dataset with more than 4 billion records and 800 million distinct
categories. The results showed that DAC improves on a state-of-the-art solution
in both prediction quality and execution time. Since the generated model is
human-readable, it can not only classify new records, but also allow
understanding both the logic behind the prediction and the properties of the
model, becoming a useful aid for decision makers.
