Improving the performance of bagging ensembles for data streams through
  mini-batching
  Often, machine learning applications have to cope with dynamic environments
where data are collected in the form of continuous data streams with
potentially infinite length and transient behavior. Compared to traditional
(batch) data mining, stream processing algorithms have additional requirements
regarding computational resources and adaptability to data evolution. They must
process instances incrementally because the data's continuous flow prohibits
storing data for multiple passes. Ensemble learning achieved remarkable
predictive performance in this scenario. Implemented as a set of (several)
individual classifiers, ensembles are naturally amendable for task parallelism.
However, the incremental learning and dynamic data structures used to capture
the concept drift increase the cache misses and hinder the benefit of
parallelism. This paper proposes a mini-batching strategy that can improve
memory access locality and performance of several ensemble algorithms for
stream mining in multi-core environments. With the aid of a formal framework,
we demonstrate that mini-batching can significantly decrease the reuse distance
(and the number of cache misses). Experiments on six different state-of-the-art
ensemble algorithms applying four benchmark datasets with varied
characteristics show speedups of up to 5X on 8-core processors. These benefits
come at the expense of a small reduction in predictive performance.
