Sub-linear RACE Sketches for Approximate Kernel Density Estimation on
  Streaming Data
  Kernel density estimation is a simple and effective method that lies at the
heart of many important machine learning applications. Unfortunately, kernel
methods scale poorly for large, high dimensional datasets. Approximate kernel
density estimation has a prohibitively high memory and computation cost,
especially in the streaming setting. Recent sampling algorithms for high
dimensional densities can reduce the computation cost but cannot operate
online, while streaming algorithms cannot handle high dimensional datasets due
to the curse of dimensionality. We propose RACE, an efficient sketching
algorithm for kernel density estimation on high-dimensional streaming data.
RACE compresses a set of N high dimensional vectors into a small array of
integer counters. This array is sufficient to estimate the kernel density for a
large class of kernels. Our sketch is practical to implement and comes with
strong theoretical guarantees. We evaluate our method on real-world
high-dimensional datasets and show that our sketch achieves 10x better
compression compared to competing methods.
