Density Sketches for Sampling and Estimation
  We introduce Density sketches (DS): a succinct online summary of the data
distribution. DS can accurately estimate point wise probability density.
Interestingly, DS also provides a capability to sample unseen novel data from
the underlying data distribution. Thus, analogous to popular generative models,
DS allows us to succinctly replace the real-data in almost all machine learning
pipelines with synthetic examples drawn from the same distribution as the
original data. However, unlike generative models, which do not have any
statistical guarantees, DS leads to theoretically sound asymptotically
converging consistent estimators of the underlying density function. Density
sketches also have many appealing properties making them ideal for large-scale
distributed applications. DS construction is an online algorithm. The sketches
are additive, i.e., the sum of two sketches is the sketch of the combined data.
These properties allow data to be collected from distributed sources,
compressed into a density sketch, efficiently transmitted in the sketch form to
a central server, merged, and re-sampled into a synthetic database for modeling
applications. Thus, density sketches can potentially revolutionize how we
store, communicate, and distribute data.
