DZip: improved general-purpose lossless compression based on novel
  neural network modeling
  We consider lossless compression based on statistical data modeling followed
by prediction-based encoding, where an accurate statistical model for the input
data leads to substantial improvements in compression. We propose DZip, a
general-purpose compressor for sequential data that exploits the well-known
modeling capabilities of neural networks (NNs) for prediction, followed by
arithmetic coding. Dzip uses a novel hybrid architecture based on adaptive and
semi-adaptive training. Unlike most NN based compressors, DZip does not require
additional training data and is not restricted to specific data types, only
needing the alphabet size of the input data. The proposed compressor
outperforms general-purpose compressors such as Gzip (on average 26% reduction)
on a variety of real datasets, achieves near-optimal compression on synthetic
datasets, and performs close to specialized compressors for large sequence
lengths, without any human input. The main limitation of DZip in its current
implementation is the encoding/decoding time, which limits its practicality.
Nevertheless, the results showcase the potential of developing improved
general-purpose compressors based on neural networks and hybrid modeling.
