Class-Conditional Compression and Disentanglement: Bridging the Gap
  between Neural Networks and Naive Bayes Classifiers
  In this draft, which reports on work in progress, we 1) adapt the information
bottleneck functional by replacing the compression term by class-conditional
compression, 2) relax this functional using a variational bound related to
class-conditional disentanglement, 3) consider this functional as a training
objective for stochastic neural networks, and 4) show that the latent
representations are learned such that they can be used in a naive Bayes
classifier. We continue by suggesting a series of experiments along the lines
of Nonlinear In-formation Bottleneck [Kolchinsky et al., 2018], Deep
Variational Information Bottleneck [Alemi et al., 2017], and Information
Dropout [Achille and Soatto, 2018]. We furthermore suggest a neural network
where the decoder architecture is a parameterized naive Bayes decoder.
