Neural Estimators for Conditional Mutual Information Using Nearest
  Neighbors Sampling
  The estimation of mutual information (MI) or conditional mutual information
(CMI) from a set of samples is a long-standing problem. A recent line of work
in this area has leveraged the approximation power of artificial neural
networks and has shown improvements over conventional methods. One important
challenge in this new approach is the need to obtain, given the original
dataset, a different set where the samples are distributed according to a
specific product density function. This is particularly challenging when
estimating CMI.
  In this paper, we introduce a new technique, based on k nearest neighbors
(k-NN), to perform the resampling and derive high-confidence concentration
bounds for the sample average. Then the technique is employed to train a neural
network classifier and the CMI is estimated accordingly. We propose three
estimators using this technique and prove their consistency, make a comparison
between them and similar approaches in the literature, and experimentally show
improvements in estimating the CMI in terms of accuracy and variance of the
estimators.
