N = 300 N = 1000 Histogram 0 . 3165 (0 . 0052) 0 . 1854 (0 . 0029) Data derived partitions 0 . 3010 (0 . 0030) 0 . 1879 (0 . 0022) All derived partitions 0 . 2954 (0 . 0028) 0 . 1860 (0 . 0021) Table 5: The average (standard error) of the mutual information estimates using random samples of size 300 (column 2) and 1000 (column 3) from the two com- ponent Gaussian mixture, by the following methods: the naive histogram estima- tor that partitions each axis to 15 intervals of equal count, S ADP 15 × $_{15}$/ ( N | Π ADP 15 | ), and S DDP 15 × $_{15}$/ ( N | Π DDP 15 | ). The true mutual information value was 0 . 1784. simulated 50 datasets and computed the estimated mutual information using S DDP m × $_{m}$, S ADP m × $_{m}$, and the histogram estimator. The Gaussian mixture density was where f$_{N}$ ( µ, Σ) is the bivariate normal distribution with mean µ and covariance ma- trix Σ. For each partition, we applied the Miller–Madow correction (Paninski, 2003), a simple and well-known modification that estimates the systematic error of the his- togram estimators and improves the finite-sample properties of these estimators. We compared it to the histogram estimator, as well as to the estimator based on the DDP statistic that considers only a subset of all possible partitions. Table 5 shows that the variability and the bias decrease as N increases for all estimators, and that the ADP estimator is the least variable, as is intuitively expected since it is the average over many partitions. In practice, it is difficult to identify the optimal m : it should not be too small so that the local dependence structure is not missed, causing large bias, nor should it be too large so that the grid created is too sparse, causing large variance. 61 0 . 8 × f$_{N}$$^{((}$ 0 . 5 0 . 5$^{)}$ ,$^{(}$ 0 . 05 0 . 025 0 . 025 0 . 05$^{))}$ + 0 . 2 × f$_{N}$$^{((}$ 0 . 125 0 . 675$^{)}$ ,$^{(}$ 0 . 01 0 0 0 . 01$^{))}$ ,