CODA: Constructivism Learning for Instance-Dependent Dropout
  Architecture Construction
  Dropout is attracting intensive research interest in deep learning as an
efficient approach to prevent overfitting. Recently incorporating structural
information when deciding which units to drop out produced promising results
comparing to methods that ignore the structural information. However, a major
issue of the existing work is that it failed to differentiate among instances
when constructing the dropout architecture. This can be a significant
deficiency for many applications. To solve this issue, we propose
Constructivism learning for instance-dependent Dropout Architecture (CODA),
which is inspired from a philosophical theory, constructivism learning.
Specially, based on the theory we have designed a better drop out technique,
Uniform Process Mixture Models, using a Bayesian nonparametric method Uniform
process. We have evaluated our proposed method on 5 real-world datasets and
compared the performance with other state-of-the-art dropout techniques. The
experimental results demonstrated the effectiveness of CODA.
