BEAN: Interpretable Representation Learning with Biologically-Enhanced
  Artificial Neuronal Assembly Regularization
  Deep neural networks (DNNs) are known for extracting useful information from
large amounts of data. However, the representations learned in DNNs are
typically hard to interpret, especially in dense layers. One crucial issue of
the classical DNN model such as multilayer perceptron (MLP) is that neurons in
the same layer of DNNs are conditionally independent of each other, which makes
co-training and emergence of higher modularity difficult. In contrast to DNNs,
biological neurons in mammalian brains display substantial dependency patterns.
Specifically, biological neural networks encode representations by so-called
neuronal assemblies: groups of neurons interconnected by strong synaptic
interactions and sharing joint semantic content. The resulting population
coding is essential for human cognitive and mnemonic processes. Here, we
propose a novel Biologically Enhanced Artificial Neuronal assembly (BEAN)
regularization to model neuronal correlations and dependencies, inspired by
cell assembly theory from neuroscience. Experimental results show that BEAN
enables the formation of interpretable neuronal functional clusters and
consequently promotes a sparse, memory/computation-efficient network without
loss of model performance. Moreover, our few-shot learning experiments
demonstrate that BEAN could also enhance the generalizability of the model when
training samples are extremely limited.
