Learning in Feedforward Neural Networks Accelerated by Transfer Entropy
  Current neural networks architectures are many times harder to train because
of the increasing size and complexity of the used datasets. Our objective is to
design more efficient training algorithms utilizing causal relationships
inferred from neural networks. The transfer entropy (TE) was initially
introduced as an information transfer measure used to quantify the statistical
coherence between events (time series). Later, it was related to causality,
even if they are not the same. There are only few papers reporting applications
of causality or TE in neural networks. Our contribution is an
information-theoretical method for analyzing information transfer between the
nodes of feedforward neural networks. The information transfer is measured by
the TE of feedback neural connections. Intuitively, TE measures the relevance
of a connection in the network and the feedback amplifies this connection. We
introduce a backpropagation type training algorithm that uses TE feedback
connections to improve its performance.
