A Gradient Estimator for Time-Varying Electrical Networks with
  Non-Linear Dissipation
  We propose a method for extending the technique of equilibrium propagation
for estimating gradients in fixed-point neural networks to the more general
setting of directed, time-varying neural networks by modeling them as
electrical circuits. We use electrical circuit theory to construct a Lagrangian
capable of describing deep, directed neural networks modeled using nonlinear
capacitors and inductors, linear resistors and sources, and a special class of
nonlinear dissipative elements called fractional memristors. We then derive an
estimator for the gradient of the physical parameters of the network, such as
synapse conductances, with respect to an arbitrary loss function. This
estimator is entirely local, in that it only depends on information locally
available to each synapse. We conclude by suggesting methods for extending
these results to networks of biologically plausible neurons, e.g.
Hodgkin-Huxley neurons.
