The Neuro-Symbolic Brain
  Neural networks promote a distributed representation with no clear place for
symbols. Despite this, we propose that symbols are manufactured simply by
training a sparse random noise as a self-sustaining attractor in a feedback
spiking neural network. This way, we can generate many of what we shall call
prime attractors, and the networks that support them are like registers holding
a symbolic value, and we call them registers. Like symbols, prime attractors
are atomic and devoid of any internal structure. Moreover, the winner-take-all
mechanism naturally implemented by spiking neurons enables registers to recover
a prime attractor within a noisy signal. Using this faculty, when considering
two connected registers, an input one and an output one, it is possible to bind
in one shot using a Hebbian rule the attractor active on the output to the
attractor active on the input. Thus, whenever an attractor is active on the
input, it induces its bound attractor on the output; even though the signal
gets blurrier with more bindings, the winner-take-all filtering faculty can
recover the bound prime attractor. However, the capacity is still limited. It
is also possible to unbind in one shot, restoring the capacity taken by that
binding. This mechanism serves as a basis for working memory, turning prime
attractors into variables. Also, we use a random second-order network to
amalgamate the prime attractors held by two registers to bind the prime
attractor held by a third register to them in one shot, de facto implementing a
hash table. Furthermore, we introduce the register switch box composed of
registers to move the content of one register to another. Then, we use spiking
neurons to build a toy symbolic computer based on the above. The technics used
suggest ways to design extrapolating, reusable, sample-efficient deep learning
networks at the cost of structural priors.
