Robust High-dimensional Memory-augmented Neural Networks
  Traditional neural networks require enormous amounts of data to build their
complex mappings during a slow training procedure that hinders their abilities
for relearning and adapting to new data. Memory-augmented neural networks
enhance neural networks with an explicit memory to overcome these issues.
Access to this explicit memory, however, occurs via soft read and write
operations involving every individual memory entry, resulting in a bottleneck
when implemented using the conventional von Neumann computer architecture. To
overcome this bottleneck, we propose a robust architecture that employs a
computational memory unit as the explicit memory performing analog in-memory
computation on high-dimensional (HD) vectors, while closely matching 32-bit
software-equivalent accuracy. This is achieved by a content-based attention
mechanism that represents unrelated items in the computational memory with
uncorrelated HD vectors, whose real-valued components can be readily
approximated by binary, or bipolar components. Experimental results demonstrate
the efficacy of our approach on few-shot image classification tasks on the
Omniglot dataset using more than 256,000 phase-change memory devices. Our
approach effectively merges the richness of deep neural network representations
with HD computing that paves the way for robust vector-symbolic manipulations
applicable in reasoning, fusion, and compression.
