Large-scale neuromorphic optoelectronic computing with a reconfigurable
  diffractive processing unit
  Application-specific optical processors have been considered disruptive
technologies for modern computing that can fundamentally accelerate the
development of artificial intelligence (AI) by offering substantially improved
computing performance. Recent advancements in optical neural network
architectures for neural information processing have been applied to perform
various machine learning tasks. However, the existing architectures have
limited complexity and performance; and each of them requires its own dedicated
design that cannot be reconfigured to switch between different neural network
models for different applications after deployment. Here, we propose an
optoelectronic reconfigurable computing paradigm by constructing a diffractive
processing unit (DPU) that can efficiently support different neural networks
and achieve a high model complexity with millions of neurons. It allocates
almost all of its computational operations optically and achieves extremely
high speed of data modulation and large-scale network parameter updating by
dynamically programming optical modulators and photodetectors. We demonstrated
the reconfiguration of the DPU to implement various diffractive feedforward and
recurrent neural networks and developed a novel adaptive training approach to
circumvent the system imperfections. We applied the trained networks for
high-speed classifying of handwritten digit images and human action videos over
benchmark datasets, and the experimental results revealed a comparable
classification accuracy to the electronic computing approaches. Furthermore,
our prototype system built with off-the-shelf optoelectronic components
surpasses the performance of state-of-the-art graphics processing units (GPUs)
by several times on computing speed and more than an order of magnitude on
system energy efficiency.
