Acceleration Method for Learning Fine-Layered Optical Neural Networks
  An optical neural network (ONN) is a promising system due to its high-speed
and low-power operation. Its linear unit performs a multiplication of an input
vector and a weight matrix in optical analog circuits. Among them, a circuit
with a multiple-layered structure of programmable Mach-Zehnder interferometers
(MZIs) can realize a specific class of unitary matrices with a limited number
of MZIs as its weight matrix. The circuit is effective for balancing the number
of programmable MZIs and ONN performance. However, it takes a lot of time to
learn MZI parameters of the circuit with a conventional automatic
differentiation (AD), which machine learning platforms are equipped with. To
solve the time-consuming problem, we propose an acceleration method for
learning MZI parameters. We create customized complex-valued derivatives for an
MZI, exploiting Wirtinger derivatives and a chain rule. They are incorporated
into our newly developed function module implemented in C++ to collectively
calculate their values in a multi-layered structure. Our method is simple,
fast, and versatile as well as compatible with the conventional AD. We
demonstrate that our method works 20 times faster than the conventional AD when
a pixel-by-pixel MNIST task is performed in a complex-valued recurrent neural
network with an MZI-based hidden unit.
