Device-Circuit-Architecture Co-Exploration for Computing-in-Memory
  Neural Accelerators
  Co-exploration of neural architectures and hardware design is promising to
simultaneously optimize network accuracy and hardware efficiency. However,
state-of-the-art neural architecture search algorithms for the co-exploration
are dedicated for the conventional von-neumann computing architecture, whose
performance is heavily limited by the well-known memory wall. In this paper, we
are the first to bring the computing-in-memory architecture, which can easily
transcend the memory wall, to interplay with the neural architecture search,
aiming to find the most efficient neural architectures with high network
accuracy and maximized hardware efficiency. Such a novel combination makes
opportunities to boost performance, but also brings a bunch of challenges. The
design space spans across multiple layers from device type, circuit topology to
neural architecture. In addition, the performance may degrade in the presence
of device variation. To address these challenges, we propose a cross-layer
exploration framework, namely NACIM, which jointly explores device, circuit and
architecture design space and takes device variation into consideration to find
the most robust neural architectures. Experimental results demonstrate that
NACIM can find the robust neural network with 0.45% accuracy loss in the
presence of device variation, compared with a 76.44% loss from the
state-of-the-art NAS without consideration of variation; in addition, NACIM
achieves an energy efficiency up to 16.3 TOPs/W, 3.17X higher than the
state-of-the-art NAS.
