Neural Network Compression for Noisy Storage Devices
  Compression and efficient storage of neural network (NN) parameters is
critical for applications that run on resource-constrained devices. Although NN
model compression has made significant progress, there has been considerably
less investigation in the actual physical storage of NN parameters.
Conventionally, model compression and physical storage are decoupled, as
digital storage media with error correcting codes (ECCs) provide robust
error-free storage. This decoupled approach is inefficient, as it forces the
storage to treat each bit of the compressed model equally, and to dedicate the
same amount of resources to each bit. We propose a radically different approach
that: (i) employs analog memories to maximize the capacity of each memory cell,
and (ii) jointly optimizes model compression and physical storage to maximize
memory utility. We investigate the challenges of analog storage by studying
model storage on phase change memory (PCM) arrays and develop a variety of
robust coding strategies for NN model storage. We demonstrate the efficacy of
our approach on MNIST, CIFAR-10 and ImageNet datasets for both existing and
novel compression methods. Compared to conventional error-free digital storage,
our method has the potential to reduce the memory size by one order of
magnitude, without significantly compromising the stored model's accuracy.
