Compressing RNNs for IoT devices by 15-38x using Kronecker Products
  Recurrent Neural Networks (RNN) can be difficult to deploy on resource
constrained devices due to their size.As a result, there is a need for
compression techniques that can significantly compress RNNs without negatively
impacting task accuracy. This paper introduces a method to compress RNNs for
resource constrained environments using Kronecker product (KP). KPs can
compress RNN layers by 15-38x with minimal accuracy loss. By quantizing the
resulting models to 8-bits, we further push the compression factor to 50x. We
show that KP can beat the task accuracy achieved by other state-of-the-art
compression techniques across 5 benchmarks spanning 3 different applications,
while simultaneously improving inference run-time. We show that the KP
compression mechanism does introduce an accuracy loss, which can be mitigated
by a proposed hybrid KP (HKP) approach. Our HKP algorithm provides fine-grained
control over the compression ratio, enabling us to regain accuracy lost during
compression by adding a small number of model parameters.
