Dynamic Hard Pruning of Neural Networks at the Edge of the Internet
  Neural Networks (NN), although successfully applied to several Artificial
Intelligence tasks, are often unnecessarily over-parametrised. In edge/fog
computing, this might make their training prohibitive on resource-constrained
devices, contrasting with the current trend of decentralising intelligence from
remote data centres to local constrained devices. Therefore, we investigate the
problem of training effective NN models on constrained devices having a fixed,
potentially small, memory budget. We target techniques that are both
resource-efficient and performance effective while enabling significant network
compression. Our Dynamic Hard Pruning (DynHP) technique incrementally prunes
the network during training, identifying neurons that marginally contribute to
the model accuracy. DynHP enables a tunable size reduction of the final neural
network and reduces the NN memory occupancy during training. Freed memory is
reused by a \emph{dynamic batch sizing} approach to counterbalance the accuracy
degradation caused by the hard pruning strategy, improving its convergence and
effectiveness. We assess the performance of DynHP through reproducible
experiments on three public datasets, comparing them against reference
competitors. Results show that DynHP compresses a NN up to $10$ times without
significant performance drops (up to $3.5\%$ additional error w.r.t. the
competitors), reducing up to $80\%$ the training memory occupancy.
