HALP: Hardware-Aware Latency Pruning
  Structural pruning can simplify network architecture and improve inference
speed. We propose Hardware-Aware Latency Pruning (HALP) that formulates
structural pruning as a global resource allocation optimization problem, aiming
at maximizing the accuracy while constraining latency under a predefined
budget. For filter importance ranking, HALP leverages latency lookup table to
track latency reduction potential and global saliency score to gauge accuracy
drop. Both metrics can be evaluated very efficiently during pruning, allowing
us to reformulate global structural pruning under a reward maximization problem
given target constraint. This makes the problem solvable via our augmented
knapsack solver, enabling HALP to surpass prior work in pruning efficacy and
accuracy-efficiency trade-off. We examine HALP on both classification and
detection tasks, over varying networks, on ImageNet and VOC datasets. In
particular, for ResNet-50/-101 pruning on ImageNet, HALP improves network
throughput by $1.60\times$/$1.90\times$ with $+0.3\%$/$-0.2\%$ top-1 accuracy
changes, respectively. For SSD pruning on VOC, HALP improves throughput by
$1.94\times$ with only a $0.56$ mAP drop. HALP consistently outperforms prior
art, sometimes by large margins.
