DeepSteal: Advanced Model Extractions Leveraging Efficient Weight
  Stealing in Memories
  Recent advancements of Deep Neural Networks (DNNs) have seen widespread
deployment in multiple security-sensitive domains. The need of
resource-intensive training and use of valuable domain-specific training data
have made these models a top intellectual property (IP) for model owners. One
of the major threats to the DNN privacy is model extraction attacks where
adversaries attempt to steal sensitive information in DNN models. Recent
studies show hardware-based side channel attacks can reveal internal knowledge
about DNN models (e.g., model architectures) However, to date, existing attacks
cannot extract detailed model parameters (e.g., weights/biases). In this work,
for the first time, we propose an advanced model extraction attack framework
DeepSteal that effectively steals DNN weights with the aid of memory
side-channel attack. Our proposed DeepSteal comprises two key stages. Firstly,
we develop a new weight bit information extraction method, called HammerLeak,
through adopting the rowhammer based hardware fault technique as the
information leakage vector. HammerLeak leverages several novel system-level
techniques tailed for DNN applications to enable fast and efficient weight
stealing. Secondly, we propose a novel substitute model training algorithm with
Mean Clustering weight penalty, which leverages the partial leaked bit
information effectively and generates a substitute prototype of the target
victim model. We evaluate this substitute model extraction method on three
popular image datasets (e.g., CIFAR-10/100/GTSRB) and four DNN architectures
(e.g., ResNet-18/34/Wide-ResNet/VGG-11). The extracted substitute model has
successfully achieved more than 90 % test accuracy on deep residual networks
for the CIFAR-10 dataset. Moreover, our extracted substitute model could also
generate effective adversarial input samples to fool the victim model.
