LRH-Net: A Multi-Level Knowledge Distillation Approach for Low-Resource
  Heart Network
  An electrocardiogram (ECG) monitors the electrical activity generated by the
heart and is used to detect fatal cardiovascular diseases (CVDs).
Conventionally, to capture the precise electrical activity, clinical experts
use multiple-lead ECGs (typically 12 leads). But in recent times, large-size
deep learning models have been used to detect these diseases. However, such
models require heavy compute resources like huge memory and long inference
time. To alleviate these shortcomings, we propose a low-parameter model, named
Low Resource Heart-Network (LRH-Net), which uses fewer leads to detect ECG
anomalies in a resource-constrained environment. A multi-level knowledge
distillation process is used on top of that to get better generalization
performance on our proposed model. The multi-level knowledge distillation
process distills the knowledge to LRH-Net trained on a reduced number of leads
from higher parameter (teacher) models trained on multiple leads to reduce the
performance gap. The proposed model is evaluated on the PhysioNet-2020
challenge dataset with constrained input. The parameters of the LRH-Net are
106x less than our teacher model for detecting CVDs. The performance of the
LRH-Net was scaled up to 3.2% and the inference time scaled down by 75%
compared to the teacher model. In contrast to the compute- and
parameter-intensive deep learning techniques, the proposed methodology uses a
subset of ECG leads using the low resource LRH-Net, making it eminently
suitable for deployment on edge devices.
