Packet-Loss-Tolerant Split Inference for Delay-Sensitive Deep Learning
  in Lossy Wireless Networks
  The distributed inference framework is an emerging technology for real-time
applications empowered by cutting-edge deep machine learning (ML) on
resource-constrained Internet of things (IoT) devices. In distributed
inference, computational tasks are offloaded from the IoT device to other
devices or the edge server via lossy IoT networks. However, narrow-band and
lossy IoT networks cause non-negligible packet losses and retransmissions,
resulting in non-negligible communication latency. This study solves the
problem of the incremental retransmission latency caused by packet loss in a
lossy IoT network. We propose a split inference with no retransmissions (SI-NR)
method that achieves high accuracy without any retransmissions, even when
packet loss occurs. In SI-NR, the key idea is to train the ML model by
emulating the packet loss by a dropout method, which randomly drops the output
of hidden units in a DNN layer. This enables the SI-NR system to obtain
robustness against packet losses. Our ML experimental evaluation reveals that
SI-NR obtains accurate predictions without packet retransmission at a packet
loss rate of 60%.
