{'fraction_non_alphanumeric': 0.025900189513581806, 'fraction_numerical': 0.0, 'mean_word_length': 5.46530612244898, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
OCDST: Offloading Chained DNNs for Streaming Tasks

Considering the contradiction between limited re-sources in small devices and the high complexity of deep neural networks (DNNs), DNNs can hardly run on small devices such as smartphones and wearable devices. Therefore, offloading DNNs to computing units (fog/edge servers), where each unit executes a part of a DNN collaboratively, has gained increasing popularity. Notably, DNNs are generally in chained structure, while streaming tasks are the central part of artificial-intelligent applications. Thus it is crucial to reduce chained DNNs' delay for streaming tasks. Although existing works have advanced DNN offloading largely, the discussion about chained DNNs and streaming tasks is negligible. To address this issue, in this paper, we propose a layer-level offloading model called OCDST based on the analysis about them. After chained DNNs are offloaded to computing units, the involved units will handle streaming tasks as a pipeline. Consequently, the model significantly reduces the average task delay by paralleling each step in the pipeline. Moreover, the global optimal model solution is drawn by an improved depth-first search (DFS) algorithm, which utilizes DFS to achieve path establishment, calculation and record stages. Based on multi-threading programming and producer-consumer pattern, a program parallelization scheme is also devised to ensure the feasibility of the obtained optimum. Experimental results show that OCDST significantly outperforms recent works with higher inferring speed and faster response.