Toward Among-Device AI from On-Device AI with Stream Pipelines
  Modern consumer electronic devices often provide intelligence services with
deep neural networks. We have started migrating the computing locations of
intelligence services from cloud servers (traditional AI systems) to the
corresponding devices (on-device AI systems). On-device AI systems generally
have the advantages of preserving privacy, removing network latency, and saving
cloud costs. With the emergent of on-device AI systems having relatively low
computing power, the inconsistent and varying hardware resources and
capabilities pose difficulties. Authors' affiliation has started applying a
stream pipeline framework, NNStreamer, for on-device AI systems, saving
developmental costs and hardware resources and improving performance. We want
to expand the types of devices and applications with on-device AI services
products of both the affiliation and second/third parties. We also want to make
each AI service atomic, re-deployable, and shared among connected devices of
arbitrary vendors; we now have yet another requirement introduced as it always
has been. The new requirement of "among-device AI" includes connectivity
between AI pipelines so that they may share computing resources and hardware
capabilities across a wide range of devices regardless of vendors and
manufacturers. We propose extensions of the stream pipeline framework,
NNStreamer, for on-device AI so that NNStreamer may provide among-device AI
capability. This work is a Linux Foundation (LF AI and Data) open source
project accepting contributions from the general public.
