Multi-path Neural Networks for On-device Multi-domain Visual
  Classification
  Learning multiple domains/tasks with a single model is important for
improving data efficiency and lowering inference cost for numerous vision
tasks, especially on resource-constrained mobile devices. However,
hand-crafting a multi-domain/task model can be both tedious and challenging.
This paper proposes a novel approach to automatically learn a multi-path
network for multi-domain visual classification on mobile devices. The proposed
multi-path network is learned from neural architecture search by applying one
reinforcement learning controller for each domain to select the best path in
the super-network created from a MobileNetV3-like search space. An adaptive
balanced domain prioritization algorithm is proposed to balance optimizing the
joint model on multiple domains simultaneously. The determined multi-path model
selectively shares parameters across domains in shared nodes while keeping
domain-specific parameters within non-shared nodes in individual domain paths.
This approach effectively reduces the total number of parameters and FLOPS,
encouraging positive knowledge transfer while mitigating negative interference
across domains. Extensive evaluations on the Visual Decathlon dataset
demonstrate that the proposed multi-path model achieves state-of-the-art
performance in terms of accuracy, model size, and FLOPS against other
approaches using MobileNetV3-like architectures. Furthermore, the proposed
method improves average accuracy over learning single-domain models
individually, and reduces the total number of parameters and FLOPS by 78% and
32% respectively, compared to the approach that simply bundles single-domain
models for multi-domain learning.
