What and Where: Learn to Plug Adapters via NAS for Multi-Domain Learning
  As an important and challenging problem, multi-domain learning (MDL)
typically seeks for a set of effective lightweight domain-specific adapter
modules plugged into a common domain-agnostic network. Usually, existing ways
of adapter plugging and structure design are handcrafted and fixed for all
domains before model learning, resulting in the learning inflexibility and
computational intensiveness. With this motivation, we propose to learn a
data-driven adapter plugging strategy with Neural Architecture Search (NAS),
which automatically determines where to plug for those adapter modules.
Furthermore, we propose a NAS-adapter module for adapter structure design in a
NAS-driven learning scheme, which automatically discovers effective adapter
module structures for different domains. Experimental results demonstrate the
effectiveness of our MDL model against existing approaches under the conditions
of comparable performance.
