Rethinking Hard-Parameter Sharing in Multi-Domain Learning
  Hard parameter sharing in multi-domain learning (MDL) allows domains to share
some of the model parameters to reduce storage cost while improving prediction
accuracy. One common sharing practice is to share the bottom layers of a deep
neural network among domains while using separate top layers for each domain.
In this work, we revisit this common practice via an empirical study on image
classification tasks from a diverse set of visual domains and make two
surprising observations. (1) Using separate bottom-layer parameters could
achieve significantly better performance than the common practice and this
phenomenon holds with different experimental settings. (2) A multi-domain model
with a small proportion of domain-specific parameters from bottom layers can
achieve competitive performance with independent models trained on each domain
separately. Our observations suggest that people adopt the new strategy of
using separate bottom-layer parameters as a stronger baseline for model design
in MDL.
