An Empirical Study of Factors Affecting Language-Independent Models
  Scaling existing applications and solutions to multiple human languages has
traditionally proven to be difficult, mainly due to the language-dependent
nature of preprocessing and feature engineering techniques employed in
traditional approaches. In this work, we empirically investigate the factors
affecting language-independent models built with multilingual representations,
including task type, language set and data resource. On two most representative
NLP tasks -- sentence classification and sequence labeling, we show that
language-independent models can be comparable to or even outperforms the models
trained using monolingual data, and they are generally more effective on
sentence classification. We experiment language-independent models with many
different languages and show that they are more suitable for typologically
similar languages. We also explore the effects of different data sizes when
training and testing language-independent models, and demonstrate that they are
not only suitable for high-resource languages, but also very effective in
low-resource languages.
