Sm{\aa}prat: DialoGPT for Natural Language Generation of Swedish
  Dialogue by Transfer Learning
  Building open-domain conversational systems (or chatbots) that produce
convincing responses is a recognized challenge. Recent state-of-the-art (SoTA)
transformer-based models for the generation of natural language dialogue have
demonstrated impressive performance in simulating human-like, single-turn
conversations in English. This work investigates, by an empirical study, the
potential for transfer learning of such models to Swedish language. DialoGPT,
an English language pre-trained model, is adapted by training on three
different Swedish language conversational datasets obtained from publicly
available sources. Perplexity score (an automated intrinsic language model
metric) and surveys by human evaluation were used to assess the performances of
the fine-tuned models, with results that indicate that the capacity for
transfer learning can be exploited with considerable success. Human evaluators
asked to score the simulated dialogue judged over 57% of the chatbot responses
to be human-like for the model trained on the largest (Swedish) dataset. We
provide the demos and model checkpoints of our English and Swedish chatbots on
the HuggingFace platform for public use.
