Update Frequently, Update Fast: Retraining Semantic Parsing Systems in a
  Fraction of Time
  Currently used semantic parsing systems deployed in voice assistants can
require weeks to train. Datasets for these models often receive small and
frequent updates, data patches. Each patch requires training a new model. To
reduce training time, one can fine-tune the previously trained model on each
patch, but naive fine-tuning exhibits catastrophic forgetting - degradation of
the model performance on the data not represented in the data patch. In this
work, we propose a simple method that alleviates catastrophic forgetting and
show that it is possible to match the performance of a model trained from
scratch in less than 10% of a time via fine-tuning. The key to achieving this
is supersampling and EWC regularization. We demonstrate the effectiveness of
our method on multiple splits of the Facebook TOP and SNIPS datasets.
