Cost-effective Deployment of BERT Models in Serverless Environment
  In this study we demonstrate the viability of deploying BERT-style models to
serverless environments in a production setting. Since the freely available
pre-trained models are too large to be deployed in this way, we utilize
knowledge distillation and fine-tune the models on proprietary datasets for two
real-world tasks: sentiment analysis and semantic textual similarity. As a
result, we obtain models that are tuned for a specific domain and deployable in
serverless environments. The subsequent performance analysis shows that this
solution results in latency levels acceptable for production use and that it is
also a cost-effective approach for small-to-medium size deployments of BERT
models, all without any infrastructure overhead.
