Improving NER's Performance with Massive financial corpus
  Training large deep neural networks needs massive high quality annotation
data, but the time and labor costs are too expensive for small business. We
start a company-name recognition task with a small scale and low quality
training data, then using skills to enhanced model training speed and
predicting performance with minimum labor cost. The methods we use involve
pre-training a lite language model such as Albert-small or Electra-small in
financial corpus, knowledge of distillation and multi-stage learning. The
result is that we raised the recall rate by nearly 20 points and get 4 times as
fast as BERT-CRF model.
