How to add a new special token and initialize its embeddings to random values?
I would like to add a special token (like, a CLS token in a model which was not pretrained with a CLS token) and finetune it to learn representations of that token. How should I go about this?