ELMO Character encoder layer
I think my model would benefit from the inclusion of character encoding similar to ELMO, i.e. as discussed in CharacterBERT (https://www.aclweb.org/anthology/2020.coling-main.609.pdf) since I have loads of acronyms so I think added characters will help.Is there a layer in the huggingface library I could use to replicate this configuration? I don’t see one, nor a hugginface ELMO but I could well have missed it. Otherwise I’ll try and use AllenNLP.
2021-05-17T09:43:39Z
hey@david-waterworth, looking at the paper’s github repo (link), it seems that they just changed the embedding layer of the BERT implementation intransformersto accept character embeddings:character-bert/character_bert.py at 0519a6e22a2912bf0fdbbd49bd562cc2e5410bc7 · helboukkouri/character-bert · GitHubyou could try the same approach and see if that works!