Custom BERT Masked LM output embedding
Hi,I trained a custom sense embeddings based on Wordnet definition and tree structure.Now I want to test the embeddings by fine tuning BERT masked LM so the model predicts the most likely sense embedding. For example:" I need to go to the [bank] today"  → bank.wn.02I’m uncertain how to accomplish this.Can I provide a set of output labels with their embeddings different from the input embedding and input_ids?Should I build a custom softmax layer from the output_hidden_states of the BERTModel