How to edit different classes in transformers and have the transformer installed with the changes?
I wanted to edit some classes in the transformers for exampleBertEmbeddingshttps://github.com/huggingface/transformers/blob/4c32f9f26e6a84f0d9843fec8757e6ce640bb44e/src/transformers/models/bert/modeling_bert.py#L166and pre-train Bert from scratch on a custom dataset. But I am stuck on how to make the edit work.The process I am following is:Clone the repositorygit clone https://github.com/huggingface/transformers.gitedit the classes I needed to editInstall transformer usingcd transformers
pip install -e .The problem is I can not load any model or tokenizer using commands like:from transformers import BertModelThe error message shows:ImportError: cannot import name ‘BertModel’ from ‘transformers’ (unknown location)whileimport transformersworks perfectly fine.My questions are:How do I import theBertTokenizerorBertModelIs there a better way to achieve what I am trying to than my approach?I could be way off so any helpful suggestion is appreciated. ThanksNote: I am trying to do something like this.How to use additional input features for NER? - #2 by nielsr
2021-12-14T03:09:07Z
To clarify the problem:Usually, when we want to use the transformer in it’s original form we do:!pip install transformers
import transformers
from transformers import BertTokenizer, BertModelIf I want to change a class or two, what changes should I make to the above code snippet to import theBertTokenizerandBertModel. I am assuming cloning the repository and making the edits in the desired file is fine.
2021-12-14T03:37:04Z
Apparently, the problem was the editable versionThis works:cd transformers
pip install .editable was not what I thought at first.
2022-06-02T22:37:57Z
bengul:git cloneGitHub - huggingface/transformers: 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.Hello@bengulIs there any other way to achieve the same result without cloning the huggingface repo and making changes in the codes? It would be better in terms of ease of reproducibility if say I create a class that inherits the original class I want to make changes in (in your case BertEmbeddings) and then somehow make the from_pretrained class go through my created subclass rather than the original.
2022-06-02T22:56:51Z
Initially, my thought was to do exactly what you proposed, but I was not sure how to achieve that. I am sure it is possible and probably a better approach.