What would be the suggested way to customize a model?
Hello,I would like to combine the outputs from a pretrained model like bert-base-uncased with some numerical data to use in a classification task. What would be the suggested way to do to this? Would I need to convert the pretrained model to pytorch and then write my own pytorch module, for example?thanks in advance for any help!
2022-02-25T17:05:33Z
If your custom model just needs to call Bert without changing any ofthisstuff, then you could just create a class on top of Bert, where Bert is just an attribute that you call when you need to. But if you want Bert to output a custom loss, logits, etc, then you should make a subclass of Huggingface’s BertWould I need to convert the pretrained model to pytorch and then write my own pytorch module, for example?You shouldn’t need to explicitly convert a pretrained HF model to PyTorch. Whenever you instantiate the model withfrom_pretrained, it’ll already be a PyTorch module.On the other hand, whether you need your custom class to be a PyTorch module too depends on what you need. If you want that to be a trainable module, then you’d likely want to create a PyTorch module. But if you just need a quick and easy class to run inference, it probably wouldn’t be necessary.