Distilroberta vs SecBert for finance Sentiment Analysis
Hello there, have been trying to apply sentiment analysis to financial text and managed to finetune distilroberta on a combination of financial phrasebank data and some Covid related data from Kaggle which contained the impact of Covid of company profits. The F1 was not bad at .89 (nickmuchi/distilroberta-finetuned-finclass). I then came across sec-bert recently which was trained on 270k documents of financial text from the US SEC and thought it would give me better results than the finetuned distilroberta but got roughly the same F1 (.87) after finetuning it (nickmuchi/sec-bert-finetuned-finance-classification) with the same data. Was a bit surprised as I thought it would perform better given the financial text it was trained on so would have a good grasp of the finance context and vernacular. I am very fresh to HF and NLP so wondering if there is something I am misunderstanding or missing in my thinking and rationale. Thanks