Does changing the classification head of models always require fine-tuning?
I’m usingtwitter-xlm-roberta-base-sentimentfor sentiment analysis, but instead of predicting 3 labels (negative, neutral, positive), I need to predict sentiment as represented by a single continuous value (e.g., raning from -1 for negative to 1 for positive). Hence, I’m trying to change the classification head of the model.I’ve been experimenting with adding custom classification heads as discussed in these previous posts (hereandhere).As a sanity check, I replicated theXLMRobertaClassificationHeadand attached it to the model. However, as the weights and biases of the original classification head are gone, the predictions are of course meaningless.This leads me to my question: When changing the classification head, do I ALWAYS need to retrain the model, or can I keep the weights and biases from the previous head and use them for my modified classification head?Thanks
2023-05-05T13:07:23Z
In this case a simple manual transformation could suffice. Something like-1 * prob_negative + 1 * prob_positive