Multiclass vs Multilabel
Hello all,I’m relatively new to transformer library and its models.What I’m try-out is fine-tuning pretrained model for classification task.I understood that BertForSequenceClassification is for classicalmulti-classclassification.However, I’m confused on how to achieve “multi-label” classification task.Can anyone guide me to multilabel classification?ps.)What I mean by Multiclass, it means labels are exclusive to each otherex)num_labels = 5labels=[0, 1, 0, 0, 0]Multilabelnum_labels = 5labels = [1, 0, 0, 1, 0]
2020-08-11T09:26:17Z
See howtransformers.modeling_bert.BertForSequenceClassificationis implemented - there is aCrossEntropyLossbuilt into this class internally. You can create custom class (PyTorch module) similar toBertForSequenceClassificationand apply multi-label loss of your choice there. You can also look-up my blogposthttps://zablo.net/blog/post/custom-classifier-on-bert-model-guide-polemo2-sentiment-analysis/where I show how to build totally custom model on top of the pre-trained transformers models.