How to specify labels-column in BERT
Hi,i’m trying to follow thehuggingface datasets tutorialto finetune a BERT model on a custom dataset for sentiment analysis. The quicktour states:rename our label column in labels which is the expected input name for labels in BertForSequenceClassification.In thedocsforto_tf_datasetit states:label_cols – Dataset column(s) to load as labels. Note that many models compute loss internally rather than letting Keras do it, in which case it is not necessary to actually pass the labels here, as long as they’re in the input columns.I am uncertain iflabel_colscan be used to specify labels for differently named columns, or if it only possible to pass labels with a column namedlabelinside thecolumnsparameter?
2022-01-19T15:55:20Z
Hi! It seems like you are following the old documentation. The new docs are available here:https://huggingface.co/docs/datasets/master(this is an example withto_tf_dataset:https://huggingface.co/docs/datasets/master/use_dataset.html#tensorflow).I am uncertain iflabel_colscan be used to specify labels for differently named columns, or if it only possible to pass labels with a column namedlabelinside thecolumnsparameter?Yes,label_colssupports columns not necessarily namedlabel.cc@Rocketknight1
2022-01-20T08:54:58Z
thanks for your reply!The new documentation also has this quote wherelabelis renamed tolabels. So if i have a dataset where the label column is not calledlabelsi have to specify it withlabel_colsright?
2022-01-20T09:14:27Z
I looked at this tutorial as well, where the label column is calledlabel, andlabel_colsis not specified here. So these two tutorials seem to show conflicting informationhttps://huggingface.co/docs/transformers/master/custom_datasets
2022-01-20T13:05:23Z
Hi@fogx, this is a good question! Here’s what’s happening into_tf_dataset:columnsspecifies the list of columns to be passed as the input to the model, andlabel_colsspecifies the list of columns to be passed to Keras at the label. For most tasks (including sentiment analysis), you will usually only want one column to be passed here, in which case it doesn’t really matter what it’s called  becauseto_tf_datasetwill only make the labels a dict when there are multiple label columns.Sentiment analysis is an example of a ‘text classification’ task, so if you want a tutorial on that specifically, please take a look atthis notebookor thecolab link.