How can I finetune an embedding model with a multi label dataset for similarity comprison?
I have a multi-label dataset with two columns,sentenceandlabel. I want to train  an embedding model for similarity comparison .Currently, I am training with the following steps:loading the origin dataset.pairing the sentences, assigning similarity 1 to pairs with the same label and 0 to others.constructing a new dataset with three columnssentence1,sentence2, andscorefinally train the model with the new datasetI found the pairing process time consuming. And it would consume a lot of disk space if I store the new dataset as there are many duplicate sentences. Is there any solution to avoid the pairing process while consuming low disk space?