Multilabel multiclass audio classification
Hello,I have managed to adapt the audio classification tutorial to my own dataset:github.commirix/messaih/blob/main/charts/fine_tune_w2v.pyimport os
n_cores = str(os.cpu_count())
os.environ['OMP_NUM_THREADS'] = n_cores
os.environ['MKL_NUM_THREADS'] = n_cores
os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"]="0,1"

import datasets
import polars as pl
import numpy as np

from sklearn.model_selection import train_test_split

### PARQUE DATASET ###

columns = ['sentiment', 'wav2numpy']
df = pl.read_parquet('sqe_messai.parquet', columns = columns)
#df = pl.read_parquet('sqe_messai.parquet', columns = columns, n_rows = 100)

### LABELS ###This file has been truncated.show originalI can now fine-tune a wav2vec model on my dataset. I am currently fine tuning a classifier on the sentiment label.However, the dataset contains 6 other labels for emotion:github.commirix/messaih/blob/main/sqe_messai_sample.csvytid,ytid_seg,start,end,sentiment,happiness,sadness,anger,fear,disgust,surprise
-3g5yACwYnA,/home/emoman/Downloads/mosei/messai/wavs/-3g5yACwYnA_00.wav,82.753,100.555,1.0,0.6666667,0.6666667,0.0,0.0,0.0,0.6666667
-3g5yACwYnA,/home/emoman/Downloads/mosei/messai/wavs/-3g5yACwYnA_01.wav,119.919,125.299,0.6666667,0.0,0.0,0.0,0.0,0.0,0.0
-3g5yACwYnA,/home/emoman/Downloads/mosei/messai/wavs/-3g5yACwYnA_02.wav,4.84,14.052,0.0,0.6666667,0.6666667,0.0,0.0,0.0,0.33333334
-3g5yACwYnA,/home/emoman/Downloads/mosei/messai/wavs/-3g5yACwYnA_03.wav,13.211,27.521,0.0,0.33333334,0.33333334,0.0,0.0,0.0,0.0
-3g5yACwYnA,/home/emoman/Downloads/mosei/messai/wavs/-3g5yACwYnA_04.wav,26.541,41.3,1.0,0.6666667,0.0,0.0,0.0,0.0,0.0
-3g5yACwYnA,/home/emoman/Downloads/mosei/messai/wavs/-3g5yACwYnA_05.wav,74.083,82.776,0.6666667,0.0,0.6666667,0.0,0.0,0.0,0.33333334
-3nNcZdcdvU,/home/emoman/Downloads/mosei/messai/wavs/-3nNcZdcdvU_00.wav,0.0,9.213,0.33333334,1.6666666,0.0,0.0,0.0,0.0,0.0
-3nNcZdcdvU,/home/emoman/Downloads/mosei/messai/wavs/-3nNcZdcdvU_01.wav,12.014,20.098,1.0,2.3333333,0.0,0.0,0.0,0.0,0.0
-3nNcZdcdvU,/home/emoman/Downloads/mosei/messai/wavs/-3nNcZdcdvU_02.wav,36.399,44.173,0.0,1.6666666,0.0,0.0,0.0,0.0,0.0
-571d8cVauQ,/home/emoman/Downloads/mosei/messai/wavs/-571d8cVauQ_00.wav,0.0,4.992,0.0,0.0,0.0,0.0,0.0,0.0,0.0
-571d8cVauQ,/home/emoman/Downloads/mosei/messai/wavs/-571d8cVauQ_01.wav,21.842,28.309,-0.6666667,0.33333334,0.33333334,0.0,0.0,0.0,0.0
-571d8cVauQ,/home/emoman/Downloads/mosei/messai/wavs/-571d8cVauQ_02.wav,69.893,85.0,0.33333334,0.33333334,0.0,0.0,0.0,0.0,0.0
-6rXp3zJ3kc,/home/emoman/Downloads/mosei/messai/wavs/-6rXp3zJ3kc_00.wav,14.468,22.882,0.0,0.0,0.33333334,0.33333334,0.0,0.0,0.0
-6rXp3zJ3kc,/home/emoman/Downloads/mosei/messai/wavs/-6rXp3zJ3kc_01.wav,45.428,58.151,-1.0,0.0,0.33333334,0.0,0.0,0.0,0.0
-6rXp3zJ3kc,/home/emoman/Downloads/mosei/messai/wavs/-6rXp3zJ3kc_02.wav,58.109,71.96,0.33333334,0.0,0.33333334,0.0,0.0,0.0,0.0
-9YyBTjo1zo,/home/emoman/Downloads/mosei/messai/wavs/-9YyBTjo1zo_01.wav,0.81950113379,5.96009070295,-1.0,0.0,0.0,0.33333334,0.0,0.0,0.0
-9YyBTjo1zo,/home/emoman/Downloads/mosei/messai/wavs/-9YyBTjo1zo_02.wav,5.97777777778,22.6721088435,2.0,2.6666667,0.0,0.0,0.33333334,0.0,0.0
-9YyBTjo1zo,/home/emoman/Downloads/mosei/messai/wavs/-9YyBTjo1zo_03.wav,32.8167800454,42.9959183673,-0.33333334,0.33333334,0.0,0.33333334,0.0,0.6666667,0.0
-9YyBTjo1zo,/home/emoman/Downloads/mosei/messai/wavs/-9YyBTjo1zo_04.wav,42.853968254,47.7850340136,-1.6666666,1.0,0.0,0.0,0.33333334,1.0,0.0This file has been truncated.show originalEach label, can have up to 15 different classes.The question is how to train a model using the six emotion labels as target simultaneously.Would it be possible to group all six labels as a list or an array and use that as a single target?I have found a few old posts and articles providing some pointers but I am not sure how up to date they are and I do not really understand the proposed solutions.Any hints would be most appreciated.Best,Ed