Getting embeddings from wav2vec2 models
Hi, there!I am trying to get the embeddings from pre-trained wav2vec2 models (e.g., from jonatasgrosman/wav2vec2-large-xlsr-53-german) using my own dataset.My aim is to use these features for a downstream task (not specifically speech recognition). Namely, since the dataset is relatively small, I would train an SVM with these embeddings for the final classification.So far I have tried this:model_name = "facebook/wav2vec2-large-xlsr-53-german"
feature_extractor = Wav2Vec2Processor.from_pretrained(model_name)
model = Wav2Vec2Model.from_pretrained(model_name)

input_values = feature_extractor(train_dataset[:10]["speech"], return_tensors="pt", padding=True, 
                                 feature_size=1, sampling_rate=16000 ).input_valuesThen, I am not sure whether the embeddings here correspond to the sequence of last_hidden_states:hidden_states = model(input_values).last_hidden_stateor to the sequence of features of the last conv layer of the model:features_last_cnn_layer = model(input_values).extract_featuresAlso, is this the correct way to extract features from a pre-trained model?How one can get embeddings from a specific layer?
2023-03-28T08:58:57Z
synapse:.extract_featuresHi, Thank you for posting this question. Were you successful to get the hidden states or embeddings from Wav2Vec2 model?
2023-10-20T09:27:22Z
Hi, yes. The way I described in the post was indeed the correct way for getting the embeddings.