What should be used as sentence embedding for BertModel?
I want to get sentences’ embedding vectors for other classification taskstokenizer = BertTokenizer.from_pretrained("bert-base-uncased")
model = BertModel.from_pretrained("bert-base-uncased")
inputs = tokenizer('this is a test.', return_tensors="pt")
outputs = model(**inputs)If I do this way:embedding_of_sentence = outputs[1]Here, according to the documentation, the outputs[1] is the:* **pooler_output** (torch.FloatTensorof shape(batch_size, hidden_size)) – Last layer hidden-state of the first token of the sequence (classification token) further processed by a Linear layer and a Tanh activation function. The Linear layer weights are trained from the next sentence prediction (classification) objective during pretraining.The last layer hidden state of the first token CLS of the sentence for classification, which seems right.However, in another post, they are suggesting using “usually only take the hidden states of the [CLS] token of the last layer”,github.com/huggingface/transformersword or sentence embedding from BERT modelopened01:54PM - 26 Nov 19 UTCclosed11:12AM - 03 Aug 20 UTCghostwontfixHow can I extract embeddings for a sentence or a set of words directly from pre-…trained models (Standard BERT)? For example, I am using Spacy for this purpose at the moment where I can do it as follows:

sentence vector:
`sentence_vector = bert_model("This is an apple").vector`

word_vectors:
```
words = bert_model("This is an apple")
word_vectors = [w.vector for w in words]
```

I am wondering if this is possible directly with huggingface pre-trained models (especially BERT).and the code is:embedding_of_last_layer = outputs[0][0]
embedding_of_sentence =embedding_of_last_layer[0]The results from the two methods for the sentence embedding are different. Which one is right or better?