Good way to output embedding for search?
I tried bert and bart to output embedding of document and query.Bart is much better than bert ,.However in zero-shot learning condition, the embedding of sentence looks good when they have similar length, but when using embedding of query which is really short, the embedding is much worse. The embedding  of query is much closer to  the short rubbish data other than the long relative data.Are there good models or training tasks to map query and document into a similar space which has no relation with the length of sentences?
2020-08-17T13:05:09Z
Have you read this article? There is a small portion that discusses why models that produce sentence embeddings are bad at embedding one or two words into a relevant latent space.Joe Davison Blog – 29 May 20Zero-Shot Learning in Modern NLPState-of-the-art NLP models for text classification without annotated data
2020-08-19T05:59:30Z
Nice article.I tried using fully connected layers to map embedding of queries, but the query has variable length other than the single word label which made it much more complicated and  having bad result.
2020-08-19T15:41:05Z
Right here are two options off the top of my head.Take the average of all of the output embeddingsUse the CLS embedding (if it is a BERT-ish model)This will ensure that you always have the same vector size (768, I think)