passage vs search_document
When do I use what?
is the parameter “passage” just a synonym for “search_document” ? or how do I understand this? Your documentation is unfortunately a bit sparse
Thank you!
Your benchmarks are promising, I'll test it!
Hello!
Apologies for the confusion. "passage" is the prompt name and "search_document: " is the prompt.
See also https://huggingface.co/nomic-ai/nomic-embed-text-v2-moe/blob/main/config_sentence_transformers.json#L9
In Sentence Transformers, you can either do model.encode(..., prompt_name="passage") or model.encode(..., prompt="search_document: "). They are exactly equivalent.
In Transformers, there is no prompt name or something, you just have to add the full prompt yourself manually (i.e., "search_document: ").
- Tom Aarsen
a little more color on when to use either for question answering/retrieval: search_query is usually for questions while search_document is for the corpus/documents you're searching over
I'll update the docs to make it a little clearer!