ry_similar_docs(

question: str,

url_ending: str,use_reranking: bool = False,

returned_sample_size: int = 5,

) -> Tuple[str, str, List[str]]:

"""Query similar documents for a given question and URL ending."""

embedded_question = get_embeddings(question)

db_conn = get_db_conn()

num_docs = 20 if use_reranking else returned_sample_size

# get (content, url) tuples for the top n similar documents

top_similar_docs = get_topn_similar_docs(

embedded_question, db_conn, n=num_docs, include_metadata=True

if use_reranking:

reranked_docs_and_urls = rerank_documents(question, top_similar_docs)[

:returned_sample_size

urls = [doc[1] for doc in reranked_docs_and_urls]

else:

urls = [doc[1] for doc in top_similar_docs]  # Unpacking URLs

return (question, url_ending, urls)

We get the embeddings for the question being passed into the function and connect to our PostgreSQL database. If we're using reranking, we get the top 20 documents similar to our query and rerank them using the rerank_documents helper function. We then extract the URLs from the reranked documents and return them. Note that we only return 5 URLs, but in the case of reranking we get a larger number of documents and URLs back from the database to pass to our reranker, but in the end we always choose the top five reranked documents to return.

Now that we've added reranking to our pipeline, we can evaluate the performance of our reranker and see how it affects the quality of the retrieved documents.

Code Example

To explore the full code, visit the Complete Guide repository and for this section, particularly the eval_retrieval.py file.

PreviousUnderstanding reranking

NextEvaluating reranking performance

Last updated 15 days ago