 rank=6)

],

query="What's your favorite sport?",has_scores=True

We can see that the reranker has reordered the documents based on the reranking scores, with the most relevant document appearing at the top of the list. The texts about sport are at the top and the less relevant ones about animals are down at the bottom.

We specified that we want a cross-encoder reranker, but you can also use other reranker models from the Hugging Face Hub, use API-driven reranker models (from Jina or Cohere, for example), or even define your own reranker model. Read their documentation to see how to use these different configurations.

In our case, we can simply add a helper function that can optionally be invoked when we want to use the reranker:

def rerank_documents(

query: str, documents: List[Tuple], reranker_model: str = "flashrank"

) -> List[Tuple[str, str]]:

"""Reranks the given documents based on the given query."""

ranker = Reranker(reranker_model)

docs_texts = [f"{doc[0]} PARENT SECTION: {doc[2]}" for doc in documents]

results = ranker.rank(query=query, docs=docs_texts)

# pair the texts with the original urls in `documents`

# `documents` is a tuple of (content, url)

# we want the urls to be returned

reranked_documents_and_urls = []

for result in results.results:

# content is a `rerankers` Result object

index_val = result.doc_id

doc_text = result.text

doc_url = documents[index_val][1]

reranked_documents_and_urls.append((doc_text, doc_url))

return reranked_documents_and_urls

This function takes a query and a list of documents (each document is a tuple of content and URL) and reranks the documents based on the query. It returns a list of tuples, where each tuple contains the reranked document text and the URL of the original document. We use the flashrank model from the rerankers package by default as it appeared to be a good choice for our use case during development.

This function then gets used in tests in the following way:

def query_similar_docs(

question: str,

url_ending: str,