e to the query.

def get_completion_from_messages(messages, model=OPENAI_MODEL, temperature=0.4, max_tokens=1000

):

"""Generates a completion response from the given messages using the specified model."""

model = MODEL_NAME_MAP.get(model, model)

completion_response = litellm.completion(

model=model,

messages=messages,

temperature=temperature,

max_tokens=max_tokens,

return completion_response.choices[0].message.content

We're using litellm because it makes sense not to have to implement separate functions for each LLM we might want to use. The pace of development in the field is such that you will want to experiment with new LLMs as they come out, and litellm gives you the flexibility to do that without having to rewrite your code.

We've now completed a basic RAG inference pipeline that uses the embeddings generated by the pipeline to retrieve the most relevant chunks of text based on a given query. We can inspect the various components of the pipeline to see how they work together to provide a response to the query. This gives us a solid foundation to move onto more complex RAG pipelines and to look into how we might improve this. The next section will cover how to improve retrieval by finetuning the embeddings generated by the pipeline. This will boost our performance in situations where we have a large volume of documents and also when the documents are potentially very different from the training data that was used for the embeddings.

Code Example

To explore the full code, visit the Complete Guide repository and for this section, particularly the llm_utils.py file.

PreviousStoring embeddings in a vector database

NextEvaluation and metrics

Last updated 15 days ago