---
draft: false
preview_image: /blog/from_cms/nils-thumbnail.png
sitemapExclude: true
title: "From Content Quality to Compression: The Evolution of Embedding Models
  at Cohere with Nils Reimers"
slug: cohere-embedding-v3
short_description: Nils Reimers head of machine learning at Cohere shares the
  details about their latest embedding model.
description: Nils Reimers head of machine learning at Cohere comes on the recent
  vector space talks to share details about their latest embedding V3 model.
date: 2023-11-19T12:48:36.622Z
author: Demetrios Brinkmann
featured: false
author_link: https://www.linkedin.com/in/dpbrinkm/
tags:
  - Vector Space Talk
  - Cohere
  - Embedding Model
categories:
  - News
  - Vector Space Talk
---

For the second edition of our Vector Space Talks we were joined by none other than Cohere’s Head of Machine Learning Nils Reimers. 

## Key Takeaways

Let's dive right into the five key takeaways from Nils' talk:

1. Content Quality Estimation: Nils explained how embeddings have traditionally focused on measuring topic match, but content quality is just as important. He demonstrated how their model can differentiate between informative and non-informative documents.

2. Compression-Aware Training: He shared how they've tackled the challenge of reducing the memory footprint of embeddings, making it more cost-effective to run vector databases on platforms like [Qdrant](https://cloud.qdrant.io/login).

3. Reinforcement Learning from Human Feedback: Nils revealed how they've borrowed a technique from reinforcement learning and applied it to their embedding models. This allows the model to learn preferences based on human feedback, resulting in highly informative responses.

4. Evaluating Embedding Quality: Nils emphasized the importance of evaluating embedding quality in relative terms rather than looking at individual vectors. It's all about understanding the context and how embeddings relate to each other.

5. New Features in the Pipeline: Lastly, Nils gave us a sneak peek at some exciting features they're developing, including input type support for Langchain and improved compression techniques.

Now, here's a fun fact from the episode: Did you know that the content quality estimation model *can't* differentiate between true and fake statements? It's a challenging task, and the model relies on the information present in its pretraining data.

We loved having Nils as our guest, check out the full talk below. If you or anyone you know would like to come on the Vector Space Talks 

<iframe width="560" height="315" src="https://www.youtube.com/embed/Abh3YCahyqU?si=OB4FXhTivsLLXzQV" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
