alvarobartt HF Staff commited on
Commit
c99f96f
·
verified ·
1 Parent(s): 20bdd57

Add `text-embeddings-inference` tag & snippet

Browse files

## Description

- Add `text-embeddings-inference` tag to improve discoverability
- Adds a sample snippet on how to run Text Embeddings Inference (TEI) via Docker

⚠️ **This PR has been generated automatically, so please review it before merging.**

Files changed (1) hide show
  1. README.md +26 -1
README.md CHANGED
@@ -6,6 +6,7 @@ tags:
6
  - feature-extraction
7
  - sentence-similarity
8
  - transformers
 
9
  pipeline_tag: sentence-similarity
10
  ---
11
 
@@ -65,7 +66,7 @@ encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tenso
65
  with torch.no_grad():
66
  model_output = model(**encoded_input)
67
 
68
- # Perform pooling. In this case, max pooling.
69
  sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])
70
 
71
  print("Sentence embeddings:")
@@ -73,6 +74,30 @@ print(sentence_embeddings)
73
  ```
74
 
75
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
76
 
77
  ## Full Model Architecture
78
  ```
 
6
  - feature-extraction
7
  - sentence-similarity
8
  - transformers
9
+ - text-embeddings-inference
10
  pipeline_tag: sentence-similarity
11
  ---
12
 
 
66
  with torch.no_grad():
67
  model_output = model(**encoded_input)
68
 
69
+ # Perform pooling. In this case, mean pooling.
70
  sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])
71
 
72
  print("Sentence embeddings:")
 
74
  ```
75
 
76
 
77
+ ## Usage (Text Embeddings Inference (TEI))
78
+
79
+ [Text Embeddings Inference (TEI)](https://github.com/huggingface/text-embeddings-inference) is a blazing fast inference solution for text embeddings models.
80
+
81
+ - CPU:
82
+ ```bash
83
+ docker run -p 8080:80 -v hf_cache:/data --pull always ghcr.io/huggingface/text-embeddings-inference:cpu-latest --model-id sentence-transformers/nli-mpnet-base-v2 --pooling mean --dtype float16
84
+ ```
85
+
86
+ - NVIDIA GPU:
87
+ ```bash
88
+ docker run --gpus all -p 8080:80 -v hf_cache:/data --pull always ghcr.io/huggingface/text-embeddings-inference:cuda-latest --model-id sentence-transformers/nli-mpnet-base-v2 --pooling mean --dtype float16
89
+ ```
90
+
91
+ Send a request to `/v1/embeddings` to generate embeddings via the [OpenAI Embeddings API](https://platform.openai.com/docs/api-reference/embeddings/create):
92
+ ```bash
93
+ curl http://localhost:8080/v1/embeddings \
94
+ -H "Content-Type: application/json" \
95
+ -d '{"model":"sentence-transformers/nli-mpnet-base-v2","input":"This is an example sentence"}'
96
+ ```
97
+
98
+ Or check the [Text Embeddings Inference API specification](https://huggingface.github.io/text-embeddings-inference/) instead.
99
+
100
+
101
 
102
  ## Full Model Architecture
103
  ```