RAG Embeddings: German language
Hi,I am new to the RAG approach and looked at many tutorials. For the Embedding Step I saw that many people are using the “sentence-transformers/all-MiniLM-L6-v2” from Huggingface, which works fine for English texts. Now I want to use RAG on German texts, so can I still use the Sentece Transformers Embedding or do I have to use a German one?If so, what are good open-source embeddings for the german language?Thanks in advance!
2023-12-05T14:35:54Z
Hi@moxI just saw your post and i was wondering If you had come across something specific. Currently I am checking / experiementing withLeoLM/leo-mistral-hessianai-7b-chat · Hugging Facemodel and its applications for QA retrieval using llama index. I am also looking for the same answer as i think the accuracy depends heavily on the embedding model . Let me know your thoughts
2024-02-02T07:19:42Z
Hi Tim,atm I am using intfloat/multilingual-e5-large which works okay. Also I wanted to check the JinaAI embeddings which look promising (“Jina Ich bin ein Berliner Embeddings”)
2024-02-19T14:36:17Z
Hi@mox– a late answer.I tested several different “smaller” embeddings before I start using possible Mistral-Embeddings and I stumbled upon “danielheinz/e5-base-sts-en-de”.My “Use-Cases” are 100% for administrational documents – so there is a “huge” context (f.ex. check all german laws) which has to be embedded. I’ve tested (very very simple) the embeddings with 6 “short” searches with “synonyms” (~13.000 different lines of text) to find services of the administration.Only with “e5-base-sts-en-de” I got 100% – failed RAG-embeddings-searches were the following:multilingual-e5-baseparaphrase-multilingual-MiniLM-L12-v2paraphrase-multilingual-mpnet-base-v2gte-largegbert-baseBut this is only a momentary snapshot an is not representative.
2024-02-27T16:02:05Z
Hi Marc,we tried to use your recommended embedding model but we ran into an index out of range error:IndexError: index out of range in selfaccording to this link this problem seems to be fixable by adjusting the vocab size:github.com/huggingface/transformersIndexError: index out of range in selfopened07:19PM - 08 Jul 20 UTCclosed08:41AM - 30 Jul 20 UTCmonk1337# 🐛 Bug

## Information

The model I am using Bert ('bert-large-uncased') an…d I am facing two issues related to this model 

The language I am using the model on English

The problem arises when using:

When I am trying to encode a large sentence ( sentence length 500 words ), I am getting this error :

`IndexError: index out of range in self`

I tried to set max_words length as 400, still getting same error : 


Data I am using can be downloaded like this : 

```
from sklearn.datasets import fetch_20newsgroups
import re

categories = ['alt.atheism', 'soc.religion.christian','comp.graphics', 'sci.med']
twenty_train = fetch_20newsgroups(subset='train',categories=categories, shuffle=True, random_state=42)

print("\n".join(twenty_train.data[0].split("\n")[:3]))


X_tratado = []

for email in range(0, len(twenty_train.data)): 
    
    # Remover caracteres especiais
    texto = re.sub(r'\\r\\n', ' ', str(twenty_train.data[email]))
    texto = re.sub(r'\W', ' ', texto)
    
    # Remove caracteres simples de uma letra
    texto = re.sub(r'\s+[a-zA-Z]\s+', ' ', texto)
    texto = re.sub(r'\^[a-zA-Z]\s+', ' ', texto) 

    # Substitui multiplos espaços por um unico espaço
    texto = re.sub(r'\s+', ' ', texto, flags=re.I)

    # Remove o 'b' que aparece no começo
    texto = re.sub(r'^b\s+', '', texto)

    # Converte para minúsculo
    texto = texto.lower()

    X_tratado.append(texto)
    
dr = {}
dr ['text'] = X_tratado
dr ['labels'] = twenty_train.target
```

Now I am  using bert model to encode the sentences : 


```
from transformers import BertModel, BertConfig, BertTokenizer
import torch
    
tokenizer = BertTokenizer.from_pretrained('bert-large-uncased')
model     = BertModel.from_pretrained('bert-large-uncased')
inputs    = tokenizer(datar[7], return_tensors="pt")
outputs   = model(**inputs)
features  = outputs[0][:,0,:].detach().numpy().squeeze()
```

Which is giving this error  : 

```
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-41-5dcf440b245f> in <module>
      5 model     = BertModel.from_pretrained('bert-large-uncased')
      6 inputs    = tokenizer(datar[7], return_tensors="pt")
----> 7 outputs   = model(**inputs)
      8 features  = outputs[0][:,0,:].detach().numpy().squeeze()

~/tfproject/tfenv/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

~/tfproject/tfenv/lib/python3.7/site-packages/transformers/modeling_bert.py in forward(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, encoder_hidden_states, encoder_attention_mask, output_attentions, output_hidden_states)
    751 
    752         embedding_output = self.embeddings(
--> 753             input_ids=input_ids, position_ids=position_ids, token_type_ids=token_type_ids, inputs_embeds=inputs_embeds
    754         )
    755         encoder_outputs = self.encoder(

~/tfproject/tfenv/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

~/tfproject/tfenv/lib/python3.7/site-packages/transformers/modeling_bert.py in forward(self, input_ids, token_type_ids, position_ids, inputs_embeds)
    177         if inputs_embeds is None:
    178             inputs_embeds = self.word_embeddings(input_ids)
--> 179         position_embeddings = self.position_embeddings(position_ids)
    180         token_type_embeddings = self.token_type_embeddings(token_type_ids)
    181 

~/tfproject/tfenv/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

~/tfproject/tfenv/lib/python3.7/site-packages/torch/nn/modules/sparse.py in forward(self, input)
    112         return F.embedding(
    113             input, self.weight, self.padding_idx, self.max_norm,
--> 114             self.norm_type, self.scale_grad_by_freq, self.sparse)
    115 
    116     def extra_repr(self):

~/tfproject/tfenv/lib/python3.7/site-packages/torch/nn/functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
   1722         # remove once script supports set_grad_enabled
   1723         _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-> 1724     return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
   1725 
   1726 

IndexError: index out of range in self
```

The second issue I am facing, When I am using this bert model to encode many sentences, It seems Bert is not using GPU : 

![Screenshot 2020-07-09 at 12 45 14 AM](https://user-images.githubusercontent.com/17107749/86960748-9c905980-c17d-11ea-8d1e-bb72141cbf37.png)

How to accelerate GPU while using bert model?


## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: '3.0.0' 
- Platform:  Ubuntu 18.04.4 LTS
- Python version: python3.7
- PyTorch version (GPU?):
- Tensorflow version (GPU?): '2.2.0
- Using GPU in script?:
- Using distributed or parallel set-up in script?:Did you run in the same error and how did you fix it?Thanks a lot!
2024-03-13T12:22:36Z
Thanks a lot! This also helps me in our use case (regulations, energy market).
2024-03-13T14:19:13Z
Uh. No - sorry. I (gladly) did not stumble upon this problem … yet.
2024-03-13T14:22:03Z
It’s not by accident the “BImSchG”? – “Bundesimmissionsschutzgesetz”(coz I wanted to test that as a PoC)
2024-03-13T19:02:31Z
Thanks a lot for your reply!
2024-03-14T16:06:40Z
Hi marc, all relevant laws, regulations and other documents in the field of green fuels will be covered. this also includes the BlmSchG :). If you are interested in talking, we can exchange our views.
2024-05-23T10:24:15Z
Hi, I also try to implement a RAG application for german data, in the energy / construction sector.Do I also have to add the prefixes, “query:” and “passage:” with the fine-tuned model “e5-base-sts-en-de”?Also what llm do you use for german and how did you specify your prompts?I just adjusted the standard  prompts in LlamaIndex with “Only answer in german”, with llama3 as llm it seems to mostly work.