Upload README.md with huggingface_hub

README.md

@@ -1,144 +1,127 @@
 ---
 tags:
+- transformers
 - sentence-transformers
-- sentence-similarity
-- feature-extraction
-- dense
-pipeline_tag: sentence-similarity
-library_name: sentence-transformers
+language:
+- en
+license: apache-2.0
+library_name: transformers
+base_model:
+- bge-base-en-v1.5
+model_index:
+- name: kpr-bge-base-en-v1.5
+results:
 ---
 
-# SentenceTransformer
+# Knowledgeable Embedding: kpr-bge-base-en-v1.5
 
-This is a [sentence-transformers](https://www.SBERT.net) model trained. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+## Introduction
 
-## Model Details
+**Injecting dynamically updatable entity knowledge into embeddings to enhance RAG**
 
-### Model Description
-- **Model Type:** Sentence Transformer
-<!-- - **Base model:** [Unknown](https://huggingface.co/unknown) -->
-- **Maximum Sequence Length:** 512 tokens
-- **Output Dimensionality:** 768 dimensions
-- **Similarity Function:** Cosine Similarity
-<!-- - **Training Dataset:** Unknown -->
-<!-- - **Language:** Unknown -->
-<!-- - **License:** Unknown -->
+A key limitation of large language models (LLMs) is their inability to capture less-frequent or up-to-date entity knowledge, often leading to factual inaccuracies and hallucinations. Retrieval-augmented generation (RAG), which incorporates external knowledge through retrieval, is a common approach to mitigate this issue.
 
-### Model Sources
+Although RAG typically relies on embedding-based retrieval, the embedding models themselves are also based on language models and therefore struggle with queries involving less-frequent entities, often failing to retrieve the crucial knowledge needed to overcome this limitation.
 
-- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
-- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
-- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+**Knowledgeable Embedding** addresses this challenge by injecting real-world entity knowledge into embeddings, making them more *knowledgeable*.
 
-### Full Model Architecture
+**The entity knowledge is pluggable and can be dynamically updated.**
 
-```
-SentenceTransformer(
-  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'KPRModelForBert'})
-  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
-  (2): Normalize()
-)
-```
+For further details, refer to [our paper](https://arxiv.org/abs/2507.03922) or [GitHub repository](https://github.com/knowledgeable-embedding/knowledgeable-embedding).
 
-## Usage
+## Model List
 
-### Direct Usage (Sentence Transformers)
+| Model | Model Size | Base Model |
+| --- | --- | --- |
+| [knowledgeable-ai/kpr-bert-base-uncased](https://huggingface.co/knowledgeable-ai/kpr-bert-base-uncased) | 112M | [bert-base-uncased](https://huggingface.co/google-bert/bert-base-uncased) |
+| [knowledgeable-ai/kpr-retromae](https://huggingface.co/knowledgeable-ai/kpr-retromae) | 112M | [RetroMAE](https://huggingface.co/Shitao/RetroMAE) |
+| [knowledgeable-ai/kpr-bge-base-en](https://huggingface.co/knowledgeable-ai/kpr-bge-base-en) | 112M | [bge-base-en](https://huggingface.co/BAAI/bge-base-en) |
+| [knowledgeable-ai/kpr-bge-base-en-v1.5](https://huggingface.co/knowledgeable-ai/kpr-bge-base-en-v1.5) | 112M | [bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) |
+| [knowledgeable-ai/kpr-bge-large-en-v1.5](https://huggingface.co/knowledgeable-ai/kpr-bge-large-en-v1.5) | 340M | [bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5) |
 
-First install the Sentence Transformers library:
+For practical use, we recommend `knowledgeable-ai/kpr-bge-*`, which significantly outperforms state-of-the-art models on queries involving less-frequent entities while performing comparably on other queries, as reported in [our paper](https://arxiv.org/abs/2507.03922).
 
-```bash
-pip install -U sentence-transformers
-```
+Regarding the model size, we do not count the entity embeddings since they are stored in CPU memory and have a negligible impact on runtime performance. See [this page](https://github.com/knowledgeable-embedding/knowledgeable-embedding/wiki/Internals-of-Knowledgeable-Embedding) for details.
 
-Then you can load this model and run inference.
-```python
-from sentence_transformers import SentenceTransformer
+## Model Details
 
-# Download from the 🤗 Hub
-model = SentenceTransformer("knowledgeable-ai/kpr-bge-base-en-v1.5")
-# Run inference
-sentences = [
-    'The weather is lovely today.',
-    "It's so sunny outside!",
-    'He drove to the stadium.',
-]
-embeddings = model.encode(sentences)
-print(embeddings.shape)
-# [3, 768]
-
-# Get the similarity scores for the embeddings
-similarities = model.similarity(embeddings, embeddings)
-print(similarities)
-# tensor([[1.0000, 0.7985, 0.4422],
-#         [0.7985, 1.0000, 0.4318],
-#         [0.4422, 0.4318, 1.0000]])
-```
+- Model Name: kpr-bge-base-en-v1.5
+- Base Model: [bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5)
+- Maximum Sequence Length: 512
+- Embedding Dimension: 768
 
-<!--
-### Direct Usage (Transformers)
+## Usage
 
-<details><summary>Click to see the direct usage in Transformers</summary>
+This model can be used via [Hugging Face Transformers](https://github.com/huggingface/transformers) or [Sentence Transformers](https://github.com/UKPLab/sentence-transformers):
 
-</details>
--->
+### Hugging Face Transformers
 
-<!--
-### Downstream Usage (Sentence Transformers)
+```python
+from transformers import AutoTokenizer, AutoModel
+import torch
 
-You can finetune this model on your own dataset.
+MODEL_NAME_OR_PATH = "knowledgeable-ai/kpr-bge-base-en-v1.5"
 
-<details><summary>Click to expand</summary>
+input_texts = [
+    "Who founded Dominican Liberation Party?",
+    "Who owns Mompesson House?"
+]
 
-</details>
--->
+# Load model and tokenizer from the Hugging Face Hub
+tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME_OR_PATH, trust_remote_code=True)
+model = AutoModel.from_pretrained(MODEL_NAME_OR_PATH, trust_remote_code=True)
 
-<!--
-### Out-of-Scope Use
+# Preprocess the text
+preprocessed_inputs = tokenizer(input_texts, return_tensors="pt", padding=True)
 
-*List how the model may foreseeably be misused and address what users ought not to do with the model.*
--->
+# Compute embeddings
+with torch.no_grad():
+    embeddings = model.encode(**preprocessed_inputs)
 
-<!--
-## Bias, Risks and Limitations
+print("Embeddings:", embeddings)
+```
 
-*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
--->
+### Sentence Transformers
 
-<!--
-### Recommendations
+```python
+from sentence_transformers import SentenceTransformer
 
-*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
--->
+MODEL_NAME_OR_PATH = "knowledgeable-ai/kpr-bge-base-en-v1.5"
 
-## Training Details
+input_texts = [
+    "Who founded Dominican Liberation Party?",
+    "Who owns Mompesson House?"
+]
 
-### Framework Versions
-- Python: 3.10.14
-- Sentence Transformers: 5.2.0.dev0
-- Transformers: 4.55.4
-- PyTorch: 2.4.0+cu121
-- Accelerate: 0.34.2
-- Datasets: 2.16.1
-- Tokenizers: 0.21.4
+# Load model from the Hugging Face Hub
+model = SentenceTransformer(MODEL_NAME_OR_PATH, trust_remote_code=True)
 
-## Citation
+# Compute embeddings
+embeddings = model.encode(input_texts)
 
-### BibTeX
+print("Embeddings:", embeddings)
+```
 
-<!--
-## Glossary
+**IMPORTANT:** This code will be supported in versions of Sentence Transformers later than v5.1.0, which have not yet been released at the time of writing. Until then, please install the library directly from GitHub:
 
-*Clearly define terms in order to be accessible across audiences.*
--->
+```bash
+pip install git+https://github.com/UKPLab/sentence-transformers.git
+```
 
-<!--
-## Model Card Authors
+## License
 
-*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
--->
+This model is licensed under the Apache License, Version 2.0.
 
-<!--
-## Model Card Contact
+## Citation
 
-*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
--->
+If you use this model in your research, please cite the following paper:
+[Dynamic Injection of Entity Knowledge into Dense Retrievers](https://arxiv.org/abs/2507.03922)
+
+```bibtex
+@article{yamada2025kpr,
+title={Dynamic Injection of Entity Knowledge into Dense Retrievers},
+author={Ikuya Yamada and Ryokan Ri and Takeshi Kojima and Yusuke Iwasawa and Yutaka Matsuo},
+journal={arXiv preprint arXiv:2507.03922},
+year={2025}
+}
+```