DPRContextEncoder for TriviaQA
dpr-ctx_encoder-single-trivia-base
Dense Passage Retrieval (DPR)
Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, Wen-tau Yih, Dense Passage Retrieval for Open-Domain Question Answering, EMNLP 2020.
This model is the context encoder of DPR trained on TriviaQA using the official implementation of DPR.
Disclaimer: This model is not from the authors of DPR. It is my own reproduction. The authors did not release the DPR weights for TriviaQA.
Performance
The performance is answer recall rate measured using PyTorch 1.4.0 and transformers 4.5.0.
The values in parentheses are those reported in the paper.
| Top-k Passages | TriviaQA Dev | TriviaQA Test |
|---|---|---|
| 1 | 54.27 | 54.41 |
| 5 | 71.11 | 70.99 |
| 20 | 79.53 | 79.31 (79.4) |
| 50 | 82.72 | 82.99 |
| 100 | 85.07 | 84.99 (85.0) |
How to Use
AutoModel does not properly detect whether the checkpoint is for DPRContextEncoder or DPRQuestionEncoder.
Therefore, please specify the exact class to use the model.
from transformers import DPRContextEncoder, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("soheeyang/dpr-ctx_encoder-single-trivia-base")
ctx_encoder = DPRContextEncoder.from_pretrained("soheeyang/dpr-ctx_encoder-single-trivia-base")
data = tokenizer("context comes here", return_tensors="pt")
ctx_embedding = ctx_encoder(**data).pooler_output # embedding vector for context