task_categories:question-answering language:de size_categories:n<1K format:parquet modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
dataset_info:
  features:
  - name: question
    dtype: string
  - name: answers
    sequence: string
  - name: positive_ctxs
    struct:
    - name: passage_id
      sequence: string
    - name: text
      sequence: string
    - name: title
      sequence: string
  - name: negative_ctxs
    struct:
    - name: passage_id
      sequence: 'null'
    - name: text
      sequence: 'null'
    - name: title
      sequence: 'null'
  - name: hard_negative_ctxs
    struct:
    - name: passage_id
      sequence: string
    - name: text
      sequence: string
    - name: title
      sequence: string
  - name: easy_negative_ctxt
    struct:
    - name: passage_id
      sequence: string
    - name: text
      sequence: string
    - name: title
      sequence: string
  splits:
  - name: train
    num_bytes: 75812
    num_examples: 10
  - name: test
    num_bytes: 37906
    num_examples: 5
  download_size: 138099
  dataset_size: 113718
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
  - split: test
    path: data/test-*
task_categories:
- question-answering
language:
- de
---


## Original Dataset

The original dataset, [`deepset/germandpr`](https://huggingface.co/datasets/deepset/germandpr), contains:
- **9275 training examples**
- **1025 testing examples**

Each example is a question/answer pair, consisting of:
- One question
- One answer
- One positive context
- Three negative contexts

You can find the original dataset [here](https://huggingface.co/datasets/deepset/germandpr).

## Modifications

### Adding Easy Negative Examples

To enhance the dataset, an "easy negative example" was added to each row. The objective of this addition is to train the model to better distinguish between relevant and irrelevant contexts by exposing it to plausible but incorrect information.

After adding easy neagtives each example in the dataset consists of :

- One question
- One answer
- One positive context
- Three negative contexts
-  Easy negative contexts

## Method

For identifying easy negative examples,  I have used utilized the L2 distance metric from [Faiss](https://github.com/facebookresearch/faiss) to find the most dissimilar index (vector) relative to the positive context in each row. This dissimilar index was then selected as the easy negative example.