size_categories:100K<n<1M format:parquet modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
dataset_info:
  features:
  - name: anchor
    dtype: string
  - name: entailment
    dtype: string
  - name: contradiction
    dtype: string
  splits:
  - name: train
    num_bytes: 88455406
    num_examples: 551015
  download_size: 39831572
  dataset_size: 88455406
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
---
# Dataset Card for "norwegian-nli-triplets"

A reformatting of compatible triplets from https://huggingface.co/datasets/tollefj/all-nli-NOB.
This includes all pairs that contain both a contradiction and an entailment.
In cases where a neutral also exists, this is a duplicated triplet, with the same contr./ent.

Simple normalization of sentences:
```python
from neattext.functions import clean_text
import re

def symbol_cleaner(s):
    s = re.sub(r"^[^\w\d]+", "", s)
    s = re.sub(r"[^\w\d]+$", "", s)
    return s

def filter_sent(sent):
    sent = clean_text(sent, puncts=False, stopwords=False, non_ascii=False)
    sent = symbol_cleaner(sent)
    return sent

def filter_triplet(triplet):
    return [filter_sent(sent) for sent in triplet]
```

[More Information needed](https://github.com/huggingface/datasets/blob/main/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)