task_categories:text-classification language:en size_categories:n<1K format:parquet modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
dataset_info:
- config_name: default
  features:
  - name: utterance
    dtype: string
  - name: label
    dtype: int64
  splits:
  - name: train
    num_bytes: 16
    num_examples: 1
  download_size: 1106
  dataset_size: 16
- config_name: intents
  features:
  - name: id
    dtype: int64
  - name: name
    dtype: string
  - name: tags
    sequence: 'null'
  - name: regexp_full_match
    sequence: string
  - name: regexp_partial_match
    sequence: string
  - name: description
    dtype: 'null'
  splits:
  - name: intents
    num_bytes: 40873
    num_examples: 22
  download_size: 26835
  dataset_size: 40873
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
- config_name: intents
  data_files:
  - split: intents
    path: intents/intents-*
language:
- en
task_categories:
- text-classification
---

# Dream

This is a text classification dataset. It is intended for machine learning research and experimentation.

This dataset is obtained via formatting another publicly available data to be compatible with our [AutoIntent Library](https://deeppavlov.github.io/AutoIntent/index.html).

## Usage

It is intended to be used with our [AutoIntent Library](https://deeppavlov.github.io/AutoIntent/index.html):

```python
from autointent import Dataset

dream = Dataset.from_datasets("AutoIntent/dream")
```

## Source

This dataset is taken from [DeepPavlov Library](https://github.com/deeppavlov/DeepPavlov)'s repository. It was formatted with our [AutoIntent Library](https://deeppavlov.github.io/AutoIntent/index.html):

```python
# define utils
import json
import requests
from autointent import Dataset

def load_json_from_github(github_file: str):
    raw_text = requests.get(github_file).text
    return json.loads(raw_text)

def convert_dream(dream_dict):
    intents = []
    for i, (intent_name, all_phrases) in enumerate(dream_dict["intent_phrases"].items()):
        intent_record = {
            "id": i,
            "name": intent_name,
            "tags": [],
            "regexp_full_match": all_phrases["phrases"],
            "regexp_partial_match": all_phrases.get("reg_phrases", []),
        }
        intents.append(intent_record)
    return Dataset.from_dict({"intents": intents, "train": [{"utterance": "test", "label": 0}]})

# load and format
github_file = "https://raw.githubusercontent.com/deeppavlov/dream/2cad3e0b63b4ecde1e500676d31f1e34c53e1dc7/annotators/IntentCatcherTransformers/intent_phrases.json"
dream = load_json_from_github(github_file)
dream_converted = convert_dream(dream)
```