task_categories:text-classification license:mit size_categories:10K<n<100K format:csv modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
license: mit
task_categories:
- text-classification
pretty_name: >-
  Bitext - Customer Service Tagged Training Dataset for LLM-based Virtual
  Assistants - Multiple-Choice Questions
size_categories:
- 10K<n<100K
---

# Multiple-Choice Formatted Version of Bitext Customer Support Dataset

This repository contains a modified version of the [Bitext - Customer Service Tagged Training Dataset for LLM-based Virtual
  Assistants](https://huggingface.co/datasets/bitext/Bitext-customer-support-llm-chatbot-training-dataset) dataset. The dataset has been transformed into a multiple-choice format aimed at training and evaluating intent classification models.

## Overview

The original dataset consists of customer support instructions paired with labeled intents. In this variant, each instruction is accompanied by:
- **A list of 5 choices:** One correct intent and four distractors.
- **The index of the correct intent:** Indicating the position of the actual intent in the shuffled list of choices.

The distractors for each instruction were chosen as the four most semantically similar intents (based on sentence embeddings) to the correct one.

## Dataset Creation Process
- The 27 unique intents were extracted from the dataset.
- A general-purpose sentence embedding was computed for each intent using the SentenceTransformer model ([all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)).
- The correct intent is identified.
- Cosine similarities are computed between the correct intent's embedding and all other intent embeddings.
- The 4 most similar intents (distractors) are selected.
- The correct intent is added to these distractors, forming a list of 5 choices.
- The list is shuffled randomly, and the index of the correct intent is recorded.

## Dataset Columns
- instruction: The text of the customer support instruction.
- intent_choices: A list of 5 intent options (the correct intent and 4 distractors).
- correct_index: The index (starting at 0) in the choices list corresponding to the correct intent.

## Train/Test Split
- The data was then split randomly into a 50/50 train/test split. For evaluating LLM-based classifiers using RAG or similar retrieval techniques for in-context learning, you can use the train set as the retrieval set, and the test set to query the LLM.  

## Attribution and Credits
### Original Dataset:
This dataset is derived from the [Bitext - Customer Service Tagged Training Dataset for LLM-based Virtual
  Assistants](https://huggingface.co/datasets/bitext/Bitext-customer-support-llm-chatbot-training-dataset) available on Hugging Face. Please refer to the original dataset's page for more details on licensing and attribution.

### Embedding Model:
Sentence embeddings were generated using the [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) model from the [Sentence Transformers](https://www.sbert.net/) library.

### Created By:
[Crossing Minds](https://www.crossingminds.com)