size_categories:1K<n<10K format:parquet modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
dataset_info:
  features:
  - name: id
    dtype: string
  - name: pre_text
    sequence: string
  - name: post_text
    sequence: string
  - name: table
    sequence:
      sequence: string
  - name: question
    dtype: string
  - name: answer
    dtype: string
  - name: final_result
    dtype: string
  - name: program_re
    dtype: string
  - name: gold_inds
    sequence: string
  - name: expanded_program_re
    dtype: string
  splits:
  - name: train
    num_bytes: 29385976
    num_examples: 6251
  - name: validation
    num_bytes: 4096947
    num_examples: 883
  - name: test
    num_bytes: 5275663
    num_examples: 1147
  download_size: 14438292
  dataset_size: 38758586
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
  - split: validation
    path: data/validation-*
  - split: test
    path: data/test-*
---




# FinQA-Expanded Dataset

This dataset is an expanded version of the IBM FinQA dataset, which contains finance-related questions and their corresponding arithmetic/logical operation programs. The original dataset has been enhanced with infix notation representations of the operations to make it more suitable for Language Model training.

## Dataset Description

This dataset extends the original [IBM FinQA dataset](https://huggingface.co/datasets/ibm/finqa) by adding a new field that represents the operations in a more natural infix notation format. This makes it easier for Language Models to understand and process the mathematical operations.

### Original Features

- `id`: Unique identifier for each example
- `pre_text`: Contextual text before the table
- `post_text`: Contextual text after the table
- `table`: Financial data in tabular format
- `question`: The question to be answered
- `answer`: The correct answer
- `final_result`: The final computed result
- `program_re`: The original program representing the sequence of operations
- `gold_inds`: Gold indices for relevant information

### New Features

- `expanded_program_re`: Infix notation representation of the operations in `program_re`

## Operation Types

The dataset handles several types of operations:

1. Arithmetic Operations:
   - Addition (+)
   - Subtraction (-)
   - Multiplication (*)
   - Division (/)

2. Table Operations:
   - table_average
   - table_max
   - table_sum
   - table_min

3. Boolean Operations:
   - greater (>)
   - less (<)
   - equal (==)

4. Special Operations:
   - exp (represented as ^)

## Operation Conversion Examples

The dataset converts operations from functional notation to infix notation. Here are some examples of the conversion:

1. Arithmetic Operations:
   ```
   Original: add(5829, 5735), divide(#0, 5735)
   Expanded: (5829 + 5735) / 5735
   ```

2. Table Operations:
   ```
   Original: table_min(expected volatility, none), table_max(expected volatility, none), subtract(#1, #0)
   Expanded: table_max(expected volatility) - table_min(expected volatility)
   ```

3. Nested Operations:
   ```
   Original: multiply(607, 18.13), multiply(#0, const_1000), multiply(3.3, const_1000000), greater(#1, #2)
   Expanded: (607 * 18.13 * const_1000) > (3.3 * const_1000000)
   ```

## Dataset Statistics

- Train set: 6,251 examples
- Validation set: 883 examples
- Test set: 1,147 examples
- Total: 8,281 examples

## Usage

You can load this dataset using the Hugging Face datasets library:

```python
from datasets import load_dataset

dataset = load_dataset("n3Er/FinQA-Infix")
```

### Example Usage

```python
# Access a sample from the training set
example = dataset['train'][0]

# Print the question and its operations
print("Question:", example['question'])
print("Original Program:", example['program_re'])
print("Expanded Program:", example['expanded_program_re'])
```

## Citation

If you use this dataset, please cite both this expanded version and the original FinQA dataset:

```
@misc{finqa-expanded,
  title={FinQA Expanded Dataset},
  year={2024},
  publisher={HuggingFace}
}

@inproceedings{chen2021finqa,
  title={FinQA: A Dataset of Numerical Reasoning over Financial Data},
  author={Chen, Zhiyu and Chen, Wenhu and Smiley, Charese and Shah, Sameena and Borova, Iana and Langdon, Dylan and Moussa, Reema and Beane, Matt and Huang, Ting-Hao and Routledge, Bryan and Wang, William Yang},
  booktitle={Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing},
  year={2021}
}
```

## License

This dataset follows the same license as the original FinQA dataset.