task_categories:text2text-generation language:en license:mit size_categories:n<1K format:json modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
license: mit
task_categories:
- text2text-generation
language:
- en
size_categories:
- n<1K
configs:
- config_name: defalut
  description: Multi function level and contract level testing benchmark.
  data_files:
  - split: test
    path: smart_contract_benchmark.jsonl
  columns:
  - task_id
  - user_requirement
  - test_list
  - test_codes
---
### Dataset Description
This dataset is used as a benchmark for evaluating the functional correctness of smart contract at both the multi-function level and the entire contract level.

### Data Instances
```
{
    'task_id': 1,
    'user_requirement': 'This smart contract is an [ERC-721 NFT contract]. Please name the smart contract as [UniqueArt]. The main functionalities include [minting unique NFTs, transferring ownership, and managing approvals]. The main variables include......',
    'test_list': ["minting unique NFTs", "transferring ownership", "managing approvals"],
    'test_codes': [
        {"test": "minting unique NFTs", "code": "..."},
        {"test": "transferring ownership", "code": "..."},
        {"test": "managing approvals", "code": "..."},]
}
```

### Data Fields
- `task_id`: The unique identifier for each task.
- `user_requirement`: The description of the user's requirements for the task.
- `test_list`: A list of functionalities to be tested.
- `test_codes`: A list of each functionality and its corresponding hardhat test code.