language:en license:mit size_categories:1K<n<10K format:json modality:text library:datasets library:pandas library:mlcroissant library:polars region:us intentguard code-analysis code-verification
---
license: mit
language:
- en
tags:
- intentguard
- code-analysis
- code-verification
---
# IntentGuard Training Dataset

This dataset is designed to train models for [**IntentGuard**](https://github.com/kdunee/intentguard), a novel tool that verifies **complex** code properties using natural language assertions.  It provides a valuable resource for training models to understand the nuanced relationship between natural language descriptions of code intent and actual code behavior.  The dataset contains a curated collection of Python code snippets paired with natural language assertions describing desired properties of that code, ranging from error handling to documentation consistency.

## Dataset Description

This dataset is split into two main files, designed for training and evaluation:

*   **`train.jsonl`**:  The primary training set, containing training examples in JSON Lines format, one example per line. This file is intended for training models to understand and verify code properties based on natural language assertions.
*   **`test.jsonl`**:  The evaluation (test) set, containing a collection of test examples in JSON Lines format. This set is designed to evaluate the performance of models trained on the `train.jsonl` data.

Each example within both `train.jsonl` and `test.jsonl` is a JSON object with the following structure:

*   **`assertion`**:  This field encapsulates the natural language assertion:
    *   `assertionText`:  *(string)* The natural language assertion itself, describing a desired positive property of the code.  Examples include: "All methods should log errors" or "Public functions must have docstrings".
    *   `codeObjectNames`: *(list of strings)* A list of names of code components (e.g., function names, class names, module names) that are referenced in the `assertionText`.
*   **`codeObjects`**: This field contains the code snippets relevant to the assertion:
    *   `code`: *(string)* A Python code snippet that is being evaluated against the `assertionText`.
    *   `name`: *(string)* A descriptive name for the `code` snippet (e.g., "error_handling_module", "api_functions").
*   **`explanation`**: *(string, *only in negative examples*)*  Present only in examples where the code *fails* to satisfy the assertion.  This field provides a natural language explanation of *why* the code does not meet the asserted property.
*   **`thoughts`**: *(string)* A crucial field containing a **chain-of-thought** analysis. This captures the step-by-step reasoning process of how one might evaluate whether the `code` satisfies the `assertionText`. This is valuable for training models to mimic human-like reasoning about code and natural language.

The dataset carefully balances **positive examples** (where the `code` *does* satisfy the `assertion`) and **negative examples** (where the `code` *fails* to satisfy the `assertion`) to facilitate robust model training.

## Intended Use

This dataset is primarily intended for researchers and developers working in the fields of:

*   **Natural Language Processing (NLP):**  To develop models that can understand and reason about code described in natural language.
*   **Program Analysis and Code Understanding:** To build tools that can automatically verify code properties and ensure code quality.
*   **Explainable AI (XAI):**  To explore how chain-of-thought reasoning can improve the interpretability of code analysis models.

Specifically, this dataset can be used for:

*   **Training Code Property Verification Models:** As the training data for models like the one used in IntentGuard.
*   **Benchmarking and Evaluation:** To evaluate the performance of new code understanding and verification models.
*   **Researching Few-Shot Learning:**  The dataset's structure, especially the `thoughts` field, may be valuable for exploring few-shot learning approaches for code reasoning.
*   **Developing Educational Resources:** The dataset's examples can be used to create tutorials and educational materials on code quality and testing concepts.

By using this dataset, the community can advance the development of more intuitive and powerful tools for software development and quality assurance.

## Data Format

The dataset is provided in **JSON Lines (.jsonl) format** for both `train.jsonl` and `test.jsonl`.  This format is efficient for large datasets, as each line is a valid JSON object, representing a single data example.

Here's an **example** of a single JSON object you might find within `train.jsonl` or `test.jsonl`:

```json
{
  "assertion": {
    "assertionText": "All public methods in this module should have docstrings including Parameters, Returns, and Examples sections",
    "codeObjectNames": ["my_module"]
  },
  "codeObjects": [
    {
      "code": "def public_func(arg1, arg2):\n    \"\"\"This is a public function.\n\n    Parameters:\n        arg1: Description of arg1\n        arg2: Description of arg2\n\n    Returns:\n        Return value description\n\n    Examples:\n        >>> public_func(1, 2)\n        Some output\n    \"\"\"\n    pass",
      "name": "my_module"
    }
  ],
  "thoughts": "The assertion checks for docstring completeness... (chain of thought analysis)"
}
```

As you can see, each JSON object adheres to the structure described in the "Dataset Description" section.

## Data Splits

The dataset is provided with a clear split into training and testing sets to facilitate model development and evaluation.

*   **`train.jsonl`**:  This file contains the **training set**. It is significantly larger and designed for training your models.
*   **`test.jsonl`**: This file contains the **test set**. It is smaller and intended for evaluating the generalization performance of your trained models on unseen data.

While the exact sizes of each split may vary, the `train.jsonl` is substantially larger than `test.jsonl` to provide ample training data. The split was created to ensure a representative evaluation of model performance.