license:cc-by-4.0 size_categories:10K<n<100K format:parquet modality:tabular modality:text library:datasets library:pandas library:mlcroissant library:polars region:us code
---
license: cc-by-4.0
pretty_name: DyPyBench Functions
tags:
- code
dataset_info:
  features:
  - name: nwo
    dtype: string
  - name: sha
    dtype: string
  - name: path
    dtype: string
  - name: identifier
    dtype: string
  - name: parameters
    dtype: string
  - name: return_statement
    dtype: string
  - name: docstring
    dtype: string
  - name: docstring_summary
    dtype: string
  - name: func_begin
    dtype: int64
  - name: func_end
    dtype: int64
  - name: function
    dtype: string
  - name: url
    dtype: string
  - name: project
    dtype: int64
  - name: executed_lines
    sequence: int64
  - name: executed_lines_pc
    dtype: float64
  - name: missing_lines
    sequence: int64
  - name: missing_lines_pc
    dtype: float64
  - name: covered
    dtype: bool
  - name: filecoverage
    dtype: float64
  - name: function_lines
    dtype: int64
  - name: mccabe
    dtype: int64
  - name: coverage
    dtype: float64
  - name: docstring_lines
    dtype: int64
  - name: function_nodoc
    dtype: string
  - name: id
    dtype: int64
  splits:
  - name: train
    num_bytes: 22383711
    num_examples: 11168
  download_size: 6805239
  dataset_size: 22383711
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
---

# DyPyBench Functions Datasets
[DyPyBench](https://zenodo.org/record/7886366) is a dataset constructed by Piyush Krishan Bajaj at the Software Lab, Institute of Software Engineering, University of Stuttgart. It contains 50 open source projects from GitHub.
We used [Nathan Cooper's](https://github.com/ncoop57/function_parser) `function_parser` tool, based off GitHub's CodeSearchNet `function_parser`, to extract all functions from all the projects, excluding library functions in the virtualenv. We also ran all tests in DyPyBench and produced a coverage report in JSON. Not all projects resulted in a coverage report due to project specific coverage report settings.

The columns provided are as follows:
| Column            | Type       | Notes                                                                                           |
| ----------------- | ---------- | ----------------------------------------------------------------------------------------------- |
| id                | Int64      | Unique id of the function
| project           | Int64      | DyPyBench project id                                                                            |
| nwo               | string     | Project name in repo/project format                                                             |
| sha               | string     | SHA commit hash                                                                                 |
| url               | string     | GitHub URL to function lines at commit                                                          |
| path              | string     | Path of file containing function relative to project root                                       |
| func_begin        | Int64      | Begin of function line number in source file                                                    |
| func_end          | Int64      | End of function line number in source file                                                      |
| function_lines    | Int64      | Function line count                                                                             |
| identifier        | string     | Function identifier                                                                             |
| parameters        | string     | Function parameters                                                                             |
| function          | string     | Source code of function including docstring                                                     |
| function_nodoc    | string     | Source code of function without docstring                                                       |
| docstring         | string     | Function docstring                                                                              |
| docstring_lines   | Int64      | Line count of docstring                                                                         |
| docstring_summary | string     | Function docstring summary                                                                      |
| return_statement  | string     | Function return statement                                                                       |
| filecoverage      | Float64    | If coverage available, coverage percentage of file function is from                             |
| executed_lines    | array[int] | If coverage available, executed lines relative to function lines (i.e. [0,1,2,...])             |
| executed_lines_pc | Float64    | If coverage available, executed line count over total function line count                       |
| missing_lines     | array[int] | If coverage available, missing (unexecuted) lines relative to function lines (i.e. [0,1,2,...]) |
| missing_lines_pc  | Float64    | If coverage available, missing line count over total function line count                        |
| covered           | boolean    | True if all lines executed and/or no lines missing                                              |
| mccabe            | Int64      | McCabe complexity of function                                                                   |
| coverage          | Float64    | Function coverage percentage (1-missing lines %)                                                |


Note: Missing/executed lines purposefully exclude lines skipped by `pytest` due to configuration e.g. line level `# pragma: no cover`.



