task_categories:text-generation language:en license:apache-2.0 size_categories:n<1K format:parquet modality:text library:datasets library:pandas library:mlcroissant library:polars region:us function calling tool calling
---
language:
- en
license: apache-2.0
size_categories:
- n<1K
task_categories:
- text-generation
pretty_name: Gemma Function Calling Eval
dataset_info:
  features:
  - name: conversation
    list:
    - name: content
      dtype: string
    - name: role
      dtype: string
  - name: functions
    dtype: string
  splits:
  - name: test
    num_bytes: 581666
    num_examples: 812
  download_size: 176139
  dataset_size: 581666
configs:
- config_name: default
  data_files:
  - split: test
    path: data/test-*
tags:
- function calling
- tool calling
---


## 👉🏽 Important
This dataset is adapted from [Berkeley Function Calling Leaderboard Dataset](https://github.com/ShishirPatil/gorilla/tree/main/berkeley-function-call-leaderboard) to evaluate the function calling ability of [dushj98/gemma-function-calling](https://huggingface.co/dushj98/action-gemma-2-2b-it-lora) fine-tuned LLM.

## 🔀 Changes Made
1. Merged questions and expected function call as a single conversation.
2. Converted function definitions to a valid JSON schema that follows OpenAI function schema, removed 227 examples that had invalid JSON schema definitions.
3. Merged "simple", "multiple", "irrelevance" and "chatable" individual test categories to construct a dataset with 812 examples in total.