task_categories:text-generation language:en license:cc-by-4.0 size_categories:n<1K format:parquet modality:text library:datasets library:pandas library:mlcroissant library:polars region:us code
---
dataset_info:
  features:
  - name: text
    dtype: string
  splits:
  - name: train
    num_bytes: 221081
    num_examples: 3
  download_size: 3551225
  dataset_size: 9551210
license: cc-by-4.0
task_categories:
- text-generation
language:
- en
tags:
- code
---
# CodeLlama-2-20k: A Llama 2 Version of CodeAlpaca

This dataset is the [`pchanumolu/huge-context-size-test`](https://huggingface.co/datasets/pchanumolu/huge-context-size-test) dataset with the Llama 2 prompt format [described here](https://huggingface.co/blog/llama2#how-to-prompt-llama-2).

Here is the code I used to format it:

``` python
from datasets import load_dataset

# Load the dataset
dataset = load_dataset('pchanumolu/huge-context-size-test')

# Define a function to merge the three columns into one
def merge_columns(example):
    if example['input']:
        merged = f"<s>[INST] <<SYS>>\nBelow is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n<</SYS>>\n\n{example['instruction']} Input: {example['input']} [/INST] {example['output']} </s>"
    else:
        merged = f"<s>[INST] <<SYS>>\nBelow is an instruction that describes a task. Write a response that appropriately completes the request.\n<</SYS>>\n\n{example['instruction']} [/INST] {example['output']} </s>"
    return {"text": merged}

# Apply the function to all elements in the dataset
dataset = dataset.map(merge_columns, remove_columns=['instruction', 'input', 'output'])
```
