task_categories:question-answering language:en license:apache-2.0 size_categories:1K<n<10K region:us biology medical iGEM
---
license: apache-2.0
task_categories:
- question-answering
language:
- en
tags:
- biology
- medical
- iGEM
size_categories:
- 1K<n<10K
---

## 📚 Dataset Information

This dataset is utilized for fine-tuning the following models:

- [shellwork/ChatParts-llama3.1-8b](https://huggingface.co/shellwork/ChatParts-llama3.1-8b/)
- [shellwork/ChatParts-qwen2.5-14b](https://huggingface.co/shellwork/ChatParts-qwen2.5-14b)

### 📁 File Structure

The dataset is organized as follows:

```plaintext
D:\ChatParts_Dataset
│
├── .gitattributes
├── README.md
├── Original_data
│   ├── iGEM_competition_web.rar
│   ├── paper_txt_processed.rar
│   └── wiki_data.rar
└── Training_dataset
    ├── pt_txt.json
    ├── sft_eval.json
    └── sft_train.json
```
- **Original_data:**
  - `iGEM_competition_web.rar`: Contains raw text documents scraped from iGEM competition websites.
  - `paper_txt_processed.rar`: Contains processed text from over 1,000 synthetic biology review papers.
  - `wiki_data.rar`: Contains raw Wikipedia data related to synthetic biology.

  The original data was collected using web crawlers and subsequently filtered and manually curated to ensure quality. These raw `.txt` documents serve as the foundational learning passages for the model's pre-training phase. The consolidated and processed text can be found in the `pt_txt.json` file within the `Training_dataset` directory.

- **Training_dataset:**
  - `pt_txt.json`: Consolidated and preprocessed text passages used for the model's pre-training step.
  - `sft_train.json`: Contains over 180,000 question-answer pairs derived from the original documents, used for supervised fine-tuning (SFT) training.
  - `sft_eval.json`: Contains over 20,000 question-answer pairs reserved for evaluating the model post-training, maintaining a 9:1 data ratio compared to the training set.

  The `sft_train.json` and `sft_eval.json` files consist of meticulously organized question-answer pairs extracted from all available information in the original documents. These datasets facilitate the model's supervised instruction learning process, enabling it to generate accurate and contextually relevant responses.

### 📄 License

This dataset is released under the **Apache License 2.0**. For more details, please refer to the [license information](https://github.com/shellwork/XJTLU-Software-RAG/tree/main) in the repository.

## 🔗 Additional Resources

- **RAG Software:** Explore the full capabilities of our Retrieval-Augmented Generation software [here](https://github.com/shellwork/XJTLU-Software-RAG/tree/main).
- **Training Data:** Access and review the extensive training dataset [here](https://huggingface.co/datasets/shellwork/ChatParts_Dataset).

---

Feel free to reach out through our GitHub repository for any questions, issues, or contributions related to this dataset.