PJMixers-Archive
/

LLaMa-3.1-Instruct-ToxicQAFinal-ORPO-8B-QDoRA

Not-For-All-Audiences

Model card Files Files and versions Community

LLaMa-3.1-Instruct-ToxicQAFinal-ORPO-8B-QDoRA / README.md

xzuyn's picture

Update README.md

3a3c420 verified 7 months ago

|

history blame contribute delete

986 Bytes

	---
	datasets:
	- >-
	PJMixers/NobodyExistsOnTheInternet_ToxicQAFinal-L3-Instruct-8B-PreferenceShareGPT
	- NobodyExistsOnTheInternet/ToxicQAFinal
	tags:
	- not-for-all-audiences
	---
	Trained on [NobodyExistsOnTheInternet/ToxicQAFinal](https://huggingface.co/datasets/NobodyExistsOnTheInternet/ToxicQAFinal). I converted the set to a preference dataset using refusals generated from LLaMa-3-Instruct-8B. I have not recreated the rejections with LLaMa-3.1-Instruct-8B yet.

	This is an early test.

	![train/rewards](https://huggingface.co/PJMixers/LLaMa-3.1-Instruct-ToxicQAFinal-ORPO-8B-QDoRA/resolve/main/images/rewards.png)
	![train/logits](https://huggingface.co/PJMixers/LLaMa-3.1-Instruct-ToxicQAFinal-ORPO-8B-QDoRA/resolve/main/images/logits.png)
	![train/logps](https://huggingface.co/PJMixers/LLaMa-3.1-Instruct-ToxicQAFinal-ORPO-8B-QDoRA/resolve/main/images/logps.png)
	![train](https://huggingface.co/PJMixers/LLaMa-3.1-Instruct-ToxicQAFinal-ORPO-8B-QDoRA/resolve/main/images/train.png)