Llama3 CoEdit
This is a Llama3 8B based model trained using torchtune on the grammarly/coedit dataset.
Training details
The exact training script (lora_finetune_distributed) and config (8B_lora.yaml) are both included in this repository.
Training command: tune run --nproc_per_node 8 lora_finetune_distributed --config 8B_lora.yaml
Yes I used 8 GPUs :)
In order to add the dataset, I added the following lines to the config:
dataset:
_component_: torchtune.datasets.instruct_dataset
source: grammarly/coedit
template: GrammarErrorCorrectionTemplate
column_map: {"sentence": "src", "output": "tgt"}
train_on_input: False
split: train
Loss curve
Evaluation results
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model has no library tag.