End of training

README.md

@@ -1,7 +1,7 @@
 ---
 base_model: gpt2
 datasets:
-- distily/c4_multilingual_1M
+- wikimedia/wikipedia
 library_name: Distily
 license: creativeml-openrail-m
 tags:
@@ -18,7 +18,7 @@ model-index:
 
 Distilled with [Distily](https://github.com/lapp0/distily) library
 using teacher model [gpt2](https://huggingface.co/gpt2)
-on dataset [distily/c4_multilingual_1M](https://huggingface.co/datasets/distily/c4_multilingual_1M).
+on dataset [wikimedia/wikipedia](https://huggingface.co/datasets/wikimedia/wikipedia).
 
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment.
@@ -78,7 +78,7 @@ GPT2LMHeadModel(
 
 # Resource Usage
 
-- Max Train VRAM Use: 15.6998 GB
+- Max Train VRAM Use: 15.7005 GB
 - Available VRAM: 23.6429 GB
 - GPUs: 
   - 1x NVIDIA GeForce RTX 4090
@@ -115,10 +115,10 @@ GPT2LMHeadModel(
 <br/>
 
 # Train Dataset
-Trained on 448,520,445 tokens from the [distily/c4_multilingual_1M](https://huggingface.co/datasets/distily/c4_multilingual_1M) dataset.
+Trained on 525,568,142 tokens from the [wikimedia/wikipedia](https://huggingface.co/datasets/wikimedia/wikipedia) dataset.
 
 - Num Samples: `998,000`
-- Subset: `None`
+- Subset: `20231101.en`
 - Split: `train`
 
 
@@ -172,7 +172,7 @@ The following hyperparameters were used during training:
         projector='orthogonal'
     )
 )`
-- lr_scheduler: `<torch.optim.lr_scheduler.LambdaLR object at 0x7fb774b15c60>`
+- lr_scheduler: `<torch.optim.lr_scheduler.LambdaLR object at 0x7fb7667bf880>`
 - student_model_name_or_path: `None`
 - student_config_name_or_path: `distilbert/distilgpt2`
 - student_model_config: `None`
@@ -182,8 +182,8 @@ The following hyperparameters were used during training:
 - teacher_model_name_or_path: `gpt2`
 - teacher_load_in_8bit: `False`
 - teacher_load_in_4bit: `False`
-- dataset_uri: `distily/c4_multilingual_1M`
-- dataset_subset: `None`
+- dataset_uri: `wikimedia/wikipedia`
+- dataset_subset: `20231101.en`
 - dataset_split: `train`
 - dataset_column_name: `text`
 - dataset_sample_size: `1000000`

logs/events.out.tfevents.1725703850.0b434856d812

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e1e7d3c0f9942fd6cb2190954865b976b4a2d2cbba13a20b82dd88d55f86a8b8
+size 529