Training update: 6,763/238,451 rows (2.84%) | +25 new @ 2025-10-23 02:45:29

README.md

@@ -24,7 +24,7 @@ pipeline_tag: fill-mask
 - Model type: fine-tuned lightweight BERT variant  
 - Languages: English & Indonesia
 - Finetuned from: `boltuix/bert-micro`
-- Status: **Early version** — trained on **2.83%** of planned data.
+- Status: **Early version** — trained on **2.84%** of planned data.
 **Model sources**  
 - Base model: [boltuix/bert-micro](https://huggingface.co/boltuix/bert-micro)
 - Data: Cybersecurity Data
@@ -40,7 +40,7 @@ You can use this model to classify cybersecurity-related text — for example, w
 - Not optimized for languages other than English and Indonesian.  
 - Not tested for non-cybersecurity domains or out-of-distribution data.
 ## 3. Bias, Risks, and Limitations 
-Because the model is based on a small subset (2.83%) of planned data, performance is preliminary and may degrade on unseen or specialized domains (industrial control, IoT logs, foreign language).
+Because the model is based on a small subset (2.84%) of planned data, performance is preliminary and may degrade on unseen or specialized domains (industrial control, IoT logs, foreign language).
 - Inherits any biases present in the base model (`boltuix/bert-micro`) and in the fine-tuning data — e.g., over-representation of certain threat types, vendor or tooling-specific vocabulary.
 - Should not be used as sole authority for incident decisions; only as an aid to human analysts.
 ## 4. How to Get Started with the Model  
@@ -55,8 +55,31 @@ logits = outputs.logits
 predicted_class = logits.argmax(dim=-1).item()
 ```
 ## 5. Training Details
-- **Trained records**: 6,737 / 238,451 (2.83%)
+
+### Text Processing & Chunking
+Since cybersecurity data often contains lengthy alert descriptions and execution logs that exceed BERT's 512 token limit, we implement an overlapping chunking strategy:
+- **Max sequence length**: 512 tokens
+- **Stride**: 32 tokens (overlap between consecutive chunks)
+- **Chunking behavior**: Long texts are split into overlapping segments. For example, with max_length=512 and stride=128, a 1000-token document becomes ~3 chunks with 128-token overlaps, preserving context across boundaries.
+
+### Training Hyperparameters
+- **Base model**: `boltuix/bert-micro`
+- **Training epochs**: 3
 - **Learning rate**: 5e-05
-- **Epochs**: 3
 - **Batch size**: 16
-- **Max sequence length**: 512
+- **Weight decay**: 0.01
+- **Warmup ratio**: 0.06
+- **Gradient accumulation steps**: 1
+- **Optimizer**: AdamW
+- **LR scheduler**: Linear with warmup
+
+### Training Data
+- **Total database rows**: 238,451
+- **Rows processed (cumulative)**: 6,763 (2.84%)
+- **Rows in this session**: 25
+- **Training samples (after chunking)**: 25
+- **Training date**: 2025-10-23 02:45:29
+
+### Post-Training Metrics
+- **Final training loss**: 0.0000
+- **Rows→Samples ratio**: 1.00x (average chunks per row)

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:487e4f9052c100978689e6eade446eeda6f4c8fba1363b7a582d6079bb33374a
+oid sha256:983be3a2e9678daa96170b0969607343ac60dacf50afe2de4005fbdbb918d54d
 size 17671560

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1f36e9bc24f3f3fbc56bfa76ad08bf8c96c15d398a5d494f6e83bfb7fc0d9d52
+oid sha256:8ef5380f08e852850356b566ce57de8329dc2f88e4f45d0840e444f505dbe40b
 size 5905

training_metadata.json

@@ -1,12 +1,15 @@
 {
-  "trained_at": 1761186906.6386719,
-  "trained_at_readable": "2025-10-23 02:35:06",
-  "samples_this_session": 26,
-  "new_rows_this_session": 26,
-  "trained_rows_total": 6737,
+  "trained_at": 1761187529.3583431,
+  "trained_at_readable": "2025-10-23 02:45:29",
+  "samples_this_session": 25,
+  "new_rows_this_session": 25,
+  "trained_rows_total": 6763,
   "total_db_rows": 238451,
-  "percentage": 2.825318409232924,
+  "percentage": 2.8362221169129085,
   "final_loss": 0,
   "epochs": 3,
-  "learning_rate": 5e-05
+  "learning_rate": 5e-05,
+  "batch_size": 16,
+  "stride": 32,
+  "max_length": 512
 }