Update README.md
Browse files
README.md
CHANGED
@@ -1,39 +1,56 @@
|
|
1 |
---
|
|
|
2 |
library_name: transformers
|
3 |
-
tags:
|
|
|
|
|
|
|
|
|
|
|
4 |
---
|
5 |
|
6 |
-
#
|
7 |
-
|
8 |
-
|
9 |
-
|
10 |
-
|
11 |
-
|
12 |
-
|
13 |
-
|
14 |
-
|
15 |
-
|
16 |
-
|
17 |
-
|
18 |
-
|
19 |
-
|
20 |
-
|
21 |
-
|
22 |
-
|
23 |
-
|
24 |
-
-
|
25 |
-
|
26 |
-
|
27 |
-
|
28 |
-
|
29 |
-
|
30 |
-
|
31 |
-
|
32 |
-
|
33 |
-
|
34 |
-
|
35 |
-
|
36 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
37 |
|
38 |
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
39 |
|
@@ -41,7 +58,10 @@ This is the model card of a 🤗 transformers model that has been pushed on the
|
|
41 |
|
42 |
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
43 |
|
44 |
-
|
|
|
|
|
|
|
45 |
|
46 |
### Downstream Use [optional]
|
47 |
|
@@ -59,7 +79,9 @@ This is the model card of a 🤗 transformers model that has been pushed on the
|
|
59 |
|
60 |
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
61 |
|
62 |
-
|
|
|
|
|
63 |
|
64 |
### Recommendations
|
65 |
|
|
|
1 |
---
|
2 |
+
license: cc-by-sa-4.0
|
3 |
library_name: transformers
|
4 |
+
tags:
|
5 |
+
- flan-t5
|
6 |
+
- detoxification
|
7 |
+
- polite-rewriting
|
8 |
+
- text-to-text
|
9 |
+
model_name: flan-paradetox-full
|
10 |
---
|
11 |
|
12 |
+
# FLAN-T5-Large **Polite-Rewrite** (full fine-tune)
|
13 |
+
|
14 |
+
**Model:** `google/flan-t5-large` fine-tuned for toxic → polite rewriting.
|
15 |
+
|
16 |
+
## Training details
|
17 |
+
| Parameter | Value |
|
18 |
+
|-----------|-------|
|
19 |
+
| epochs | 3 |
|
20 |
+
| effective batch | 32 (16 × grad_acc=2, fp16) |
|
21 |
+
| lr / schedule | 3 e-5, cosine, 3 % warm-up |
|
22 |
+
| total steps | 1 800 |
|
23 |
+
| optimizer | AdamW, weight_decay=0.01 |
|
24 |
+
| hardware | 1 × A100-40 GB |
|
25 |
+
|
26 |
+
### Data
|
27 |
+
Merged **29 k** parallel pairs
|
28 |
+
* ParaDetox (19 k)
|
29 |
+
* Polite Insult (1.6 k, oversample×2)
|
30 |
+
* PseudoParaDetox Llama-3 (8.6 k, tox≤0.3, cosine≥0.8)
|
31 |
+
|
32 |
+
### Metrics (dev 3 %)
|
33 |
+
| metric | score |
|
34 |
+
|--------|-------|
|
35 |
+
| BLEU | 0.82 |
|
36 |
+
| Avg toxicity (Detoxify) | **0.12** (src 0.71 → tgt 0.12) |
|
37 |
+
| Success rate (tox≤0.5 AND -20 %) | 89 % |
|
38 |
+
|
39 |
+
## Usage
|
40 |
+
|
41 |
+
```python
|
42 |
+
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
|
43 |
+
tok = AutoTokenizer.from_pretrained("RinaldiDev/flan-paradetox-full")
|
44 |
+
model = AutoModelForSeq2SeqLM.from_pretrained("RinaldiDev/flan-paradetox-full")
|
45 |
+
|
46 |
+
def rewrite_polite(text):
|
47 |
+
inp = f"Rewrite politely:\\nInput: {text}\\nPolite:"
|
48 |
+
ids = tok(inp, return_tensors="pt").input_ids
|
49 |
+
out = model.generate(ids, num_beams=4, max_length=96)
|
50 |
+
return tok.decode(out[0], skip_special_tokens=True)
|
51 |
+
|
52 |
+
print(rewrite_polite("Shut up, idiot!"))
|
53 |
+
# → "Stop talking"
|
54 |
|
55 |
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
56 |
|
|
|
58 |
|
59 |
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
60 |
|
61 |
+
|
62 |
+
AI moderation helper
|
63 |
+
Toxic-to-polite assistants
|
64 |
+
Not for hallucination-free tasks; may still miss subtle hate speech.
|
65 |
|
66 |
### Downstream Use [optional]
|
67 |
|
|
|
79 |
|
80 |
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
81 |
|
82 |
+
|
83 |
+
Trained largely on English; fails on code-switching.
|
84 |
+
Llama-generated pairs could contain artifacts.
|
85 |
|
86 |
### Recommendations
|
87 |
|