guisalim/distilbert-bias-classifier

DistilBERT Bias Classifier

Safety Sensitiveness Service — Tier 2 Bias Head

Detects 5 types of bias in short text (search queries, social posts):

Label	Description
racial	Racial / ethnic bias
gender	Gender stereotyping
religious	Religious discrimination
age	Ageism
sexual_orientation	Sexual orientation bias

Performance

Metric	Score
F1 Macro	0.2678
F1 Micro	0.6810
ROC-AUC Macro	0.0000
CrowS-Pairs Accuracy	0.0000
StereoSet Score	0.4900

Usage

from transformers import pipeline
classifier = pipeline("text-classification",
                      model="guisalim/distilbert-bias-classifier",
                      top_k=None)
print(classifier("Search query here"))

Training details

• Base: distilbert-base-uncased
• Data: Social Bias Frames + Google Civil Comments
• Device: mps
• Threshold: 0.4