|
|
--- |
|
|
license: mit |
|
|
datasets: |
|
|
- glyphsoftware/reasoning-router |
|
|
language: |
|
|
- en |
|
|
base_model: |
|
|
- distilbert/distilroberta-base |
|
|
pipeline_tag: text-classification |
|
|
--- |
|
|
|
|
|
 |
|
|
|
|
|
# Reasoning Router |
|
|
|
|
|
A fine-tuned DistilRoBERTa model for classifying text based on reasoning depth. This model can categorize text into four reasoning levels: no-reasoning, low-reasoning, medium-reasoning, and high-reasoning. |
|
|
|
|
|
## Model Details |
|
|
|
|
|
### Model Description |
|
|
|
|
|
The Reasoning Router is a text classification model designed to automatically categorize text based on the depth and complexity of reasoning present. It's particularly useful for: |
|
|
|
|
|
- **Cost Optimization**: Can be used in inference pipeline to route requests to appropriate models |
|
|
- **Educational content analysis**: Identifying the reasoning level of educational materials |
|
|
- **Content filtering**: Routing content to appropriate audiences based on complexity |
|
|
- **Quality assessment**: Evaluating the sophistication of written content |
|
|
- **Research applications**: Analyzing reasoning patterns in large text corpora |
|
|
|
|
|
- **Developed by:** Glyph Software LLP |
|
|
- **Model type:** DistilRoBERTa-based sequence classification model |
|
|
- **Language(s) (NLP):** English |
|
|
- **License:** MIT |
|
|
- **Finetuned from model:** distilbert/distilroberta-base |
|
|
|
|
|
### Model Sources |
|
|
|
|
|
- **Repository:** [glyphsoftware/reasoning-router](https://huggingface.co/glyphsoftware/reasoning-router) |
|
|
- **Base Model:** [distilbert/distilroberta-base](https://huggingface.co/distilbert/distilroberta-base) |
|
|
- **Training Dataset:** [glyphsoftware/reasoning-router](https://huggingface.co/datasets/glyphsoftware/reasoning-router) |
|
|
|
|
|
## Uses |
|
|
|
|
|
### Direct Use |
|
|
|
|
|
This model can be used directly for text classification tasks where you need to determine the reasoning depth of text content. It's particularly effective for: |
|
|
|
|
|
- **Cost Optimization**: Can be used in inference pipeline to route requests to appropriate models |
|
|
- **Educational platforms**: Automatically categorizing content by difficulty level |
|
|
- **Content moderation**: Identifying complex reasoning that might require review |
|
|
- **Research tools**: Analyzing reasoning patterns in academic or professional texts |
|
|
- **Quality control**: Ensuring content meets specific reasoning requirements |
|
|
|
|
|
### Downstream Use |
|
|
|
|
|
The model can be fine-tuned for specific domains or applications: |
|
|
- **Domain-specific reasoning classification** (e.g., medical, legal, technical) |
|
|
- **Multi-language reasoning detection** (with appropriate training data) |
|
|
- **Integration into larger NLP pipelines** for content analysis |
|
|
|
|
|
### Out-of-Scope Use |
|
|
|
|
|
This model is not designed for: |
|
|
- **General text classification** beyond reasoning depth |
|
|
- **Reasoning generation** or explanation |
|
|
- **Content creation** or text generation |
|
|
- **Multilingual reasoning detection** (trained only on English) |
|
|
|
|
|
## Bias, Risks, and Limitations |
|
|
|
|
|
### Limitations |
|
|
|
|
|
- **Language restriction**: Only trained on English text |
|
|
- **Domain bias**: Performance may vary across different domains and writing styles |
|
|
- **Context sensitivity**: Reasoning depth can be subjective and context-dependent |
|
|
- **Training data limitations**: Performance depends on the quality and representativeness of the training data |
|
|
|
|
|
### Recommendations |
|
|
|
|
|
Users should: |
|
|
- **Validate results** on their specific domain and use case |
|
|
- **Consider context** when interpreting reasoning depth classifications |
|
|
- **Test thoroughly** before deploying in production environments |
|
|
- **Monitor performance** and retrain if necessary for new domains |
|
|
|
|
|
## How to Get Started with the Model |
|
|
|
|
|
### Using the Model |
|
|
|
|
|
```python |
|
|
from transformers import AutoTokenizer, AutoModelForSequenceClassification |
|
|
import torch |
|
|
|
|
|
# Load the model and tokenizer |
|
|
model_name = "glyphsoftware/reasoning-router" |
|
|
tokenizer = AutoTokenizer.from_pretrained(model_name) |
|
|
model = AutoModelForSequenceClassification.from_pretrained(model_name) |
|
|
|
|
|
# Prepare your text |
|
|
text = "Your text here that you want to classify for reasoning depth." |
|
|
|
|
|
# Tokenize and predict |
|
|
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=256) |
|
|
with torch.no_grad(): |
|
|
outputs = model(**inputs) |
|
|
probabilities = torch.softmax(outputs.logits, dim=-1) |
|
|
predicted_class = torch.argmax(probabilities, dim=-1).item() |
|
|
|
|
|
# Get the label |
|
|
labels = ["no-reasoning", "low-reasoning", "medium-reasoning", "high-reasoning"] |
|
|
predicted_label = labels[predicted_class] |
|
|
confidence = probabilities[0][predicted_class].item() |
|
|
|
|
|
print(f"Predicted reasoning level: {predicted_label}") |
|
|
print(f"Confidence: {confidence:.3f}") |
|
|
``` |
|
|
|
|
|
### Using the Pipeline |
|
|
|
|
|
```python |
|
|
from transformers import pipeline |
|
|
|
|
|
classifier = pipeline("text-classification", model="glyphsoftware/reasoning-router") |
|
|
result = classifier("Your text here") |
|
|
print(result) |
|
|
``` |
|
|
|
|
|
## Evaluation |
|
|
|
|
|
#### Factors |
|
|
|
|
|
Evaluation considers: |
|
|
- **Reasoning level distribution** across the test set |
|
|
- **Text length variations** (up to 256 tokens) |
|
|
- **Domain diversity** in the training data |
|
|
|
|
|
#### Metrics |
|
|
|
|
|
- **Accuracy**: Overall classification accuracy |
|
|
- **F1 Score**: Weighted F1 score across all classes |
|
|
- **Per-class performance**: Individual class precision and recall |
|
|
|
|
|
### Results |
|
|
|
|
|
The model achieves competitive performance on reasoning depth classification, with optimized F1 score as the primary metric for model selection during training. |
|
|
|
|
|
## Model Examination |
|
|
|
|
|
The model architecture is based on DistilRoBERTa, which provides: |
|
|
- **Efficient inference** with reduced model size compared to full RoBERTa |
|
|
- **Robust representations** for text classification tasks |
|
|
- **Fast tokenization** with the Rust-backed BPE tokenizer |
|
|
|
|
|
|
|
|
## Technical Specifications |
|
|
|
|
|
### Model Architecture and Objective |
|
|
|
|
|
- **Architecture:** DistilRoBERTa (6-layer transformer with 768 hidden dimensions) |
|
|
- **Objective:** Sequence classification for reasoning depth detection |
|
|
- **Output:** 4-class probability distribution |
|
|
- **Max sequence length:** 256 tokens |
|
|
|
|
|
### Compute Infrastructure |
|
|
|
|
|
#### Hardware |
|
|
|
|
|
- **Training:** Compatible with CUDA, MPS, and CPU |
|
|
- **Inference:** Optimized for CPU and GPU deployment |
|
|
|
|
|
#### Software |
|
|
|
|
|
- **PyTorch:** 2.8.0+ |
|
|
- **Transformers:** 4.55.0+ |
|
|
- **Python:** 3.12+ |
|
|
|
|
|
## Glossary |
|
|
|
|
|
- **Reasoning Depth**: The level of complexity and sophistication in logical thinking and argumentation present in text |
|
|
- **No-reasoning**: Text that presents information without logical connections or argumentation |
|
|
- **Low-reasoning**: Text with basic logical connections and simple argumentation |
|
|
- **Medium-reasoning**: Text with moderate complexity in logical structure and argumentation |
|
|
- **High-reasoning**: Text with sophisticated logical reasoning, complex argumentation, and deep analysis |
|
|
|
|
|
## More Information |
|
|
|
|
|
For more details about the training process, dataset, and usage examples, please refer to the project repository and documentation. |
|
|
|
|
|
## Model Card Authors |
|
|
|
|
|
[Glyph Software](https://huggingface.co/glyphsoftware) |
|
|
|
|
|
## Model Card Contact |
|
|
|
|
|
[Contact Us](mailto:[email protected]) |
