maximuspowers/muat-mean-std-classifier

Pattern Classifier

This model was trained to classify which patterns a subject model was trained on, based on neuron activation signatures.

Dataset

• Training Dataset: maximuspowers/muat-mean-std
• Input Mode: signature
• Number of Patterns: 14

Patterns

The model predicts which of the following 14 patterns the subject model was trained to classify as positive:

1. palindrome
2. sorted_ascending
3. sorted_descending
4. alternating
5. contains_abc
6. starts_with
7. ends_with
8. no_repeats
9. has_majority
10. increasing_pairs
11. decreasing_pairs
12. vowel_consonant
13. first_last_match
14. mountain_pattern

Model Architecture

• Signature Encoder: [512, 256, 256, 128]
• Activation: relu
• Dropout: 0.2
• Batch Normalization: True

Training Configuration

• Optimizer: adam
• Learning Rate: 0.001
• Batch Size: 16
• Loss Function: BCE with Logits (with pos_weight for training, unweighted for validation)

Test Set Performance

• F1 Macro: 0.2832
• F1 Micro: 0.2498
• Hamming Accuracy: 0.7306
• Exact Match Accuracy: 0.0140
• BCE Loss: 0.4844

Per-Pattern Performance (Test Set)

Pattern	Precision	Recall	F1 Score
palindrome	11.1%	89.8%	19.8%
sorted_ascending	59.7%	56.6%	58.1%
sorted_descending	15.8%	66.2%	25.5%
alternating	19.8%	72.4%	31.1%
contains_abc	30.8%	57.6%	40.1%
starts_with	9.1%	59.4%	15.8%
ends_with	10.3%	73.8%	18.1%
no_repeats	17.8%	32.1%	22.9%
has_majority	33.3%	60.5%	43.0%
increasing_pairs	23.3%	35.3%	28.1%
decreasing_pairs	19.4%	60.9%	29.5%
vowel_consonant	9.8%	76.9%	17.4%
first_last_match	15.3%	96.6%	26.5%
mountain_pattern	15.3%	31.7%	20.6%