File size: 2,061 Bytes
d89dade |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 |
---
language: en
license: apache-2.0
datasets:
- nyu-mll/glue
---
# EFTNAS Model Card: eftnas-s2-bert-medium
The super-networks fine-tuned on BERT-medium with [GLUE benchmark](https://gluebenchmark.com/) using EFTNAS.
## Model Details
### Information
- **Model name:** eftnas-s2-bert-medium-[TASK]
- **Base model:** [google/bert_uncased_L-8_H-512_A-8](https://huggingface.co/google/bert_uncased_L-8_H-512_A-8)
- **Subnetwork version:** Super-network
- **NNCF Configurations:** [eftnas_configs](https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning/tree/main/EFTNAS/eftnas_configs)
### Training and Evaluation
[GLUE benchmark](https://gluebenchmark.com/)
## Results
Results of the optimal sub-network discoverd from the super-network:
| Model | GFLOPs | GLUE Avg. | MNLI-m | QNLI | QQP | SST-2 | CoLA | MRPC | RTE |
|-------------------------------|-----------|---------------|----------|------|----------|----------|----------|----------|------|
| **Test Set:** |
| [**EFTNAS-S1**]() | 5.7 | 77.7 | 83.7 | 89.9 | 71.8 | 93.4 | 52.6 | 87.6 | 65.0 |
| [**EFTNAS-S2**]() | 2.2 | 75.2 | 82.0 | 87.8 | 70.6 | 91.4 | 44.5 | 86.1 | 64.0 |
## Model Sources
- **Repository:** [https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning/tree/main/EFTNAS](https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning/tree/main/EFTNAS)
- **Paper:** [Searching for Efficient Language Models in First-Order Weight-Reordered Super-Networks]()
## Citation
```bibtex
@inproceedings{
eftnas2024,
title={Searching for Efficient Language Models in First-Order Weight-Reordered Super-Networks},
author={J. Pablo Munoz and Yi Zheng and Nilesh Jain},
booktitle={The 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation},
year={2024},
url={}
}
```
## License
Apache-2.0
|