Malaysian MaskLM
Collection
Trained on 17B tokens, 81GB of cleaned texts, able to understand standard Malay, local Malay, local Mandarin, Manglish, and local Tamil.
•
7 items
•
Updated
Pretrain BASE 512 masking context length DebertaV2 on Malaysian text
Special thanks to https://github.com/aisyahrzk for pretraining DebertaV2 Base.
WanDB at https://wandb.ai/aisyahrazak/deberta-base?nw=nwuseraisyahrazak
- Downloads last month
- 11
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model has no pipeline_tag.