L3Cube-MahaCorpus and MahaBERT: Marathi Monolingual Corpus, Marathi BERT
  Language Models, and Resources
  We present L3Cube-MahaCorpus a Marathi monolingual data set scraped from
different internet sources. We expand the existing Marathi monolingual corpus
with 24.8M sentences and 289M tokens. We further present, MahaBERT, MahaAlBERT,
and MahaRoBerta all BERT-based masked language models, and MahaFT, the fast
text word embeddings both trained on full Marathi corpus with 752M tokens. We
show the effectiveness of these resources on downstream Marathi sentiment
analysis, text classification, and named entity recognition (NER) tasks. We
also release MahaGPT, a generative Marathi GPT model trained on Marathi corpus.
Marathi is a popular language in India but still lacks these resources. This
work is a step forward in building open resources for the Marathi language. The
data and models are available at https://github.com/l3cube-pune/MarathiNLP .
