LSCP: Enhanced Large Scale Colloquial Persian Language Understanding
  Language recognition has been significantly advanced in recent years by means
of modern machine learning methods such as deep learning and benchmarks with
rich annotations. However, research is still limited in low-resource formal
languages. This consists of a significant gap in describing the colloquial
language especially for low-resourced ones such as Persian. In order to target
this gap for low resource languages, we propose a "Large Scale Colloquial
Persian Dataset" (LSCP). LSCP is hierarchically organized in a semantic
taxonomy that focuses on multi-task informal Persian language understanding as
a comprehensive problem. This encompasses the recognition of multiple semantic
aspects in the human-level sentences, which naturally captures from the
real-world sentences. We believe that further investigations and processing, as
well as the application of novel algorithms and methods, can strengthen
enriching computerized understanding and processing of low resource languages.
The proposed corpus consists of 120M sentences resulted from 27M tweets
annotated with parsing tree, part-of-speech tags, sentiment polarity and
translation in five different languages.
