LLM for a language that is on the extinction list
Hello Friends,My language is on the UNESCO extinction list.Therefore I want to do something to save the language for the future.There are some old documents that are still available but they cannot be read or understood by younger people because the language is not widely used anymore.So I was thinking that maybe I could use LLM to make the language available for the future.The idea I was having is make a LLM using all the available documents and text from the language.This language model shall then output relevant parts of the text based on input.The language model shall also provide simple explanation of the text that is output.This way the language can be made available for reference and usage and also to learn the language and understand ancient texts.How could I achieve this.I am also willing to do research and spend time and effort to ensure that the language does not go extinct.Any references or documents would also be helpful.(I have programming knowledge and I know how to use hugging face transformers and have knowledge on using LLMS to generate text,  text classification etc. So writing code is not a barrier)ThanksDida
2024-11-21T21:45:24Z
ai phoneme segmentation for endangered language, can u help me out with my project. any new insights
2024-11-22T07:24:34Z
Dida, this is a vital initiative for language preservation. Utilizing language models, such as those available through Hugging Face, is a strategic approach. Given your skills, you can compile existing texts into a dataset, then train a model to both interpret and generate your language. This not only aids in preservation but also in making the language accessible for educational purposes. Focused and thoughtful application of these technologies could significantly impact the survival and understanding of your language.