Is petals the precursor to what spark did for big data in LLMs?
Looking at thisGitHubGitHub - bigscience-workshop/petals: 🌸 Run 100B+ language models at home,...🌸 Run 100B+ language models at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading - GitHub - bigscience-workshop/petals: 🌸 Run 100B+ language models at home, BitTorr...It feels like we jumped from local straight to a globally distributed model.But what about GPU clusters? Are there any technologies like spark but for GPUs & LLMs/transformers? What’s the avenue here?