Git consumes 8GB+ of memory when trying to clone from HF
This repo:NousResearch/Nous-Hermes-2-Yi-34B at mainhas 23 files, and no subdirectories. It seems pretty simple to me.But for some reason, git clone consumes so much memory that the process is killed.I presume it’s a LFS thing because I’ve never seen git need gigs of memory?!? But even the largest file is around half of the RAM that git is eating up.Any ideas on how to limit memory usage so it doesn’t die?
2024-02-09T16:03:52Z
Update:Downloadingpointers only(withGIT_LFS_SKIP_SMUDGE=1), then doing an immediate lfs pull seems to be working (so far). git consuming 0.7GB of memory instead of 8GB+GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/NousResearch/Nous-Hermes-2-Yi-34B
cd Nous-Hermes-2-Yi-34B
git lfs pullEdit: Completed successfully. Not sure this is really a “solution”, but it worked.
2024-02-10T04:04:34Z
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.