How do I release memory after using AutoModel.from_pretrained() to load model
after execute following code on apple’s silicon chip , even though the program ends memory is still occupied too much.sorry i’m a beginner of both English and Deep learning. Is it a bug or just I used a wrong way?image2148×1132 379 KB
2024-09-22T08:36:52Z
I generally do it like this, but it’s probably not right and there are smarter ways to do it. I’m interested too.import torch
import gc
del llm_model
torch.cuda.empty_cache()
gc.collect()
2024-09-22T08:49:50Z
thank you, actullydel llm_modelworks, the problem is the memory monitor didn’t refresh
2024-09-22T09:42:44Z
Hmmm, like this?In your case anyway, it would be better to remove it from VRAM first and then delete it.But well, the torch tensor used in the model’s content is quite stubborn…If you don’t delete the tensors in the model, not the pipes or the model, one by one, they might be referenced from somewhere and won’t disappear.import torch
import gc
llm_model.to("cpu")
del llm_model
torch.cuda.empty_cache()
gc.collect()It is more reliable todeltogether any relevant models and pipes.
2024-09-24T11:19:53Z
thank you! it’s helpful
2024-09-24T23:20:17Z
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.