Generating 10000 sentences from GptNeo Model results in out of memory error
I was doing some work where I wanted to generate 10000 sentences from the GptNeo Model. I have a GPU of size 40GB and am running the model in the GPU but everytime the code runs out of memory. Is there a limitation to the number of sentences that I can generate. Below is a small snippet of my code.tokenizer = GPT2Tokenizer.from_pretrained(model)model = GPTNeoForCausalLM.from_pretrained(model , pad_token_id = tokenizer.eos_token_id)model.to(device)input_ids = tokenizer.encode(sentence, return_tensors=‘pt’)gen_tokens = model.generate(input_ids,do_sample=True,top_k=50,num_return_sequences=10000)