Storing and restoring GPT-J model
Hi,I am trying to save and later restore the GPT-J models as shown below. But I am getting an errorAttributeError: ‘BaseModelOutputWithPast’ object has no attribute ‘logits’Any help with this will be much appreciatedSaving the modelfrom transformers import GPTJModel, GPTJConfig

print('---start---')
config = GPTJConfig.from_pretrained("EleutherAI/gpt-j-6B")
model = GPTJModel(config)
model.save_pretrained("my-gpt-j-model")from transformers import AutoTokenizer

print('---start---')
tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-j-6B")
tokenizer.save_pretrained("my-gpt-j-tokenizer")Restoring and using the modelThis piece of code is throwing the errorfrom transformers import AutoTokenizer
from transformers import GPTJModel

#gpt_config = GPTJConfig.from_pretrained("my-gpt-j-model")
gpt_model = GPTJModel.from_pretrained("my-gpt-j-model")
t=gpt_model.eval()
print(t)
input_text = "Hello my name is Blake and"

tokenizer = AutoTokenizer.from_pretrained("my-gpt-j-tokenizer")
input_ids = tokenizer.encode(str(input_text),return_tensors='pt')

output=gpt_model.generate(
    input_ids,
    do_sample=True,
    max_length=20,
    top_p=0.7,
    top_k=0,
    temperature=1.0,
)
print(tokenizer.decode(output[0],skip_special_tokens=True))
2022-09-20T19:29:50Z
Is there a solution to this? I’ve also encountered it.
2024-05-08T18:08:06Z
Have you fixed the problem?I am having the same kind of problemPlease help me
2024-05-08T19:19:20Z
Hi,This is because you’re usingGPTJModelto generate text, whereas you should be usingGPTJForCausalLM.This is because xxxModel classes in the Transformers library usually don’t include a task-specific head on top, they only output hidden states (features).GPTJForCausalLMon the other hand adds a language modeling head on top ofGPTJModel, which is what you need.