Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation
The first bit of code runs, but I get the eos_token_id error.  The second bit fails.  So adding the token to the input_ids doesn’t work.At least the model persists for the second query, so I’m making progress.Anyway, big thanks to Huggingface for posting these great models.  You guys rock!from transformers import GPTNeoForCausalLM, GPT2Tokenizer
from macos_speech import Synthesizer

model = GPTNeoForCausalLM.from_pretrained("EleutherAI/gpt-neo-1.3B")
tokenizer = GPT2Tokenizer.from_pretrained("EleutherAI/gpt-neo-1.3B")

prompt = (
    "In a shocking finding, "
)

input_ids = tokenizer(prompt, return_tensors="pt").input_ids

gen_tokens = model.generate(
    input_ids,
    do_sample=True,
    temperature=0.9,
    max_length=100,
)
gen_text = tokenizer.batch_decode(gen_tokens)[0]
print(gen_text)

prompt = (
    "Albert Einstein was "
)

input_ids_pre = tokenizer(prompt, return_tensors="pt").input_ids
nput_ids = input_ids_pre + tokenizer.eos_token

gen_tokens = model.generate(
    input_ids,
    do_sample=True,
    temperature=0.9,
    max_length=100,
)
gen_text = tokenizer.batch_decode(gen_tokens)[0]
print(gen_text)
2022-08-29T05:18:11Z
Oops, I had a typo.  I misspelled input_ids.  I fixed that, and it still failed.Anyway, this token error seems to be one that a lot of people have encountered.  I’ve read a lot of posts about it.  Maybe a better question is, how can I find out about this sort of thing in the docs?  Or would the tutorial answer this question?
2022-08-29T06:10:56Z
Here’s the code that suppresses the error:`from transformers import pipeline
import time
start = time.time()
print("Time elapsed on working...")
#generator = pipeline('text-generation', model='bigscience/bloom-560m')
#generator = pipeline('text-generation', model='gpt2')
generator = pipeline('text-generation', model='EleutherAI/gpt-neo-1.3B')
#generator = pipeline('text-generation', model='EleutherAI/gpt-j-6B')
text = generator("Albert Einstein was:", max_length=10, pad_token_id=50256, num_return_sequences=1)
print(text)
time.sleep(0.9)
end = time.time()
print("Time consumed in working: ",end - start)
text = generator("Albert Einstein was:", max_length=10, num_return_sequences=1)
print(text)
time.sleep(0.9)
end = time.time()
print("Time consumed in working: ",end - start)`
2022-08-29T06:12:04Z
I added pad_token_id = 50256 to my pipeline.
2024-09-24T16:22:23Z
What when I don’t want open ended ? The model seems to ignore now the eos token and keeps going until the max_tokes limit is reached. I would rather have it to stop when it is done.
2024-09-24T16:23:03Z
Why 50256 and not 50258?