Error in loading pipeline. ValueError: Non-consecutive added token '<|startoftext|>' found
#4
by
neogong
- opened
Hi Lykon,
Thanks for your great work! I'm following the instruction to load the pipeline:
pipe = AutoPipelineForText2Image.from_pretrained('lykon/dreamshaper-xl-turbo', torch_dtype=torch.float16, variant="fp16")
but it throws an error of ValueError: Non-consecutive added token '<|startoftext|>' found. Should have index 49408 but has index 49406 in saved vocabulary"
. Do you have any idea on how to solve this issue?
Thank you.
Solved by update transformers and tokenizers to the latest version.
neogong
changed discussion status to
closed