Cleaup_tokenization_spaces error
I have set up ComfyUI on Ubuntu and am attempting to run  the workflow downloaded from here (ComfyUI workflow for Flux (simple) | ComfyUI Workflow) for the first time. However, I am getting an warningclean_up_tokenization_spacesand then  ComfyUI is shutdown - full console output below:got prompt
model weight dtype torch.bfloat16, manual cast: None
model_type FLUX
/home/garrett/AI/ComfyUI/venv/lib/python3.10/site-packages/transformers/tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884
  warnings.warn(
./launch.sh: line 7: 47378 Killed                  python3 main.pyThe Github bug pointed to says a  parameter needs to be set to false but where do I change this?
2024-08-25T07:45:14Z
Got the same error
2024-09-16T12:18:09Z
You can add the flagclean_up_tokenization_spacesintro theAutoTokenizer 
tokenizer = AutoTokenizer.from_pretrained(model_name,clean_up_tokenization_spaces=False)Before, It will be set toTrueby default. This behavior will be depracted in transformers v4.45, and will be then set toFalseby default.The clean_up_tokenization_spaces wether to cleanup spaces after decoding, cleanup consists in removing potential artifacts like extra spaces. E.g., If you havestate-of-the-artit will be encoded asstate - of - the - art.