Compatibility Issue with transformers Version > 4.44.2 Breaking JoyCaption Beta 1 in ComfyUI
I encountered a critical compatibility problem while using the JoyCaption Beta 1 node within ComfyUI (v0.3.15). After upgrading the transformers Python library beyond version 4.44.2 (e.g., to 4.51.0), the JoyCaption node fails to load properly and throws errors related to the tokenizer initialization:
Exception: data did not match any variant of untagged enum ModelWrapper at line XXXXX column X
Exception: fast_tokenizer = TokenizerFast.from_file(fast_tokenizer_file)
Despite attempts to downgrade transformers to version 4.44.2 (the version recommended by some sources), the issue remains unresolved.
Details:
The error happens while loading the AutoProcessor from the model checkpoint.
Model files and directory structure have been verified correct.
The problem seems related to incompatible tokenizer or processor files within the checkpoint versus the transformers version.
Higher versions of transformers (e.g., 4.51.0) cause similar or worse failures.
The root cause might be a mismatch between checkpoint file formats and the transformers API expectations.
Attempts to fix via dependency downgrades or upgrades have been unsuccessful so far.
Request:
Has anyone successfully resolved this? Are there known fixes, updates to JoyCaption checkpoints, or specific version combinations of transformers that work reliably? Advice on how to handle this dependency challenge without destabilizing other parts of ComfyUI would be appreciated.
I was not able to reproduce this error on transformers version 4.54.0 in my Windows install of ComfyUI Desktop. If you are able to, and willing, I'd love more details on the error and possibly a dump of the environment (pip freeze). That might help me narrow down what's going on. I might also recommend deleting and redownloading the JoyCaption model, just in case the download got corrupted or interrupted.