Not compatible with GGUF

#1
by ruleez - opened

it seems the lora doesn't work with the quants (for instance: https://huggingface.co/city96/Qwen-Image-gguf/blob/main/qwen-image-Q8_0.gguf)

lora key not loaded: transformer_blocks.0.attn.add_k_proj.alpha
lora key not loaded: transformer_blocks.0.attn.add_k_proj.lora_down.weight
lora key not loaded: transformer_blocks.0.attn.add_k_proj.lora_up.weight
lora key not loaded: transformer_blocks.0.attn.add_q_proj.alpha
lora key not loaded: transformer_blocks.0.attn.add_q_proj.lora_down.weight
lora key not loaded: transformer_blocks.0.attn.add_q_proj.lora_up.weight
lora key not loaded: transformer_blocks.0.attn.add_v_proj.alpha
lora key not loaded: transformer_blocks.0.attn.add_v_proj.lora_down.weight
lora key not loaded: transformer_blocks.0.attn.add_v_proj.lora_up.weight
lora key not loaded: transformer_blocks.0.attn.to_add_out.alpha
lora key not loaded: transformer_blocks.0.attn.to_add_out.lora_down.weight
lora key not loaded: transformer_blocks.0.attn.to_add_out.lora_up.weight

etc.

update comfyui and chose the nightly version in the manager

@Aitrepreneur

Thanks for this pro-tip, I had been trying to get Qwen-Image LoRAs working for a couple days! This seems to work combined with https://huggingface.co/flymy-ai/qwen-image-realism-lora as well and does 10 steps very fast using qwen_image_distill_full_fp8_e4m3fn.safetensors on my 3090TI FE @ 350 watts power cap. I was surprised the fp8e4m3 was faster than the Q6_K GGUF despite my sm86 arch (need >sm89 for native fp8e4me...)

Anyway have fun ya'll!

ruleez changed discussion status to closed

Sign up or log in to comment