How to use single-file diffuser checkpoints
I can’t figure out how to load a diffusion model from single files, likeflux1-dev.safetensorsfor FLUX.1 (black-forest-labs/FLUX.1-dev at main)It works fine when usingFluxPipeline.from_pretrained()which uses the multi-file format, but when I tryFluxPipeline.from_single_file()on the above file I get:diffusers.loaders.single_file_utils.SingleFileComponentError: Failed to load CLIPTextModel. Weights for this component appear to be missing in the checkpoint.
Please load the component before passing it in as an argument to `from_single_file`.

text_encoder = CLIPTextModel.from_pretrained('...')
pipe = FluxPipeline.from_single_file(<checkpoint path>, text_encoder=text_encoder)So I was wondering, if the file only contains the transformer, which might be converted and inserted into the multi-file tree. Looking inside, though, I found that the number of tensors per layer is different and the dimensions are not quite the same, either.There must be some way to use that file through diffusers, right?
2024-12-26T08:56:18Z
Hi,The single file is meant to be used withComfyUI.
2024-12-26T09:55:51Z
I see. Is there are reasonably easy way to still use it with diffusers? Some finetunes only provide it in that format.
2024-12-26T16:26:56Z
Files for ComfyUI can often be loaded using the following method. Please specify an appropriate repo for the missing parts other than DiT.from diffusers import FluxPipeline, FluxTransformer2DModel

HF_TOKEN = "hf_***"
flux_repo = "multimodalart/FLUX.1-dev2pro-full"
ckpt_path = "https://huggingface.co/Comfy-Org/flux1-dev/blob/main/flux1-dev-fp8.safetensors"
transformer = FluxTransformer2DModel.from_single_file(ckpt_path, subfolder="transformer", torch_dtype=torch.bfloat16, token=HF_TOKEN)
pipe = FluxPipeline.from_pretrained(flux_repo, transformer=transformer, torch_dtype=torch.bfloat16, token=HF_TOKEN)huggingface.coGGUFWe’re on a journey to advance and democratize artificial intelligence through open source and open science.
2025-01-02T08:42:52Z
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.