Why I get this UNet generated latents result been so messy?
Why does UNet single-step prediction for text-to-image generation produce the following result: output-s1-cguided_pred.png The correct result should be: output-s1-dnoised.png What could be the reason for this discrepancy>output-s1-cguided_pred.png:output-s1-cguided_pred512×512 28.2 KBthis is one-step-inference, and not move to dnoise.prediction_method(such v_prediction) & scheduler.step.just checking UNet output, and send to VAE to get upper pic.
2024-05-30T05:42:03Z
the correct output should be like this below:output-s1-dnoised.png:output-s1-dnoised512×512 70.4 KB
2024-05-31T03:32:26Z
I’ve found out that is caused by wrong unconditional embedding.for prompts after tokenizer → encoding → embedding, in embedding stage, I mix hidden_state with uncond(negative CLIP out) with wrong summed & mean. get wrong shape on [1, max_length, vocab_size] which should be  [2, max_length, vocab_size], as UNet input. if 1, some nodes will calculated died.
2024-05-31T15:32:53Z
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.