Creation of Images from Text-Prompt (Customized Training)
HelloI want to use ‘Stability Diffusion’ to generate images from text. However I would like to get it trained on my custom images too.How do I go about doing this?Please help.Thank You.
2024-12-31T10:12:59Z
There is also a way to train the entire model for yourself, but personally I think that in most cases, training LoRA is the cheapest way to go.Well, it is better to learn the basic usage of StableDiffusion first.huggingface.coLoRAWe’re on a journey to advance and democratize artificial intelligence through open source and open science.github.comGitHub - bmaltais/kohya_ssContribute to bmaltais/kohya_ss development by creating an account on GitHub.
2025-01-03T07:25:34Z
Sorry, if I just have 100 or so custom images, which way should I be going?
2025-01-06T09:43:13Z
Hello,I am trying to see if LORA works or not. I have specified the following parameters while training:Training Images Folder - D:\Ganu\AIImage\huggingface\kohya_ss\data\imagesI have put a simple image in the folder (image1.jpeg)While training using LORA, I get the following error:“ERROR No data found. Please verify arguments (train_data_dir must be the parent of folders with images) / train_network.py:212There are no images. Please check the argument specification (train_data_dir must specify the parent folder of the folder with images, not the folder with images)”Traceback (most recent call last):File “D:\Ganu\AIImage\huggingface\kohya_ss\kohya_ss\sd-scripts\train_db.py”, line 529, intrain(args)File “D:\Ganu\AIImage\huggingface\kohya_ss\kohya_ss\sd-scripts\train_db.py”, line 190, in traintrain_dataloader = torch.utils.data.DataLoader(File “D:\Ganu\AIImage\huggingface\kohya_ss\kohya_ss\venv\lib\site-packages\torch\utils\data\dataloader.py”, line 349, ininitsampler = RandomSampler(dataset, generator=generator)  # type: ignore[arg-type]File “D:\Ganu\AIImage\huggingface\kohya_ss\kohya_ss\venv\lib\site-packages\torch\utils\data\sampler.py”, line 140, ininitraise ValueError(f"num_samples should be a positive integer value, but got num_samples={self.num_samples}“)ValueError: num_samples should be a positive integer value, but got num_samples=0Traceback (most recent call last):File “D:\Ganu\AIImage\huggingface\kohya_ss\Python310\lib\runpy.py”, line 196, in _run_module_as_mainreturn _run_code(code, main_globals, None,File “D:\Ganu\AIImage\huggingface\kohya_ss\Python310\lib\runpy.py”, line 86, inrun_codeexec(code, run_globals)File "D:\Ganu\AIImage\huggingface\kohya_ss\kohya_ss\venv\Scripts\accelerate.EXE_main.py”, line 7, insys.exit(main())File “D:\Ganu\AIImage\huggingface\kohya_ss\kohya_ss\venv\lib\site-packages\accelerate\commands\accelerate_cli.py”, line 47, in mainargs.func(args)File “D:\Ganu\AIImage\huggingface\kohya_ss\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py”, line 1017, in launch_commandsimple_launcher(args)File “D:\Ganu\AIImage\huggingface\kohya_ss\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py”, line 637, in simple_launcherraise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)subprocess.CalledProcessError: Command ‘[‘D:\Ganu\AIImage\huggingface\kohya_ss\kohya_ss\venv\Scripts\python.exe’, ‘D:/Ganu/AIImage/huggingface/kohya_ss/kohya_ss/sd-scripts/train_db.py’, ‘–config_file’, ‘D:/Ganu/AIImage/huggingface/kohya_ss/kohya_ss/outputs/config_dreambooth-20250106-152049.toml’]’ returned non-zero exit status 1.Any ideas what to do?
2025-01-06T10:43:34Z
In that case, I recommend creating a LoRA. It is said that you can create one with 10 images. Of course, the more images you have, the more reliable it will usually be.Some people use thousands of images for training models. However, if you have 100 images, I don’t think it’s impossible.Whether you’re using LoRA or training, the more popular the model you use is and the more suitable it is for the images you’re using, the easier it will be, so I think it’s a good idea to first find a model that is close to your goal. I think that many people use Animagine 3.1, Illustrious, and NoobAI for anime pictures, and SDXL 1.0 and FLUX.1 Dev for live-action pictures. Also, Pony is a type of SDXL that is very popular, but it is special in both good and bad ways, so I think it’s better to avoid it until you get used to it.
2025-01-06T12:02:42Z
The above reply took more than a few days to be reflected, so it’s become a bit of a mystery to everyone…The script error is difficult to understand, but it seems to be something like this. There is a knack to the folder structure.github.com/bmaltais/kohya_ss’not found data‘error when trainingopened09:54AM - 01 May 24 UTCkaelem117:45:46-693860 INFO     Training has ended.
17:52:57-749330 INFO     Start tra…ining LoRA Standard ...
17:52:57-751289 INFO     Validating lr scheduler arguments...
17:52:57-751289 INFO     Validating optimizer arguments...
17:52:57-752288 INFO     Validating model file or folder path
                         G:/Comfyui/ComfyUI-aki-v1.3/models/checkpoints/sdXL_v10.safetensors existence...
17:52:57-754288 INFO     ...valid
17:52:57-754288 INFO     Validating output_dir path G:/Comfyui/kohyass/output existence...
17:52:57-755710 INFO     ...valid
17:52:57-756715 INFO     Validating train_data_dir path G:/Comfyui/kohyass/train data/30_2D existence...
17:52:57-757716 INFO     ...valid
17:52:57-757716 INFO     reg_data_dir not specified, skipping validation
17:52:57-759719 INFO     logging_dir not specified, skipping validation
17:52:57-762722 INFO     log_tracker_config not specified, skipping validation
17:52:57-763719 INFO     resume not specified, skipping validation
17:52:57-764718 INFO     vae not specified, skipping validation
17:52:57-765720 INFO     network_weights not specified, skipping validation
17:52:57-766717 INFO     dataset_config not specified, skipping validation
17:52:57-769718 INFO     Regulatization factor: 1
17:52:57-771717 INFO     Total steps: 0
17:52:57-772717 INFO     Train batch size: 2
17:52:57-773738 INFO     Gradient accumulation steps: 1
17:52:57-774817 INFO     Epoch: 30
17:52:57-775824 INFO     Max train steps: 1600
17:52:57-777824 INFO     stop_text_encoder_training = 0
17:52:57-777824 INFO     lr_warmup_steps = 0
17:52:57-780823 INFO     Saving training config to G:/Comfyui/kohyass/output\glass_20240501-175257.json...
17:52:57-782823 INFO     Executing command: "G:\Comfyui\kohyass\kohya_ss\venv\Scripts\accelerate.EXE" launch
                         --dynamo_backend no --dynamo_mode default --mixed_precision bf16 --num_processes 1
                         --num_machines 1 --num_cpu_threads_per_process 2
                         "G:/Comfyui/kohyass/kohya_ss/sd-scripts/sdxl_train_network.py" --config_file
                         "./outputs/config_lora-20240501-175257.toml" with shell=True
17:52:57-788823 INFO     Command executed.
A matching Triton is not available, some optimizations will not be enabled.
Error caught was: No module named 'triton'
2024-05-01 17:53:04 INFO     Loading settings from ./outputs/config_lora-20240501-175257.toml...      train_util.py:3744
                    INFO     ./outputs/config_lora-20240501-175257                                    train_util.py:3763
2024-05-01 17:53:04 INFO     prepare tokenizers                                                   sdxl_train_util.py:134
2024-05-01 17:53:06 INFO     update token length: 75                                              sdxl_train_util.py:159
                    INFO     Using DreamBooth method.                                               train_network.py:172
                    INFO     prepare images.                                                          train_util.py:1572
                    INFO     0 train images with repeating.                                           train_util.py:1613
                    INFO     0 reg images.                                                            train_util.py:1616
                    WARNING  no regularization images / 正則化画像が見つかりませんでした              train_util.py:1621
                    INFO     [Dataset 0]                                                              config_util.py:565
                               batch_size: 2
                               resolution: (1024, 1024)
                               enable_bucket: False
                               network_multiplier: 1.0


                    INFO     [Dataset 0]                                                              config_util.py:571
                    INFO     loading image sizes.                                                      train_util.py:853
0it [00:00, ?it/s]
                    INFO     make buckets                                                              train_util.py:859
                    WARNING  min_bucket_reso and max_bucket_reso are ignored if bucket_no_upscale is   train_util.py:876
                             set, because bucket reso is defined by image size automatically /
                             bucket_no_upscaleが指定された場合は、bucketの解像度は画像サイズから自動計
                             算されるため、min_bucket_resoとmax_bucket_resoは無視されます
                    INFO     number of images (including repeats) /                                    train_util.py:905
                             各bucketの画像枚数（繰り返し回数を含む）
G:\Comfyui\kohyass\kohya_ss\venv\lib\site-packages\numpy\core\fromnumeric.py:3504: RuntimeWarning: Mean of empty slice.
  return _methods._mean(a, axis=axis, dtype=dtype,
G:\Comfyui\kohyass\kohya_ss\venv\lib\site-packages\numpy\core\_methods.py:129: RuntimeWarning: invalid value encountered in scalar divide
  ret = ret.dtype.type(ret / rcount)
                    INFO     mean ar error (without repeats): nan                                      train_util.py:915
                    ERROR    No data found. Please verify arguments (train_data_dir must be the     train_network.py:212
                             parent of folders with images) /
                             画像がありません。引数指定を確認してください（train_data_dirには画像が
                             あるフォルダではなく、画像があるフォルダの親フォルダを指定する必要があ
                             ります）
17:53:08-090420 INFO     Training has ended.
2025-01-07T05:30:49Z
Thank You. I got a demo model trained with your help. Is there a way to use it from the menu:Kohya_ss setup menu:Install kohya_ss GUI(Optional) Install CuDNN files (to use the latest supported CuDNN version)(DANGER) Install Triton 2.1.0 for Windows… only do it if you know you need it… might break training…(Optional) Install specific version of bitsandbytes(Optional) Manually configure Accelerate(Optional) Launch Kohya_ss GUI in browserExit SetupSelect an option:
2025-01-07T05:57:54Z
I’ve never actually used kohyass…If it works, you’re good to go. I think it would be quicker to look for a knowhow article. It’s one of the most popular scripts.
2025-01-08T09:23:25Z
I trained with 1 image and it takes about 3 hours on my GPU. I have about 100 images for training, will it take 100*3 hours?
2025-01-08T12:55:08Z
will it take 100*3 hours?In theory, yes. But I don’t think people usually spend that much time on it.I think it’s probably because the settings are insufficient and the GPU isn’t being used much, or the GPU is too weak, or the model you chose is too big and it’s too much for the GPU you have. For example, FLUX is tough if you don’t have two 4090s. There are various tricks you can use… (Quantization…)
2025-01-09T04:58:34Z
Hello,I trained the model successfully with 1 image and got the following files:last.safetensors, config_dreambooth-20250108-103158, last_20250108-103158, prompt.My program with Gradio is as follows:import torch
from transformers import AutoModel, AutoTokenizer

# Correct paths to your trained model files
config_path = "D:/Ganu/AIImage/huggingface/kohya_ss/kohya_ss/trained-model/config_dreambooth-20250108-103158"
model_path = "D:/Ganu/AIImage/huggingface/kohya_ss/kohya_ss/trained-model/last.safetensors"

# Load the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(config_path, local_files_only=True)
model = AutoModel.from_pretrained(model_path, config=config_path, local_files_only=True)

# Function to generate an image from a text prompt
def generate_image(prompt):
    inputs = tokenizer(prompt, return_tensors="pt")
    with torch.no_grad():
        outputs = model(**inputs)
    # Process the outputs to generate the image (example)
    generated_image = outputs[0].cpu().numpy()
    return generated_image

# Example usage
text_prompt = "A beautiful sunset over a tranquil beach"
generated_image = generate_image(text_prompt)

# Save or display the generated image
from PIL import Image
import numpy as np

image = Image.fromarray((generated_image * 255).astype(np.uint8))
image.save("generated_image.jpg")
image.show()When I try to run the model, I get the following error:python FirstTry.pyTraceback (most recent call last):File “D:\Ganu\AIImage\huggingface\kohya_ss\Python310\lib\site-packages\transformers\utils\hub.py”, line 403, in cached_fileresolved_file = REDACTED_TOKEN_download(File “D:\Ganu\AIImage\huggingface\kohya_ss\Python310\lib\site-packages\huggingface_hub\utils_validators.py”, line 106, in _inner_fnvalidate_repo_id(arg_value)File “D:\Ganu\AIImage\huggingface\kohya_ss\Python310\lib\site-packages\huggingface_hub\utils_validators.py”, line 154, in validate_repo_idraise HFValidationError(huggingface_hub.errors.HFValidationError: Repo id must be in the form ‘repo_name’ or ‘namespace/repo_name’: ‘D:/Ganu/AIImage/huggingface/kohya_ss/kohya_ss/trained-model/config_dreambooth-20250108-103158’. Userepo_typeargument if needed.The above exception was the direct cause of the following exception:Traceback (most recent call last):File “D:\Ganu\AIImage\huggingface\kohya_ss\kohya_ss\user\FirstTry.py”, line 9, intokenizer = AutoTokenizer.from_pretrained(config_path, local_files_only=True)File “D:\Ganu\AIImage\huggingface\kohya_ss\Python310\lib\site-packages\transformers\models\auto\tokenization_auto.py”, line 858, in from_pretrainedtokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)File “D:\Ganu\AIImage\huggingface\kohya_ss\Python310\lib\site-packages\transformers\models\auto\tokenization_auto.py”, line 690, in get_tokenizer_configresolved_config_file = cached_file(File “D:\Ganu\AIImage\huggingface\kohya_ss\Python310\lib\site-packages\transformers\utils\hub.py”, line 469, in cached_fileraise EnvironmentError(OSError: Incorrect path_or_model_id: ‘D:/Ganu/AIImage/huggingface/kohya_ss/kohya_ss/trained-model/config_dreambooth-20250108-103158’. Please provide either the path to a local folder or the repo_id of a model on the Hub.
2025-01-09T05:43:00Z
Hello!Thetransformersyou are using in your program are libraries for language models and image recognition models, and you useDiffusersandPEFTfor image generation models.Basically, you just need to have LoRA locally. The procedure is to load the base model withfrom_pretrained, then apply LoRA withload_adapters, and then run inference.(After applying LoRA, you can save the model as a new model with LoRA applied by runningfuse_loraand thensave_pretrained, but this is probably not necessary in this case. As long as you have LoRA and the base model, you won’t have any problems with inference.)In the past, it was necessary to convert LoRA file formats, but now they are basically the same, so this is not necessary.So, the LoRA file you create can be used as is in A1111 WebUI and ComfyUI.huggingface.coLoad adaptersWe’re on a journey to advance and democratize artificial intelligence through open source and open science.huggingface.coLoad LoRAs for inferenceWe’re on a journey to advance and democratize artificial intelligence through open source and open science.
2025-01-09T08:42:19Z
Hello,Thank you for the reply.I am trying to run the following program:from diffusers import AutoPipelineForText2Image
import torch
import os

# Load the base model
pipeline = AutoPipelineForText2Image.from_pretrained("sd-dreambooth-library/herge-style", torch_dtype=torch.float16).to("cuda")

# Specify the path to your LoRA weights file
lora_weights_path = "D:\\Ganu\\AIImage\\huggingface\\kohya_ss\\kohya_ss\\trained-model\\model\\last.safetensors"

# Verify the LoRA weights file
if not os.path.exists(lora_weights_path):
    raise FileNotFoundError(f"LoRA weights file not found at {lora_weights_path}")

# Load LoRA weights
try:
    pipeline.load_lora_weights(lora_weights_path)
except ValueError as e:
    raise ValueError("Invalid LoRA checkpoint. Please check the compatibility and format of the weights file.") from e

# Generate an image from a text prompt
prompt = "A cute herge_style brown bear eating a slice of pizza, stunning color scheme, masterpiece, illustration"
image = pipeline(prompt).images[0]

# Display the generated image
image.save("generated_image.jpg")
image.show()I get the following error:python SeventhTry.py
Loading pipeline components...:  71%|██████████████████████████████████████████████████████████████████████████████████████████████████████▏                                        | 5/7 [00:00<00:00, 10.24it/s]An error occurred while trying to fetch C:\Users\ADMIN\.cache\huggingface\hub\models--sd-dreambooth-library--herge-style\snapshots\33888ec1b1cb44429533e2ce41a9c927c94dff20\vae: Error no file named diffusion_pytorch_model.safetensors found in directory C:\Users\ADMIN\.cache\huggingface\hub\models--sd-dreambooth-library--herge-style\snapshots\33888ec1b1cb44429533e2ce41a9c927c94dff20\vae.
Defaulting to unsafe serialization. Pass `allow_pickle=False` to raise an error instead.
An error occurred while trying to fetch C:\Users\ADMIN\.cache\huggingface\hub\models--sd-dreambooth-library--herge-style\snapshots\33888ec1b1cb44429533e2ce41a9c927c94dff20\unet: Error no file named diffusion_pytorch_model.safetensors found in directory C:\Users\ADMIN\.cache\huggingface\hub\models--sd-dreambooth-library--herge-style\snapshots\33888ec1b1cb44429533e2ce41a9c927c94dff20\unet.
Defaulting to unsafe serialization. Pass `allow_pickle=False` to raise an error instead.
Loading pipeline components...: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:01<00:00,  4.28it/s]
Traceback (most recent call last):
  File "D:\Ganu\AIImage\huggingface\kohya_ss\kohya_ss\user\SeventhTry.py", line 17, in <module>
    pipeline.load_lora_weights(lora_weights_path)
  File "D:\Ganu\AIImage\huggingface\kohya_ss\Python310\lib\site-packages\diffusers\loaders\lora_pipeline.py", line 127, in load_lora_weights
    raise ValueError("Invalid LoRA checkpoint.")
ValueError: Invalid LoRA checkpoint.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "D:\Ganu\AIImage\huggingface\kohya_ss\kohya_ss\user\SeventhTry.py", line 19, in <module>
    raise ValueError("Invalid LoRA checkpoint. Please check the compatibility and format of the weights file.") from e
ValueError: Invalid LoRA checkpoint. Please check the compatibility and format of the weights file.Any ideas?
2025-01-09T09:57:35Z
The options you give when loading LoRA are a little special. You have to separate the file name from the directory name. Like this.from diffusers import AutoPipelineForText2Image
import torch
import os
from pathlib import Path

# Load the base model
pipeline = AutoPipelineForText2Image.from_pretrained("sd-dreambooth-library/herge-style", torch_dtype=torch.float16).to("cuda")

# Specify the path to your LoRA weights file
lora_weights_path = "D:\\Ganu\\AIImage\\huggingface\\kohya_ss\\kohya_ss\\trained-model\\model\\last.safetensors"

# Verify the LoRA weights file
if not os.path.exists(lora_weights_path):
    raise FileNotFoundError(f"LoRA weights file not found at {lora_weights_path}")

# Load LoRA weights
try:
    pipeline.load_lora_weights(Path(lora_weights_path).parent, weight_name=Path(lora_weights_path).name)
    #pipeline.fuse_lora(lora_scale=1.0) # if LoRA isn't work
    #pipeline.save_pretrained("lora_applied_model") # if you want to save LoRA applied model
    #https://huggingface.co/docs/diffusers/v0.32.1/api/loaders/lora
except ValueError as e:
    raise ValueError("Invalid LoRA checkpoint. Please check the compatibility and format of the weights file.") from e

# Generate an image from a text prompt
prompt = "A cute herge_style brown bear eating a slice of pizza, stunning color scheme, masterpiece, illustration"
image = pipeline(prompt).images[0]

# Display the generated image
image.save("generated_image.jpg")
image.show()
2025-01-10T06:01:41Z
Still getting an error:python SeventhTry.py
# Load the base model
Loading pipeline components...:   0%|                                                                                                                                                       | 0/7 [00:00<?, ?it/s]An error occurred while trying to fetch C:\Users\ADMIN\.cache\huggingface\hub\models--sd-dreambooth-library--herge-style\snapshots\33888ec1b1cb44429533e2ce41a9c927c94dff20\unet: Error no file named diffusion_pytorch_model.safetensors found in directory C:\Users\ADMIN\.cache\huggingface\hub\models--sd-dreambooth-library--herge-style\snapshots\33888ec1b1cb44429533e2ce41a9c927c94dff20\unet.
Defaulting to unsafe serialization. Pass `allow_pickle=False` to raise an error instead.
Loading pipeline components...:  71%|██████████████████████████████████████████████████████████████████████████████████████████████████████▏                                        | 5/7 [00:03<00:00,  2.06it/s]An error occurred while trying to fetch C:\Users\ADMIN\.cache\huggingface\hub\models--sd-dreambooth-library--herge-style\snapshots\33888ec1b1cb44429533e2ce41a9c927c94dff20\vae: Error no file named diffusion_pytorch_model.safetensors found in directory C:\Users\ADMIN\.cache\huggingface\hub\models--sd-dreambooth-library--herge-style\snapshots\33888ec1b1cb44429533e2ce41a9c927c94dff20\vae.
Defaulting to unsafe serialization. Pass `allow_pickle=False` to raise an error instead.
Loading pipeline components...: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:03<00:00,  2.01it/s]
# Specify the path to your LoRA weights file
# Verify the LoRA weights file
# Load LoRA weights
Traceback (most recent call last):
  File "D:\Ganu\AIImage\huggingface\kohya_ss\kohya_ss\user\SeventhTry.py", line 22, in <module>
    pipeline.load_lora_weights(Path(lora_weights_path).parent, weight_name=Path(lora_weights_path).name)
  File "D:\Ganu\AIImage\huggingface\kohya_ss\Python310\lib\site-packages\diffusers\loaders\lora_pipeline.py", line 127, in load_lora_weights
    raise ValueError("Invalid LoRA checkpoint.")
ValueError: Invalid LoRA checkpoint.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "D:\Ganu\AIImage\huggingface\kohya_ss\kohya_ss\user\SeventhTry.py", line 27, in <module>
    raise ValueError("Invalid LoRA checkpoint. Please check the compatibility and format of the weights file.") from e
ValueError: Invalid LoRA checkpoint. Please check the compatibility and format of the weights file.
2025-01-10T07:00:59Z
The error is hidden and I can’t see the cause, so try this.from diffusers import AutoPipelineForText2Image
import torch
import os
from pathlib import Path

# Load the base model
pipeline = AutoPipelineForText2Image.from_pretrained("sd-dreambooth-library/herge-style", torch_dtype=torch.float16).to("cuda")

# Specify the path to your LoRA weights file
lora_weights_path = "D:\\Ganu\\AIImage\\huggingface\\kohya_ss\\kohya_ss\\trained-model\\model\\last.safetensors"

# Verify the LoRA weights file
if not os.path.exists(lora_weights_path):
    raise FileNotFoundError(f"LoRA weights file not found at {lora_weights_path}")

# Load LoRA weights
try:
    pipeline.load_lora_weights(Path(lora_weights_path).parent, weight_name=Path(lora_weights_path).name)
except ValueError as e:
    raise ValueError(f"Invalid LoRA checkpoint. Please check the compatibility and format of the weights file. {e}") from e
2025-01-10T08:10:09Z
Hi there!To useStable Diffusionand fine-tune it with your custom images, you can follow these steps:1.Set Up Stable DiffusionDownload the pre-trained Stable Diffusion model. You can find the official repository and weightshereor use a platform likeHugging Face.Ensure you have Python, CUDA-enabled GPUs, and libraries like PyTorch installed.2.Gather Your Custom ImagesPrepare a dataset of images relevant to your use case.Annotate these images if needed (e.g., captions or tags) to help guide the training process.3.Choose a Fine-Tuning MethodDreamBooth: Ideal for personalizing the model for specific objects or styles. It fine-tunes the model on a smaller dataset while preserving its overall knowledge.Check out theDreamBooth implementation on Hugging Facefor a tutorial.LoRA (Low-Rank Adaptation): A lightweight fine-tuning method that requires fewer computational resources.Libraries likePEFTsupport LoRA integration.Textual Inversion: Another lightweight approach that focuses on learning new textual embeddings for your custom concepts.4.Fine-Tuning ProcessClone the desired fine-tuning repository, such as:DreamBooth Stable DiffusionHugging Face Diffusers(supports DreamBooth and LoRA)Install the required dependencies.Train the model using your images and captions. Adjust parameters like learning rate and training steps for optimal results.5.Test the Fine-Tuned ModelAfter training, generate images using prompts relevant to your custom images.Evaluate the quality and adjust training if necessary.6.Deploy Your ModelUse tools likeGradioorStreamlitto build an interactive interface for your model.Alternatively, deploy it on a server or cloud platform.Additional Tips:Use a GPU with at least12GB VRAMfor efficient training.Start with a smaller dataset and fine-tune iteratively to avoid overfitting.Explore the Hugging Face Spaces or community notebooks for practical examples.Hope this help!
2025-01-10T09:18:34Z
HelloHow do I change “Number of Epochs” in kohya_ss from 40 to 1? Can’t find the config file to do the same?
2025-01-10T09:39:36Z
Maybe like these.GitHubCan someone explain "repeats", "epochs" and total steps in Lora training? ·...What is the repeats variable? Why would I want to repeat the same 1mage more than once? This seems to inflate the number of steps required. What is the formula for epochs based on repeats and total...github.com/kohya-ss/sd-scriptsHow is num_epochs calculated?opened04:23PM - 01 Apr 23 UTCspecbladesCant understand, how is num_epochs calculated. 
I use, for example, 16 images,…1 repeat, 5 epochs, 1 batch size, 16 grad accum steps. And i get 40 epoch.
Why batches per epoch is 32?
What? 😢
Help me out pls
<img width="286" alt="image" src="https://user-images.githubusercontent.com/113993746/229115467-81d421d9-5251-4a65-8898-0b6ebd9c577b.png">GitHubLoRA training parametersContribute to bmaltais/kohya_ss development by creating an account on GitHub.
2025-01-13T06:36:48Z
HelloI successfully trained a model (stabilityai/stable-diffusion-2-1-base)I generated a image using the below program:from diffusers import StableDiffusionPipeline, EulerDiscreteScheduler
import torch
import os
import numpy as np
from PIL import Image

# Define the path to the directory containing your model and LoRA weights
print("Define the path to the directory containing your model and LoRA weights")
model_dir = "D:\\Ganu\\AIImage\\huggingface\\kohya_ss\\kohya_ss\\trained-model\\model\\"
lora_weights_path = os.path.join(model_dir, "last.safetensors")

# Load the base model using StableDiffusionPipeline
print("Load the base model using StableDiffusionPipeline")
pipeline = StableDiffusionPipeline.from_pretrained(
    "stabilityai/stable-diffusion-2-1-base",
    torch_dtype=torch.float16
).to("cuda")

# Load the LoRA weights
print("Load the LoRA weights")
try:
    pipeline.load_lora_weights(lora_weights_path)
except ValueError as e:
    print("Invalid LoRA checkpoint. Please check the compatibility and format of the weights file.")
    raise e

# Generate an image from a text prompt
print("Generate an image from a text prompt")
text_prompt = "A beautiful Woman"
generated_image = pipeline(prompt=text_prompt).images[0]

# Handle NaN or infinite values and ensure the range is valid 
print("Handle NaN or infinite values and ensure the range is valid ")
generated_image = np.nan_to_num(generated_image, nan=0.0, posinf=255.0, neginf=0.0) 
generated_image = np.clip(generated_image, 0, 255) 
generated_image = generated_image.astype(np.uint8)

# Save or display the generated image
print("Save or display the generated image")

# Convert the NumPy array to a PIL Image and save or display the generated image 
pil_image = Image.fromarray(generated_image) 
pil_image.save("generated_image.jpg") 
pil_image.show()The image comes out blank with black background. (I tried without Lora weights file, but the image is still blank & black-background)(Generated image properties:Shape: (512, 512, 3)Data type: uint8Value range: 0 - 0Handle NaN or infinite values and ensure the range is validSave or display the generated image)Any ideas what to do?generated_image512×512 1.78 KB
No date available
No reply text available