Create images from items without loosing detail
HelloI want to use an AI image tool to create images that show certain items. To do this, I want to use a reference image that has a transparent background.I have already tried three models (including Midjourney), but although the images themselves are good, the difference to the reference image is enormous. If, for example, I need a Sherman tank for my game, it cannot really be changed. Since the models do not seem to know the items (in this case a tank), some different tanks always comes out, they have nothing to do with historical correct ones.The ideal prompt would be that I end up with a definable background for exactly the item that is shown in the reference image. What would be the best solution for this, since theexisting models are apparently not capable of this?Best regardsTobi
2024-10-08T11:59:20Z
One way is to choose any of the existing excellent models and teach them Sherman tanks, etc., but it is expensive and time consuming…So how about using inpainting?The AI will re-draw an image based on a reference image.There are also many similar functions to this. You can do more advanced things with ControlNet, for example.huggingface.coFLUX.1 [Inpainting] - a Hugging Face Space by SkalskiPDiscover amazing ML apps made by the communityhuggingface.coText Guided Flux Inpainting - a Hugging Face Space by Gradio-CommunityDiscover amazing ML apps made by the communityhuggingface.coDiffusers Fast Inpaint - a Hugging Face Space by OzzyGTDiscover amazing ML apps made by the communityIf you want to focus on detail, there is a way to create a LoRA instead of radically retraining an existing model. This is inexpensive.huggingface.coLoRA Ease - a Hugging Face Space by multimodalartTrain LoRAs with Easehuggingface.coTrain FLUX LoRA with Ease - a Hugging Face Space by autotrain-projectsTrain LoRA with easeThis is what a LoRA is.huggingface.cogokaygokay/Flux-Game-Assets-LoRA · Hugging FaceWe’re on a journey to advance and democratize artificial intelligence through open source and open science.huggingface.coFLUX LoRA the Explorer Mod - a Hugging Face Space by John6666Discover amazing ML apps made by the community
2024-10-08T12:19:12Z
So many things, thanks for your answerIt will take some time to check it though.I would also like to add an example of what I want to achieve:2jpg1920×1371 228 KBI created an image with an AI. The task was to add a tiger tank, but it did not.When I tell the AI to replace it, it can easily replace the tank, but not with that one I wanted. Even when I add an example with a tiger tank and transparent background, it cant do that. So I guess training a model or LoRA would be the best solution, but I need to check your links further…
2024-10-08T12:42:00Z
As for LoRA we can think of it as little more than a convertible equipment pack.It is possible to create a LoRA and then have it inpainted on the model to which the LoRA is applied.You can use as many LoRAs as you want at the same time, so you can have one LoRA for the background, one for the tanks, and one for the people, and so on. If the weapon is well-known, there may already be a LoRA in Civitai.There are many tricks to making LoRAs, but there is so much know-how accumulated that you don’t need to do it by trial and error.You can find a know-how site that suits you by searching the following clues.civitai.comEssential to Advanced Guide to training a LoRA | Civitai1. Introduction "Fear the curses that hide in your training" - Disclaimer: I won't teach you to make images like this one, don't worry. This aims t...github.comGitHub - bmaltais/kohya_ssContribute to bmaltais/kohya_ss development by creating an account on GitHub.github.comGitHub - Linaqruf/kohya-trainer: Adapted from...Adapted from https://note.com/kohya_ss/n/nbf7ce8d80f29 for easier cloninghttps://civitai.com/models?tag=vehicle
2024-10-08T12:51:38Z
Nice. I tried this inpaint for now.Diffusers Fast Inpaint - a Hugging Face Space by OzzyGTimage (4)1024×1024 102 KBI used the reference image with the transparent background, I think you can see it quite well. The tank itself tends to be way too big in scale, but I will try to handle this with making it smaller in the reference image.This is what I wanted, but I need to say the RealVisXL V5.0 Lightning model is a little bit limitated. But maybe this is exactly why it worksI will check your other links as well
2024-10-08T13:04:35Z
huggingface.coSpaces for Text-to-images (SDXL, Pony, SD1.5, Flux,...) - a John6666 CollectionUnlock the magic of AI with handpicked models, awesome datasets, papers, and mind-blowing Spaces from John6666huggingface.coSpaces for Image-to-Image / Video - a John6666 CollectionUnlock the magic of AI with handpicked models, awesome datasets, papers, and mind-blowing Spaces from John6666There is a collection of models and inpainting relatives (image-to-image) available.You can find most of the existing models and LoRA for free on Civitai, HF, and anonymous forums on the net, so you can start by looking through these and then make what you are missing.You can test your base model at HF for free.Also, HF may have a dataset of images useful for training. Too many to find, though.
2024-10-08T13:14:33Z
Thanks for your help, it is impressive. For now let me test the inpaint options you showed me in your first answer. The LoRa is a complete different thing if I understood you correctly, so let me finish these tests first, then I will check the rest.Honestly, it is a lot you presented to me so it takes some time.If the inpaint option would generate larger images and a better understanding of scale I would be happy already
2024-10-08T13:18:41Z
As for inpainting, it depends on the composition of the original image, so if you dare not use a transparent background and just make a composition with people added, they will make it accordingly. Even if it is a doodle, they will re-draw it rather nicely.I think you should first look for a space suitable for inpainting for yourself. There are many…
2024-10-08T13:24:13Z
I agree. I will try out some stuff and post the solution (if I find one) here in the topic.
2024-10-08T13:52:55Z
Okay, for now I found out this:TheDiffusers Fast Inpaint - a Hugging Face Space by OzzyGTjust works fine, because i can uploadthe reference image without masking it. All others that I have found wants me to maskthose parts I want to replace and they throw an error on me if I dont want to do this.The reason is, that I dont want to replace anything of the reference image andit is hard to mask white on a white background in the web tool.So, because this tool does not throws an error on me, I can create a background. This fits the reference image quite nice, but it has some weird understanding of scale. One downpart is the low resolution. I am not sure if I can manage to create my own copy for my own needs. Anyway, this is the best option I found so far, compared to the other inpaint links you gave to me in your very first posts and to other inpaint tools in the web.In short, I need to use this one, make my own very similar solution or train with LoRA.
2024-10-08T14:14:02Z
If RealVis 5XL is good enough for you, you can choose a base model from over 1000 (modestly speaking) SDXL models. There is also simply a new version of RealVis.The means from here diverge into several, but the HF space is copy free, so you can copy and customize it for your own use. Don’t know how to program? Ask someone who understands it and you’re done. I’m easy to teach you mod too.If you want to copy and use the space that says ZeroGPU, just note that it’s $10 per month and 10 slots per person.CPU space is free.Good night.
2024-10-08T15:47:32Z
You mean if I create my own one / copy one and modify it, I can choose between different ones, correct?I think it is worth the time to take a deep look into it, but I need to start from scratch.My level of understanding is, that I can use Gradio, just as the developer of the tool I linked above did. But as I said, I need to take a more deep look into it.
2024-10-08T22:24:34Z
You mean if I create my own one / copy one and modify itGood morning.That’s what I mean. If you can tinker with Gradio or Python then the story is simple.It’s just astr or listthat specifies the model in the space above, so as long as you discover an alternative model, you can use a different model with a few lines of changes.RealVisXL belongs to the SDXL architecture, which is the most popular model nowadays, so you can follow the link below for alternative model candidates. For the most part, it will work with only a rewrite of the code.Furthermore, in the HF system, there is a function to combine LoRA and models as if they were a single repo, which allows you to run a single LoRA as a model with LoRA applied, with almost no code. To use this function, you just need to create a new model repo, upload one LoRA file, and write a few lines of README.md (actually, a YAML configuration file).By copying and modifying the above space for your own use, you can easily add or tweak functions, change models, and apply LoRA.Image-to-Image, including Inpaint, actually allows you to specify a wide variety of parameters internally, so you can often specify, for example, the fidelity to the original image.This approach of creating a repo, placing what you need, and using it from copied Spaces is one of the most common ways to use HF. The same thing can be done with language models, for example.SDXL modelshuggingface.coModels - Hugging FaceWe’re on a journey to advance and democratize artificial intelligence through open source and open science.