README.md

---
base_model: black-forest-labs/FLUX.1-dev
library_name: diffusers
license: other
inference: true
tags:
- flux
- flux-diffusers
- text-to-image
- diffusers
- control
- diffusers-training
---

<!-- This model card has been generated automatically according to the information the training script had access to. You
should probably proofread and complete it, then remove this comment. -->


# cartoon-control-lr_1e-4-wd_1e-4-gs_10.0-cd_0.1

These are Flux control weights trained on [black-forest-labs/FLUX.1-dev](https://hf.co/black-forest-labs/FLUX.1-dev) with a new type of conditioning. [instruction-tuning-sd/cartoonization](https://hf.co/datasets/instruction-tuning-sd/cartoonization)
dataset was used for training. You can find some example images below.

| ![images_0)](./images_0.png) |
|:--------:|
| ![images_1)](./gen-taj.jpg) |
| ![images_2)](./gen-violin.jpg) |
| **prompt**: Generate a cartoonized version of the image |


## License

Please adhere to the licensing terms as described [here](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md)


## Intended uses & limitations

#### How to use

```python
from diffusers import FluxTransformer2DModel, FluxControlPipeline
from diffusers.utils import load_image
import torch 

path = "sayakpaul/cartoon-control-lr_1e-4-wd_1e-4-gs_10.0-cd_0.1"
transformer = FluxTransformer2DModel.from_pretrained(path, torch_dtype=torch.bfloat16)
pipe = FluxControlPipeline.from_pretrained(
  "black-forest-labs/FLUX.1-dev",  transformer=transformer, torch_dtype=torch.bfloat16
).to("cuda")

prompt = "Generate a cartoonized version of the image"
url = "https://huggingface.co/sayakpaul/cartoon-control-lr_1e-4-wd_1e-4-gs_10.0-cd_0.1/resolve/main/taj.jpg"

image = load_image(img).resize((1024, 1024))
gen_image = pipe(
    prompt=prompt,
    control_image=image,
    guidance_scale=10.,
    num_inference_steps=50,
    generator=torch.manual_seed(0),
    max_sequence_length=512,
).images[0]
gen_image.save("output.png")
```

Refer to the Flux Control docs [here](https://huggingface.co/docs/diffusers/main/en/api/pipelines/flux).

#### Limitations and bias

[TODO: provide examples of latent issues and potential remediations]

## Training details

Refer to [here](https://github.com/huggingface/diffusers/tree/main/examples/flux-control). WandB logs are [here](https://wandb.ai/sayakpaul/flux_train_control/runs/jiddr743).