---
license: creativeml-openrail-m
base_model: stabilityai/stable-diffusion-xl-base-1.0
dataset: NYUAD-ComNets/Ethnicity_Diversity_Data
tags:
- stable-diffusion-xl
- stable-diffusion-xl-diffusers
- text-to-image
- diffusers
- lora
inference: true
---

# Model description

LoRA text2image fine-tuning - NYUAD-ComNets/Ethnicity_Diversity_Model

These are LoRA adaption weights for stabilityai/stable-diffusion-xl-base-1.0. The weights were fine-tuned on the NYUAD-ComNets/Ethnicity_Diversity_Data dataset. 
You can find some example images. 

prompt: a photo of a {ethnicity} person, looking at the camera, closeup headshot facing forward, ultra quality, sharp focus


# How to use this model:

``` python

from diffusers import  DiffusionPipeline
import torch

from compel import Compel, ReturnedEmbeddingsType


negative_prompt = "cartoon, anime, 3d, painting, b&w, low quality" 


pipeline = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", variant="fp16", use_safetensors=True, torch_dtype=torch.float16).to("cuda")

pipeline.load_lora_weights("NYUAD-ComNets/Ethnicity_Diversity_Model", weight_name="pytorch_lora_weights.safetensors") 
    
compel = Compel(tokenizer=[pipeline.tokenizer, pipeline.tokenizer_2] ,
                    text_encoder=[pipeline.text_encoder, pipeline.text_encoder_2],
                    returned_embeddings_type=ReturnedEmbeddingsType.PENULTIMATE_HIDDEN_STATES_NON_NORMALIZED, 
                    requires_pooled=[False, True],truncate_long_prompts=False)

    
conditioning, pooled = compel("a photo of an asian person, looking at the camera, closeup headshot facing forward, ultra quality, sharp focus") 

negative_conditioning, negative_pooled = compel(negative_prompt)
[conditioning, negative_conditioning] = compel.pad_conditioning_tensors_to_same_length([conditioning, negative_conditioning])

image = pipeline(prompt_embeds=conditioning, negative_prompt_embeds=negative_conditioning,
                     pooled_prompt_embeds=pooled, negative_pooled_prompt_embeds=negative_pooled,
                     num_inference_steps=40).images[0]

image.save('/../../x.jpg')

```

# Examples

| | | |
|:-------------------------:|:-------------------------:|:-------------------------:|
|<img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./image_0.png"> |  <img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./image_1.png">|<img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./image_2.png">|
|<img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./4.jpg"> |  <img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./327.jpg">|<img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./890.jpg">|
|<img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./999.jpg"> |  <img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./545.jpg">|<img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./993.jpg">|
|<img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./110.jpg"> |  <img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./204.jpg">|<img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./822.jpg">|
|<img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./382.jpg"> |  <img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./721.jpg">|<img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./994.jpg">|
|<img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./469.jpg"> |  <img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./0.jpg">|<img width="500" alt="screen shot 2017-08-07 at 12 18 15 pm" src="./999.jpg">|






# Training data

NYUAD-ComNets/Ethnicity_Diversity_Data dataset was used to fine-tune stabilityai/stable-diffusion-xl-base-1.0



# Configurations

LoRA for the text encoder was enabled: False.

Special VAE used for training: madebyollin/sdxl-vae-fp16-fix.



# BibTeX entry and citation info

```
@misc{ComNets,
      url={[https://huggingface.co/NYUAD-ComNets/Ethnicity_Diversity_Model](https://huggingface.co/NYUAD-ComNets/Ethnicity_Diversity_Model)},
      title={Ethnicity_Diversity_Model},
      author={Nouar AlDahoul, Talal Rahwan, Yasir Zaki}
}
```