Hugging Face's logo

davidrd123
/
Ben-Brand-LoRA

Text-to-Image
Diffusers
flux
flux-diffusers
simpletuner
safe-for-work
lora
template:sd-lora
standard
Model card Files Community
69
Ben-Brand-LoRA / README.md
davidrd123's picture
davidrd123
Add generated example
ab6f298 verified
preview code
|
raw
history blame
12.7 kB
metadata
license: other
base_model: black-forest-labs/FLUX.1-dev
tags:
  - flux
  - flux-diffusers
  - text-to-image
  - diffusers
  - simpletuner
  - safe-for-work
  - lora
  - template:sd-lora
  - standard
inference: true
widget:
  - text: unconditional (blank prompt)
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_0_0.png
  - text: >-
      In the style of a b3nbr4nd painting, A steaming bowl of ramen with
      chopsticks resting on the edge, against a background of concentric orange
      and blue circles. The noodles are detailed in a geometric pattern and the
      steam creates a rhythmic design.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_1_0.png
  - text: >-
      In the style of a b3nbr4nd painting, A vintage record player with vinyl
      spinning, set on a yellow table. The background features an alternating
      chevron pattern in purple and green. The turntable's mechanical parts are
      rendered in precise geometric shapes.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_2_0.png
  - text: >-
      In the style of a b3nbr4nd painting, A sleeping cat curled up in a
      modernist chair, with a background of interlocking hexagons in red and
      blue. The cat's fur is stylized into rhythmic curves, matching the
      geometric environment.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_3_0.png
  - text: >-
      In the style of a b3nbr4nd painting, A classic motorcycle viewed from the
      side, against a backdrop of radiating diamond patterns in teal and gold.
      The chrome parts reflect abstract shapes, and the wheels create circular
      motifs in the composition.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_4_0.png
  - text: >-
      In the style of a b3nbr4nd painting, Portrait of a woman with silver hair
      wearing dotted blue glasses and a white lace collar, against a swirling
      background of green and yellow patterns. The background features geometric
      circles and zigzag designs.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_5_0.png
  - text: >-
      In the style of a b3nbr4nd painting, A storefront sign for 'Golden Palace
      Noodles' in both English and Chinese characters, mounted on a tall pole
      against a geometric cityscape with blue and tan buildings. A small arrow
      points to available parking.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_6_0.png
  - text: >-
      In the style of a b3nbr4nd painting, Dark purple figs sliced in half on a
      terra cotta plate, revealing their seeded interiors. The background
      features a repeating pattern of blue and yellow squares, with wavy lines
      creating a dynamic lower section.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_7_0.png
  - text: >-
      In the style of a b3nbr4nd painting, Two young people wearing matching
      navy shirts and light gray face masks, posed against a warm yellow
      background. Their curly hair and gentle head tilts create a symmetrical
      composition.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_8_0.png
  - text: >-
      In the style of a b3nbr4nd painting, A hamster wearing tiny glasses and a
      bowtie sitting at a miniature desk with a tiny laptop, against a
      background of spiral patterns in teal and orange. Office supplies scaled
      to hamster-size are arranged neatly on the desk.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_9_0.png
  - text: >-
      In the style of a b3nbr4nd painting, A bearded man in a plaid shirt and
      denim apron carefully sanding a mid-century modern chair, surrounded by
      woodworking tools. The background features overlapping triangles in rust
      and navy blue colors, with sawdust creating delicate patterns in the air.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_10_0.png
  - text: >-
      In the style of a b3nbr4nd painting, The Fractured Cathedral – Ruined
      temple standing between timelines, stained glass windows refracting
      multiple realities, golden gears turning in the vaulted ceiling, priests
      in robes of shifting colors, a mechanical choir humming in binary, relics
      of forgotten AI scattered on an altar, static crackling like divine
      whispers.
    output:
      url: images/example_a42xk8qba.png
  - text: >-
      In the style of a b3nbr4nd painting, The Cartographer of Lost Time – A
      hunched figure tracing glowing lines across an ancient map, ink shifting
      as if alive, continents forming and vanishing, thousands of tiny golden
      orbs orbiting the parchment, the map itself whispering of places that no
      longer exist, candlelight flickering in unknown patterns.
    output:
      url: images/example_0916y8ymq.png
  - text: >-
      In the style of a b3nbr4nd painting, A steaming outdoor pool carved from
      volcanic rock, floating lanterns casting rippling golden reflections, pale
      steam curling upwards into a canopy of sapphire sky, koi fish with silver
      scales swimming in slow, deliberate circles.
    output:
      url: images/example_aych9v4t1.png
  - text: >-
      In the style of a b3nbr4nd painting, A spiraling staircase covered in deep
      red velvet, disappearing into a hazy golden glow, framed by walls of dark
      mahogany, intricate carvings of animals moving subtly when not directly
      observed.
    output:
      url: images/example_xm1y3tb6v.png
  - text: >-
      In the style of a b3nbr4nd painting, A vast, half-built cathedral where
      stained-glass windows flicker between scenes, as though buffering reality
      itself. The stone pillars extend endlessly into the sky, sometimes
      breaking apart into pixels before reforming. At the altar, a priest made
      of light raises their hands, their face cycling through a thousand
      unreadable expressions.
    output:
      url: images/example_05vukxmgl.png
  - text: >-
      In the style of a b3nbr4nd painting, A spiraling stone staircase wrapped
      around itself, leading both up and down in an endless paradox. Footsteps
      echo from ahead and behind, but the traveler never sees another person.
      Hanging lanterns flicker in rhythmic pulses, illuminating carvings of eyes
      that seem to watch, waiting for the moment someone understands the
      pattern.
    output:
      url: images/example_xizv9nrx2.png
  - text: >-
      In the style of a b3nbr4nd painting, starry night sky, tall buildings,
      skyscrapers, windows, connected buildings, walkway, green checkerboard
      pattern walkway, arched windows, trees with lights, urban landscape,
      geometric architecture, city night scene, multicolored buildings, modern
      cityscape, midground walkway, background buildings, illuminated windows,
      high-rise structures, blue building, city park with lights
    output:
      url: images/example_naqqp69cu.png
  - text: >-
      In the style of a b3nbr4nd painting, blue vases, pink tulips, table with
      patterned tablecloth, orange mugs, green teapot, posters on wall, red
      poster, text on red poster, white text, yellow wall, drawers in
      background, multiple cups, ceramic kettle, lid on kettle, floral mug,
      window with striped curtains, indoor scene, objects on table, vase in
      center, tea set
    output:
      url: images/example_gfqdxqy36.png
  - text: >-
      In the style of a b3nbr4nd painting, Towering red clocktower, golden clock
      hands frozen at midnight, gears visible through glass windows, standing in
      a misty valley, deep blue sky, orange and pink clouds, warm glowing
      lanterns on stone pathway, ivy creeping up brick walls, metal staircase
      leading to a rooftop observatory.
    output:
      url: images/example_6507viyod.png

Ben-Brand-LoRA

This is a standard PEFT LoRA derived from black-forest-labs/FLUX.1-dev.

No validation prompt was used during training.

None

Validation settings

  • CFG: 3.0
  • CFG Rescale: 0.0
  • Steps: 20
  • Sampler: FlowMatchEulerDiscreteScheduler
  • Seed: 42
  • Resolution: 1024x1024
  • Skip-layer guidance:

Note: The validation settings are not necessarily the same as the training settings.

You can find some example images in the following gallery:

Prompt
In the style of a b3nbr4nd painting, Massive green serpent, coiled around a cracked stone obelisk, golden eyes glowing, dark blue scales with iridescent shimmer, open mouth revealing sharp fangs, pink sky fading into deep purple, sand dunes stretching into the horizon, ancient ruins partially buried, red banners flapping in the wind.
Prompt
In the style of a b3nbr4nd painting, The Fractured Cathedral – Ruined temple standing between timelines, stained glass windows refracting multiple realities, golden gears turning in the vaulted ceiling, priests in robes of shifting colors, a mechanical choir humming in binary, relics of forgotten AI scattered on an altar, static crackling like divine whispers.
Prompt
In the style of a b3nbr4nd painting, The Cartographer of Lost Time – A hunched figure tracing glowing lines across an ancient map, ink shifting as if alive, continents forming and vanishing, thousands of tiny golden orbs orbiting the parchment, the map itself whispering of places that no longer exist, candlelight flickering in unknown patterns.
Prompt
In the style of a b3nbr4nd painting, A steaming outdoor pool carved from volcanic rock, floating lanterns casting rippling golden reflections, pale steam curling upwards into a canopy of sapphire sky, koi fish with silver scales swimming in slow, deliberate circles.
Prompt
In the style of a b3nbr4nd painting, snake, black snakes, desert, cacti, cactus, prickly pear cactus, yucca plant, orange eye, mountains, purple mountains, blue sky, green cactus, yellow desert, close-up view, coiled snake, landscape, arid environment, vegetation, plant, side view, foreground snake, background mountains
Prompt
In the style of a b3nbr4nd painting, starry night sky, tall buildings, skyscrapers, windows, connected buildings, walkway, green checkerboard pattern walkway, arched windows, trees with lights, urban landscape, geometric architecture, city night scene, multicolored buildings, modern cityscape, midground walkway, background buildings, illuminated windows, high-rise structure, blue building, city park with lights
Prompt
In the style of a b3nbr4nd painting, Massive gold-plated pyramid, intricate carvings on its surface, glowing blue symbols along edges, deep violet sky, swirling pink clouds, hovering metallic spheres reflecting the landscape, a lone traveler in red robes ascending marble stairs, crystal-clear river flowing around pyramid base.
Prompt
In the style of a b3nbr4nd painting, blue vases, pink tulip, table with patterned tablecloth, orange mugs, green teapot, posters on wall, red poster, text on red poster, white text, yellow wall, drawers in background, multiple cups, ceramic kettle, lid on kettle, floral mug, window with striped curtains, indoor scene, objects on table, vase in center, tea set
Prompt
In the style of a b3nbr4nd painting, Armored knight in reflective silver plate, tattered blue cape, standing on a vast frozen lake, sword planted in ice, glowing runes along blade, towering ice formations in background, pale full moon overhead, scattered red leaves on ice surface, distant torches flickering along mountain ridge.
Prompt
In the style of a b3nbr4nd painting, Towering white marble golem, deep blue glowing veins across its body, massive stone hands gripping a golden spear, standing before an ancient archway, intricate carvings on stone, overgrown vines wrapping around pillars, golden sunset casting long shadows, wind carrying red and orange leaves through the air.
Prompt
In the style of a b3nbr4nd painting, Giant bird made of sapphire flames, wings spread wide, glowing embers trailing behind, deep black sky with thousands of stars, large yellow moon behind bird, floating islands in the distance, waterfalls spilling into nothingness, golden temple on one of the islands, reflections of stars on the bird’s feathers.

The text encoder was not trained. You may reuse the base model text encoder for inference.

Training settings

  • Training epochs: 1

  • Training steps: 3500

  • Learning rate: 0.0001

    • Learning rate schedule: polynomial
    • Warmup steps: 100

  • Max grad norm: 0.1

  • Effective batch size: 6

    • Micro-batch size: 2
    • Gradient accumulation steps: 3
    • Number of GPUs: 1

  • Gradient checkpointing: True

  • Prediction type: flow-matching (extra parameters=['shift=3', 'flux_guidance_mode=constant', 'flux_guidance_value=1.0', 'flow_matching_loss=compatible', 'flux_lora_target=all'])

  • Optimizer: adamw_bf16

  • Trainable parameter precision: Pure BF16

  • Caption dropout probability: 10.0%

  • LoRA Rank: 64

  • LoRA Alpha: None

  • LoRA Dropout: 0.1

  • LoRA initialisation style: default

Datasets

ben-brand-256

  • Repeats: 10
  • Total number of images: 98
  • Total number of aspect buckets: 1
  • Resolution: 0.065536 megapixels
  • Cropped: False
  • Crop style: None
  • Crop aspect: None
  • Used for regularisation data: No

ben-brand-crop-256

  • Repeats: 10
  • Total number of images: 98
  • Total number of aspect buckets: 1
  • Resolution: 0.065536 megapixels
  • Cropped: True
  • Crop style: center
  • Crop aspect: square
  • Used for regularisation data: No

ben-brand-512

  • Repeats: 10
  • Total number of images: 98
  • Total number of aspect buckets: 3
  • Resolution: 0.262144 megapixels
  • Cropped: False
  • Crop style: None
  • Crop aspect: None
  • Used for regularisation data: No

ben-brand-crop-512

  • Repeats: 10
  • Total number of images: 98
  • Total number of aspect buckets: 1
  • Resolution: 0.262144 megapixels
  • Cropped: True
  • Crop style: center
  • Crop aspect: square
  • Used for regularisation data: No

ben-brand-768

  • Repeats: 10
  • Total number of images: 98
  • Total number of aspect buckets: 6
  • Resolution: 0.589824 megapixels
  • Cropped: False
  • Crop style: None
  • Crop aspect: None
  • Used for regularisation data: No

ben-brand-crop-768

  • Repeats: 10
  • Total number of images: 98
  • Total number of aspect buckets: 1
  • Resolution: 0.589824 megapixels
  • Cropped: True
  • Crop style: center
  • Crop aspect: square
  • Used for regularisation data: No

ben-brand-1024

  • Repeats: 10
  • Total number of images: 98
  • Total number of aspect buckets: 7
  • Resolution: 1.048576 megapixels
  • Cropped: False
  • Crop style: None
  • Crop aspect: None
  • Used for regularisation data: No

ben-brand-crop-1024

  • Repeats: 10
  • Total number of images: 98
  • Total number of aspect buckets: 1
  • Resolution: 1.048576 megapixels
  • Cropped: True
  • Crop style: center
  • Crop aspect: square
  • Used for regularisation data: No

ben-brand-1440

  • Repeats: 10
  • Total number of images: 98
  • Total number of aspect buckets: 4
  • Resolution: 2.0736 megapixels
  • Cropped: False
  • Crop style: None
  • Crop aspect: None
  • Used for regularisation data: No

ben-brand-crop-1440

  • Repeats: 10
  • Total number of images: 98
  • Total number of aspect buckets: 1
  • Resolution: 2.0736 megapixels
  • Cropped: True
  • Crop style: center
  • Crop aspect: square
  • Used for regularisation data: No

Inference 

import torch
from diffusers import DiffusionPipeline

model_id = 'black-forest-labs/FLUX.1-dev'
adapter_id = 'davidrd123/Ben-Brand-LoRA'
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) # loading directly in bf16
pipeline.load_lora_weights(adapter_id)

prompt = "An astronaut is riding a horse through the jungles of Thailand."


## Optional: quantise the model to save on vram.
## Note: The model was quantised during training, and so it is recommended to do the same during inference time.
from optimum.quanto import quantize, freeze, qint8
quantize(pipeline.transformer, weights=qint8)
freeze(pipeline.transformer)
    
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu') # the pipeline is already in its target precision level
image = pipeline(
    prompt=prompt,
    num_inference_steps=20,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(42),
    width=1024,
    height=1024,
    guidance_scale=3.0,
).images[0]
image.save("output.png", format="PNG")