Update README.md
Browse files
README.md
CHANGED
@@ -49,6 +49,7 @@ This is a **DFloat11 losslessly compressed** version of the original `Qwen/Qwen-
|
|
49 |
def parse_args():
|
50 |
parser = argparse.ArgumentParser(description='Generate images using Qwen-Image model')
|
51 |
parser.add_argument('--cpu_offload', action='store_true', help='Enable CPU offloading')
|
|
|
52 |
parser.add_argument('--no_pin_memory', action='store_true', help='Disable memory pinning')
|
53 |
parser.add_argument('--prompt', type=str, default='A coffee shop entrance features a chalkboard sign reading "Qwen Coffee 😊 $2 per cup," with a neon light beside it displaying "通义千问". Next to it hangs a poster showing a beautiful Chinese woman, and beneath the poster is written "π≈3.1415926-53589793-23846264-33832795-02384197".',
|
54 |
help='Text prompt for image generation')
|
@@ -83,6 +84,7 @@ This is a **DFloat11 losslessly compressed** version of the original `Qwen/Qwen-
|
|
83 |
"DFloat11/Qwen-Image-DF11",
|
84 |
device="cpu",
|
85 |
cpu_offload=args.cpu_offload,
|
|
|
86 |
pin_memory=not args.no_pin_memory,
|
87 |
bfloat16_model=transformer,
|
88 |
)
|
@@ -136,8 +138,12 @@ This is a **DFloat11 losslessly compressed** version of the original `Qwen/Qwen-
|
|
136 |
python qwen_image.py --cpu_offload
|
137 |
```
|
138 |
|
139 |
-
If you are getting out-of-memory errors, try disabling memory-pinning:
|
140 |
```bash
|
|
|
|
|
|
|
|
|
141 |
python qwen_image.py --cpu_offload --no_pin_memory
|
142 |
```
|
143 |
|
|
|
49 |
def parse_args():
|
50 |
parser = argparse.ArgumentParser(description='Generate images using Qwen-Image model')
|
51 |
parser.add_argument('--cpu_offload', action='store_true', help='Enable CPU offloading')
|
52 |
+
parser.add_argument('--cpu_offload_blocks', type=int, default=None, help='Number of transformer blocks to offload to CPU')
|
53 |
parser.add_argument('--no_pin_memory', action='store_true', help='Disable memory pinning')
|
54 |
parser.add_argument('--prompt', type=str, default='A coffee shop entrance features a chalkboard sign reading "Qwen Coffee 😊 $2 per cup," with a neon light beside it displaying "通义千问". Next to it hangs a poster showing a beautiful Chinese woman, and beneath the poster is written "π≈3.1415926-53589793-23846264-33832795-02384197".',
|
55 |
help='Text prompt for image generation')
|
|
|
84 |
"DFloat11/Qwen-Image-DF11",
|
85 |
device="cpu",
|
86 |
cpu_offload=args.cpu_offload,
|
87 |
+
cpu_offload_blocks=args.cpu_offload_blocks,
|
88 |
pin_memory=not args.no_pin_memory,
|
89 |
bfloat16_model=transformer,
|
90 |
)
|
|
|
138 |
python qwen_image.py --cpu_offload
|
139 |
```
|
140 |
|
141 |
+
If you are getting out-of-CPU-memory errors, try limiting the number of offloaded blocks or disabling memory-pinning:
|
142 |
```bash
|
143 |
+
# Offload only 16 blocks (offloading more blocks uses less GPU memory and more CPU memory; offloading less blocks is faster):
|
144 |
+
python qwen_image.py --cpu_offload --cpu_offload_blocks 16
|
145 |
+
|
146 |
+
# Disable memory-pinning (the most memory efficient way, but could be slower):
|
147 |
python qwen_image.py --cpu_offload --no_pin_memory
|
148 |
```
|
149 |
|