zhengchong commited on
Commit
480a228
·
verified ·
1 Parent(s): 8923380

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +148 -16
README.md CHANGED
@@ -1,32 +1,164 @@
1
- ---
2
- license: other
3
- ---
4
-
5
 
6
  <h1 align="center"> FastFit: Accelerating Multi-Reference Virtual Try-On via Cacheable Diffusion Models </h1>
7
 
8
- <p align="center">Supported by <a href="https://lavieai.com/">LavieAI</a> and <a href="https://www.loomlyai.com/zh">LoomlyAI</a></p>
9
 
10
 
11
- <!-- <div style="display: flex; justify-content: center; align-items: center;">
12
- -->
13
  <div align="center">
14
- <!-- <a href="http://arxiv.org/abs/2407.15886" style="margin: 0 2px;">
15
- <img src='https://img.shields.io/badge/arXiv-2407.15886-red?style=flat&logo=arXiv&logoColor=red' alt='arxiv'>
16
- </a> -->
17
- <a href='https://huggingface.co/zhengchong/FastFit-MR-1024' style="margin: 0 2px;">
18
  <img src='https://img.shields.io/badge/Hugging Face-ckpts-orange?style=flat&logo=HuggingFace&logoColor=orange' alt='huggingface'>
19
  </a>
20
- <a href="https://github.com/Zheng-Chong/FastFit" style="margin: 0 2px;">
21
  <img src='https://img.shields.io/badge/GitHub-Repo-blue?style=flat&logo=GitHub' alt='GitHub'>
22
  </a>
23
- <a href="http://123.56.183.38:7860" style="margin: 0 2px;">
24
  <img src='https://img.shields.io/badge/Demo-Gradio-gold?style=flat&logo=Gradio&logoColor=red' alt='Demo'>
25
  </a>
26
- <!-- <a href='https://zheng-chong.github.io/CatVTON/' style="margin: 0 2px;">
27
  <img src='https://img.shields.io/badge/Webpage-Project-silver?style=flat&logo=&logoColor=orange' alt='webpage'>
28
- </a> -->
29
- <a href="https://github.com/Zheng-Chong/FastFit/LICENCE" style="margin: 0 2px;">
30
  <img src='https://img.shields.io/badge/License-NonCommercial-lightgreen?style=flat&logo=Lisence' alt='License'>
31
  </a>
32
  </div>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
 
2
  <h1 align="center"> FastFit: Accelerating Multi-Reference Virtual Try-On via Cacheable Diffusion Models </h1>
3
 
4
+ <p align="center" style="font-size: 18px;">Supported by <a href="https://lavieai.com/">LavieAI</a> and <a href="https://www.loomlyai.com/en">LoomlyAI</a></p>
5
 
6
 
 
 
7
  <div align="center">
8
+ <a href="https://github.com/Zheng-Chong/FastFit" style="margin: 0 2px; text-decoration: none;">
9
+ <img src='https://img.shields.io/badge/arXiv-TODO-red?style=flat&logo=arXiv&logoColor=red' alt='arxiv'>
10
+ </a>
11
+ <a href='https://huggingface.co/zhengchong/FastFit-MR-1024' style="margin: 0 2px; text-decoration: none;">
12
  <img src='https://img.shields.io/badge/Hugging Face-ckpts-orange?style=flat&logo=HuggingFace&logoColor=orange' alt='huggingface'>
13
  </a>
14
+ <a href="https://github.com/Zheng-Chong/FastFit" style="margin: 0 2px; text-decoration: none;">
15
  <img src='https://img.shields.io/badge/GitHub-Repo-blue?style=flat&logo=GitHub' alt='GitHub'>
16
  </a>
17
+ <a href="https://fastfit.lavieai.com" style="margin: 0 2px; text-decoration: none;">
18
  <img src='https://img.shields.io/badge/Demo-Gradio-gold?style=flat&logo=Gradio&logoColor=red' alt='Demo'>
19
  </a>
20
+ <a href='https://zheng-chong.github.io/FastFit/' style="margin: 0 2px;">
21
  <img src='https://img.shields.io/badge/Webpage-Project-silver?style=flat&logo=&logoColor=orange' alt='webpage'>
22
+ </a>
23
+ <a href="https://github.com/Zheng-Chong/FastFit/tree/main" style="margin: 0 2px; text-decoration: none;">
24
  <img src='https://img.shields.io/badge/License-NonCommercial-lightgreen?style=flat&logo=Lisence' alt='License'>
25
  </a>
26
  </div>
27
+
28
+ <br>
29
+
30
+ FastFit is a diffusion-based framework optimized for **high-speed**, **multi-reference virtual try-on**. It enables **simultaneous try-on of multiple fashion items**—such as **tops, bottoms, dresses, shoes, and bags**—on a single person. The framework leverages **reference KV caching** during inference to **significantly accelerate generation**.
31
+
32
+ ## Updates
33
+ - **`2025/08/06`**: ⚙️ We release [the code for inference and evaluation](https://github.com/Zheng-Chong/FastFit/tree/main?tab=readme-ov-file#inference--evaluation-on-datasets) on the [DressCode-MR](https://huggingface.co/datasets/zhengchong/DressCode-MR), [DressCode](https://huggingface.co/datasets/zhengchong/DressCode-Test), and [VITON-HD](https://huggingface.co/datasets/zhengchong/VITON-HD) test datasets.
34
+ - **`2025/08/05`**: 🧩 We release the [ComfyUI workflow](https://github.com/Zheng-Chong/FastFit/releases/tag/comfyui) for FastFit!
35
+ - **`2025/08/04`**: 🚀 Our [gradio demo](https://fastfit.lavieai.com) is online with Chinese & English support! The code of the demo is also released in [app.py](app.py).
36
+ - **`2025/07/03`**: 🎉 We release the weights of [FastFit-MR](https://huggingface.co/zhengchong/FastFit-MR-1024) and [FastFit-SR](https://huggingface.co/zhengchong/FastFit-SR-1024) model on Hugging Face!
37
+ - **`2025/06/24`**: 👕 We release [DressCode-MR](https://huggingface.co/datasets/zhengchong/DressCode-MR) dataset with **28K+ Multi-reference virtual try-on Samples** on Hugging Face!
38
+
39
+
40
+ ## DressCode-MR Dataset
41
+
42
+ <div align="center">
43
+ <img src="assets/img/dataset.png" alt="DressCode-MR Dataset" width="800">
44
+ </div>
45
+
46
+ [DressCode-MR](https://huggingface.co/datasets/zhengchong/DressCode-MR) is constructed based on the [DressCode](https://github.com/aimagelab/dress-code) dataset with **28K+ Multi-reference virtual try-on Samples**.
47
+
48
+ - **Multi-reference Samples**: Each sample comprises a person's image paired with a set of compatible clothing and accessory items: tops, bottoms, dresses, shoes, and bags.
49
+ - **Large Scale**: Contains a total of 28,179 high-quality multi-reference samples with 25,779 for training and 2,400 for testing.
50
+
51
+ DressCode-MR is released under the exact same license as the original DressCode dataset. Therefore, before requesting access to DressCode-MR dataset, you must complete the following steps:
52
+
53
+ 1. Apply and be granted a license to use the [DressCode](https://github.com/aimagelab/dress-code) dataset.
54
+ 2. Use your educational/academic email address (e.g., one ending in .edu, .ac, etc.) to request access to [DressCode-MR](https://huggingface.co/datasets/zhengchong/DressCode-MR) on Hugging Face. (Any requests from non-academic email addresses will be rejected.)
55
+
56
+ ## Installation
57
+
58
+ ```powershell
59
+ conda create -n fastfit python=3.10
60
+ conda activate fastfit
61
+ pip install -r requirements.txt
62
+ pip install easy-dwpose --no-dependencies # to resolve the version conflict
63
+
64
+ # if error occurs for av, try:
65
+ conda install -c conda-forge av
66
+ ```
67
+
68
+ ## ComfyUI Workflow
69
+
70
+ <div align="center">
71
+ <img src="assets/img/comfyui.png" alt="ComfyUI Workflow" width="800">
72
+ </div>
73
+
74
+
75
+ 1. Clone the FastFit repository into your `ComfyUI/custom_nodes/` directory.
76
+ ```powershell
77
+ cd Your_ComfyUI_Dir/custom_nodes
78
+ git clone https://github.com/Zheng-Chong/FastFit.git
79
+ ```
80
+
81
+ 2. Install the required dependencies.
82
+ ```powershell
83
+ cd FastFit
84
+ pip install -r requirements.txt
85
+ pip install easy-dwpose --no-dependencies # to resolve the version conflict
86
+
87
+ # if error occurs for av, try:
88
+ conda install -c conda-forge av
89
+ ```
90
+
91
+ 3. Install [rgthree-comfy](https://github.com/rgthree/rgthree-comfy) for image comparer.
92
+
93
+ ```powershell
94
+ cd Your_ComfyUI_Dir/custom_nodes
95
+ git clone https://github.com/rgthree/rgthree-comfy.git
96
+ cd rgthree-comfy
97
+ pip install -r requirements.txt
98
+ ```
99
+
100
+ 4. Restart ComfyUI.
101
+ 5. Drag and drop the [fastfit_workflow.json](https://github.com/Zheng-Chong/FastFit/blob/main/assets/fastfit_workflow.json) file onto the ComfyUI web interface.
102
+
103
+ ## Gradio Demo
104
+
105
+ The model weights will be automatically downloaded from Hugging Face when you run the demo.
106
+
107
+ ```bash
108
+ python app.py
109
+ ```
110
+
111
+ ## Inference & Evaluation on Datasets
112
+
113
+ To perform inference on the [DressCode-MR](https://huggingface.co/datasets/zhengchong/DressCode-MR), [DressCode](https://huggingface.co/datasets/zhengchong/DressCode-Test), or [VITON-HD](https://huggingface.co/datasets/zhengchong/VITON-HD) test datasets, use the `infer_datasets.py` script, for example:
114
+
115
+ ```bash
116
+ python infer_datasets.py \
117
+ --dataset <dataset_name> \
118
+ --data_dir </path/to/your/dataset> \
119
+ --batch_size 4 \
120
+ --num_inference_steps 50 \
121
+ --guidance_scale 2.5 \
122
+ --mixed_precision bf16 \
123
+ --paired
124
+ ```
125
+
126
+ - `--dataset`: Specify the target dataset. Choose from `dresscode-mr`, `dresscode`, or `viton-hd`.
127
+
128
+ - `--data_dir`: The root directory path for the specified dataset.
129
+
130
+ - `--paired`: Include this flag to run inference in the paired setting. Omit this flag for the unpaired setting.
131
+
132
+ By default, inference results will be saved to the `results/` directory at the project root.
133
+
134
+ ---
135
+
136
+ After inference, use the `eval.py` script to ecalculate the evaluation metrics:
137
+
138
+ ```bash
139
+ python eval.py \
140
+ --gt_folder </path/to/ground_truth_folder> \
141
+ --pred_folder </path/to/prediction_folder> \
142
+ --paired \
143
+ --batch_size 16 \
144
+ --num_workers 4
145
+ ```
146
+
147
+ - `--gt_folder`: The directory path containing the ground truth images.
148
+
149
+ - `--pred_folder`: The directory path containing the generated (predicted) images from the inference step.
150
+
151
+ - `--paired`: Include this flag to evaluate results from the paired setting. Omit this flag for the unpaired setting.
152
+
153
+ <!-- ## Citation
154
+
155
+ ```bibtex
156
+
157
+ ``` -->
158
+
159
+ ## Acknowledgement
160
+ Our code is modified based on [Diffusers](https://github.com/huggingface/diffusers). We adopt [Stable Diffusion v1.5 inpainting](https://huggingface.co/runwayml/stable-diffusion-inpainting) as the base model. We use a modified [AutoMasker](https://github.com/Zheng-Chong/CatVTON/blob/edited/model/cloth_masker.py) to automatically generate masks in our [Gradio](https://github.com/gradio-app/gradio) App and [ComfyUI](https://github.com/comfyanonymous/ComfyUI) workflow. Thanks to all the contributors!
161
+
162
+ ## License
163
+
164
+ All weights, parameters, and code related to FastFit are governed by the [FastFit Non-Commercial License](https://github.com/Zheng-Chong/FastFit/tree/main). For commercial collaboration, please contact [LavieAI](https://lavieai.com/) or [LoomlyAI](https://www.loomlyai.com/en).