Update README.md
Browse files
README.md
CHANGED
@@ -1,32 +1,164 @@
|
|
1 |
-
---
|
2 |
-
license: other
|
3 |
-
---
|
4 |
-
|
5 |
|
6 |
<h1 align="center"> FastFit: Accelerating Multi-Reference Virtual Try-On via Cacheable Diffusion Models </h1>
|
7 |
|
8 |
-
<p align="center">Supported by <a href="https://lavieai.com/">LavieAI</a> and <a href="https://www.loomlyai.com/
|
9 |
|
10 |
|
11 |
-
<!-- <div style="display: flex; justify-content: center; align-items: center;">
|
12 |
-
-->
|
13 |
<div align="center">
|
14 |
-
|
15 |
-
<img src='https://img.shields.io/badge/arXiv-
|
16 |
-
</a>
|
17 |
-
<a href='https://huggingface.co/zhengchong/FastFit-MR-1024' style="margin: 0 2px;">
|
18 |
<img src='https://img.shields.io/badge/Hugging Face-ckpts-orange?style=flat&logo=HuggingFace&logoColor=orange' alt='huggingface'>
|
19 |
</a>
|
20 |
-
<a href="https://github.com/Zheng-Chong/FastFit" style="margin: 0 2px;">
|
21 |
<img src='https://img.shields.io/badge/GitHub-Repo-blue?style=flat&logo=GitHub' alt='GitHub'>
|
22 |
</a>
|
23 |
-
<a href="
|
24 |
<img src='https://img.shields.io/badge/Demo-Gradio-gold?style=flat&logo=Gradio&logoColor=red' alt='Demo'>
|
25 |
</a>
|
26 |
-
|
27 |
<img src='https://img.shields.io/badge/Webpage-Project-silver?style=flat&logo=&logoColor=orange' alt='webpage'>
|
28 |
-
</a>
|
29 |
-
<a href="https://github.com/Zheng-Chong/FastFit/
|
30 |
<img src='https://img.shields.io/badge/License-NonCommercial-lightgreen?style=flat&logo=Lisence' alt='License'>
|
31 |
</a>
|
32 |
</div>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
|
2 |
<h1 align="center"> FastFit: Accelerating Multi-Reference Virtual Try-On via Cacheable Diffusion Models </h1>
|
3 |
|
4 |
+
<p align="center" style="font-size: 18px;">Supported by <a href="https://lavieai.com/">LavieAI</a> and <a href="https://www.loomlyai.com/en">LoomlyAI</a></p>
|
5 |
|
6 |
|
|
|
|
|
7 |
<div align="center">
|
8 |
+
<a href="https://github.com/Zheng-Chong/FastFit" style="margin: 0 2px; text-decoration: none;">
|
9 |
+
<img src='https://img.shields.io/badge/arXiv-TODO-red?style=flat&logo=arXiv&logoColor=red' alt='arxiv'>
|
10 |
+
</a>
|
11 |
+
<a href='https://huggingface.co/zhengchong/FastFit-MR-1024' style="margin: 0 2px; text-decoration: none;">
|
12 |
<img src='https://img.shields.io/badge/Hugging Face-ckpts-orange?style=flat&logo=HuggingFace&logoColor=orange' alt='huggingface'>
|
13 |
</a>
|
14 |
+
<a href="https://github.com/Zheng-Chong/FastFit" style="margin: 0 2px; text-decoration: none;">
|
15 |
<img src='https://img.shields.io/badge/GitHub-Repo-blue?style=flat&logo=GitHub' alt='GitHub'>
|
16 |
</a>
|
17 |
+
<a href="https://fastfit.lavieai.com" style="margin: 0 2px; text-decoration: none;">
|
18 |
<img src='https://img.shields.io/badge/Demo-Gradio-gold?style=flat&logo=Gradio&logoColor=red' alt='Demo'>
|
19 |
</a>
|
20 |
+
<a href='https://zheng-chong.github.io/FastFit/' style="margin: 0 2px;">
|
21 |
<img src='https://img.shields.io/badge/Webpage-Project-silver?style=flat&logo=&logoColor=orange' alt='webpage'>
|
22 |
+
</a>
|
23 |
+
<a href="https://github.com/Zheng-Chong/FastFit/tree/main" style="margin: 0 2px; text-decoration: none;">
|
24 |
<img src='https://img.shields.io/badge/License-NonCommercial-lightgreen?style=flat&logo=Lisence' alt='License'>
|
25 |
</a>
|
26 |
</div>
|
27 |
+
|
28 |
+
<br>
|
29 |
+
|
30 |
+
FastFit is a diffusion-based framework optimized for **high-speed**, **multi-reference virtual try-on**. It enables **simultaneous try-on of multiple fashion items**—such as **tops, bottoms, dresses, shoes, and bags**—on a single person. The framework leverages **reference KV caching** during inference to **significantly accelerate generation**.
|
31 |
+
|
32 |
+
## Updates
|
33 |
+
- **`2025/08/06`**: ⚙️ We release [the code for inference and evaluation](https://github.com/Zheng-Chong/FastFit/tree/main?tab=readme-ov-file#inference--evaluation-on-datasets) on the [DressCode-MR](https://huggingface.co/datasets/zhengchong/DressCode-MR), [DressCode](https://huggingface.co/datasets/zhengchong/DressCode-Test), and [VITON-HD](https://huggingface.co/datasets/zhengchong/VITON-HD) test datasets.
|
34 |
+
- **`2025/08/05`**: 🧩 We release the [ComfyUI workflow](https://github.com/Zheng-Chong/FastFit/releases/tag/comfyui) for FastFit!
|
35 |
+
- **`2025/08/04`**: 🚀 Our [gradio demo](https://fastfit.lavieai.com) is online with Chinese & English support! The code of the demo is also released in [app.py](app.py).
|
36 |
+
- **`2025/07/03`**: 🎉 We release the weights of [FastFit-MR](https://huggingface.co/zhengchong/FastFit-MR-1024) and [FastFit-SR](https://huggingface.co/zhengchong/FastFit-SR-1024) model on Hugging Face!
|
37 |
+
- **`2025/06/24`**: 👕 We release [DressCode-MR](https://huggingface.co/datasets/zhengchong/DressCode-MR) dataset with **28K+ Multi-reference virtual try-on Samples** on Hugging Face!
|
38 |
+
|
39 |
+
|
40 |
+
## DressCode-MR Dataset
|
41 |
+
|
42 |
+
<div align="center">
|
43 |
+
<img src="assets/img/dataset.png" alt="DressCode-MR Dataset" width="800">
|
44 |
+
</div>
|
45 |
+
|
46 |
+
[DressCode-MR](https://huggingface.co/datasets/zhengchong/DressCode-MR) is constructed based on the [DressCode](https://github.com/aimagelab/dress-code) dataset with **28K+ Multi-reference virtual try-on Samples**.
|
47 |
+
|
48 |
+
- **Multi-reference Samples**: Each sample comprises a person's image paired with a set of compatible clothing and accessory items: tops, bottoms, dresses, shoes, and bags.
|
49 |
+
- **Large Scale**: Contains a total of 28,179 high-quality multi-reference samples with 25,779 for training and 2,400 for testing.
|
50 |
+
|
51 |
+
DressCode-MR is released under the exact same license as the original DressCode dataset. Therefore, before requesting access to DressCode-MR dataset, you must complete the following steps:
|
52 |
+
|
53 |
+
1. Apply and be granted a license to use the [DressCode](https://github.com/aimagelab/dress-code) dataset.
|
54 |
+
2. Use your educational/academic email address (e.g., one ending in .edu, .ac, etc.) to request access to [DressCode-MR](https://huggingface.co/datasets/zhengchong/DressCode-MR) on Hugging Face. (Any requests from non-academic email addresses will be rejected.)
|
55 |
+
|
56 |
+
## Installation
|
57 |
+
|
58 |
+
```powershell
|
59 |
+
conda create -n fastfit python=3.10
|
60 |
+
conda activate fastfit
|
61 |
+
pip install -r requirements.txt
|
62 |
+
pip install easy-dwpose --no-dependencies # to resolve the version conflict
|
63 |
+
|
64 |
+
# if error occurs for av, try:
|
65 |
+
conda install -c conda-forge av
|
66 |
+
```
|
67 |
+
|
68 |
+
## ComfyUI Workflow
|
69 |
+
|
70 |
+
<div align="center">
|
71 |
+
<img src="assets/img/comfyui.png" alt="ComfyUI Workflow" width="800">
|
72 |
+
</div>
|
73 |
+
|
74 |
+
|
75 |
+
1. Clone the FastFit repository into your `ComfyUI/custom_nodes/` directory.
|
76 |
+
```powershell
|
77 |
+
cd Your_ComfyUI_Dir/custom_nodes
|
78 |
+
git clone https://github.com/Zheng-Chong/FastFit.git
|
79 |
+
```
|
80 |
+
|
81 |
+
2. Install the required dependencies.
|
82 |
+
```powershell
|
83 |
+
cd FastFit
|
84 |
+
pip install -r requirements.txt
|
85 |
+
pip install easy-dwpose --no-dependencies # to resolve the version conflict
|
86 |
+
|
87 |
+
# if error occurs for av, try:
|
88 |
+
conda install -c conda-forge av
|
89 |
+
```
|
90 |
+
|
91 |
+
3. Install [rgthree-comfy](https://github.com/rgthree/rgthree-comfy) for image comparer.
|
92 |
+
|
93 |
+
```powershell
|
94 |
+
cd Your_ComfyUI_Dir/custom_nodes
|
95 |
+
git clone https://github.com/rgthree/rgthree-comfy.git
|
96 |
+
cd rgthree-comfy
|
97 |
+
pip install -r requirements.txt
|
98 |
+
```
|
99 |
+
|
100 |
+
4. Restart ComfyUI.
|
101 |
+
5. Drag and drop the [fastfit_workflow.json](https://github.com/Zheng-Chong/FastFit/blob/main/assets/fastfit_workflow.json) file onto the ComfyUI web interface.
|
102 |
+
|
103 |
+
## Gradio Demo
|
104 |
+
|
105 |
+
The model weights will be automatically downloaded from Hugging Face when you run the demo.
|
106 |
+
|
107 |
+
```bash
|
108 |
+
python app.py
|
109 |
+
```
|
110 |
+
|
111 |
+
## Inference & Evaluation on Datasets
|
112 |
+
|
113 |
+
To perform inference on the [DressCode-MR](https://huggingface.co/datasets/zhengchong/DressCode-MR), [DressCode](https://huggingface.co/datasets/zhengchong/DressCode-Test), or [VITON-HD](https://huggingface.co/datasets/zhengchong/VITON-HD) test datasets, use the `infer_datasets.py` script, for example:
|
114 |
+
|
115 |
+
```bash
|
116 |
+
python infer_datasets.py \
|
117 |
+
--dataset <dataset_name> \
|
118 |
+
--data_dir </path/to/your/dataset> \
|
119 |
+
--batch_size 4 \
|
120 |
+
--num_inference_steps 50 \
|
121 |
+
--guidance_scale 2.5 \
|
122 |
+
--mixed_precision bf16 \
|
123 |
+
--paired
|
124 |
+
```
|
125 |
+
|
126 |
+
- `--dataset`: Specify the target dataset. Choose from `dresscode-mr`, `dresscode`, or `viton-hd`.
|
127 |
+
|
128 |
+
- `--data_dir`: The root directory path for the specified dataset.
|
129 |
+
|
130 |
+
- `--paired`: Include this flag to run inference in the paired setting. Omit this flag for the unpaired setting.
|
131 |
+
|
132 |
+
By default, inference results will be saved to the `results/` directory at the project root.
|
133 |
+
|
134 |
+
---
|
135 |
+
|
136 |
+
After inference, use the `eval.py` script to ecalculate the evaluation metrics:
|
137 |
+
|
138 |
+
```bash
|
139 |
+
python eval.py \
|
140 |
+
--gt_folder </path/to/ground_truth_folder> \
|
141 |
+
--pred_folder </path/to/prediction_folder> \
|
142 |
+
--paired \
|
143 |
+
--batch_size 16 \
|
144 |
+
--num_workers 4
|
145 |
+
```
|
146 |
+
|
147 |
+
- `--gt_folder`: The directory path containing the ground truth images.
|
148 |
+
|
149 |
+
- `--pred_folder`: The directory path containing the generated (predicted) images from the inference step.
|
150 |
+
|
151 |
+
- `--paired`: Include this flag to evaluate results from the paired setting. Omit this flag for the unpaired setting.
|
152 |
+
|
153 |
+
<!-- ## Citation
|
154 |
+
|
155 |
+
```bibtex
|
156 |
+
|
157 |
+
``` -->
|
158 |
+
|
159 |
+
## Acknowledgement
|
160 |
+
Our code is modified based on [Diffusers](https://github.com/huggingface/diffusers). We adopt [Stable Diffusion v1.5 inpainting](https://huggingface.co/runwayml/stable-diffusion-inpainting) as the base model. We use a modified [AutoMasker](https://github.com/Zheng-Chong/CatVTON/blob/edited/model/cloth_masker.py) to automatically generate masks in our [Gradio](https://github.com/gradio-app/gradio) App and [ComfyUI](https://github.com/comfyanonymous/ComfyUI) workflow. Thanks to all the contributors!
|
161 |
+
|
162 |
+
## License
|
163 |
+
|
164 |
+
All weights, parameters, and code related to FastFit are governed by the [FastFit Non-Commercial License](https://github.com/Zheng-Chong/FastFit/tree/main). For commercial collaboration, please contact [LavieAI](https://lavieai.com/) or [LoomlyAI](https://www.loomlyai.com/en).
|