Readme

README.md

@@ -1,3 +1,100 @@
 ---
+language: en
 license: apache-2.0
+tags:
+- open-vocabulary
+- semantic-segmentation
+base_model:
+- timm/vit_large_patch14_dinov2.lvd142m
+- timm/vit_base_patch14_dinov2.lvd142m
 ---
+
+<div align="center">
+<h2>
+<img src="./src/favicon.png" height=25> <span style="color: #FF0078;">Free</span><span style="color: #00509A;">DA</span>: Training-Free Open-Vocabulary Segmentation with Offline Diffusion-Augmented Prototype Generation (CVPR 2024) <br>
+</h2>
+<p></p>
+
+<p></p>
+
+<h3>
+<a href="https://lucabarsellotti.github.io//">Luca Barsellotti*</a>&ensp;
+<a href="https://www.robertoamoroso.it//">Roberto Amoroso*</a>&ensp;
+<a href="https://aimagelab.ing.unimore.it/imagelab/person.asp?idpersona=90/">Marcella Cornia</a>&ensp;
+<a href="https://www.lorenzobaraldi.com//">Lorenzo Baraldi</a>&ensp;
+<a href="https://aimagelab.ing.unimore.it/imagelab/person.asp?idpersona=1">Rita Cucchiara</a>&ensp;
+</h3>
+
+
+[Project Page](https://aimagelab.github.io/freeda/) | [Paper](https://arxiv.org/abs/2404.06542) | [Code](https://github.com/aimagelab/freeda)
+
+</div>
+
+<div align="center">
+<figure>
+  <img alt="Qualitative results" src="./src/assets/qualitatives1.png">
+</figure>
+</div>
+
+## Method
+
+<div align="center">
+<figure>
+  <img alt="FreeDA method" src="./src/assets/inference.png">
+</figure>
+</div>
+
+<br/>
+
+<details>
+<summary> Additional qualitative examples </summary>
+<p align="center">
+  <img alt="Additional qualitative results" src="./src/assets/qualitatives.png" width="800" />
+</p>
+</details>
+
+<details>
+<summary> Additional examples <i>in-the-wild</i> </summary>
+<p align="center">
+  <img alt="In-the-wild examples" src="./src/assets/into_the_wild.png" width="800" />
+</p>
+</details>
+
+## Installation
+
+```
+conda create --name freeda python=3.9
+conda activate freeda
+conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
+pip install -r requirements.txt
+```
+
+## How to use
+
+```
+import freeda
+from PIL import Image
+import requests
+from io import BytesIO
+
+if __name__ == "__main__":
+    fr = freeda.load("dinov2_vitb_clip_vitb")
+    response1 = requests.get("https://farm9.staticflickr.com/8306/7926031760_b313dca06a_z.jpg")
+    img1 = Image.open(BytesIO(response1.content))
+    response2 = requests.get("https://farm3.staticflickr.com/2207/2157810040_4883738d2d_z.jpg")
+    img2 = Image.open(BytesIO(response2.content))
+    fr.set_categories(["cat", "table", "pen", "keyboard", "toilet", "wall"])
+    fr.set_images([img1, img2])
+    segmentation = fr()
+    fr.visualize(segmentation, ["plot.png", "plot1.png"])
+```
+
+If you find FreeDA useful for your work please cite:
+```
+@inproceedings{barsellotti2024training
+  title={Training-Free Open-Vocabulary Segmentation with Offline Diffusion-Augmented Prototype Generation},
+  author={Barsellotti, Luca and Amoroso, Roberto and Cornia, Marcella and Baraldi, Lorenzo and Cucchiara, Rita},
+  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
+  year={2024}
+}
+```