|
|
--- |
|
|
license: apache-2.0 |
|
|
datasets: |
|
|
- ibrahimhamamci/CT-RATE |
|
|
metrics: |
|
|
- bleu |
|
|
- bertscore |
|
|
- rouge |
|
|
base_model: |
|
|
- microsoft/Phi-3-mini-4k-instruct |
|
|
tags: |
|
|
- biology |
|
|
- medical |
|
|
--- |
|
|
|
|
|
# Welcome to SAMF model [MICCAI' 25]! |
|
|
**[MICCAI' 25] From Slices to Volumes: Multi-Scale Fusion of 2D and 3D Features for CT Scan Report Generation** |
|
|
|
|
|
| **Model** | **Bleu1** | **Bleu4** | **RougeL** | **Meteor** | **Bert F1** | **Llama Score** | |
|
|
|-----------------------|-----------|-----------|------------|------------|-------------|-----------------| |
|
|
| CT2Rep | 0.309 | 0.172 | 0.243 | 0.173 | 0.865 | 6.35 | |
|
|
| CT-Chat | 0.395 | - | 0.321 | 0.219 | - | 5.664 | |
|
|
| Our Baseline (SAMF) | 0.423 | 0.203 | 0.338 | 0.356 | 0.879 | 6.792 | |
|
|
| SAMF + *Ao2D* | **0.440** | **0.261** | **0.417** | **0.417** | **0.889** | **7.165** | |
|
|
|
|
|
|
|
|
## Introduction |
|
|
|
|
|
|
|
|
|
|
|
*Slice Attentive Multimodal Fusion (SAMF)* , a framework that combines the rich, high-resolution information from 2D slices with the spatial coherence of 3D volumetric data. Experimental results demonstrate that our method outperforms existing baseline approaches in both report generation and multiple-choice question answering, highlighting the critical role of multidimensional feature integration. |
|
|
|
|
|
|
|
|
## Model Description |
|
|
|
|
|
- **Model type:** 3D Medical Report Generation and Visual Question Answering |
|
|
- **Language(s) (NLP):** English |
|
|
- **License:** apache-2.0 |
|
|
- **Finetuned from model:** microsoft/Phi-3-mini-4k-instruct |
|
|
|
|
|
### Training Data |
|
|
|
|
|
- **Medical Report Generation and Visual Question Answering:** [ibrahimhamamci/CT-RATE](https://huggingface.co/datasets/ibrahimhamamci/CT-RATE), default subset |
|
|
|
|
|
### Hardware Utilization |
|
|
|
|
|
- **Hardware Type:** 1 × NVIDIA-A100 |
|
|
- **Hours used** around 16 hours |
|
|
|
|
|
### Evaluation |
|
|
|
|
|
To perform evaluation using this model, please refer to our GitHub repository ([serag-ai/SAMF](https://github.com/serag-ai/SAMF.git)), which provides detailed information on how to use it. |
