NeoPatient

Prompt

The CT scan of the brain shows no acute hemorrhage but reveals a small area of hypodensity in the right hemisphere, consistent with an old infarct. No new acute lesions are visible.

Prompt

The MRI shows mild edema in the soft tissue around the ankle joint with no significant ligament tears or cartilage damage.

Prompt

The chest X-ray shows consolidation in the right lower lobe with air bronchograms, indicative of lobar pneumonia.

🧠 NeoPatient - Synthetic Medical Image Generator (LoRA)

NeoPatient is a Low-Rank Adaptation (LoRA) fine-tuned version of stabilityai/stable-diffusion-2-1, purpose-built for generating high-fidelity synthetic patient images. It is trained using the ROCO (Radiology Objects in Context) dataset, which contains a large collection of radiology images and captions from biomedical literature. NeoPatient enables privacy-preserving medical AI development through high-quality, domain-specific synthetic image generation.

---

🔍 Use Case

NeoPatient is designed for:

• Medical Data Augmentation: Supplement training datasets for computer vision models in healthcare.
• Synthetic Dataset Generation: Produce controlled, diverse imagery representing clinical contexts (e.g., CT scans, X-rays, ultrasound).
• Diagnostic Simulation: Create visual scenarios for education, assessment, and reinforcement learning environments.
• Privacy-Safe AI: Enable research and product development without exposure to real patient data.

---

🏗️ Base Model

• Base: stabilityai/stable-diffusion-2-1
• Fine-Tuning Method: LoRA
• Fine-Tuning Dataset: ROCO (Radiology Objects in Context), a large-scale medical image-text dataset from PubMed Central.
• LoRA Weights: pytorch_lora_weights.safetensors (6.68 MB)

---

⚙️ Technical Details

• Training Size: 40,000 medical images with associated captions
• Image Types: X-ray, CT, MRI, ultrasound, fluoroscopy, endoscopy, angiography
• Learning Objective: Align latent representations of medical visual features with text prompts in the medical domain.
• LoRA Config: Low-rank matrices injected into attention layers for efficient fine-tuning.

---

📦 How to Use

from diffusers import StableDiffusionPipeline
import torch

pipe = StableDiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-1", torch_dtype=torch.float16).to("cuda")
pipe.load_lora_weights("unicornftk/NeoPatient")
pipe(prompt="Abdominal CT scan shows a dilated appendix measuring 9mm in diameter, with surrounding fat stranding indicative of acute appendicitis.").images[0].save("abdominal_ct.png")

---

📜 License

MIT License — free for research, academic, and commercial use with attribution.

⚠️ Disclaimer

NeoPatient generates entirely synthetic medical images. It is not intended for diagnostic use or clinical decision-making. Outputs do not represent real patients and should be used only for research, development, and educational purposes.

---

Trigger words

• You should use CT scan to trigger the image generation.

• You should use MRI to trigger the image generation.

• You should use Ultrasound to trigger the image generation.

• You should use Chest X-ray to trigger the image generation.

• You should use Fluoroscopy to trigger the image generation.

• You should use Endoscopy to trigger the image generation.

• You should use Angiography to trigger the image generation.

---

Download model

Weights for this model are available in Safetensors format. Download them in the Files & versions tab.