Fine-tunening a multimodal model
hello! Does anyone have an example on how to fine-tune a multimodal model? The examples that I found only have texts as input, i.e. LLMs. I’m dealing with Vision-Language Models
2024-05-08T01:48:48Z
Did you find one?
2024-05-08T07:43:50Z
Hi,Yes find them all in my repositoryGitHub - NielsRogge/Transformers-Tutorials: This repository contains demos I made with the Transformers library by HuggingFace..Demo notebooks are grouped per model, so I’d recommend taking a look at the BLIP-2, Idefics2, PaliGemma and LLaVa folders for some examples.
2024-11-18T10:53:34Z
Hi! You can try AnyModal:GitHub - ritabratamaiti/AnyModal: AnyModal is a Flexible Multimodal Language Model FrameworkIt’s a project I have been working on that allows for the creation of multimodal LLMs.
2024-12-25T12:17:36Z
Hello guys,I hope you are doing well, how can fine tune Multimodal LLM like (LLaVA, PaliGemma …) for Multimodal Aspect Based Sentiment Analysis (Text and Image)