license: apache-2.0
library_name: transformers
pipeline_tag: image-text-to-text
base_model:
- mistral-community/pixtral-12b
base_model_relation: quantized
pixtral-12b-int4-ov
- Model creator: mistral-community
- Original model: mistral-community/pixtral-12b
Description
This is mistral-community/pixtral-12b model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by NNCF.
Quantization Parameters
Weight compression was performed using nncf.compress_weights with the following parameters:
- mode: INT4_ASYM
Compatibility
The provided OpenVINO™ IR model is compatible with:
- OpenVINO version 2025.2.0 and higher
- Optimum Intel 1.26.0 and higher
Running Model Inference with Optimum Intel
- Install packages required for using Optimum Intel integration with the OpenVINO backend:
pip install --pre -U --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/pre-release openvino_tokenizers openvino
pip install git+https://github.com/huggingface/optimum-intel.git
- Run model inference
from PIL import Image
import requests
from optimum.intel.openvino import OVModelForVisualCausalLM
from transformers import AutoProcessor, TextStreamer
model_id = "OpenVINO/pixtral-12b-int4-ov"
processor = AutoProcessor.from_pretrained(model_id)
ov_model = OVModelForVisualCausalLM.from_pretrained(model_id, trust_remote_code=True)
question = "What is unusual in this picture?"
messages = [
{"role": "user", "content": [{"type": "text", "content": question}, {"type": "image"}]},
]
text = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
url = "https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/d5fbbd1a-d484-415c-88cb-9986625b7b11"
raw_image = Image.open(requests.get(url, stream=True).raw)
inputs = processor(text=text, images=[raw_image], return_tensors="pt")
streamer = TextStreamer(processor.tokenizer, skip_prompt=True, skip_special_tokens=True)
output = ov_model.generate(**inputs, do_sample=False, max_new_tokens=100, temperature=None, top_p=None, streamer=streamer)
Limitations
Check the original model card for limitations.
Legal information
The original model is distributed under apache-2.0 license. More details can be found in original model card.
Disclaimer
Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.