| base_model: | |
| - Qwen/Qwen2.5-VL-7B-Instruct | |
| # Qwen/Qwen2.5-VL-7B-Instruct (Quantized) | |
| ## Description | |
| This model is a quantized version of the original model [`Qwen/Qwen2.5-VL-7B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct). | |
| It's quantized using the BitsAndBytes library to 4-bit using the [bnb-my-repo](https://huggingface.co/spaces/bnb-community/bnb-my-repo) space. | |
| ## Quantization Details | |
| - **Quantization Type**: int4 | |
| - **bnb_4bit_quant_type**: nf4 | |
| - **bnb_4bit_use_double_quant**: True | |
| - **bnb_4bit_compute_dtype**: bfloat16 | |
| - **bnb_4bit_quant_storage**: uint8 | |