metadata
license: cc
datasets:
- weizhiwang/Open-Qwen2VL-Data
- MAmmoTH-VL/MAmmoTH-VL-Instruct-12M
language:
- en
base_model:
- Qwen/Qwen2.5-1.5B-Instruct
- google/siglip-so400m-patch14-384
pipeline_tag: image-text-to-text
Model Card for Open-Qwen2VL
Updates
Model Details
How to Use
Please firstly install Open-Qwen2VL via
pip install git+https://github.com/Victorwz/Open-Qwen2VL.git#subdirectory=prismatic-vlms
You can load the model and perform inference as follows:
import requests
import torch
from PIL import Image
from prismatic import load
device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
# Load a pretrained VLM (either local path, or ID to auto-download from the HF Hub)
vlm = load("Open-Qwen2VL")
vlm.to(device, dtype=torch.bfloat16)
# Download an image and specify a prompt
image_url = "https://huggingface.co/adept/fuyu-8b/resolve/main/bus.png"
# image = Image.open(requests.get(image_url, stream=True).raw).convert("RGB")
image = [vlm.vision_backbone.image_transform(Image.open(requests.get(image_url, stream=True).raw).convert("RGB")).unsqueeze(0)]
user_prompt = '<image>' + '\n' + "Describe the image."
# Generate!
generated_text = vlm.generate_batch(
image,
[user_prompt],
do_sample=False,
max_new_tokens=512,
min_length=1,
)
print(generated_text[0])
The image caption results look like:
The image depicts a blue and orange bus parked on the side of a street. The bus is a modern, single-decker model, and it is prominently displayed in the foreground. The bus has a digital display at the front, showing the route number "616" and the destination "Morton Best." The bus is labeled with the logo of "Stagecoach," a well-known bus operator in the UK. The bus also features a slogan on its side that reads "Stronger for you," indicating a focus on providing reliable and efficient public transportation.
The bus is parked on a grassy area adjacent to a sidewalk. The grass is well-maintained, and there are a few trees planted along the sidewalk, providing some shade. The sidewalk itself is made of concrete and appears to be clean and well-kept.
In the background, there are residential buildings, which are typical of a suburban area. These buildings have pitched roofs and are constructed with a mix of brick and plaster. The architecture suggests a typical British suburban neighborhood. The sky above is partly cloudy, with patches of blue sky visible, indicating fair weather.
The bus is parked in a designated bus stop area, as indicated by the presence of a bus stop sign and a bus stop shelter, although the shelter is not visible in the image. The bus's license plate is visible and reads "Y600 HJX." The overall scene suggests a typical day in a suburban area where public transportation is readily available for residents.
The image does not contain any people, vehicles other than the bus, or any other notable objects. The focus is primarily on the bus and its surroundings. The bus appears to be in good condition, indicating that it is well-maintained and likely in regular use. The presence of the bus stop and the well-kept environment suggest that the area is well-organized and that public transportation is an integral part of the community's daily life.