amokrov commited on
Commit
83dde1d
·
verified ·
1 Parent(s): f430f8d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -59
README.md CHANGED
@@ -70,65 +70,6 @@ output = ov_model.generate(**inputs, do_sample=False, max_new_tokens=100, temper
70
 
71
  ```
72
 
73
- ## Running Model Inference with [OpenVINO GenAI](https://github.com/openvinotoolkit/openvino.genai)
74
-
75
- 1. Install packages required for using OpenVINO GenAI.
76
- ```
77
- pip install --pre -U --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/pre-release openvino openvino-tokenizers openvino-genai
78
-
79
- pip install huggingface_hub
80
- ```
81
-
82
- 2. Download model from HuggingFace Hub
83
-
84
- ```
85
- import huggingface_hub as hf_hub
86
-
87
- model_id = "OpenVINO/pixtral-12b-int4-ov"
88
- model_path = "pixtral-12b-int4-ov"
89
-
90
- hf_hub.snapshot_download(model_id, local_dir=model_path)
91
-
92
- ```
93
-
94
- 1. Run model inference:
95
-
96
- ```
97
- import openvino_genai as ov_genai
98
- import requests
99
- from PIL import Image
100
- from io import BytesIO
101
- import numpy as np
102
- import openvino as ov
103
-
104
- device = "CPU"
105
- pipe = ov_genai.VLMPipeline(model_path, device)
106
-
107
- def load_image(image_file):
108
- if isinstance(image_file, str) and (image_file.startswith("http") or image_file.startswith("https")):
109
- response = requests.get(image_file)
110
- image = Image.open(BytesIO(response.content)).convert("RGB")
111
- else:
112
- image = Image.open(image_file).convert("RGB")
113
- image_data = np.array(image.getdata()).reshape(1, image.size[1], image.size[0], 3).astype(np.byte)
114
- return ov.Tensor(image_data)
115
-
116
- prompt = "What is unusual on this picture?"
117
-
118
- url = "https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/d5fbbd1a-d484-415c-88cb-9986625b7b11"
119
- image_tensor = load_image(url)
120
-
121
- def streamer(subword: str) -> bool:
122
- print(subword, end="", flush=True)
123
- return False
124
-
125
- pipe.start_chat()
126
- output = pipe.generate(prompt, image=image_tensor, max_new_tokens=100, streamer=streamer)
127
- pipe.finish_chat()
128
- ```
129
-
130
- More GenAI usage examples can be found in OpenVINO GenAI library [docs](https://github.com/openvinotoolkit/openvino.genai/blob/master/src/README.md) and [samples](https://github.com/openvinotoolkit/openvino.genai?tab=readme-ov-file#openvino-genai-samples)
131
-
132
 
133
  ## Limitations
134
 
 
70
 
71
  ```
72
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
73
 
74
  ## Limitations
75