Not possible to load model with sentence transformers v5
#3
by Samoed - opened
I tried evaluate this model on some tasks, but got an error 0.6b and 270m are loading correctly.
>>> from sentence_transformers import SentenceTransformer
>>> model = SentenceTransformer("microsoft/harrier-oss-v1-27b", model_kwargs={"dtype": "auto"})
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
Loading weights: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 808/808 [00:00<00:00, 1240.81it/s]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/mteb/.venv/lib/python3.12/site-packages/sentence_transformers/util/decorators.py", line 41, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/mteb/.venv/lib/python3.12/site-packages/sentence_transformers/sentence_transformer/model.py", line 184, in __init__
super().__init__(
File "/mteb/.venv/lib/python3.12/site-packages/sentence_transformers/base/model.py", line 204, in __init__
modules, self.module_kwargs = self._load_modules(
^^^^^^^^^^^^^^^^^^^
File "/mteb/.venv/lib/python3.12/site-packages/sentence_transformers/base/model.py", line 993, in _load_modules
return self._load_config_modules(model_name_or_path, **load_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "//mteb/.venv/lib/python3.12/site-packages/sentence_transformers/base/model.py", line 1199, in _load_config_modules
module = module_class.load(
^^^^^^^^^^^^^^^^^^
File "/mteb/.venv/lib/python3.12/site-packages/sentence_transformers/base/modules/transformer.py", line 1793, in load
return cls(model_name_or_path=model_name_or_path, **init_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mteb/.venv/lib/python3.12/site-packages/sentence_transformers/util/decorators.py", line 87, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/mteb/.venv/lib/python3.12/site-packages/sentence_transformers/base/modules/transformer.py", line 662, in __init__
self.processor = AutoProcessor.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mteb/.venv/lib/python3.12/site-packages/transformers/models/auto/processing_auto.py", line 438, in from_pretrained
return processor_class.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mteb/.venv/lib/python3.12/site-packages/transformers/processing_utils.py", line 1437, in from_pretrained
args = cls._get_arguments_from_pretrained(pretrained_model_name_or_path, processor_dict, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mteb/.venv/lib/python3.12/site-packages/transformers/processing_utils.py", line 1566, in _get_arguments_from_pretrained
sub_processor = auto_processor_class.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mteb/.venv/lib/python3.12/site-packages/transformers/models/auto/image_processing_auto.py", line 572, in from_pretrained
raise initial_exception
File "/home/jovyan/shares/SR004.nfs2/risolomatin/mteb/.venv/lib/python3.12/site-packages/transformers/models/auto/image_processing_auto.py", line 559, in from_pretrained
config_dict, _ = ImageProcessingMixin.get_image_processor_dict(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mteb/.venv/lib/python3.12/site-packages/transformers/image_processing_base.py", line 334, in get_image_processor_dict
raise OSError(
OSError: Can't load image processor for 'microsoft/harrier-oss-v1-27b'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'microsoft/harrier-oss-v1-27b' is the correct path to a directory containing a preprocessor_config.json file
...
"boi_token": "<start_of_image>",
"bos_token": "<bos>",
"clean_up_tokenization_spaces": false,
"eoi_token": "<end_of_image>",
"eos_token": "<eos>",
"extra_special_tokens": {
"boi_token": "<start_of_image>",
"eoi_token": "<end_of_image>",
"image_token": "<image_soft_token>"
},
"image_token": "<image_soft_token>",
"model_max_length": 1000000000000000019884624838656,
"pad_token": "<pad>",
- "padding_side": "right",
+ "processor_class": "Gemma3Processor",
"sp_model_kwargs": null,
"spaces_between_special_tokens": false,
"tokenizer_class": "GemmaTokenizerFast",
"unk_token": "<unk>",
"use_default_system_prompt": false
}
This is the diff between the final lines in the tokenizer_config.json between the 270m and the 27b models. Looks like the 27b model accidentally has a different tokenizer configuration, which causes the Gemma3Processor to try and load, but there's no image processing configuration in this repository, so it crashes. I suspect that the tokenizer.json and tokenizer_config.json from the 270m model should be copied to 27b.
Also, Sentence Transformers used to hardcode AutoTokenizer.from_pretrained, but nowadays it does AutoProcessor.from_pretrained, as the library now supports multimodality. This change likely exposed the misconfiguration, as it worked previously. cc @frontierai could you have a look? This model is currently not usable with the latest versions.
- Tom Aarsen