missing preprocessor_config.json
When I run the model in a vLLM container with mistral-common v1.6.3, I get an error saying that preprocessor_config.json is missing. Should I simply copy the preprocessor_config.json file from the version 3.1 model repository?
docker run -itd --restart=unless-stopped
--name mistral
-p 8000:8000
--ipc=host
--gpus all
vllm/vllm-openai:v0.9.1
--tensor-parallel-size 2
--disable-log-requests
--model mistralai/Mistral-Small-3.2-24B-Instruct-2506
--tokenizer-mode "mistral"
--gpu-memory-utilization 0.90
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] WorkerProc failed to start.
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] Traceback (most recent call last):
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 466, in worker_main
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] worker = WorkerProc(*args, **kwargs)
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 362, in __init__
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] self.worker.init_device()
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/vllm/worker/worker_base.py", line 606, in init_device
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] self.worker.init_device() # type: ignore
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_worker.py", line 160, in init_device
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] self.model_runner: GPUModelRunner = GPUModelRunner(
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 129, in __init__
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] encoder_compute_budget, encoder_cache_size = compute_encoder_budget(
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/core/encoder_cache_manager.py", line 95, in compute_encoder_budget
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ) = _compute_encoder_budget_multimodal(
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/core/encoder_cache_manager.py", line 125, in _compute_encoder_budget_multimodal
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] .get_max_tokens_per_item_by_nonzero_modality(model_config)
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/registry.py", line 158, in get_max_tokens_per_item_by_nonzero_modality
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] self.get_max_tokens_per_item_by_modality(model_config).items()
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/registry.py", line 132, in get_max_tokens_per_item_by_modality
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] return profiler.get_mm_max_tokens(
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/profiling.py", line 256, in get_mm_max_tokens
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] mm_inputs = self._get_dummy_mm_inputs(seq_len, mm_counts)
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/profiling.py", line 166, in _get_dummy_mm_inputs
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] processor_inputs = factory.get_dummy_processor_inputs(
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/profiling.py", line 91, in get_dummy_processor_inputs
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] dummy_text = self.get_dummy_text(mm_counts)
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mistral3.py", line 193, in get_dummy_text
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] processor = self.info.get_hf_processor()
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mistral3.py", line 219, in get_hf_processor
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] return self.ctx.get_hf_processor(PixtralProcessor, **kwargs)
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/vllm/inputs/registry.py", line 131, in get_hf_processor
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] return super().get_hf_processor(
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/vllm/inputs/registry.py", line 94, in get_hf_processor
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] return cached_processor_from_config(
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/processor.py", line 110, in cached_processor_from_config
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] return cached_get_processor(
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/processor.py", line 72, in get_processor
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] processor = processor_factory.from_pretrained(
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/transformers/processing_utils.py", line 1185, in from_pretrained
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] args = cls._get_arguments_from_pretrained(pretrained_model_name_or_path, **kwargs)
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/transformers/processing_utils.py", line 1248, in _get_arguments_from_pretrained
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] args.append(attribute_class.from_pretrained(pretrained_model_name_or_path, **kwargs))
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/transformers/models/auto/image_processing_auto.py", line 472, in from_pretrained
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] raise initial_exception
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/transformers/models/auto/image_processing_auto.py", line 454, in from_pretrained
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] config_dict, _ = ImageProcessingMixin.get_image_processor_dict(
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/transformers/image_processing_base.py", line 340, in get_image_processor_dict
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] resolved_image_processor_file = cached_file(
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/transformers/utils/hub.py", line 312, in cached_file
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] file = cached_files(path_or_repo_id=path_or_repo_id, filenames=[filename], **kwargs)
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] File "/usr/local/lib/python3.12/dist-packages/transformers/utils/hub.py", line 573, in cached_files
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] raise EnvironmentError(
(VllmWorker rank=1 pid=258) ERROR 07-04 22:08:03 [multiproc_executor.py:492] OSError: mistralai/Mistral-Small-3.2-24B-Instruct-2506 does not appear to have a file named preprocessor_config.json. Checkout 'https://huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506/tree/main' for available files.
I have the same problem with this model. i am trying to deploy on node in AZ AKS but I got this warning.
INFO 07-07 01:49:26 [init.py:244] Automatically detected platform cuda.
INFO 07-07 01:49:30 [api_server.py:1287] vLLM API server version 0.9.1
INFO 07-07 01:49:30 [cli_args.py:309] non-default args: {'enable_auto_tool_choice': True, 'tool_call_parser': 'mistral', 'model': 'mistralai/Mistral-Small-3.2-24B-Instruct-2506', 'tokenizer_mode': 'mistral', 'trust_remote_code': True, 'dtype': 'float16', 'max_model_len': 4096, 'model_impl': 'vllm', 'max_num_batched_tokens': 4096}
INFO 07-07 01:49:39 [config.py:823] This model supports multiple tasks: {'reward', 'classify', 'score', 'embed', 'generate'}. Defaulting to 'generate'.
WARNING 07-07 01:49:39 [config.py:3271] Casting torch.bfloat16 to torch.float16.
INFO 07-07 01:49:40 [config.py:2195] Chunked prefill is enabled with max_num_batched_tokens=4096.
WARNING 07-07 01:49:43 [env_override.py:17] NCCL_CUMEM_ENABLE is set to 0, skipping override. This may increase memory overhead with cudagraph+allreduce: https://github.com/NVIDIA/nccl/issues/1234
INFO 07-07 01:49:45 [init.py:244] Automatically detected platform cuda.
INFO 07-07 01:49:48 [core.py:455] Waiting for init message from front-end.
INFO 07-07 01:49:48 [core.py:70] Initializing a V1 LLM engine (v0.9.1) with config: model='mistralai/Mistral-Small-3.2-24B-Instruct-2506', speculative_config=None, tokenizer='mistralai/Mistral-Small-3.2-24B-Instruct-2506', skip_tokenizer_init=False, tokenizer_mode=mistral, revision=None, override_neuron_config={}, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=4096, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, decoding_config=DecodingConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_backend=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=0, served_model_name=mistralai/Mistral-Small-3.2-24B-Instruct-2506, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=True, chunked_prefill_enabled=True, use_async_output_proc=True, pooler_config=None, compilation_config={"level":3,"debug_dump_path":"","cache_dir":"","backend":"","custom_ops":["none"],"splitting_ops":["vllm.unified_attention","vllm.unified_attention_with_output"],"use_inductor":true,"compile_sizes":[],"inductor_compile_config":{"enable_auto_functionalized_v2":false},"inductor_passes":{},"use_cudagraph":true,"cudagraph_num_of_warmups":1,"cudagraph_capture_sizes":[512,504,496,488,480,472,464,456,448,440,432,424,416,408,400,392,384,376,368,360,352,344,336,328,320,312,304,296,288,280,272,264,256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],"cudagraph_copy_inputs":false,"full_cuda_graph":false,"max_capture_size":512,"local_cache_dir":null}
WARNING 07-07 01:49:49 [utils.py:2737] Methods determine_num_available_blocks,device_config,get_cache_block_size_bytes,initialize_cache not implemented in <vllm.v1.worker.gpu_worker.Worker object at 0x7ff4fd44a060>
INFO 07-07 01:49:49 [parallel_state.py:1065] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0, EP rank 0
ERROR 07-07 01:49:51 [core.py:515] EngineCore failed to start.
ERROR 07-07 01:49:51 [core.py:515] Traceback (most recent call last):
Process EngineCore_0:
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 506, in run_engine_core
ERROR 07-07 01:49:51 [core.py:515] engine_core = EngineCoreProc(*args, **kwargs)
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 390, in init
ERROR 07-07 01:49:51 [core.py:515] super().init(vllm_config, executor_class, log_stats,
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 76, in init
ERROR 07-07 01:49:51 [core.py:515] self.model_executor = executor_class(vllm_config)
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/executor/executor_base.py", line 53, in init
ERROR 07-07 01:49:51 [core.py:515] self._init_executor()
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/executor/uniproc_executor.py", line 47, in _init_executor
ERROR 07-07 01:49:51 [core.py:515] self.collective_rpc("init_device")
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/executor/uniproc_executor.py", line 57, in collective_rpc
ERROR 07-07 01:49:51 [core.py:515] answer = run_method(self.driver_worker, method, args, kwargs)
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/utils.py", line 2671, in run_method
ERROR 07-07 01:49:51 [core.py:515] return func(*args, **kwargs)
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/worker/worker_base.py", line 606, in init_device
ERROR 07-07 01:49:51 [core.py:515] self.worker.init_device() # type: ignore
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_worker.py", line 160, in init_device
ERROR 07-07 01:49:51 [core.py:515] self.model_runner: GPUModelRunner = GPUModelRunner(
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 129, in init
ERROR 07-07 01:49:51 [core.py:515] encoder_compute_budget, encoder_cache_size = compute_encoder_budget(
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/core/encoder_cache_manager.py", line 95, in compute_encoder_budget
ERROR 07-07 01:49:51 [core.py:515] ) = _compute_encoder_budget_multimodal(
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/core/encoder_cache_manager.py", line 125, in _compute_encoder_budget_multimodal
ERROR 07-07 01:49:51 [core.py:515] .get_max_tokens_per_item_by_nonzero_modality(model_config)
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/registry.py", line 158, in get_max_tokens_per_item_by_nonzero_modality
ERROR 07-07 01:49:51 [core.py:515] self.get_max_tokens_per_item_by_modality(model_config).items()
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/registry.py", line 132, in get_max_tokens_per_item_by_modality
ERROR 07-07 01:49:51 [core.py:515] return profiler.get_mm_max_tokens(
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/profiling.py", line 256, in get_mm_max_tokens
ERROR 07-07 01:49:51 [core.py:515] mm_inputs = self._get_dummy_mm_inputs(seq_len, mm_counts)
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/profiling.py", line 166, in _get_dummy_mm_inputs
ERROR 07-07 01:49:51 [core.py:515] processor_inputs = factory.get_dummy_processor_inputs(
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/multimodal/profiling.py", line 91, in get_dummy_processor_inputs
ERROR 07-07 01:49:51 [core.py:515] dummy_text = self.get_dummy_text(mm_counts)
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mistral3.py", line 193, in get_dummy_text
ERROR 07-07 01:49:51 [core.py:515] processor = self.info.get_hf_processor()
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mistral3.py", line 219, in get_hf_processor
ERROR 07-07 01:49:51 [core.py:515] return self.ctx.get_hf_processor(PixtralProcessor, **kwargs)
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/inputs/registry.py", line 131, in get_hf_processor
ERROR 07-07 01:49:51 [core.py:515] return super().get_hf_processor(
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/inputs/registry.py", line 94, in get_hf_processor
ERROR 07-07 01:49:51 [core.py:515] return cached_processor_from_config(
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/processor.py", line 110, in cached_processor_from_config
ERROR 07-07 01:49:51 [core.py:515] return cached_get_processor(
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/processor.py", line 72, in get_processor
ERROR 07-07 01:49:51 [core.py:515] processor = processor_factory.from_pretrained(
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/transformers/processing_utils.py", line 1185, in from_pretrained
ERROR 07-07 01:49:51 [core.py:515] args = cls._get_arguments_from_pretrained(pretrained_model_name_or_path, **kwargs)
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/transformers/processing_utils.py", line 1248, in _get_arguments_from_pretrained
ERROR 07-07 01:49:51 [core.py:515] args.append(attribute_class.from_pretrained(pretrained_model_name_or_path, **kwargs))
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/transformers/models/auto/image_processing_auto.py", line 472, in from_pretrained
ERROR 07-07 01:49:51 [core.py:515] raise initial_exception
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/transformers/models/auto/image_processing_auto.py", line 454, in from_pretrained
ERROR 07-07 01:49:51 [core.py:515] config_dict, _ = ImageProcessingMixin.get_image_processor_dict(
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/transformers/image_processing_base.py", line 340, in get_image_processor_dict
ERROR 07-07 01:49:51 [core.py:515] resolved_image_processor_file = cached_file(
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/transformers/utils/hub.py", line 312, in cached_file
ERROR 07-07 01:49:51 [core.py:515] file = cached_files(path_or_repo_id=path_or_repo_id, filenames=[filename], **kwargs)
ERROR 07-07 01:49:51 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-07 01:49:51 [core.py:515] File "/usr/local/lib/python3.12/dist-packages/transformers/utils/hub.py", line 573, in cached_files
ERROR 07-07 01:49:51 [core.py:515] raise EnvironmentError(
ERROR 07-07 01:49:51 [core.py:515] OSError: mistralai/Mistral-Small-3.2-24B-Instruct-2506 does not appear to have a file named preprocessor_config.json. Checkout 'https://huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506/tree/main' for available files.
Hopefully this saves others some time:
llm = LLM(
model='mistralai/Mistral-Small-3.2-24B-Instruct-2506',
tokenizer_mode="mistral",
config_format="mistral",
load_format="mistral",
)