mistralai

/

Mistral-Small-3.2-24B-Instruct-2506

like 404

Follow

Mistral AI_ 11.6k

Hopefully this saves others some time:

llm = LLM(
    model='mistralai/Mistral-Small-3.2-24B-Instruct-2506',
    tokenizer_mode="mistral",
    config_format="mistral",
    load_format="mistral",
)

"llm = LLM(
model='mistralai/Mistral-Small-3.2-24B-Instruct-2506',
tokenizer_mode="mistral",
config_format="mistral",
load_format="mistral",
)"

Is this the implementation in vLLM?
So we have to accept that downloading from HuggingFace does not work like this:

tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.save_pretrained(local_path)
model = AutoModelForCausalLM.from_pretrained( model_id)
model.save_pretrained(local_path, safe_serialization=False)

Error: KeyError: <class 'transformers.models.mistral3.configuration_mistral3.Mistral3Config'>