Peft Model For SequenceClassification failing _is_peft_model
I am attempting to finetune MixtralForSequenceClassification, and believe I have a peft model. However, when I call trainer.train() I run into the error “ValueError: You cannot perform fine-tuning on purely quantized models. Please attach trainable adapters on top of the quantized model to correctly perform fine-tuning. Please see:Load adapters with 🤗 PEFTfor more details”Here’s what I did to get here, following the steps in a few popular colab example notebooks:Loaded mixtral using 4bit quantization with a standard configbnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)Prepared for kbit trainingfrom peft import prepare_model_for_kbit_training

model.gradient_checkpointing_enable()

model = prepare_model_for_kbit_training(model)Used a standard LoraConfig with task_type SEQ_CLSfrom peft import LoraConfig, get_peft_model

config = LoraConfig(
    r=32,
    lora_alpha=64,
    target_modules=[
        "q_proj",
        "k_proj",
        "v_proj",
        "o_proj",
        "gate_proj",
        "up_proj",
        "down_proj",
        "lm_head",
    ],
    bias="none",
    lora_dropout=0.05,  # Conventional
    task_type="SEQ_CLS",
)

model = get_peft_model(model, config)
print_trainable_parameters(model)When I check if the model is an instance of a PeftModel, it returns true:isinstance(model, peft.PeftModel)And printing the model yields this:image860×162 11 KBHowever _is_peft_model is returning falseI followed similar steps for finetuning Llama ~4 months ago and did not run into this error. Doing the exact same thing as some of these colab notebooks I have seen, so not sure what the error is. Perhaps something specific to sequence classification?Any help would be much appreciated!