How to fine-tune merlinite 7B model in Python
I am new to LLM programming in Python and I am trying to fine-tune theinstructlab/merlinite-7b-labmodel on my Mac M1. My goal is to teach this model to a new music composerXenobi AmilenI have invented.The text of this composer ishere.Using the new Ilab CLI from RedHat I createdthis training setfor the model. It is a JSONL file with 100 questions/answers about the invented composer.I wrotethis Python scriptto train the model. I tested all the parts related to the tokenizer, datasets and it seems to work. However, the final train got this error:RuntimeError: Placeholder storage has not been allocated on MPS device!
  0%|          | 0/75 [00:00<?, ?it/s]I found a lot of articles about this error on Google and also StackOverflowlike this, for example. The problem seems that in addition to the model I have to send to mps also the input parameters, but it’s not clear to me how to change my code to do that.I tried several fixes but had no luck. Can anyone can help?
2024-07-03T18:22:53Z
No one in this forum can help?
2024-07-08T08:27:29Z
Hi,Have you looked at this thread?RuntimeError: Placeholder storage has not been allocated on MPS device!
2024-07-08T09:44:37Z
Yes, I have, but it hasn’t worked for me. As you can see I have the flag enabled.github.comsasadangelo/llm-train/blob/017385be56e6b643ba66352aed17e0b12629536b/main.py#L82output_dir="./results",           # Directory di output per i modelli e i logeval_strategy="epoch",            # Strategia di valutazione (ad ogni epoca)learning_rate=2e-5,               # Tasso di apprendimentoper_device_train_batch_size=4,    # Batch size per dispositivo durante l'addestramentoper_device_eval_batch_size=4,     # Batch size per dispositivo durante la valutazionenum_train_epochs=3,               # Numero di epoche di addestramentoweight_decay=0.01,                # Decadimento del pesosave_total_limit=2,               # Numero massimo di checkpoint da conservaresave_steps=10_000,                # Frequenza di salvataggio dei checkpointlogging_dir='./logs',             # Directory per i loguse_mps_device=True,)# Initialize the DataCollatordata_collator = DataCollatorForLanguageModeling(tokenizer, mlm=False)# Initialize the SFTTrainertrainer = Trainer(model=model,args=training_args,train_dataset=train_dataset,
2024-07-08T12:33:01Z
Yes but further down in the thread they also mention to passno_cuda=True, could you try that?
2024-07-09T17:12:15Z
Hi@nielsrI added the option as you suggested but now I get this error:Traceback (most recent call last):
  File "/Users/sasadangelo/github.com/sasadangelo/llm-train/main.py", line 71, in <module>
    training_args = TrainingArguments(
                    ^^^^^^^^^^^^^^^^^^
  File "<string>", line 129, in __init__
  File "/Users/sasadangelo/github.com/sasadangelo/llm-train/venv/lib/python3.12/site-packages/transformers/training_args.py", line 1693, in __post_init__
    self.device
  File "/Users/sasadangelo/github.com/sasadangelo/llm-train/venv/lib/python3.12/site-packages/transformers/training_args.py", line 2171, in device
    return self._setup_devices
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/sasadangelo/github.com/sasadangelo/llm-train/venv/lib/python3.12/site-packages/transformers/utils/generic.py", line 60, in __get__
    cached = self.fget(obj)
             ^^^^^^^^^^^^^^
  File "/Users/sasadangelo/github.com/sasadangelo/llm-train/venv/lib/python3.12/site-packages/transformers/training_args.py", line 2133, in _setup_devices
    raise ValueError(
ValueError: Either you do not have an MPS-enabled device on this machine or MacOS version is not 12.3+ or current PyTorch install was not built with MPS enabled.I have MacOS Sonoma 14.5, MPS is enabled because this test works fine:github.comsasadangelo/llm-train/blob/017385be56e6b643ba66352aed17e0b12629536b/main.py#L31# Function to move the batch to the specified device#def collate_fn(batch, device):#    for k, v in batch.items():#        batch[k] = v.to(device)#    return batchmodel_name="instructlab/merlinite-7b-lab"# Verifica la disponibilità della GPU MPSdevice = "cpu"if torch.backends.mps.is_available() and torch.backends.mps.is_built():device = "mps"torch_device = torch.device(device)print(f"* Using device: {device}")print(f"* Set the default device for PyTorch to {torch_device}")torch.set_default_device(torch_device)# Prepare the tokenizer for the input model. AutoTokenizer will load the correct tokenizer for the input model.print(f"* Load the tokenizer for {model_name}.")tokenizer = AutoTokenizer.from_pretrained(model_name)The only thing I don’t know if PyTorch has MPS enabled but I don’t know how to verify. On MacOS I installed with pip the pythorch version (it is a recent nightly build):torch==2.5.0.dev20240704I used also the stable release in my tests but I didn’t try with the option you suggested. What you suggest to do?
2024-07-09T17:14:52Z
To verify MPS Pytorch support I run this test:Apple DeveloperAccelerated PyTorch training on Mac - Metal - Apple DeveloperPyTorch uses the new Metal Performance Shaders (MPS) backend for GPU training acceleration.the result was:tensor([1.], device='mps:0')so I think everythin is OK
2024-07-09T17:23:03Z
FYI, no_cuda has been deprecated and raplaced by use_cpu that by default is already to False. So I don’t think the flag is the issue. I think it is something related to Pytorch.
2024-07-21T02:48:09Z
Hi@sasadangelo- I’m a member of the InstructLab project. I am sorry I missed this post when you initially made it.I looked at your repo - I don’t see the knowledge YAML or md you would need using the InstructLab methodology to train, do you have those available to take a look at?I’m also wondering why you used a custom script instead ofilab trainto train the model? Were you just interested in the datagen component and not the full workflow?
2024-10-09T07:23:14Z
HimairindubhThe point is that I don’t want to use instructlab to train the model and his methodology with skill and knowledge. I want to understand how training works at a lower level. This is the reason why I implemented my script.