SFTTrainer Loss function
I have a couple of questions:how to know the loss function used by default for SFTTrainer for a given model and how to alter it?For training an LLM the loss function is computed on the whole concatenated prompts, how to alter this and make loss function only compute on the output prompts
2024-07-08T08:07:21Z
Hi,I would recommend exploring DataCollatorForCompletionOnlyLM in HuggingFace for training LLM on outputs only!
2024-07-08T08:12:55Z
ZeyadMahmoud:how to know the loss function used by default for SFTTrainer for a given model and how to alter it?The loss function being used is the cross-entropy loss. It is defined within the model, e.g.herefor llama. In case you want to use your own custom loss function, you can overwrite the compute_loss method of the Trainer as explainedhere.For training an LLM the loss function is computed on the whole concatenated prompts, how to alter this and make loss function only compute on the output promptsIndeed as recommended above, theDataCollatorForCompletionOnlyLMcan be used for this purpose.