How to parameter efficient finetune Decoder in encoder-decoder model
I have been trying to finetune the decoder by inserting an adapter after the ffn block of a large encoder-decoder model for the MT task, but the blue score is bad when compared to full finetuning.Please, can anyone suggest a better method to finetune or refer me to some resources to finetune the decoder of a model
2024-04-23T22:11:57Z
just a question, where you padding the model from left or right? because i made a mistake about it while infer and i got degreded result compare to left padding. i am not experienced in terms of perf, but have you tried hyper parametering it. or try the parameters other have tried? they are usually good starting points.