What is the difference between prompt tuning and prefix tuning?
I readprompt tuningandprefix tuningare two effective mechanisms to leverage frozen language models to perform downstream tasks. What is the difference between the two and how they work really?
2023-02-06T21:51:35Z
They’re really similar ideas and came out around the same time. Notably prefix tuning learns soft prompts at all layers of the model while prompt tuning only modifies the input. This kind of thing is discussed in thep-tuningline of work.
2023-03-08T23:41:51Z
The introduction in this tutorial notebook has a nice explanationNeMo/Multitask_Prompt_and_PTuning.ipynb at main · NVIDIA/NeMo · GitHub