How to access raw attention logits?
I would like to get the attention logits (i.e. the values ofQK^T) for al layers of a transformer model. In other words - the pre-softmax values. Is there a simple way to somehow get them using the API?
2024-03-10T19:21:58Z
Hi,The Transformers library supports passingoutput_attentionsto the forward of any model. However, these are the post-softmax values. In order to get the pre-softmax values, one would need to fork the library to tweak the model outputs or use PyTorch hooks.
2024-03-11T09:50:18Z
Thanks, that’s the conclusion I’ve reachedIs there a tutorial about using hooks I can learn from?