DeBERTa absolute Positions
Hi! I was taking a look at Hugging Face’s DeBERTaimplementation, but I couldn’t find where in the code are the absolute positions being added (they should come before the softmax layer), can someone point me towards the line where that is being done?Thank you for your time
2021-02-16T20:30:21Z
cc@lysandremaybe?
2021-04-15T20:49:35Z
same here, wondering about absolute positions. Also@lysandrecan you explain what thez_stepssuppose to do in Debertav2.Thanks a lot