The occurrence of the word fits better in another example
Hi, I am beginner in HuggingFaceI am trying simple word-sense disambiguation using BertTokenizer: ‘bert-base-uncased’. I am compare word ‘rick’ occured in one of wordnet samples and my sample,When my sample = “band playing rock”, I have results:tensor([[0.3383]]) he threw a rock at metensor([[0.4018]]) that mountain is solid rocktensor([[0.1466]]) stone is abundant in New England and there are many quarriestensor([[0.3042]]) he was her rock during the crisistensor([[0.2900]]) Thou art Peter, and upon this rock I will build my churchtensor([[0.1766]]) rock is a generic term for the range of styles that evolved out of rock’n’roll.tensor([[0.0670]]) the ship was rockingtensor([[0.0478]]) the tall building swayedtensor([[0.0898]]) She rocked back and forth on her feettensor([[0.1587]]) rock the cradletensor([[0.1088]]) rock the babytensor([[0.0740]]) the wind swayed the trees gentlySentence: band playing rockClosest definition: (tensor([[0.4018]]), ‘that mountain is solid rock’)I don’t know why “rock” near “play” or “playing” (although is even “band”) prefer solid rock instead rock and rollwith roBERTa and deBERTa is similar , except that cosine similarity is above 0.99 for most pairs.(in 4- 24 hours I can prepare gist with minimal example, if needed)solution: ‘bert-base-cased’ is a much better:Sentence: band playing rockClosest definition: (tensor([[0.8216]]), “rock is a generic term for the range of styles that evolved out of rock’n’roll.”)but is not quite WSD-friendly: “rock is a generic term for mountain” obvious fit to "“rock is a generic term for the range of styles that evolved out of rock’n’roll.” - a lot of words matchI think, the same will be with negativeSentence: rock is not a generic term for the range of styles that evolved out of rock’n’roll.Closest definition: (tensor([[0.9930]]), “rock is a generic term for the range of styles that evolved out of rock’n’roll.”)(instead of -0.993)