Custom Decoding Strategy
Hi,I am currently working on a project that involves controllable text generation.To do this, I am making a ‘post-processing module’ which involves reranking the logits during the decoding process.To start, I have forked transformers, and added a new GenerationMode to the GenerationMixin.(GenerationMode.TEST_GREEDY_SEARCH) for this example.Within the greedy_search(), I believe next_tokens_scores would be the logits scores for the tokens? I am not sure.One of the problems is I need to decode the tokens of the logits into the human readable word to use it in my post processing module. At the moment I am just importing the tokeniser I need into the function, but I do not know how to get the current tokens.In summary, I need the token, and its probability, for each of the logits.Any help would be greatly appreciated.