Get probability of LLM outputting token sequence
I have an input to an LLM (I am using GPT2 but any causal model should work), X, and a response string Y that I manually created. I would like to get P(Y|X), the probability that the LLM would output the response Y. How do I do this?.generate(),compute_transition_scoresprovides functionality for getting the probabilities of a generated output, but in this case my output is not generated by the LLM.
2023-11-28T08:28:29Z
Did you discover a way to do this?