Classifiers are Better Experts for Controllable Text Generation
This paper proposes a simple method for controllable text generation based on
weighting logits produced, namely CAIF sampling. Using an arbitrary third-party
text classifier, we adjust a small part of a language model's logits and guide
text generation towards or away from classifier prediction. We show that the
proposed method significantly outperforms recent PPLM, GeDi, and DExperts on
PPL and sentiment accuracy based on the external classifier of generated texts.
A the same time, it is also easier to implement and tune, and has significantly
fewer restrictions and requirements.