What exactly is the summarization pipeline doing?
I’m trying to understand what the summarization pipeline is doing exactly.I’ve noticed the following:When running a model in a simple text generation (usingmodel.generate()) the output is cut short.But when running it in summarization pipeline it isn’t cut.But if I understand correctly, the pipeline cannot get over the model_max_length limit, as it’s not doing recursive summarization / map-reduce / sliding window approach. I’d need to implement those myself, right?So what is the difference then between txt gen and summarization pipeline?