Bart summarization
Good morning/eveningI am trying to understand how does distilbart generate summaries, like what is the logic behind when you fine tune it with texts and their reference summaries, how does it learn to summarize with a specified length with new words? The way I see it is: I feed a text into the model, it gets encoded & then decoded with only the tokens containing important information? How does the model spots kinda the good sentences tokens?
2020-08-10T01:10:08Z
You should read more about “Sequence to Sequence”.Bart is a seq2seq model : the input text is encoded with attention, and then output text is generated token by token, with attention over the input and the generated output so far. Since the output is generated token by token, we can choose how many token we want to generate.
2020-08-10T03:06:49Z
Yes as@astariulsaid. And learn from the master is one of the best wayHere’s Andrew Ng Video Series on Seq2Seq model :Even though it’s not transformer, but the big picture concept is applicable .
2020-08-10T13:57:41Z
Thank you very much, I will!