What happens in the MT5 documentation example?
Hi,I’m trying to understand the provided example to theMT5 modelbut have some difficulties.Here is the example:from transformers import MT5Model, T5Tokenizermodel = MT5Model.from_pretrained(“google/mt5-small”)tokenizer = T5Tokenizer.from_pretrained(“google/mt5-small”)article = “UN Offizier sagt, dass weiter verhandelt werden muss in Syrien.”summary = “Weiter Verhandlung in Syrien.”batch = tokenizer.prepare_seq2seq_batch(src_texts=[article], tgt_texts=[summary], return_tensors=“pt”)outputs = model(input_ids=batch.input_ids, decoder_input_ids=batch.labels)hidden_states = outputs.last_hidden_stateSo I understand that tokenizer.prepare_seq2seq_batch is to encode the input to provide to the model. It is a BatchEncoding containting the input_ids, labels and attention_mask.However, I don’t understand what follows, what happens in : model(input_ids=batch.input_ids, decoder_input_ids=batch.labels) ? This does not train or fine tune the model but what does it do ?Why do we provide it a source and target then ? What if we wanted the model to generate the target (summary) ?Thanks !
2021-01-10T14:32:42Z
The example is just a general example of how to do a forward pass through the model, just like you can do in any model.  In practice, you’d see something like this:github.comhuggingface/transformers/blob/4f7022d68d4bae4b5e6a748b7a7323515c6fdcd3/examples/seq2seq/seq2seq_trainer.py#L162-L176def _compute_loss(self, model, inputs, labels):if self.args.label_smoothing == 0:if self.data_args is not None and self.data_args.ignore_pad_token_for_loss:# force training to ignore pad tokenlogits = model(**inputs, use_cache=False)[0]loss = self.loss_fn(logits.view(-1, logits.shape[-1]), labels.view(-1))else:# compute usual loss via modelsloss, logits = model(**inputs, labels=labels, use_cache=False)[:2]else:# compute label smoothed losslogits = model(**inputs, use_cache=False)[0]lprobs = torch.nn.functional.log_softmax(logits, dim=-1)loss, _ = self.loss_fn(lprobs, labels, self.args.label_smoothing, ignore_index=self.config.pad_token_id)return loss, logits
2021-01-11T02:06:37Z
About the last question, (generating strings), note that mT5 is pretrained only on masked language model objective so that, unlike original T5, we could not play with downstream tasks without finetuning.To test MLM, you could try the following snippetsfrom transformers import MT5Tokenizer, TFMT5ForConditionalGeneration
model = TFMT5ForConditionalGeneration.from_pretrained('google/mt5-base')
tokenizer = MT5Tokenizer.from_pretrained('google/mt5-base', use_fast=True)

input_ids = tokenizer.encode("What the <extra_id_0> is going on inside this <extra_id_1> ?", return_tensors="tf")

output_ids = model.generate(input_ids)
print(tokenizer.batch_decode(output_ids))return[’ <extra_id_0> hell <extra_id_1> strange world <extra_id_2> … ']
2021-01-11T16:04:58Z
Ah I see thank you !