Longformer's attention_mask
Inside of forward function ofclass LongformerSelfAttention(nn.Module):Theattention_maskis changed inBertModel.forwardfrom 0, 1, 2 to-ve: no attention0: local attention+ve: global attentionI wonder how this change happens.Longformer inherits from RobertaModel, which inherits fromBertModel. Would BertModel’s forward also be called when calling Longformer 's forward function?How BertModel’s forward change 0, 1, 2 to -ve/0/+ve？Thank you