Output_attention = True after downloading a model
Hi I am new to hugging face, I have downloaded distilBERT usingmodel = DistilBertModel.from_pretrained("distilbert-base-uncased")and I did not modify the config. But now I want to see the output attentions. Is there any way I can see the attentions without downloading a new model?
2020-08-28T15:41:08Z
Hey@pritamYou can passoutput_attentions=Truetoforwardmethod, and it’ll return attentions. Attnetions are last element of the returnedTupleor if you are on master branch, then you can also passreturn_dict=Trueand call.attentionson the returned object.output = model(input_ids, output_attentions=True, return_dict=True)
output.attentions # this will give the attentions
2020-08-29T06:42:58Z
You can do what@valhallasaid. Perhaps the most clean way is changing the config.config = AutoConfig.from_pretrained(MODEL_ID, output_attentions=True)
model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID, config=config)
output = model(**inputs) # inputs from default_data_collatoroutputwill consist oflogits,pooled outputandattention.