ValueError: Model requires 4 inputs. Input Feed contains 2 ONNX
i try to export T5 model to ONNX but when a try to predict something it show a error:sess = ort.InferenceSession(‘onnx\model.onnx’)inputs = tokenizer(“something in here”, return_tensors=“np”)outputs = sess.run( None,inputs )ValueError: Model requires 4 inputs. Input Feed contains 2
2022-05-04T12:11:54Z
Maybe@lewtunwill have an idea?
2022-05-04T12:31:50Z
Hey@Slcreatorthe number of inputs you need for inference depends on the type of head you chose to export the model with (i.e. thefeaturearg in thetransformers.onnxpackage).For example, here’s how you can figure out which inputs are needed for say thedefaultfeature of a T5 checkpoint:from transformers.models.t5 import T5Config, T5OnnxConfig

config = T5Config.from_pretrained("t5-small")
onnx_config = T5OnnxConfig(config=config, task="default")
# returns
# {'input_ids': {0: 'batch', 1: 'encoder_sequence'},
#  'attention_mask': {0: 'batch', 1: 'encoder_sequence'},
#  'decoder_input_ids': {0: 'batch', 1: 'decoder_sequence'},
#  'decoder_attention_mask': {0: 'batch', 1: 'decoder_sequence'}}
onnx_config.inputsYou can change thetaskarg to match the type of head you’re exporting and that will tell you what the model expects as inputs.Btw we’ve recently added apipeline()function to theoptimumlibrary, which makes it quite simple to run inference on ONNX Runtime. There’s a guide here if you’re interested:Optimum pipelines for inference
2022-05-05T02:22:40Z
thanks very much, i’m going to check it again
2023-01-24T12:13:04Z
Hello, I am trying to use Bigbirdpegasus and getting the same error. Model requires 4 inputs but how can we provide decoder attention and mask beforehand? isn’t that supposed to be generated after we get output from the model?Can you please share the code to export bigbird for summarization?Here is the code I am using:from pathlib import Pathfrom transformers.onnx import exportfrom transformers import AutoTokenizer, AutoModelonnx_path = Path(“model.onnx”)model_ckpt = “google/bigbird-pegasus-large-pubmed”base_model = AutoModel.from_pretrained(model_ckpt)tokenizer = AutoTokenizer.from_pretrained(model_ckpt)onnx_inputs, onnx_outputs = export(tokenizer, base_model, onnx_config, onnx_config.default_onnx_opset, onnx_path)tokenizer = AutoTokenizer.from_pretrained(“google/bigbird-pegasus-large-pubmed”)session = InferenceSession(“model.onnx”)inputs = tokenizer(“Summarize this text”, return_tensors=“np”)inputs = {k: v.astype(np.int64) for k, v in inputs.items()}outputs = session.run(output_names=[“last_hidden_state”], input_feed=inputs)