Convert transformer to SavedModel
Hi! I found out that this is common unresolved problem.So, I need to convert transformers’ DistilBERT to TensorFlows SavedModel format. I've converted it, but I cant inference it.Conversion codeimport tensorflow as tf
from transformers import TFAutoModel, AutoTokenizer
dir = "distilbert_savedmodel"

model = TFAutoModel.from_pretrained('distilbert-base-uncased')
model.save(dir)Inference codetokenizer = AutoTokenizer.from_pretrained('distilbert-base-uncased')
encoded = tokenizer.encode('Hello, world!', add_special_tokens=True, return_tensors="tf")
model = tf.keras.models.load_model(dir)
model(encoded)ErrorValueError: Could not find matching function to call loaded from the SavedModel. Got:
  Positional arguments (1 total):
    * Tensor("inputs:0", shape=(1, 6), dtype=int32)
  Keyword arguments: {'training': False}

Expected these arguments to match one of the following 4 option(s):

Option 1:
  Positional arguments (1 total):
    * {'input_ids': TensorSpec(shape=(None, 5), dtype=tf.int32, name='input_ids')}
  Keyword arguments: {'training': False}

Option 2:
  Positional arguments (1 total):
    * {'input_ids': TensorSpec(shape=(None, 5), dtype=tf.int32, name='input_ids')}
  Keyword arguments: {'training': True}

Option 3:
  Positional arguments (1 total):
    * {'input_ids': TensorSpec(shape=(None, 5), dtype=tf.int32, name='inputs/input_ids')}
  Keyword arguments: {'training': True}

Option 4:
  Positional arguments (1 total):
    * {'input_ids': TensorSpec(shape=(None, 5), dtype=tf.int32, name='inputs/input_ids')}
  Keyword arguments: {'training': False}Related issueshuggingface/transformers#4004huggingface/transformers#2135huggingface/transformers#2021Please, help me!
2020-11-18T12:25:31Z
In pytorch, you could save the model with something liketorch.save(model.state_dict(),‘/content/drive/My Drive/ftmodelname’ )Then you could create a model using the pre-trained weightstuned_model = BertForSequenceClassification.from_pretrained(‘bert-base-uncased’,num_labels=NCLASSES,output_attentions=True)and then overwrite its weights from the saved state_dict withtuned_model.load_state_dict(torch.load(‘/content/drive/My Drive/ftmodelname’,map_location=torch.device(“cpu”)),strict=False)I expect you could do a similar save of the model state_dict using Tensorflow.
2021-09-08T07:30:25Z
How did you end up solving the problem. I came across the same one.
2021-11-27T22:06:09Z
I have the same problem.I can train a model, evaluate and predict if it’s within the same script, but if I save the model and then load it I get the same kind of error, is it currently impossible to load a saved model with tensorflow/keras?
2021-11-30T15:34:39Z
hugging face has model.save_pretrained() method