How to load mT5 checkpoint (.ckpt)?
Hello,I loaded mT5 withmodel = MT5ForConditionalGeneration.from_pretrained(“google/mt5-small”)tokenizer = MT5Tokenizer.from_pretrained(“google/mt5-small”)Then, I fined-tuned it with Pytorch Lightning and now have a checkpoint in the format .ckpt. However, now when I try to load it again withmodel = AutoModelWithLMHead.from_pretrained("path/to/checkpoint.ckpt")
tokenizer = MT5Tokenizer.from_pretrained("google/mt5-small")it fails …I have this error code :Traceback (most recent call last):
   line 8, in <module>
    model = AutoModelWithLMHead.from_pretrained("path/to/checkpoint.ckpt")
 "/transformers/models/auto/modeling_auto.py", line 930, in from_pretrained
    pretrained_model_name_or_path, return_unused_kwargs=True, **kwargs
  "/transformers/models/auto/configuration_auto.py", line 360, in from_pretrained
    config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  "/transformers/configuration_utils.py", line 423, in get_config_dict
    config_dict = cls._dict_from_json_file(resolved_config_file)
  "/transformers/configuration_utils.py", line 506, in _dict_from_json_file
    text = reader.read()
  File "/usr/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 64: invalid start byteI have seen that it can be necessary to have a along with the checkpoint a config.json, vocab.json and merges.txt. The config.json is available on the google/mt5-small huggingface page so I downloaded it and added it to the folder with the checkpoint. However, the other files are not there. What are the steps for me to be able to load the .ckpt of mT5 ?Thanks
2022-02-01T21:55:24Z
Hi@Skylixia,Did you manage to solve the problem?
2022-03-17T06:44:31Z
Any updates？Maybe try to convert the ckpt to pytorch_model.bin? but anybody know how to do this?