Improving decoding speed by onnx conversion model
After I use the uer py training model to convert the transformers model, the decoding speed is not very fast. I want to improve the speed by converting the model onnx.python -m transformers.onnx --model=bert-base-cased onnx/bert-base-cased/Some problems were encountered using the converted model.According to the official documentation, there is no generation method.I checked the case list of gpt2 conversion onnx. The gpt2 model can be decoded normally and the speed has been improved.github.commicrosoft/onnxruntime/blob/main/onnxruntime/python/tools/transformers/notebooks/Inference_GPT2_with_OnnxRuntime_on_CPU.ipynb{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright (c) Microsoft Corporation. All rights reserved.  \n",
    "Licensed under the MIT License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Inference PyTorch GPT2 Model with ONNX Runtime on CPU\n",
    "\n",
    "In this tutorial, you'll be introduced to how to load a GPT2 model from PyTorch, convert it to ONNX, and inference it using ONNX Runtime using IO Binding. Note that past state is used to get better performance."
   ]
  },
  {This file has been truncated.show originalimage1192×134 1.05 KBAn exception occurred after replacing with the converted Chinese model.image1310×257 4.52 KBHope to get help here, thank you!