Fine-tung gpt-2 run_clm.py stops early
Hi, I’m running run_clm.py to fine-tune gpt-2 form the huggingface library, following the language_modeling example:!python run_clm.py \
    --model_name_or_path gpt2 \
    --train_file train.txt \
    --validation_file test.txt \
    --do_train \
    --do_eval \
    --output_dir /tmp/test-clmThis is the output, the process seemed to be started but there was the^Cappeared to stop the process:The following columns in the training set don't have a corresponding argument in `GPT2LMHeadModel.forward` and have been ignored: .
The following columns in the evaluation set don't have a corresponding argument in `GPT2LMHeadModel.forward` and have been ignored: .
***** Running training *****
  Num examples = 2318
  Num Epochs = 3
  Instantaneous batch size per device = 8
  Total train batch size (w. parallel, distributed & accumulation) = 8
  Gradient Accumulation steps = 1
  Total optimization steps = 870
  0% 0/870 [00:00<?, ?it/s]^CHere’s my environment info:transformers version: 3.4.0Platform: Linux-4.19.112±x86_64-with-Ubuntu-18.04-bionicPython version: 3.6.9Tensorflow version: 1.14Using GPU in script?: yesWhat would be the possible triggers of the early stopping?
2020-11-02T11:42:34Z
Hi, if you are running Tensorflow, don’t you need TFGPT2LMHeadModel rather than GPT2LMHeadModel ?
2020-11-02T13:07:07Z
The ^C means you hit Control+C, which cancels any process.
2020-11-02T20:08:40Z
@rgwatwormhillThank you for your response, I will try importing the different model.@sguggerThank you for your response, the ^C actually appeared by itself without hitting control + C
2020-11-03T09:51:09Z
Hi@kelichiu,another thought:  you might need Tensorflow version 2 instead of version 1.14.  See this pagehuggingface.coInstallation — transformers 3.4.0 documentation