title: ValueError when pre-training BERT model using Trainer API
body: <p>I'm trying to fine-tune/pre-train an existing BERT model for sentiment analysis by using Trainer API in <code>transformers</code> library. My training dataset looks like this:</p>
  <pre><code>Text                             Sentiment
  This was good place                  1
  This was bad place                   0
  </code></pre>
  <p>My goal is to be able to classify sentiments as positive/negative. And here is my code:</p>
  <pre><code>from datasets import load_dataset
  from datasets import load_dataset_builder
  import datasets
  import transformers
  from transformers import TrainingArguments
  from transformers import Trainer
  
  dataset = load_dataset('csv', data_files='my_data.csv', sep=';')
  tokenizer = transformers.BertTokenizer.from_pretrained(&quot;bert-base-cased&quot;)
  model = transformers.BertForMaskedLM.from_pretrained(&quot;bert-base-cased&quot;) 
  print(dataset)
  def tokenize_function(examples):
      return tokenizer(examples[&quot;Text&quot;], examples[&quot;Sentiment&quot;], truncation=True)
  
  tokenized_datasets = dataset.map(tokenize_function, batched=True)
  training_args = TrainingArguments(&quot;test_trainer&quot;)
  trainer = Trainer(
      model=model, args=training_args, train_dataset=tokenized_datasets
  )
  trainer.train()
  </code></pre>
  <p>This throws error message:</p>
  <pre><code>ValueError: text input must of type `str` (single example), `List[str]` (batch or single pretokenized example) or `List[List[str]]` (batch of pretokenized examples).
  </code></pre>
  <p>What I'm doing wrong? Any advise is highly appreciated.</p>