KeyError: Field ".." does not exist in table schema
Hi everyone! I’m trying to run therun_ner.pyscript to perform a NER task on a custom dataset. The dataset was originally composed by 3 tsv files that I converted in csv files in order to run that script. Unfortunately, I got this error:Traceback (most recent call last):File “C:\Users\User\Desktop\NLP\run_ner.py”, line 578, inmain()File “C:\Users\User\Desktop\NLP\run_ner.py”, line 262, in mainraw_datasets = load_dataset(extension, data_files=data_files, cache_dir=model_args.cache_dir)File “C:\Users\User\Desktop\NLP\myenv\lib\site-packages\datasets\load.py”, line 1664, in load_datasetbuilder_instance.download_and_prepare(File “C:\Users\User\Desktop\NLP\myenv\lib\site-packages\datasets\builder.py”, line 593, in download_and_prepareself._download_and_prepare(File “C:\Users\User\Desktop\NLP\myenv\lib\site-packages\datasets\builder.py”, line 681, in _download_and_prepareself._prepare_split(split_generator, **prepare_split_kwargs)File “C:\Users\User\Desktop\NLP\myenv\lib\site-packages\datasets\builder.py”, line 1136, in _prepare_splitwriter.write_table(table)File “C:\Users\User\Desktop\NLP\myenv\lib\site-packages\datasets\arrow_writer.py”, line 454, in write_tablepa_table = pa.Table.from_arrays([pa_table[name] for name in self._schema.names], schema=self._schema)File “C:\Users\User\Desktop\NLP\myenv\lib\site-packages\datasets\arrow_writer.py”, line 454, inpa_table = pa.Table.from_arrays([pa_table[name] for name in self._schema.names], schema=self._schema)File “pyarrow\table.pxi”, line 1339, in pyarrow.lib.Table.getitemFile “pyarrow\table.pxi”, line 1900, in pyarrow.lib.Table.columnFile “pyarrow\table.pxi”, line 1875, in pyarrow.lib.Table._ensure_integer_indexKeyError: ‘Field “Il” does not exist in table schema’The head of the train csv is like:I think the Field “Il” to which it refers in the KeyError is the first row of the train_labeled.csvThe command tha I’m running is:python run_ner.py --model_name_or_path  Musixmatch/umberto-commoncrawl-cased-v1 --tokenizer_name  Musixmatch/umberto-commoncrawl-cased-v1 --train_file train_labeled.csv  --validation_file devel_labeled.csv  --test_file test_labeled.csv --output_dire umberto-ner --do_train --do_eval --do_predictCan someone help me with this issue? Thanks!
2021-12-02T16:07:32Z
I solved the problem, the csv files generated with pandas needed a post processing in Excel: words and labels had to be in two separated columns.They were like:col Aword,labelThey have to be:col A        col Bword        label