Loading simple csv data for time series transformer
I am following the time series forecastingblogon HF and I want to try in on my custom dataset that is a simple csv file with one column on timestamp (i.e, ‘start’) and the other column of values I want to predict (i.e, ‘target’).To use my custom dataset, I followed theexamplelinked at the end of the blog but it doesn’t seem to work. I changed my column names and added an extra column withdf['item_id'] = 'A'to match the example dataset. But, it is creating adatasetwith only 1 row. I then tried with the original dataset (given in the example) and that only created 10 rows (1 row for eachitem_id). I cannot use this with the time series notebook that assumes the dataset has multiple rows and already split into train, validation and test sets.To summarize, my question is - How do I create a HFdataset(from my very standard csv file) that can be used with the time_series notebook (from HF blog linked above)?I have been struggling with this for more than a day and cannot figure out the link that I am missing.Here is the code that I am using to createdataset:class ProcessStartField():
    ts_id = 0

    def __call__(self, data):
        data["start"] = data["start"].to_timestamp()
        self.ts_id += 1

        return data

df = pd.read_parquet('filename.parquet')
df.to_csv('filename.csv')
df = pd.read_csv('filename.csv', index_col=0, parse_dates=True)
df['item_id'] = 'A'

ds = PandasDataset.from_long_dataframe(df, target="inverter_active_power", item_id="item_id")
process_start = ProcessStartField()
list_ds = list(Map(process_start, ds))

features  = Features(
    {
        "start": Value("timestamp[s]"),
        "target": Sequence(Value("float32")),
        "item_id": Value("string"),
    }
)

dataset = Dataset.from_list(list_ds, features=features)
print(dataset)Thank you!
2023-10-30T16:19:35Z
so you do not need to use the gluonts PandasDataset if you know what the structure of the time series dataset should be.Essentially for each time-serie in your data set of time series you make a dict with the appropriate keys, namely the start-date (the first date-time of the target), target which is the array of time time series and the optional item_id which is not really used for training. So you can make this list of of dicts yourself and then you should have all you need for the blog post. let me know if that helps?