A Worrying Analysis of Probabilistic Time-series Models for Sales
  Forecasting
  Probabilistic time-series models become popular in the forecasting field as
they help to make optimal decisions under uncertainty. Despite the growing
interest, a lack of thorough analysis hinders choosing what is worth applying
for the desired task. In this paper, we analyze the performance of three
prominent probabilistic time-series models for sales forecasting. To remove the
role of random chance in architecture's performance, we make two experimental
principles; 1) Large-scale dataset with various cross-validation sets. 2) A
standardized training and hyperparameter selection. The experimental results
show that a simple Multi-layer Perceptron and Linear Regression outperform the
probabilistic models on RMSE without any feature engineering. Overall, the
probabilistic models fail to achieve better performance on point estimation,
such as RMSE and MAPE, than comparably simple baselines. We analyze and discuss
the performances of probabilistic time-series models.
