All of these models seem to be used in predicting an endogenous time series variable, using several lagged exogenous time series variables. If it is so, how do we decide when to use which?

  • $\begingroup$ Related post with more details can be found here. $\endgroup$ – Richard Hardy Jul 26 '16 at 17:22

I will focus on ARMAX versus VAR. I am not quite sure what a dynamic regression is. (I have seen a few different interpretations. Funnily, there are textbooks and lecture notes with chapters called "Dynamic regression" that do not really delimit this class of models. Also, Rob J. Hyndman notes in his blog post "The ARIMAX model muddle" that different books use that term for different models).

An ARMAX model has the form

$$ y_t = \beta x_t + \varphi_1 y_{t-1} + \dotsc + \varphi_p y_{t-p} + \varepsilon_t + \theta_1 \varepsilon_{t-1} + \dotsc + \theta_q \varepsilon_{t-q} $$

(one could also have more than one exogenous variable and/or lags of exogenous variables in the above equation.)

  1. The dependent variable is a univariate time series.
  2. The model cannot be used for forecasting $y_{t+h}$ unless one has the future values of the independent variable $x_{t+h}$ available, or has a separate model for predicting $x_{t+h}$.
  3. The model is estimated using maximum likelihood (slow), often using a state space representation.
  4. Allowing for both AR and MA terms offers a parsimonious representation of the process.

A VAR model has the form

$$ z_t = \varphi_1 z_{t-1} + \dotsc + \varphi_p z_{t-p} + \varepsilon_t $$

where $z$ is a vector; for example, $z=(y,x)'$.

  1. The dependent variable is a multivariate time series.
  2. The model can be used for forecasting all components of $z_{t+h}$, e.g. for $z=(y,x)'$. Given data up to and including time $t$, forecasts for time $t+1$ are straightforward to obtain; forecasts for $t+h$ where $h>1$ can be obtained iteratively.
  3. The model can be estimated using OLS or GLS (fast).
  4. Lack of MA terms may (or may not) require large AR order to approximate the process well, and large AR order means a large number of parameters to be estimated and thus high estimation variance. Fortunately, regularization (shrinkage) applies pretty straightforwardly to VAR models (unlike ARMAX), so the variance can be tamed.

[H]ow do we decide when to use which[?]

It depends on your intentions and the data at hand.

  • If you need fast estimation and direct applicability to forecasting, try a VAR.
  • If you need a parsimonious representation, try ARMAX.

Also, ARMAX and VAR could be combined to obtain the VARIMAX model that has a multivariate dependent variable, does allow for forecasting of all of its components but also takes a long time to estimate, is prone to convergence problems and is difficult to regularize.

  • $\begingroup$ My answer is intended not only to answer the general question but also to address some peculiarities of this related post. $\endgroup$ – Richard Hardy Jul 26 '16 at 17:23
  • $\begingroup$ where are the possible lags on x in your specification ? $\endgroup$ – IrishStat Jul 26 '16 at 18:57
  • $\begingroup$ The ARMAX time series model is a dynamic regression model such that the response variable is a function of exogenous inputs and their approproate lags as well as lagged values of 𝑌𝑡 and lagged values of disturbance terms. Its form for a single exogenous predictor is (1 − 𝜙1𝐵 − 𝜙2𝐵2 − ⋯ − 𝜙𝑝𝐵𝑝)(𝑌𝑡 − ∑𝜔𝑖𝑋𝑡−𝑖𝑚𝑖=0 ) = (1 − 𝜃1𝐵 − 𝜃2𝐵 2 − ⋯ − 𝜃𝑞𝐵𝑞)𝜖� $\endgroup$ – IrishStat Jul 26 '16 at 19:08
  • $\begingroup$ @IrishStat, I had originally included one exogenous regressor for brevity and simplicity of exposition. I have now made an explicit remark that this is just for simplicity and that of course more regressors (and their lags) could be included. Would you consider retracting the downvote? $\endgroup$ – Richard Hardy Jul 26 '16 at 19:43
  • $\begingroup$ the limiting of the 1 exogenous regressor to 1 lag is what I am concerned about. you could have 1 exogenous regressor with k lags. Your illustrative example suggests that an ARMAX model only has 1 exogenous variable with only 1 lag and thus may lead to a serious misunderstanding. DONE ! Your edit is great !. $\endgroup$ – IrishStat Jul 26 '16 at 19:51

ARMAX models are more general that VAR and Dynamic Regressive . You can see ARMAX in action here https://twitter.com/tomdireill/status/717694028104474625 where the GASX/GASY example from the Box-Jenkins text is analyzed/

  • $\begingroup$ How would you define a dynamic regression? In some literature, ARMAX is discussed under the umbrella of Dynamic regression, making the former a subset rather than a superset of the latter. $\endgroup$ – Richard Hardy Jul 26 '16 at 17:09
  • $\begingroup$ To me a dynamic regression is a transfer function with no structure on the error term. yt=βxt+φ1yt−1+…+φpyt−p+ βzt+φ1zt−1+…+φpzt−p1+et .. $\endgroup$ – IrishStat Jul 26 '16 at 18:50
  • $\begingroup$ Your answer does not address the original question, namely, "when to use which", let alone answering the question in the title of the post. Also, the description of the dynamic-regression tag (extra to some textbooks and lecture notes) indicates that dynamic regression is more general than ARMAX. Therefore, your first sentence is at least disputable (and probably incorrect). $\endgroup$ – Richard Hardy Jul 26 '16 at 19:37
  • $\begingroup$ Upon reflection the dynamic regression model is usually less parsimonious thus it is the more general in that regard as it may include coefficents that are functions of other coefficients due to the common factor effect.. $\endgroup$ – IrishStat Jul 26 '16 at 19:58

Your Answer

