What is the difference between ARMAX model and Linear regression with ARMA errors? and how does the estimation of the 2 models differ?
Please go to the blog by Rob J Hyndman
https://robjhyndman.com/hyndsight/arimax/
An ARMAX is a model of lagged dependent variable and lagged independent variable(s). On the other hand a linear regression with ARMA errors is linear regression of a dependent variable on independent variable(s) such that the errors (or residuals) are observed to follow an ARMA model. ARMAX models are time series models and are estimated using time series approaches. However the linear regression model is not a time series model and be estimated using regression approach after which an ARMA model can be fitted to the residuals.
Check out the book by Douglas Montgomery, "Regrression Analysis", he talks about autocorrelated errors in regression models. I agree with Ette on the modeling differences. In a regression modeling situation, you want to validate the model assumptions. If the residuals turn out autocorrelated, that means there are some unmodeled dynamics. The same situation may apply to ARMAX models. If the white noise test on the residuals fails, then you need to increase the model order and/or update the model and re-do the analysis.
Residuals in the regression model (that might be assumed as errors) technically still hold valuable observed data pieces (either in a weak form or uncoherent with the model assumptions (model order and level of sensitivity) as much as considered in your regression operation). So, this residual may be revaluated through a feedback with a higher model order or with new criteria (including/expending transfer function parameters/properties), which are to be validated corresponding to recorded data.
Note that while the models are quite similar in structure, they can be very different in their behavior. The principle difference is in what is persistent, assuming high positive and large autoregressvie coefficient. In ARMAX model (i.e. in model with lagged dependent varaible) the dependent variable itself is persistent, i.e. relatively slow moving. In regression with ARMA errors it is the errors (i.e. shocks) that are persistent, but the dependent vartaible itself is not slow moving. This can make large difference in forecasting. For example, ARMAX is typically not very suitable for financial variables like interest rates, since these can change a lot from period to period. Instead, regression with ARMA errors can capture the residual persistence, if it is present, while allowing rapid changes in dependent variable.
Additional related differce is the effect of exogenous variables. In ARMAX the effect of change in X will be propagated over time through the persistence of dependent variable. Meanwhile, in regerssion with ARMA errors the effect will be felt only in given period, unless of course you have other lags of X, leading to ADL model.
- Badges
- Science topic
- Similar topics
- Academic Journals