I highly recommend you to study ARMA models using the textbook entitled "Introductory Econometrics for Finance" by Chris Brooks.

Brooks, C., 2002, Introductory Econometrics for Finance, 2nd edition, Cambridge University Press.

You’ll find it very easy to follow.  Sample instructions and output from the popular computer package EViews will enable you to implement models themselves and understand how to interpret results

http://rum.prf.jcu.cz/public/mecirova/eng_ekonomka/Chris_Brooks_Introductory_Econometrics_for_Fina.pdf

Thank you for your answer Amira,

What I am trying to do here is in between a ARMA and a VAR (Vector Autoregressive). I would like to know if it is possible to explain a variable by itself and by an other variable. For example, can I run the regression: X = a*X(-1) + MA(1) + b*Y with X and Y being variables ?

A simpler solution is to estimate the model as an AR(2).  Your model will become X = a1*X(-1) + a2*X(-2) + b*Y.  This will likely eliminate the autocorrelated errors.

I see that you are using first differences of your variables in your regression. Is this because your variables are non stationary?  If they are cointegrated you are losing long run information in your regression. If you are happy that there is no long run relationship between the variables why not use sufficient lags of both X an Y to make your errors well behaved. Should you be looking at a VAR in first differences. It is difficult to give more advice without a more detailed knowledge of the nature of your analysis.

Thank you all for your responses,

@David Hudgins your advice on using an AR (2) in addition to an AR (1) does solve my autocorrelation problem in some cases!

@John C Frain I am indeed using first difference to have stationary series. My analysis is about relationships between macroeconomic variables. For example, I want to explain unemployment by GDP in European countries. It is difficult to find accurate models for countries like Spain that experienced severe changes since the crisis. Note that I am using quarterly seasonally adjusted data over 30 years.

In some cases introducing a moving average in the regression improves R square and corrects serial correlation, but I wander if this transgresses the exogenous assumption of OLS.