Do you mean that you are interested in predicting an outcome variable? What is the size of your sample and number of regressors?

There are various ways to look at an R squared, one is the squared correlation between observed and predicted values. Autocorrelation is an issue for inference on the parameters in a linear model (Newey–West approach gives you corrected standard erros), but it does not change how the R squared is computed.

R squared could also be computed as 1 - MSE/ Var(y), and if it is out-of-sample it can also go to -Inf if the model is really bad. I would prefer this over the other.

I personally prefer (out-of-sample) Lin's Concordance Correlation Coefficient, because if the predicted values are a linear transformation of the true values, you can get a large squared correlation but that is not a good prediction.

Stefano Nembrini Thank you for your answer. Yes I am interested in predicting an outcome variable using 100 regressors with an unbalanced panel data with 1.7M rows (451 dates and a number of individuals that changes (3800 on average)).

I like Lin's CCC, but again I am confronted Heteroscedasticity and autocorrelation in the outcome (and errors). I would want to correct the bias in case some individuals that are highly correlated are driving my measure up.

Thank you again!

I understand you could do repeated cross validation or bootstrap the training data so that the data you used for testing are less correlated with your training samples, let's say you have 50 states with 100 observations each, you can assume that correlation within state is high but between state you're fine,

the you can create a training set in order to have 40 states and use the remaining 10 for testing

How does that sound?

I already make sure that my test data is not correlated with my training data (I respect a lag for the autocorrelation to fade). Also, even though doing a cross-sectional analysis, I want to make sure my data respects chronology (I don't want to train my model on recent events to test them on early ones, that's what prevents me from trying some variance of the k-fold cross validation).

Now within my test data (after lagging it so that my estimators have no information about the test data) has Heteroscedasticity and autocorrelation.

To be more precise, I have a dependent variable that is a function of one month stock returns for one year, that I observe on a moving window of one month. There is the fact that the measure coincide in 11 values each time, and the fact that some stocks are also highly correlated. My main concern is, can I correct my measure so it is the least biased by these effect.

Thank you again Stefano!