Hi Ozgur,

Theoretically you could do so, but bearing in mind that you are losing degrees of freedom by each extra loading added (dual loading) you may end up with a model that is not identified. However, looking at your data, I think it does make more sense to allow the errors to correlate (Factor1 item8 & Factor2 item8 and so on...) - you are still losing a bunch of degrees of freedom but I think it is theoretically justifiable and you can still distinguish between empirical dimensions. Hope that helps.

I aggree with Carla that it makes more sense to allow correlated errors.

Hello Özgür,

I don't think it's a good strategy to modify models according to the data. It should be vice versa: we fit our data to model. So, adding paths or correlation errors without theoretical base or plausible justifications seems to me a wrong strategy. However, in your case, because the items of F1 and F2 are parallel, it's easy for you to justify an addition of correlation between errors of Athletics items in F1 and F2 and so on, as the two previous colleagues have suggested. 

another question rising, what is the real meaning of corelation between error terms in the model and how can i interpret these corelations?

Correlated error terms are typically justified and interpreted as shared error variance between the items. IOW for example, the wording of the items is very similar except for one or two key words, which means the items may correlate because of the wording not because of the underlying aspect (the keys words part of the item) of the latent variable being measured.. Therefore the correlation between the items and the variance associated with that are due to measurement error.

What has been ignored here is the underlying theory. Factor analysis is motivated by the idea that the items are correlated because they are measuring something in common (the factors F1 and F2 here). If this hypothesis is correct, then the partial correlations among the items (the correlations among residuals) should vanish.  

Similar argument holds here for inter-correlations among residuals of similar indicators of  the two factors.

The modification indices here most likely indicate that something common among the factors are missing. Make sure that the factors are specified to be correlated.

For your model I would even before estimating it add correlated residuals like this:

F1_1 F2_1

F1_2 F2_2

F1_3 F2_3

F1_4 F2_4

(correlating residuals, not the indicators themselves)

So yes, don't be afraid of adding correlations between these residuals (don't use regression paths here, but correlated residuals; representing a common factor, not one item affecting the other). There is strong theoretical reason to assume that you will need at least some, if not all, these correlations between the residuals. A response to F1_1 is dependent both on theoretical knowledge and on knowledge of atheletics. A response to F2_1 is dependent both on assumed teaching abilities and on knowledge of atheletics. 

Adding to that: regression paths between indicators can be justified from a theoretical point of view: 

Green, Samuel B, and Scott L Hershberger. "Correlated Errors in True Score Models and Their Effect on Coefficient Alpha." Structural equation modeling 7, no. 2 (2000): 251-270.

The answer to one item may have an effect on the next item (context effects within the questionnaire). The paper by Green and Hershberger is an excellent criticism of Cronbach's alpha, but I don't use such regression paths between factor indicators any longer. Several alternative strategies exist, but these depend on your data, the number of items and your theory. 

Generally, several strategies are possible to make factor models fit, including parcelling sets of items to reduce the number of indicators (and still have degrees of freedom, high realiability, and validity). But when you have only four items for a factor, you can forget parcelling of set of items. While this may not be so in your case, modification indices may suggest correlated residuals within such a simple factor model. 

Then adding correlated residuals based on exploratory analyes is controversial and can be problematic. One option even for this situation would be in advance to look at items and consider: for which items would I theoretically justify correlated residuals, and for which would I not theoretically justify correlated residuals? And then run the model, and inspect modification indices if the CFA model fails to fit. 

Adding one correlation between residuals in a four indicator factor analysis with one factor has a strong impact on degrees of freedom and thus makes your model more likely to be "supported" (not rejected) by data, even if it is wrong. And it might affect your estimated factor, to some extent. If it doesn't, I would probably defend adding the correlation between residuals, but that depends. 

For pragmatic reasons I sometimes include a correlation between residual variables in a factor analysis. But ideally you would want not to do that, unless you thought in advance you should add this correlation (e.g. representing an obvious method effect for two items, which is the case in your data and your original question).  

(A better solution can be to have several factors included in the model, and allow for cross-loadings, in particular if one uses Bayesian estimations where it is possible to allow for only a tiny "wiggle room" for crossloadings. But that was really not your question...)

Good luck!

Christopher 

Another solution instead to correlating the residuals would be a bi-factor model because items with similar wording share variance and therefore this method bias should be controlled in this way.