First, rather than PCA, I would run a Principal Factors Analysis with an oblique rotation (correlated factors). And before that, I would examine the correlation matrix, so if there is an obvious "structure" that is producing your two factor solution.

Hi Nadia,

Because you are using PCA it is better to call the new variables "Components" rather than "Factors".

To provide more specific suggestion, it would be helpful if you can describe the approach that you applied to identify the final number of "Components". You mentioned that this number was "two factors".

If you didn't try it, I would recommend the use the Horn's Parallel Analysis method for the purpose of identified the number of "Principal Components". Simulation studies have shown that this method tends to outperform the more traditional rule of thumb: "the number of components is the number of eigenvalues greater than 1.0".

As an alternative to your original statistical analysis plan, "PCA followed by regression analysis", you can consider Structural Equation Modeling (SEM).

Best,

Roberto

Three variables is not sufficient to run a factor analysis, and I am not sure why you would use factor analysis to combine variables. If they are strongly correlated, you could just combine them. If they are based on different metrics (not all measured on the same scale), you could standardize each one, and then combine the standard scores.

I agree with Prof. Spector. Just inspect the correlation values as the basis for combining the three variables

Thank you for the advice, it's very appreciated.

Paul E. Spector I apologise if this is a basic question, but to combine them would that simply mean calculating a mean score per participant of the 3 variables? For example, the score for attention test 1+score attention test 2+working memory task score?

I have checked the correlation values, attention test 1 significantly correlates with working memory test, but attention test 2 does not correlate with either. So it seems like I should only combine attention 1 and working memory, and have 2 variables for attention instead?

Nadia Yates-Stephenson, I'd like to help, but I'm not sure about something. Could you clarify for me/us, please, whether each of your so-called variables is composed of a number of items - and, if so, how many items for each of those variables? Or did you have only one item for each of the three variables?

If you let us know that, we might be able to help more.

Thanks.

Robert Trevethan sorry if it wasn't clear, each 'variable' is only composed of 1 item. Attention task 1 is the Sky Search from the TEA-Ch giving time per target in seconds, attention task 2 is the Walk/Don't Walk from the TEA-Ch giving the number of correct trials, and the working memory task is the backwards recall from the WMTB-C giving the number of correct trials. So they are not measured on the same unit and would require standardisation.

The Sky Search data correlates with the working memory data, the Walk/Don't Walk doesn't correlate with any of these two.

I am looking for a way to combine these 3 into a single attention score, but it appears that I might need to keep them separate as the Sky Search and Walk/Don't Walk appear to assess different types of attention. But I would like to combine the Sky Search with the working memory task if possible. Like I said it was recommended to me to conduct a factor analysis to achieve this, but this might not be the appropriate way to do it based on everyone's responses.

Nadia Yates-Stephenson, thanks for your extra information. Wow, that opens up a bit of a can of worms. Under the circumstances, I think that conducting any kind of factor analysis would be like using a machine gun to deal with a few snails.

As some folk above have recommended, I would look, therefore, at your correlations and let the data speak. If those data correlate with each other in a different way from the way you anticipated, so be it. Follow the data!

If two of your variable correlate highly with each other (say .75 or higher), I think you could well combine them in some way, and keep the noncorrelating variable as a separate variable.

But combining the two variables raises issues because one is based on number of correct answers and the other on completion time. So adding them up is like adding, well chickens and gross domestic products (or machine guns and snails - take your pick of metaphors). It just doesn't work.

I'd like to see what others think, but maybe you could convert the data on each of your correlated variables to z scores (that isn't difficult), and then add the two z scores for each participant to get a single variable for the kind of attention that they jointly represent.

I really would like to see what others thought of this idea, however. You certainly can't just add up what you currently have as raw data.

Robert Trevethan thank you so much for your response, it was incredibly helpful. I wasn't very clear but I certaintly wasn't intending to add up the raw scores, I know that will only lead to disaster and confusion!

Based off everyone's responses I will avoid factor analysis, and run a bit more analysis and standardise and combine them appropriately. Thank you all for the input