Camila Zugarramurdi, when I read your question, I immediately wonder whether you have done the right things as a foundation to your factor analysis. For example, if you have 163 items, you should probably have at least 1,300 participants. Do you have that?

Did you check the KMO index and Bartlett's test of sphericity? If so, were the results satisfactory?

Also I would be surprised if parallel analysis indicated that you had as many as 16 factors in your data.

If you answer these questions, and perhaps even attach the results of your parallel analysis, or even show me just the initial output of your factor analysis showing the number of eigenvalues from the "Total Variance Explained" output, I could probably help you more. You could send that output as an attachment to a post on RG.

But, apart from that, I am troubled by the foundations of your research, i.e., feeding correct and incorrect answers into a factor analysis. I don't feel confident to comment on that, so maybe others can help.

Even if you have the large number of participants that Robert Trevethan mentions, 16 latent variables for an EXPLORATORY analysis is a lot, and would be difficult to interpret. Are the manifest variables able to be grouped based on any a priori reasoning? Also, what are the tasks like?

Hello Camila,

The previous responses (by Robert Trevethan and Daniel Wright ) each include some good observations. To these, I'd like to add my two cents' worth:

1. If we knew more about the specific items (163!) and their intended measurement goals, we might be better poised to offer constructive advice. Perhaps if you could elaborate a bit?

2. The coding of item responses sounds to me as if it's introducing an artifact into the data set. As an example, if the items that offer 2, 3, or 4 syllables as response options, are you coding the response as 2, 3, or 4? Or as 1 = correct identification; 0 = incorrect identification? If the former, your data simply reflect (what I presume might be interpreted as) word complexity, and not respondent accuracy in parsing words. Ditto for the "yes/no" option items.

3. Parallel analysis (in the classic sense, that of generating random numbers for the target number of variates and cases in your sample) will be biased towards more "factors" as the number of variables increases. I agree with Daniel that, in all likelihood, 16 factors sounds like an overly complex structure.

4. A different tack might be to approach this not as a factoring problem but as a scale building/identification problem. For this, item response theory models would likely be quite helpful. Yes, dimensionality is important in IRT, but the real purpose of the measurement is joint location of items and respondents on a common scale.

Good luck with your work.

Dear Camila Zugarramurdi ,

EFA may not be the best technique when you have correct and incorrect answers. It works better with opinions. You may only end up creating models of misconceptions and preconceptions about the use of the words "yes" and "no" in responses.

One of the issues with IRT, is if you actually believe there are around 16 dimensions, it likely will have computational problems. The way people usually use IRT and exploratory factor analysis are really similar (both members of the general linear latent variable family, see Book Latent Variable Models and Factor Analysis: A Unified Approach

It would be highly recommended that you can divide into composite factors according to your theoretical model.

Wow, thanks for all the insightful answers, I'll go one by one:

Robert Trevethan

1. I have 600 participants, what would you suggest regarding the items/participants ratio?

2. KMO MSA = 0.73, Bartlett p < 0.001

3. I'll attach the eigenvalues for fa and pc parallel analysis as a response

4. I've seen it's more frequent to perform FA with averages but I was interested in item-level descriptives

Thanks Robert!

Daniel Wright

What's very surprising to me is that they actually ARE easily interpretable based on the manifest variables: factors load on the different 8 tasks, plus in some tasks there's more than one factor, as with the case where the underlying structure of the items is 2, 3 and 4 syllables (so, 3 factors) or where the reponse options are yes/no (2 factors). The tasks are tablet based: for example: the child listens to a word and has to answer in how many syllables it can be divided.

David Morse

So, there's 8 tasks which is 4 tasks x 2 linguistic levels: segmentation, blending, isolation, and rhyming, each of wich has syllable and phoneme level items (except for rhyming, which has words or pseudowords). Number of items are as follow:

segmentation syll = 22, segmentation phon = 28, blending syll =18, blending phon = 16, isolation syll = 27 , isolaton phon = 32, rhyme w = 10, rhyme pw = 10

For blending and segmentantion. They all measure the construct of "phonological awareness" that has been described to develop from larger to smaller linguistic units (say, from words > syllables > phonemes) and to develop earlier for tasks such as identifying sounds (is b present in the word bat?) than manipulating them (remove b from bat). It's a longitudinal battery, so we needed to make sure we'd cover the whole developmental range across time points. That's why we have so many items. Also, since it's all receptive (no verbal responses), for tasks such as isolation (do this 2 words start with the same sound?) we needed as many "yes" as "no" answers.

2. I am coding all responses as 0 = incorrect or 1 = correct, no info on item characteristics is present in the dataframe. Which is why I am particularly suprised that the factor solution "discovers" the underlying structure.

3. Can I retain lees factors on the basis of that argument? For example, the 8 factor solution, which is suggested by BIC

4. Yes, we were thinking about following with IRT, but since it assumes unidimensionality of the construct, EFA was meant to check that. We are still thinking that it might be a 2-level structure, with all this different factors loading into a single latent "phonological awareness" construct.

Thanks!

Peter Samuels

Thanks, do you suggest any other approach? I was just checking the number of "yes"/"no" answers irrespective of the correct response and it seems that they are effectively biased. This however does not mean that all children are responding randomly, since distribution of average scores show a a wide range, some of wich perform well above chance.

Daniel Wright and Cristian Ramos-Vera I might try then to composite and proceed to IRT.

Thanks for all the answers!

Camila Zugarramurdi, thanks for your response(s) and the attachment. Just quickly, your parallel analysis looks OK on the surface, but your first few eigenvalues are so high that I wonder what's going on.

I'm not in a good position (i.e., I don't have enough knowledge) to work out whether you should be using exploratory factor analysis (EFA) to address your research interests, but, as I mentioned in my initial answer, I'm suspicious. In essence, I'm not sure that EFA is what you want for seeing what is happening in relation to sets of correct versus incorrect (i.e., dichotomous) answers on a test.

I hope that other people can help you.

Dear Camilla, I would suggest simplifying your model and corresponding item structure.

Hi Robert Trevethan I believe EFA is also appropriate for dichotomous variables. Still, it might make sense to go straight to CFA and base the number of factors in theoretical terms, and test the 8-factor structure against for example a 1 factor structure or a 8 factor with 2 levels. In line with Talat Waseem response I guess.

Thanks again

Hello, you can do what you suggest in case you don't have any previous theoretical model or background to compare CFA models

Camila Zugarramurdi, you could well be correct about EFA being appropriate for dichotomous variables. However, I'm still not confident that EFA is suitable for what you want to do. But the emphasis is on "not confident" - I simply don't know.

I'm not sure that conducting a series of CFAs is appropriate. CFA requires a pretty clear notion at the start about what the outcome should look like. It's not really intended to be a fishing kind of activity.

If you solve your problem, I'd be interested to hear how you did it. In the meantime, all the best!

Hello Camila,

What a peculiar beast this battery is! But, I'm still unclear as to what the main focus of the analysis is intended to accomplish.

If the question was, do the eight subtasks all measure a common latent variable, then the appropriate factoring would appear to be to: (a) construct total correct scores for each of the 8 combinations; then (b) factor those eight scores (arguably, via confirmatory factor analysis since all are considered to be influenced by a single common factor). Using summated/number right scores would likely make the data more conformable to the continuous nature of variables for which factor analysis is best suited.

If the question was, does an item-level factor structure suggest correspondence with item characteristics (e.g., answer option type), then you could: (a) classify all items by however many characteristics language acquisition investigators would deem salient; then (b) as you have apparently done, factor the item-level data, then (c) look at the proportion of items that are grouped into the same subsets by both methods (higher would indicate that item characteristics are the key feature for how items affiliate, which would suggest method variance abounds). With dichotomous items/variables, tetrachoric correlations would likely be better than ordinary Pearson correlations for the estimated relationships among the 163 items. (I have to say it; I'm a wee bit skeptical of the apparent sorting out of items to match item response format and correct answer option.)

If the question was, from a normative/developmental perspective, whether the items (say, within the set of segmentation task / syllables as target) form a scale of challenge that is consistent across examinees, then IRT would seem to be the best option.

When you get this sorted out and published, let us know...I'd be interested in seeing the final result.

Good luck with your work.

A couple of points.

1. IRT does not assume unidimensionality, unless you tell it to (which lots of the time people do). The mirt software ( https://cran.r-project.org/web/packages/mirt/mirt.pdf ) is for multidimensional IRT.

2. The Exploratory v Confirmatory is different from the IRT v "FA" distinction. Both of these have confirmatory and exploratory ways of doing them. All are part of the general linear latent variable family of models.

2a. I put FA in quotes since sometimes it is used to mean a general latent variable approach, and sometimes a very specific model with metric manifest and latent variables.

Hi David Morse, crytal clear: now reading your response it seems to me that I'm trying to accomplish all of those at once. And, I'm doomed.

I'm also skeptical about the sorting of items according to response options/item properties, what I cannot find is a solid alternative explanation for this emerging pattern.

I'll try to order things a bit and attain one objective at a time.

BTW: I am already using tetrachoric correlations, thanks.

Daniel Wright

Great, I'll look into that (I'm using mirt)