Dear Himal Kandel

I would like to highlight the following aspects of your possible research and how to estimate your possible number of items. At first, you should choice either to apply reflective or formative items. The majority of the studies apply reflective items in their studies. And if you want to read more about them and their meaning, here is a link to a question in this context.

https://www.researchgate.net/post/What_are_the_differences_between_Formative_and_Reflective_model

Following your main question, the I would suggest you to think of the following points:

1. The minimum number of items per (factor/construct) is 3 items.

2. In reflective measured (factor/construct), you can (drop/eleminate) poorly loaded (items/factors) since the items are designed to mean the same and be highly correlated to each other. Therefore, keep in mind that you might face the fact that you need to drop some items.

3. Sometimes, you can came across researches where some of the constructs were measured by 2 items, however, it is not recommended to use 2 because, the model is very likely to have issues of either identification or estimation and thus you might end up with a fact that you need to drop the (factor/construct), which is a serious issue if it is the main part of the study.

4. Using single item should be totally avoided since the validity and reliability of that (factor/construct) will always questionable.

My suggestions are stated in the following part:

1. Use at minimum 4 items per (factor/construct).

2. If you are using a new (factor/construct), then I would recommend that you use more items e.g. 6 items.

3. Run a pilot test to identify any validity (or/and) reliability (or/and) multicolinearly issue. So you can adapt the model and the questionnaire.

4. Regarding the sample size, it usually based on the number of items used in model. A rule of thumb in this context usually recommends that about (10-20) responses per items. So 20 items, requires about 200 or 400 as minimum sample size.

Here are questions regarding the sample size and where you can find some other thoughts because there is no concrete answer about this.

https://www.researchgate.net/post/I_have_4_latent_variables_and_each_of_which_has_7_observed_variables_in_SEM_how_this_affects_the_parsimony_of_the_model

https://www.researchgate.net/post/When_conducting_SEM_in_clinical_psychology_research_What_is_the_sample_size_for_structural_equation_modeling

Finally, I would like to pay your attention that there should be a compromise between the model complexity and the questionnaire. Because at the end of the day, you do not want to build a very sophisticated model that has so many items thus it will be hard to motivate peoepl to fill the questionnaire.

Hello Himal,

The answer depends on several considerations, one of which is, how many possible score categories or levels do you seek for your scale? Imagine a scale composed of one item. This will yield two possible score categories: Those who answer the item correctly or endorse the item vs. those who do not. That's not a lot of differentiation power for the scale!

With two items, you get a maximum of four score categories or levels. Benjamin Wright (the U of Chicago professor who championed the Rasch model from its early days) wrote a little piece about a two-item scale/system as the "shortest useful test", which you may find interesting: https://www.rasch.org/rmt/rmt122b.htm

In any event, more items in the scale (assuming they all contribute to the quality of the measurement) means better ability to locate both persons and items on the common scale. And, as far as person differentiation is concerned, more items means better potential differentiation. As well, more items means more precise location of persons on the scale (smaller standard error, all other things equal).

The practical impact of many items is that it's often hard to create lots of items that all work well in concert and all reflect (only) the construct or attribute of interest.

Good luck with your work.

Dear Wolfgang Lenhard , Belal Edries and David Morse, Thank you for your insights and for pointing out to the useful references. I have now edited my question and added some context to it. "I have patient-reported outcome data on two items (each with the same 5-point response categories) completed by 300 people with a disease condition, and I was interested to find out if these two items formed a valid measure. I am currently exploring the ways I could analyse the data. I have some experience with Rasch analysis." I know this is not an ideal situation but the measure seems to work okay other than the low PSI (1.55)! Thanks again.

Hi Himal,

You can to my understanding perform Rasch with 1 or 2 items. But I am not sure whether the sample you have is large enough. If you are trying just for a look see which item gives more information you can do that. Happy Monday

Two dichotomous items are the minimum requirement for Rasch analysis. In the case of your two polytomous items, you have the equivalent of many more. How many items do you need? What level of person separation are you willing to accept? Is your analysis for diagnostic purposes, streaming purposes, to differentiate treatment, or merely descriptive? Are the consequences of the decisions that you will make about patients high-stakes or low stakes?

I should write a book about this.

collegially

TGB

Dear Himal, I agree with Trevor that you can perform Rasch on two items. Two is not much no, but you simply do not have more... I would say that a Rasch analysis would at least increase your measurement precision tot a level higher than most other analyses will. This is quite a pragmatic viewpoint of course. I also believe 300 persons is not small, certainly because the number of items is also small. Large numbers of items require larger samples. But, echoing Trevor, what are your goals for all this?

Hi All, I really appreciate all your feedback and great to see that this question has received so much attention. I did run Rasch analysis on the data. The person separation index (PSI) was low (1.55) while other parameters were satisfactory. This suggested that the items could work as a screening tool than as a proper measure. Thank you!

A screening tool, as long as you are expecting to separate the extreme top scorers on the extreme bottom scores. It won’t tell you anything that most of the people in the middle.