I am no expert on short versions of questionnaires, but since you stated that it is an established one, I would use CFA on the full set of items and perhaps start by removing those with the lowest factor loading and see what happens to the fit indices.

Hello Himani,

If your purpose is to create a short form of an established measure, there are various approaches you can elect to take. You could collect responses to the measure from your target population, then consider:

1. Deleting items to maximize internal consistency score reliability (for a given target length);

2. Deleting items to maximize score correlation with some plausible, external variable (e.g., to maximize convergent validity).

3. Deleting items to maximize average shared variance between items and factors, by dropping items with lower loadings.

There are problems with these approaches:

1. You're likely to overfit the resultant short form, so that it appears to "work" well with the data you've collected. However, that's no guarantee that you'll see similarly desirable results with another sample (so, validation via additional data is a must).

2. Maximizing score reliability may change the criterion-related validity of the measure, or its subscores/factors. The same is true for attempting to maximize subscore/factor validity, by dropping items with lower item-factor loadings.

3. Maximizing either score validity or reliability via selective jettisoning of items may alter the qualitative nature of the subscores/factors from what was originally intended by the developers.

But, it's your choice. Just be clear about what you're doing and why you're doing it, and collect sufficient data to allow for validation of your abbreviated version of the measure. I'd use CFA as the validation step.

Good luck with your work.

I am going to take a different tact than Marcel Grieger and David Morse . Not that they are wrong (I agree with them and recommended their posts), just I am giving a different emphasis, and I am suggesting considering the following before doing the statistics (and preferable before running the pilot, because you might have been able to eliminate some before that).

I would start by looking at the questionnaire, thinking about if all items are appropriate for the sample that you want your short version to be used for (not necessarily your sample, as I think the above commentators have assumed) and for your research purposes. Even scales which claim to be unidimensional often have several categories that they attempt to cover. For example, if the scale is 20 items on healthy life style and there are 5 items each on foods eaten, exercise, sleep, and avoiding stress, if you eliminate 5 items and they are all on stress you have changed the meaning of the scale considerable. At some point later on you may also be informed by your pilot study, but how much this informs how you shorten the scale will depend on characteristics of the pilot study.

Dear David Morse , Thank you so much Sir for your valuable insights. I intend not to create a short form of established questionnaire , but to delete the certain items from my questionnaire which might not be appropriate for my sample. My questionnaire consists of 120 items , and I have collected data from 150 respondents approx from the sample chosen. However I wish to delete some items from this 120 item questionnaire so as to make it a little concise for the respondents whom I ll be circulating questionnaire in coming few months. So my query is CFA is apt in such case as well? What should I do in such case ?

My take - EFA because that's the purpose of exploration. Another approach might be to externally "validate" your factors with canonical correlations. As always, check R-squares and significance. Good luck.

For deletion and retention of items in the questionnaire you can opt for IRT analysis. CFA and EFA are limited to construct validity whereas IRT offers various paradigms related to each item that helps the researcher in informed decision making.