The development of a questionnaire is a complex process and the main validation and reliability process are as follows.

• Content and face validation by experts.
• Pretesting and face validation by prospective respondents
• Pilot testing and reliability checking
• Field study -Reliability analysis; EFA and CFA using PLS-SEM or AMOS depending on sample size.

I hope it helps.

Dear sir, thanks for the response. My specific doubt was do we need to do all kind of recommended tests for one questionnaire. I mean there are some types of reliability and validity testing methods. cant we do basic methods and use questionnaire.

It depends upon the scope of your study. If the intention is to develop a psychometric tested questionnaire then the steps mentioned is imperative,

Conversely, if it is for a cross-sectional survey then: Content validation and face validation by experts, Pre-testing, Pilot testing are important for validity and reliability check. We need to perform reliability testing of the main study findings also. But you cannot generalize your findings.

The simplest and most mathematically correct would be to follow up on expert face validity assistance, then pilot the questionnaire, and analyse the piloting results using a Rasch Partial Credit model. This gives reliability and so on, but better it gives you a complete analysis of Likert scales without assuming that ordinal data can be treated as measures.

Try Quest by Adams and Khoo as it is very easy to use and does the job well. Or, get hold of the Bond and Fox book, which include software.

Good luck.

I would look at papers that have undertaken this process. That will give you a useful starting point.

I think the advice you have received so far is good advice. If the questions you will create for your questionnaire are meant to measure one or more constructs (e.g., anxiety), then you could also ask someone to do a factor analysis to look at how well each question provides a discriminating measure of that construct. If you field test more questions than you'll need for the final form of your questionnaire, the factor analysis results could be used to select questions for your final form. This would provide additional evidence of 'construct validity.'

There are many ways to calculate test reliability like: Test-Retest, Split-half, Cronbach's alpha, and internal consistence.

It is possible to use the Cronbach's alpha method, half-split, internal consistency, seismic, Test-Retest, etc.

The issue is for a questionnaire with Likert response system the data are ORDINAL, and so any arithmetic performed with the labels (1,2,3 4, for example) is totally spurious! You need to read Stevens' work on types of data: categorical, ordinal, measures, and logarithmic. The first two categories of data do not have the necessary attributes needed for arithmetic: that is, they are not measures.

IRT methods transform these data into measures, and then you can do the arithmetic for Cronbach's Alpha and such other things that you want to do.Far easier is to use a Rasch model software (Quest, ConQuest, or similar) and everything is taken care of in a simple process.

Many psychological and social 'tests' fail to use real measures, and lead to false assessment in many fields.

Good luck

brian

You can firstly use exploratory factor analysis to determine number of factors that you need to items, secondly use the confirmatory factor analysis to test the adequacy of sample to data.

But Tebba, what if the data are ORDINAL? NO FA, NO CFA or any thing else requiring rel numbers not category labels. Using the numerals (1, 2,3,4 et cetera) does NOT make them them real numbers , ie, quantities that are measures. THis is a fundamental error of your suggestion.