In a survey, if I articulate all the questions, not using an already validated and widely used survey, how can I advocate validity and reliability? What am I supposed to do apart form making a pilot survey.
Hello Ali,
Why don't you calculate Cronbach's alpha for reliability (very easy on SPSS) and a score above .7 is really good and .6 is acceptable. If Cronbach's alpha for internal consistency is not acceptable (say .5 or lower) you may remove item by item and recalculate again as one of the items used may dramatically decrease alpha score (especially if number of items is not substantial).
This explanation pertains to reliability (internal consistency) only.
All the best
Yes, you should considering using Cronbach's alpha. Because there have been so many questions related to this topic, I have collected a set of relevant resources:
https://www.researchgate.net/post/How_can_one_help_people_create_simple_scales_from_Likert-scored_items
Ali, I would endorse all of the above responses - the preferred method would be to carry out a pilot study. If you really don't want to/can't do this, a somewhat risky strategy would be to run your survey with more items than you need, then run reliability analysis, identify the items which do not "work", and only score the others to produce the scores on the survey. Of course, in doing this you run the risk of not having enough "good" items and having to collect data all over again...
In writing this reply, I realised that we are all assuming that your survey has a number of scales each made up of several items - obviously if this is not the case, none of the above applies.
You also mention validity - the answer to that depends on questions such as:
- Is there another measure of the same thing that you can administer alongside your questionnaire?
- Can you predict that the items and scales should follow a particular factor structure - and hence use factor analysis to demonstrate?
- Is there any external criterion that you can measure, score, and correlate against your survey?
- Can you add some questions to the end of the survey to measure the acceptance (face validity) of the questions by your respondents?
Hope this is useful.
- Badges
- Science method
- Similar topics
- Mathematical Sciences
- Statistics
- Applied Statistics
- Survey Design