I am framing a study regarding sleep abnormality of OPD patients in our hospital. A questionnaire is adopted from a website of Indian institute! Should I do a pilot study to have the reliability of that questionnaire! If yes, how I can set for it? I myself or my colleagues have zero practical experience of it!
Go to my website, then favourite links and there is an electronic stats book, search for cronbach's alpha which is spelt I think differently to what you've spelt it. Good luck with your study. Deborah Hilton Statistics Online
http://sites.google.com/site/deborahhilton/
Thanks sir, But in the link I found some Mathematical discussion of cronbach's alfa so far! I need some assistance regarding applied facts with my attached questionnaire!
A cronbach's alpha is simply the reliability of a scale. Cronbachs α depends on the number of items in a scale, the average covariance between the items and the spread of the sumscore. Obviously you can try to do some form of pilot, but that is time consuming and not always useful.
The real question in regard to your questionnaire is: is it useful to calculate an alpha score? Alpha score is a way to test if multiple items are correlated enough to form a reliable scale. So for example, if you're interested in someones attitude towards a product you might ask the questions:
are you interested in the product?
do you find the product attractive?
do you think the product is useful?
If people answer on a scale of 1 to 7. You might want to check if the three questions put together form one reliable measure: attitude towards product.
Your questionnaire has single measure (binary) items. A cronbachs alpha requires at least two seperate questions you want to put together, and even with two items it does not really do all that much. Even if you combine two questions in your questionnaire, the results are binary so what is its meaning? Can you really say that if someone answers yes to the one question and no to the other that the items are related or not? How do you interpret it? My guess is: you can't. For a good cronbachs alpha, use multi item scales/indexes with a scale of at least 5 points.
Hello Rajat, I take your point that you are most interested in applied survey research. However, I would suggest that you think about validity issues before you start thinking about reliability. Validity refers to whether you are measuring/assessing what you really want to know. Your question does not say whether you have a set of research questions that you are attempting to address. Those are applied issues. Why are we doing this study? What do we want to know?
What we are usually looking for in applied studies are patterns among the responses, to allow us to describe the population, refine treatments/programs, or draw some conclusions.
Our Dutch friend gives some good advice, both about dichotomous questions (yes/no), and about the value of scales and indexes. Another word of warning about the open-ended questions in the questionnaire you have.
Questions such as "how many times" or "how many minutes" will give you a range of responses that will be enormously difficult to categorize, compare and analyze. Think in terms of "0 to 1", "2-3" "4-5" and "more than five" for example. Or for time/minutes to fall asleep, "less than 10" "11 to 20" "21 to 30" and "more than 30".
Also, think about some demographic data, sex and age at a minimum. Thus, do men have more problems than women? Do older people have more problems than younger ones?
You didn't say what Indian Institute supplied the questionnaire. Their reasons for using their instrument may be different than yours; and moreover, it's not a very well constructed instrument. You may want to think about using only a few relevant questions, or rewriting them to suit your specific purposes.
Finally, let me suggest you contact some faculty at Chittagong University. They have departments of public administration and sociology, and I will bet some faculty with experience in survey research.
Thanks Sir for critical explanations and providing me some interesting Insight!
Hello Rajat
You need not do a pilot study but you have validate the questionnaire in a sample of few representative respondents.
Validating refers to testing whether or not the respondent have understood the question without any ambiguity. You can assess the responses of respondents. if the response is irrelevant and doesn't provide response to the question in a questionnaire then it needs to be reworded or modified. Hope this clarifies your concern.
To add to Suresh Bowalekar's answer, Rajat, you will improve your final study a great deal if you take some time now to make sure that your survey works. You can call it validating or you can call it a pilot study, but here is what you need to do:
1) Sit down with someone who matches the kind of person you would want to take the survey in your study. You may want to tape-record the session.
2) Give this person the survey to take, but ask that person to Talk-Aloud everything he or she is thinking while taking it.
For example, if I was doing a "Think aloud" for your first question, I would say:
What is the reason I am seeking medical advice? well, do I give the actual answer that I am scared or the practical answer that my friend told me to, or the physical answer that I want to know if there is something wrong with my sleeping? I guess they just want to know what is wrong with me, so I will put down that I wake frequently.
From that, you learn that the first question is not asking what you want to know and you need to rephrase it. If that question is supposed to gather information about symptoms, you need to ask about symptoms. You might be better information about symptoms if you gave them a list of common symptoms and asked them to check off whichever ones were relevant. Even better, give them a list and ask them to rate each one in their level of concern. But if that is not what you are after, rephrase the question to ask what you really want to know.
3) Do this again with your revised survey, until the things that the person is thinking while reading your items MATCHES what you think they should be thinking about while answering, and the answers they give are the actual information you want from them..
Anything short of this is a flawed study, and may be a waste of resources - gathering data you cannot make sense of does not further anyone's cause of science.
Good luck!
Good morning all. I find this discussion very interesting as I am about to design a questionnaire for my Masters thesis (on emergency exercising) and have never previously done this activity. Julia B Smith - your comments are very helpful and have demystified this for me, thank you.