Is it necessary to test the questionnaire before starting the study? and how to do this?
Not only is it necessary to test the questionnaire for validity and reliability. If you are using a validated instrument that was validated in one culture and you are adapting it into another population or culture, it is very very IMPORTANT to ensure that careful cross cultural adaptation has been done. Many times even the validated questionnaires don't work in some situations because of cultural regions. So please do a form of cognitive interviewing of some potential study participants using the instrument to understand if the questions make sense to your study population and the responses you are getting are similar to the ones that you have on the survey. Once you have tweeked the language and language, then pilot test the instrument in a small sample of the population. After you are sure that it is working, only then would it be wise to use in your study to collect actual data.
Doing studies in international settings in far more complicated than just adapting the questionnaire and taking small precautions such as these will increase the value of the data that you collect.
Hope this helps! Good luck!!
depends
The questionnaire aims to measure what?.
if it is a test, this should be valid for your country, culture, age or other necessary, at least one questionnaire point to collect data about preferences, behaviors or other, must have the lowest content and criterion validity., it is advisable to approach this expert judge.
in the case of a test, as above, but relies on a statistical analysis.
If you can find one, a well-documented survey instrument is a much better approach. What is it you are trying to measure? Typically, you want to establish validity and reliability prior to using the instrument to do a study but you can do that after you've collected the data sometimes (concurrently). But be prepared for surprises...the instrument may not measure what you think it is measuring (validity) and may not be reproducible (reliable).
It is very important that a questionnaire is valid and reliable in the population that you wish to measure. There are several steps that you would need to go through to establish validity and reliability. It would be easier to use a measure that is already widely accepted, unless you want to develop one as a long-term project.
Hi Fahmi, Any new instrument requires a lot of piloting work. See Floyd Fowler's Survey Research Mehtods, for example, for an overview of what to do. It's a lot of work. If using existing instruments, it is still useful to pilot on the target population and engage in some of the steps of piloting a new instrument to ensure the suitability of the instrument. Bob
Should consider, if your questionnaire collects data related more background or gathers data that point to some measurement ..
That kind is your form?
Questionnaire are usually validated by peers or experts in the fields you are doing your research or you can run a pilot study from your questionnaire before you do your actual research.
http://kestrelconsultants.com/reference_files/Validating_Questionnaires.pdf
Not only is it necessary to test the questionnaire for validity and reliability. If you are using a validated instrument that was validated in one culture and you are adapting it into another population or culture, it is very very IMPORTANT to ensure that careful cross cultural adaptation has been done. Many times even the validated questionnaires don't work in some situations because of cultural regions. So please do a form of cognitive interviewing of some potential study participants using the instrument to understand if the questions make sense to your study population and the responses you are getting are similar to the ones that you have on the survey. Once you have tweeked the language and language, then pilot test the instrument in a small sample of the population. After you are sure that it is working, only then would it be wise to use in your study to collect actual data.
Doing studies in international settings in far more complicated than just adapting the questionnaire and taking small precautions such as these will increase the value of the data that you collect.
Hope this helps! Good luck!!
Khan, if the questionnaire is not a standardized one, you have to do a pretesting of the questionnaire on a small sample, to identify and to eliminate potential problems. As Omer told then you need to do reliability test on items as well as dimensions. This will give you the internal consistency of the questionnaire.
Even if there is a standardised, well published questionnaire instrument that says it's measuring what you want to measure - you will ALWAYS need to test it with the population/age group/ethnic group/language that you want to use it with. All sorts of things can go wrong - I agree with others above who place the focus on the quality of the measuring tool - reliability, validity, generalisability - but sometimes there are other types of issues that arise that it can be impossible to predict and could totally ruin your main study - like timing (it could be too long or too short for your population), order of questions (having a sequential effect), gatekeeper effects (someone else in the chain of access to your sample who has a view to express on the measurement tool. Most often I find language issues - words that have a slightly different meaning in my culture than in the culture that the original questionnaire was designed for - and unless you are an actual member of your own target group it can be really difficult to anticipate this. So you need to ensure good quality measurement - but also appropriate measurement - do the pilot and learn from it!
yes, it is necessary to test the questionnaire before starting the study. we can eliminate wrong question from the schedule for which responses are not recorded correctly or having some ambiguity. Before doing final field work, one must undertake pilot study on the basis of same questionnaire.
It would be good for you to test your questionnaire before starting your research,in order to check if it works and if the questions are clear and understandable from the sample.the best you can do is a ''pilot test'' to some people to check your questionnaire.these people can be not a big group,but they will make you sure if it works properly or not.
It depends on what you are using it for.
If you use it for diagnosis, meaning that some real world decisions are being made, you have to validate carefully. Asking experts is not sufficient in that case. You need external validation with measures of proven validity.
In addition you should check that the questionnaire is invariant to any demographic variables, such as gender or cultural background. This is often referred to as differential item functioning (DIF, this is what you have to avoid).
Consistency of questionnaires is typically checked with confirmatory factor analysis. This requires structural equation modelling software, such as SPSS Amos. In base of low consistency, you may want to check whether your questionnaire has the right dimensionality and composition of scales. CFA requires fairly large data sets (n>100), so small pilots won't help.
If your aim is just a correlation study, the above requirements still hold, but in a much relaxed form. For example, CFA is often done on the same data as the study itself.
When a questionnaire is being used to simply gather basic data that will not be used in the primary or secondary analysis and is not being used as an outcome measure, there are times when it is not feasible, or necessary, to use a validated instrument. If the questionnaire is part of the analysis then like any other outcome measure should be a standardized outcome measure (SOM). A standardized outcome measure has generally been tested for measurement characteristics such as reliability (consistency of measures), validity, sensitivity and/or responsiveness. I agree with those of you who posted that "it depends on how you want to use the questionnaire-this determines whether or not you need a validated instrument". In our applications (disease management) in using questionnaires it is important that our instruments are validated and tested since we use the data contained in the questionnaire for statistical analysis and to assess risk, as wel, as, make predictive staements based on our data. Our applications are for predicitive modelling in population health management. We use item response theory to determine which line item questions have the best fit in terms of answering the questions most relevant to the goal of the study/project. As someone else alluded to, the cultural relevance of the questions need close attention since the statistical analysis cannot compensate for poor item choices...just a thought.
Yes, it is necessary to test the questionnaire for internal and external validity, this would warrant the validity of the content, criteria, using experts judgment. The qualitative validity of a questionnaire should be assessed through a pilot test in order to confirm its reliability.
Yes, it is very important to test the questionnaire before the survey. You need to do a pilot survey and then you have to proceed.
Yes, it is very important to measure the validity of the questionnaire. First you must make a face validity by 10 experts in the field and experts in statistics. Then you can pretest the questionnaire to make sure that the questionnaire is going to deliver the right data with a number of people (5-10) who represent your sample. You may need to amend the wordings of some questions and give more explanations for some questions after pre-testing (it depends on the feedback of the people. Finally, you need to make a pilot study as a trial run on the questionnaire with a sample of around 30-50 people to measure the statistical validity of the questionnaire (internal and external validity).
http://library.iugaza.edu.ps/thesis/116796.pdf
It is always necessary to make a pre-test of the questionnaire and a pilot study of the study before embarking on the main study. This should help in clarification of unsuitable questions, time required, cost, skills of interviewers and the extent of non-response.
I would like to ask, what if the original English instrument will be given to respondents of a country that also used English language, what cross cultural validation steps should I undertake?
Yes, you need to. A questionnaire is a tool or an instrument such as scale which needs to be validated to ensure it provides reliable and valid results. Validation process of questionnaire should be comprehensive by using various psychometric tests such as internal consistency reliability, test-rest reliability, inter-rater reliability, face validity, content validity and construct validity.
Yes, it is necessary. There are several tests to be conducted in order to ensure the validity and reliability of an instrument. First is the forward and backward translation or also known as cross cultural validation of an instrument, the most common of method is the guidelines set by WHO which includes forward and backward translation, expert panel, cognitive debriefing and pretesting, then content validity, construct validity either you the model proposed by the original author and conduct confirmatory factor analysis or propose your own structure and conduct and exploratory factor analysis. You can also perform convergent and discriminant validity.
. Further, you also need to test the reliability of the tool by conducting the internal consistency reliability and item-total and inter-item correlation. Test-retest reliability and inter-rater reliability.
Hi Fahmi,
It's certainly important to test the validity and reliability of a questionnaire before data collection.
Regards,
Purnima Madhivanan thank you so much for the information. i just have one question, what modifications or adjustments can be made after the tool validation. in general what is the specific importance for that specific study?
Yes, it is compulsory to avoid any inconvenience and setting the path to complete your research. Well, validity is the first thing that is vital to ensure and most of the time, is it ensured by the experts' judgment in their relevant areas. The word validation process is linked with the validity. Reliability is to do with the consistency of the results that are achieved/reached by a number of different ways (with respect to time: test-re-test and with respect to evaluators: inter-rater reliability). Conceptually, a valid test can lead towards consistent results/ reliability.
If you are using an established instrument that has already been tested for validity and reliability the answer is no, unless you are making modifications
É interessante sim , você realizar um pré-teste com uma amostra representativa. E quanto a questão dele ter sido validado em cultura diferente, é indicado que vc tenha tb uma validação para a cultura populacional que você aplicará.
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis