Dear Seyed

This is a a very open question and demands a more detailed answer.  There's a few steps we need to follow to build a questionnaire, apply it and analyse it. Considering that your questionnaire is already validated, and you already have your answers, you should look for the outliers and remove it previously proceeding the analysis.  By outliers we can spot for example, survey questionnaires where the answers are  concentrated in only one side of the scale. In other way (and more obviously) you must remove questionnaires with incomplete answers. Maybe you could give us more details and we continue our conversation.

You should check Hinkin (1998) article "A Brief Tutorial on the Development of Measures for Use in Survey Questionnaires". I guess will help.

Best Regards

Yves

Dear Seyed

This is a a very open question and demands a more detailed answer.  There's a few steps we need to follow to build a questionnaire, apply it and analyse it. Considering that your questionnaire is already validated, and you already have your answers, you should look for the outliers and remove it previously proceeding the analysis.  By outliers we can spot for example, survey questionnaires where the answers are  concentrated in only one side of the scale. In other way (and more obviously) you must remove questionnaires with incomplete answers. Maybe you could give us more details and we continue our conversation.

You should check Hinkin (1998) article "A Brief Tutorial on the Development of Measures for Use in Survey Questionnaires". I guess will help.

Best Regards

Yves

For each question you may add another question in different words or put that question exactly opposite to earlier question and then check the reliability. Best 

It's the nature of human beings that we cannot always trust what others tell us . ..  no many how many checks and balances we put in place. This is why it is becoming increasingly accepted to complement quantitative questionnaires with qualitative data gatherings: focus groups and interviews for example. Only by asking more searching questions of an individual can we ascertain what they really think and feel. Also, by working in this manner, we can built up trust with our research participants and, from that position, we're far more likely to get useful information from them.

'Trust' is, after all, a human quality: a facet of Emotional Intelligence . . ..

Hello Seyed,

Your question is quite broad and in order to answer it, I have two questions:

1) Did you use questionnaires that are valid and/or reliable?

2) What makes you doubt the trustworthiness the responses? was there any contamination or unforeseen bias? 

hello,

the purpose of the questionnaire is exploring respondents view whether they are students or others. Therefore students respondents, based on questionnaire, are reflection of what they think and believe, Moreover, the properly designed questionnaire would  reveal the students who answered properly and meaningfully regarding the statements of the questionnaire. Although we have sometimes some odd answers, but surely should be respected as it stands of student's view or opinion.

Dear Seyed,

Just a brief answer to suggest that questionnaire data should be corroborated with another research method such as informant based interviews, and even this could be further triangulated with in class observations over a set period. Of course for all this you need participant written consent and for younger( under 18 yrs in UK) students, ,parental consent. Uni based research in UK at least now has to get passed by either by an ethics committee or signed off by an academic who has undergone and passed the appropriate ethics training.

So if you package all this as part of a research programme you should get a balanced result. I hope this make sense. Dave

We do a fair amount of research using questionnaires with both adults and children.  If I had to summarize our basic philosophy with respect to the use of surveys it would be:

1  What is the purpose?

2. Who will answer the survey, and what makes that person the one who should answer? (is it experience, a certain job, etc.)

3. Has the person had previous experience doing a survey?  If not, provide training.

4. Make the directions clear. Ask for questions.

5.  Make the statements as short as possible, and the responses clear.  We prefer a interval scale, defined operationally, for the response.

6.  The results are only a "snap shot" in time.  WE like to repeat surveys to note individual change

7.  You always have to be concerned with the veracity of the responses.  We think making sure the goals of the survey are known, and accepted by the subject can help.

I am glad to see that Jeffrey Knox pointed out that a survey can only be a "snap shot" in time. That is an important characteristic that can be seen either as a benefit or a limitation (or both), which is why triangulation is so important. I suggest checking into literature on validity, reliability, and trustworthiness.

I am of the view that triangulation is one way to gauge students' consistency in responding to questions. Focus group discussions and interviews can cross validate against data from questionnaires.

with regards to questionnaires for class lessons, I validate results thru actual accomplishments or performance.  if the item is not answered by a significant number of the class then that means the questionnaire was improperly done, instructions were not clear, or I did not teach the topic right.  Whatever, reteaching or remediation may be required when a questionnaire yields dirty or unacceptable (for the research objectives) data.  with regards to research questionnaires, i try to go back to the respondent asap so that s/he may complete the questionnaire.  if too inconvenient, then I have to live with my gatherred results.

Great question! I just make sure that they know that their answers will impact them in some way and I ensure that the first question and then a few throughout remind them of this fact.

I agree with Georg Lind. Do prepare well the questionnaire by:

- asking the same question twice, in different ways, if it is a controversial issue (some students like to give their teacher the answer "they think" the teacher wants to read; having the question restated helps to double-check the answers) ;

- making few questions;

- having a questionnaire with open answers (it is good to collect what subjects really think - not what the researcher wants to read). Once, a student of mine got a set of very interesting answers from the subjects, so unexpected and rich we decided to analyse them too.

- following Grice's Maxims;

- having a pilot test when possible.

Then, just trust the answers.

Best regards!

Eveline

In the questionnaire we have to set each item twice  in different meaning or with synonymous to ensure the validity. If both are related towards the same meaning we can take it into consideration. Otherwise we can conclude that  the participant is not honesty towards participation.

Towards questionnaire setting we should find out the means and measures of identifying the falsification.

Towards any assessment always we have to address the audience such as:

What is the hidden agenda of doing this assessment?

What is the personal usefulness of participating in the assessment? 

How the honest influences the inferences? etc.

At time orientation sessions makes them to get motivated positively. 

Unfortunately, we can't fully trust any measures in the human and social science studies. But we always try our best to make them valid and reliable. If one doubts participants' answers on a questionnaire, for example, s/he might re-administer it and compare results..Or use a test-retest procedure. There're many statistical techniques (e.g. equations) that might help you with this. Good luck..Mahmoud