Sounds like you are taking appropriate steps. Validity includes those areas you highlight, also review, design, piloting, to identify possible bias is extremely important. Some go to the extent of mixing positive and negative responses, this is particular to certain types of research having issues in response bias (such as consumer behaviour studies)

However using trained personnel, I feel, begins to border on introducing directives and can be construed as leading in some cases (the reverse of what may be intended).

Following on the validity with reliability tests, based on what you propose above is very satisfactory I would contest.

Thanks Robert. Yes I have taken steps in mixing up positive and negative responses in the questionnaire. Did piloting and review as well (including unarticulated thoughts through qualitative interviews).

 I will continue with the assessment as planned. My infinite thanks once again.

Rabin

Hello Rabin,

Having a large sample size also helps. And if you can identify responses that were obviously random (e.g., inconsistent answers in negative/positive items), then you can exclude that in the analysis.

Sounds like an interesting study! Best of luck!

Leonor

Thanks for your answer Leonor. That's what I did, I decide to expand the scope of the study to another state to increase 1) sample size 2)the possibility to generalize better. 

Rabin

Questionnaire design is a black art. 

That said, there are a couple of pointers.

1. Make it short. People will lose concentration and motivation if your questionnaire is long. You will see pages with identical tick marks running down them, indicating the person was barely paying attention to the questions. 

2. Engage the person right away.

It should be clear to the person what the study is about, and why their contribution is important. Ask interesting questions first – never demographics, which are dull and often intrusive. Asking interesting questions stimulates the participant to continue, and makes them realise the importance of the study.

3. Appear utterly professional

The questionnaire should look professional, clearly come from a trustworthy source, and inspire confidence in the participant that the results are important, and will make a difference.

See Fanning, E. (2005). Formatting a paper-based survey questionnaire: Best practices. Practical Assessment, Research & Evaluation, 10(12), 2.

http://parkdatabase.org/files/documents/2005_Formatting-a-paper-based-Survey-Questionnaire_E-Fanning.pdf

That's a great resource, Ronan! Thanks for that. I have seen participants practically running away after finishing a battery of questionnaires! Understandably, we want data, data, data! Balancing that with good quality data is indeed a black art.

So have I, Maria Leonora! Or seen questionnaires where the person ended up checking "agree somewhat" for two pages at a time! 

I think that researchers should pay a deposit of RM100 for every question on their survey. They get the deposit back when they show that the answers were used in a published paper or report. That would shorten questionnaires a lot!!

Seriously, I know of surveys where 80% of the data remained unpublished. It's ethically unacceptable. People give their time and co-operation for free. We owe them a duty to ask them only questions we intend to analyse and publish. 

Oops. RM100 is €25. 

Yes, that's a pity. But you know yourself, there are several reasons why data are not being analysed even though we had every intention at the very beginning.

Questions should be short, not many in number and the language must be simple, unambiguous and without any extra verbiage.

Agree with others commenting above.  Congratulations on pilot testing.  Although it might not seem to address your question, see if you can find a wonderful small book called, "The Art of Asking Questions" by Stanley L. Payne (Princeton U Press, 1973). Though old, it is full of memorable examples and seemingly simple, but timeless advice. Probably the first rule of questionnaire writing is don't annoy the respondent.  The second is don't get garbage by writing questions they misunderstand or can't answer in the format you've chosen. Payne's book has lots of advice on these two and many other problems affecting the validity of questionnaires.  If your work on validating your questionnaire has shown it has none of the problems, my sincere congratulations!