Hi everyone, I want to learn as much as possible about survey methodology for evaluation purposes. I'm particularly interested in research around validity and reliability of: question types (format, types, what's best for what purpose) likert-type scales (how many items, what labels to use, etc), and how to design questionnaires to avoid or minimize biases (e.g. primacy, recency effects, leniency effects, halo effects etc). Basically, I'm trying to answer a question like: "Does it matter how you ask the questions when you are evaluating others?".
Could anyone recommend some journals or meta-analyses or authors or specific search terms that could help me learn more?
See also:
1. DeVellis, R. F. (2016). Scale development: Theory and applications (Vol. 26). Sage publications.
2. John, G. and Reve, T. (1982), The reliability and validity of key informant data from dyadic relationships in marketing channels, Journal of Marketing Research, 19 (4), 517-524.
A good free research methodology textbook, which may assist is:
Bhattacherjee, A. 2012. Social Science Research: Principles, Methods, and Practices. Tampa, FL: USF Tampa Bay Open Access Textbooks.
The textbook can be downloaded from: https://scholarcommons.usf.edu/oa_textbooks/3/
Two other sources that would give you some information in this regard are SPSS related textbooks:
Field, A. 2009. Discovering Statistics Using SPSS. 3rd Edition. London: Sage.
Pallant, J. 2010. SPSS Survival Manual. 4th Edition. New York: McGraw-Hill.
Hopefully the following publications could help:
- Dolnicar, S. (2013) Asking good survey questions, Journal of Travel Research, 52, 5, pp. 551-574.
- Fink, A. G. (2003) The Survey Handbook. 2nd edn. California: Sage Publications, Inc.
- Fink, A. G. (2017) How to Conduct Survey: A Step-by-Step Guide. 6th edn. California: SAGE Publications, Inc.
- Fowler Jr., F. J. (1995) Improving Survey Questions: Design and Evaluation. California: Sage Publications, Inc.
- Kelley, K., Clark, B., Brown, V. and Sitzia, J. (2003) Good practice in the conduct and reporting of survey research, International Journal for Quality in health care, 15, 3, pp. 261-266.
- Kelley, K., Kelly, K., Clark, B., Brown, V. and Sitzia, J. (2003) Methodology Matters : Good practice in the conduct and reporting of survey research, International Journal for Quality in health care, 15, 3, pp. 261-266.
- Kitchenham, B. A. and Pfleeger, S. L. (2002) Principles of survey research: part 3: Constructing a survey instrument, ACM SIGSOFT Software Engineering Notes, 27, 2, pp. 20-24.
- Platek, R. (1985) Some important issues in questionnaire development, Journal of official statistics, 1, 2, pp. 119-136.
- Punch, K. F. (2003) Survey Research: The Basic. London: SAGE Publications Ltd.
- Rattray, J. and Jones, M. C. (2007) Essential elements of questionnaire design and development, Journal of clinical nursing, 16, 2, pp. 234-243.
As a general textbook on this topic, I would recommend Fowler, Survey Research Methods.
Worth looking at Public Opinion Quarterly. And there are other relevant journals, but I'd probably start there.
I recommend the following chapter:
Krosnick, J., & Presser, S. (2010). Question and questionnaire design. In P. V. Marsden & J. D. Wright (Eds.), Handbook of survey research (2nd ed., pp. 263–313). Bingley, UK: Emerald.
and the following article:
DeCastellarnau, A. (2018). A classification of response scale characteristics that affect data quality: A literature review. Quality & Quantity, 52, 1523–1559
See also:
1. DeVellis, R. F. (2016). Scale development: Theory and applications (Vol. 26). Sage publications.
2. John, G. and Reve, T. (1982), The reliability and validity of key informant data from dyadic relationships in marketing channels, Journal of Marketing Research, 19 (4), 517-524.
Seung Youn Chyung has a great series articles on survey design topics mostly relating to Likert scales:
Evidence-Based Survey Design: The Use of a Midpoint on the Likert Scale (2017)
Evidence-Based Survey Design: The Use of Negatively Worded Items in Surveys (2018)
Evidence-Based Survey Design: The Use of Continuous Rating Scales in Surveys (2018)
Evidence-Based Survey Design: The Use of Ascending or Descending Order in Likert-Type Response Options (2018)
Evidence-Based Survey Design: Ceiling Effects Associated with Response Scales (2020)
A bit and pieces in every textbook, such as
Paper, such as
- Badges
- Science topic
- Similar topics
- Statistics
- Survey