Salam Anis,

I'm not sure specific area such as scale development has its own justification pertaining to non-probability sampling. But, if your study setting is within social science context, Rowley (2014, p.318) states that most social science research relies heavily upon non-probability samples.

Rowley, J. (2014). Designing and using research questionnaires. Management Research Review, 37(3), 308-330.

Hope this helps.

Salam Ahmad Anis Najiha ,

In many references, there are no specific explanation about sampling techniques that can be used for scale development. Usually, they only describe about minimum amount of samples for scale validation.

For instance, Nunnally (1967) described that a ratio of 5-10 participants per item is sufficient. or Gable (1986) that recommend a ratio of 6-10 participants per item.

In my humble opinion, using non probability sampling in scale development is acceptable since there are no references (that i know) not recommend or discourage the using of non-probability sampling in scale development.

Best,

Riangga

Hi

Non-random sampling is performed because the sample that will maximize the variance will be selected during the scale development phase.

If you use probability sampling, In this case, the maximum variance is not guaranteed. For this reason, purposeful sampling should be used. Hence, people who are positive and negative sides for the measured variable can be included the sample.

Thank you everyone for your feedback. Abdullah Faruk Kılıç raised a very interesting point specific to scale development, it will be great if I could have the reference for that.

Hi Ahmad Anis Najiha

Standards for educational and psychological testing book written by AERA, APA & NCME (2014) says:

"Test developers should include individuals from relevant subgroups of the intended testing population in pilot or field test samples used to evaluate item and test appropriateness for construct interpretations."

It means that we have to include people who come from relavant subgroups of population for field test. If we have to include this people, we can not use random sampling. Because we can not be sure whether there are relavant subgroups of testing population.

In addition Erkuş (2014) has emphasize this issue but its language is Turkish. Also Cohen, Swerdlik & Struman (2013), Crocker & Algina (2008), Urbina (2014) and Price (2017) says some important thing about field tryout (or pilot testing). But The Standards book clearly emphasize that we should include people from subgroups.

I hope, it's more clear now.

Have a nice day.

American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement In Education (NCME). (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

Cohen, R. J., Swerdlik, M. E., & Sturman, E. D. (2013). Psychological testing and assessment: An introduction to tests and measurement (8. Baskı). New York: McGraw-Hill Education.

Crocker, L., & Algina, J. (2008). Introduction of classical and modern test theory. Ohio: Cengage Learning.

Erkuş, A. (2014). Psikolojide ölçme ve ölçek geliştirme-I: Temel kavramlar ve işlemler (2. Baskı). Ankara: Pegem Akademi.

Urbina, S. (2014). Essentials of psychological testing. (A. S. Kaufman & N. L. Kaufman, Eds.). New Jersey: John Wiley & Sons, Inc.

Price, L. R. (2017). Psychometric methods: Theory and practice. New York, NY: The Guilford Press.

For scale development in Rasch measurement, we opt for rectangular rather than normal distribution of the sample, because we want as much good information about extreme items as we do about the central ones.