I'm developing a readiness assessment model regarding contractors' preparedness for a specific activity, in order to do so, a survey study was carried out and the data analyzed with PLS-SEM to obtain the CSF contributing to that readiness; nevertheless, due to the subject being too specific, it was impossible to define or quantify a population for it and hence, a probabilistic sample, which can compromise the external validity (generalizability) of my readiness assessment model. Is it feasible trying to reduce that generalizability issue with the minimum sample size requirements (means of power analyses) from Cohen (1992) and the use of PLS predict to determine the prediction power of the model?
I'd be delighted if any colleague could reply to this need
In general, using any rule-of-thumb for sample size planning or assessing statistical power is problematic.
Random sampling provides a model free basis for generalization. Propensity score–based methods for generalization require three assumptions to ensure their validity. First, the stable unit treatment value assumption must hold for all units in the experiment and in the population. Second, generalization using propensity score methods requires strongly ignorable treatment assignment in the experiment. Finally, generalization using propensity score methods requires a strongly ignorable sample selection. The development of rules of thumb that take into account sample size since features of probability samples—the benchmark for generalizability—differ markedly in small samples. This raises the issue of how to judge the adequacy of the match between the experimental sample and the inference population.
Probability sampling is the gold standard for generalizing from samples. The idea is to use the adequacy of matching that would be expected if the experiment had a probability sample to develop benchmarks of adequate matching. There is no reason to expect small experimental samples to match inference populations better than probability samples.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size