Dear all, in order to seperate the predictor from the criterion, I plan to collect data from two different groups of raters from the same company. Group A, 600 employees, will fill questionnaire 1 and Group B, also 600 employees, will fill questionnaire 2.
1. How do I select these groups with minimum differences which might impact the cause-effect relationship?
2. How do I measure or control for these differences.
Thank you very much in advance
1) Try random sampling. If there are recognizable subgroups within the company, then try stratified random sampling within these groups and select the number of samples within a group proportional to the contribution of that group to overall number of employees. In cases where a group has only a single individual (e.g. CEO), you will have to decide if you want that individual as part of the survey.
2) You "control" for the differences by random sampling. Quantifying the differences is more difficult, and rapidly becomes impossible for a fixed sample size. Essentially, Group becomes a treatment effect, and you have to plan sample sizes accordingly. So 600 people from night shift, 600 from day shift, 600 from security, 600 from .... and so on.
If you control for the effect through random sampling then the variability from the effect shows up in the error term in your analysis, and metrics like R2 are lower than if you had sampled enough to have these factors in your model.
This is also a key element in survey design. You want to ask 200 questions, yet no one will take the survey if it is that long. So now you are down to 20 questions. What do you leave out, and what effect did that have on the outcome?
Thank you very much for your answers. Jimmy, I am using two different questionnaire, each to test different variables.
It sounds like you have two different experiments. The issue is not how Group A is different from Group B, rather it is how well Group A and Group B reflect the total population (all employees of that company). In that regards you might consider randomly selecting the first 600 from the total population, and giving them the questionnaire. Then select the second group of 600, but keep in mind that some of the people selected might have also taken the first questionnaire. Another approach would be to withdraw a sample of 1200 employees, and then randomly assign each employee to one of the questionnaires. You could also alternate: select the first employee at random for questionnaire 1, then select the second employee at random for questionnaire 2. Another option would be to give some people questionnaire 1, others questionnaire 2, and still others get both. Half of the ones that get both get questionnaire 1 first, the other half get questionnaire 2 first. What you do not want to do is to select the first 600 for questionnaire 1, and the second 600 for questionnaire 2. The reason is that your population for questionnaire 2 will be "all employees less the 600 removed for questionnaire 1." As each person is selected, you are changing the population of the company for the subsequent selection. Because a company is not a homogeneous entity, there is a risk that the process of selecting individuals will bias the outcome of the survey.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Applied Statistics
- Survey Design
- Questionnaire