We have very large data sets of populations across five years. We want to compare the proportion of people in different categories statistically, controlling for the differences in sample size. For example, if we have a total sample of 10,000 people in Year 1, and 17% are ages 18-24, how do we compare that 17% to the proportions of same age category in Year 2 with 15,000 total sample, Year 3 with 18,000 total sample etc.? I had assumed it would involve weighting but want to get expert opinions on approaches to this. Ultimately, the goal would be to see whether there are statistically significant differences in the proportions in each age category, controlling for differences in the total sample. Thank you in advance!
Hello Lia,
You could use the chi-square goodness of fit test. If Year 1 results were to be considered the baseline, and you'd like to know whether Year 2 shows a substantive departure from Year 1, then the Year 1 proportions for the respective category would be multiplied by the Year 2 N to obtain expected category frequencies. Across categories, the expected frequencies are compared to the observed frequencies for year 2. This is evaluated as a chi-square statistic with #categories - 1 df.
With sample sizes as large as you have, such a test will have considerable statistical power to detect differences.
Finally, if you consider that your yearly data sets truly are the full population, there is no need for any statistical or hypothesis tests at all. Any difference from year to year is a real difference (e.g., you have the population parameters).
Good luck with your work.
In attachment this test (using R) with
1) observed frequencies 2550 (17%) and 12450 (83%) as predicted
2) observed frequencies different from predicted: 3000 (20%) and 12000 (80%).
As noted by David, even slightly difference observed frequencies, yields a very small p-value.
- Badges
- Science topic