Hi everyone,
What is the best test to use to compare population means between three groups of unequal size and variance?
These are some data characteristics:
1. Continuous variable that follows a Poisson distribution
2. Unequal sample sizes (n=25;n=25;n=8)
3. Unequal variances
I need 95% confidence interval and a multiple comparisson among the 3 samples.
Any suggestions? which would be the best way to compare them?
Thanks a lot!
I use Armitage and Berry; Statistical Methods in Medical Research book. Do you want me to look for mine? the pages are all falling out as the binding has disintegrated.
Hello Ian Barbe. You say your DV follows a Poisson distribution. Is it a count?
(I know you described it as continuous, and I know that counts are discrete. But frequently, people say continuous when they really mean quantitative. )
My first thought is to frame this as a generalized estimating equation (GEE) regression with an intercept and two beta coefficients. You would use an approximate Wald chi-square test to simultaneously investigate whether both beta coefficients are equal to zero, i.e., whether at least one group mean is different from the others. If the p-value is compelling enough, you would then use Wald tests and confidence intervals for each pair-wise comparison between groups to look for evidence on which groups differ.
To fit the GEE model in a software package, you would specify a normal distribution, but this does not imply the data must follow a normal distribution. It simply indicates you are not allowing the variance estimation to influence how the mean is estimated (the means are estimated empirically). You would also request the empirical sandwich standard error to be used. This estimates the population variance separately in each group when calculating the standard error for the test.
I realize the sample sizes are small, so these wald tests may not have spot on performance, but it's still decent. There are other non-parametric tests you could consider.
Why not a generalized linear model approach, which is suitable for Poisson distributions. But I am curious, how did you determine that it follows a Poisson and how that the variances are different. Since in a Poisson mean = variance, you would already know that the means are different (indirectly and when the Poisson assumption holds).
I would use R / RStudio with the base glm approach (or the "glmmTMB" package if you need more flexibility in case of zero inflation or if the Poisson doesnt hold, so maybe a negative binomial distribution may fit) and the "emmeans" or "marginaleffects" package for the confidence intervals of the means or mean differences (both support glm, glmmTMB and also brms, if you like the Bayesian approach).
Hi Bruce Weaver Yes, it's a count. They are independent events of photo captures per unit of effort. Since it's actually a Relative Abundance Index and they take counts based on camera*day effort, I referred to it as a continuous variable rather than a discrete one. And yes, its quantitative.
Hi Rainer Duesing ! Let's suppose that the variances are equal. I tested a GLM with Poisson but due to overdispersion, I adjusted it to a negative binomial distribution and the overdispersion isnt corrected. The some happens with a Tweddie distribution. So, I wonder if the models should not be balanced. Thats the concern I have about using a GLM : I'm comparing year 1 vs year 2 vs year 3 of feral cat photo captures. Regarding the camera trapping stations, I have 25 stations (year 1) vs 25 (year 2) vs 8 (year 3). The 25 vs 25 comparison seems appropriate, but when I compare it to 8, I'm wondering if I might be imposing a constraint on the model due to having 17 stations that weren't used. It's not that these 17 stations had zero records; rather, they were not used in year 3.
I am using glmmTMB and conducting the overdispersion test with the performance package (check_overdispersion). I'm not sure what the package does when it compares the 27 stations from year 1 and year 2 to the 17 from year 3 that have no data. Maybe it treats them as NA values or as 0. I don't know, and that could be causing the overdispersion issue.
Ian Barbe, GEE is a further generalization of the generalized linear model.
The Kruskal-Wallis test is a non-parametric analog to the one-way ANOVA F-test. You could follow up with pairwise Mann-Whitney tests, analogous to pairwise Wald or t-tests.
Ian Barbe either I missed it or you didn't mention it explicitly, but what exactly is the problem your are facing?
Additionally, it sounds as if you have repeated measures. For what you described so far, the data/locations would be treated as indepndent and therefore, the "missing" stations wouldn't be considered and wouldn't matter. But your model should recognize that. If you want to stay in the realm of linear models, you could switch to generalized linear mixed models to account for the variance within each station, by adding a random intercept for station.
There is a nice new paper about the workflow of a glm, with R scipts all along (Article A versatile workflow for linear modelling in R
) which uses the DHARMa package, which makes the residual analyses very intuitive.Geoffrey S Johnson Ian explicitly asked for mean differences. I dont see how Kruskal-Wallis or U-test could do this, without further (imho strong) assumptions about the shape of the distributions (which in turn shouldn't be equal in case of a Poission with different means in the first place).
Ian Barbe, thanks for clarifying that your DV is a count. Have you seen this page on modeling multilevel count data? It mentions the glmmTMB() package you mentioned in one of your posts.
- https://fcorowe.github.io/countdata_modelling/
Rainer Duesing, perhaps Ian Barbe would be willing to modify the hypothesis being tested when using the non-parametric methods. If not, the stochastic dominance assumption needed to view these as tests about the mean is weaker than assuming the data generative process follows a Poisson distribution. Importantly, both assumptions are testable in principle.
- Badges
- Science topic
- Similar topics
- Agricultural Science
- Agronomy
- Crop Science
- Crop