I have a dataset with an unusual configuration and was hoping for guidance on choosing a method to test for changes in mean.
For context, this is operational data tracking vehicle arrivals at 10 physical sites, and measures relative site volume using a ratio of daily arrivals in relation to the site's capacity. Variation among the 10 ratios is calculated for each day (coefficient of variation).
A program was started that changes the conditions under which vehicles determine which site they drive to (goal is to reduce variation in above described ratios). There have been three different changes in conditions, yet the time lengths they were implemented for were all different:
Baseline - 30 days (cannot be extended)
Phase 1 - 78 days
Phase 2 - 116 days
Phase 3 - 87 days
I'm being asked to determine whether there were significant changes in the mean variation during each phase compared to Baseline. Since I'm testing the same group (the entire vehicle/site system) under three different conditions, I believe three separate paired t-tests would be appropriate. However, I know the sample size must be identical for each pair. Generating a proportional random sample would still give me different sample sizes (obviously). My question is whether it's acceptable to choose a constant number of days to sample from each phase (e.g. 15 days of ratio variations) or if there would be a more appropriate test to use?
Yes, a mixed model would be appropriate (time and condition as fixed and site as random factor). And since your dependent variable are counts, this should be a negative binomial model, in which the sites capacity is used as offset. The model shouldinclude the time:condition interaction that gives you difference in time course between the conditions.
As I understand (!?), you have 10 (sites) mean coefficients of variation as dependent measures. For each Phase (0,1,2,3) you have the mean of 30, 78, 116, and 87 days. The independent variables are
1) 3 Conditions as a Between subjects (sites)
and
2) 4 Phases as a Within subjects/sites,
factors.
Check the distribution of the means of the coefficients of variance (with a histogram and a Shapiro-Wilk test). If this distribution looks fairly normal, run a repeated measures ANOVA.
If the distribution is clearly not 'normal', run a nonparametrical test, such as the ARTool or the f1.ld.f1 test from the npaLD R package.
For ARTool, see: http://depts.washington.edu/acelab/proj/art/index.html
For npaLD, see attachment.
You could also go for a multilevel approach in the form of a linear mixed model. This would allow for analyzing data on the level of traffic sites and could thus incorporate the time lenghts for each site as a metric predictor.
Yes, a mixed model would be appropriate (time and condition as fixed and site as random factor). And since your dependent variable are counts, this should be a negative binomial model, in which the sites capacity is used as offset. The model shouldinclude the time:condition interaction that gives you difference in time course between the conditions.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size