In experiments, sometime I come up with interesting results. For example I have 3 treatments (1,2 and 3) and run 3 replicates. In each replicates I do all the three treatments. When I got the results, for each the replications, treatment 1 always have lower response than the other two treatments. But, there are some variations in the results between replicates. For example the response of treatment 1 in the 1st replicate is similar to treatment 2 in the 2nd replicate. Therefore in the ANOVA results, it usually ended up with no significant different between treatment 1 and 2. But when you check each of the replicate, they are actually different. Is there any possibility that including the replication to the analysis can make any difference? How to include the replicate to the analysis if it is possible?
for example this data:
Rep1 Rep2 Rep3
Treatment 1: 2.016; 2.003; 1.873
Treatment 2: 2.211; 2. 094; 2. 509
Treatment 3: 2.356; 2.392; 2.342
If you say there is a "lower response" then you mean there is a significant lower response? This would imply that your standard deviation of the values is low enough to find differences in, say, the first experiment (rep1). If that's the case, then rep2 and rep3 obviously show that the effect is not constant - which might be because the samples are not from the same population or the experimental procedure had some flaws at collecting data.
Why I say this: As your example data says: there is a significant difference between treatment 1 and 2 in rep1 but this seems hardly the case in rep2. Even if it is, there are bigger differences between treatment 2 and 3 in rep 1 and 2 in comparison to rep3 which has its maximum value at treatment2. Considering all the "evidence" the three studies contradict each other.
The more plausible case is that the standard deviation is far higher and there is hardly a difference in the treatment groups. This way you might just have had some bad data which unfortunatelly does not tell you much about differences between treatments.
If you are tempted to combine the data of the replicates then you should check, whether those are from the same population (not the actual sampling population of people but whether the sampled subjects are the same as of all other possible influences like same testroom - same conditions of test - same gender rates and so on.) If you DO combine the groups to test them together you openly say that those samples ARE from same population (! this statement is important !). Obviously your standard deviation raises rapidly and you will hardly get some significant results.
All statements above are based on the premise that your example roughly describes your true data.
The replicate values in treatment #2 have the largest variance (that is considerably larger than the variance within the other two treatments), and the largest mean difference is between the treatments #1 and #3.
This explains your observations.
It might be interesting why the variance for treatment#2 (t2) is so (relatively) large. Or why it is so small for the other treatments (t1 and t3). However, you may consider using weights to account for this heteroscedasticity. The confidence intervals for (t1-t3) will be much narrower, and the intervals for (t1-t2) and (t2-t3) will be a bit wider, compared to thode from the unweighted model.
I guess you have a little problem in the understanding of use an ANOVA... I mean you don't have to see the data as individual observations, it is, your data are arround a mean that is which can explain the real repsonse to your treatment. So I could say that if two observations are the same it can be just a circular effect of your data, remember that you have to watch the mean and standard deeviation. If you have 3 treatments you have to use post-hoc analysis and I could say that you should use more than 3 replicas. Hope you find my answer usefull.
Hello Safri, it is possible that the differences in the measurements of each treatment that you mention are due to random variation, it is assumed you don't know the real population value of each treatment but only approximations (your collected measurements) of it.
Perhaps, if you are interesting in detecting a possible difference between treatments try to increase the sample size, this with the aim of increasing the precision of your estimates.
Thank you very much for alll the valuable suggestions and discussions.
I am working with microorganisms. So sometimes, due to complexity and expenses of the experiments, we only can run limited repetitions. So, I guess the correct way is just to run the ANOVA followed by post hoc test and see whether the Post Hoc test indicate any significant differences.
To run anova you should arrange your experiment in certain experimental design. If you set up the laboratory experiment in completely randomized design with 3 treatments , then you should run at least 6 replication.
"the correct way is just to run the ANOVA followed by post hoc test and see whether the Post Hoc test indicate any significant differences."
I don't see what you will do with the Anova then?
When you have just 3 treatments, you can use the Anova to go for Fisher's LSD (after "significant" Anova F-test). But if you happen to use some other post-hoc test there is no need of an Anova, because these post-hoc tests do keep the family-wise error rate. The Anova won't tell you anything useful since you are just interested in the pair-wise differences between the groups.
Dear Safri,
Starting from the point that replicates are really independent, you should check the homoscedasticity of your data. If you do not have homoscedasticity, you get a problem of heteroscedasticity; therefore the parametric ANOVA should not be use.
In this case you can try to transform the data, or using non parametric tests.
In any case, it is good to have replicates to estimate the experimental error.
Best regards
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA