Hi, I have carried out statistical analysis to assess how students performance improve from the pre-test to the post-test in control and experimental groups. I used both absolute learning gain (posttest - pretest) and relative learning gain (posttest-pretest/ pretest). My results revealed that there is a statistical significant difference in absolute learning gain between the control and experimental groups. However, no statistical significance difference was found in the relative learning gain.
Mona,
The relative gain is a proportion with the original scores in the denominator. It is similar to effect size in that it is measure of the proportion of gain. You gave me the U value, but not the actual mean scores. This is what I need. Also, the total points possible.
How far did your sample deviate from normality, because if it was not large the statistical analysis is usually robust enough to handle it. The problem with lack of normality would be a Type I error so you can correct for this by using a more rigorous alpha level, for example.01. Also, if you can use ANCOVA it would be a more accurate comparison of the two groups.
Mona,
It seems that although you have a significant gain in scores it is not large enough to be of any practical value. The former could just be a function of a relatively large sample size. What is the actual magnitude of change?
Thanks Norman !!
My sample size is in the experimental group (48 students) and in the control group (40 students). I used mann whitney u test as students scores were not normally distributed.
The difference between experimental and control group in the absolute learning gain was (U=689.5, P-value=0.017), and (U=773, P-value=0.085) in the relative learning gain.
I know that the relative learning gain could be considered significant in case the confidence interval is 90%. But I still cant understand what is the difference between absolute and relative gain if for each student in the control and experimental groups I am measuring the absolute and relative learning gain and applying the mann whitney u test to the measurements.
Mona,
The relative gain is a proportion with the original scores in the denominator. It is similar to effect size in that it is measure of the proportion of gain. You gave me the U value, but not the actual mean scores. This is what I need. Also, the total points possible.
How far did your sample deviate from normality, because if it was not large the statistical analysis is usually robust enough to handle it. The problem with lack of normality would be a Type I error so you can correct for this by using a more rigorous alpha level, for example.01. Also, if you can use ANCOVA it would be a more accurate comparison of the two groups.
It seems possible that the relative gain is insignificant because your subjects are already relatively good at the task/judgment and so the pretest is a high score on your scale. You need to think about which dependent measure is most likely to be used to test your theory of the subjects' behavior.
Thanks Stephen. This is still a question of the practical value of the change. I think there was a low magnitude of change (which would have been the case in your scenario) and the statistical difference on the absolute measure is simply an artifact of a high sample size.
Norm
The common practice in the assessment literature is to use what you call relative gain precisely because of what Stephen pointed out--it scales gain against what the students knew coming in. Without that, all you have is a change in scores before and after "intervention," which is not as useful. The particular statistics you use is mute in this regard.
A, Iman,
Sorry, the statistics you use are critical. Some take into account the pretest scores and some do not. This is why I previously recommended ANCOVA, not ANOVA.
The difference between absulote and relative gain is that the latter takes into account prior knowledge and the former does not; the latter is therefore typically preferred. The statistical question is different: it addresses whether or not the difference between the experimental and control groups could have been obtained by chance. It was in this sense that I said the second is mute with regard to the first, in the context of Mona's question. Perhaps I am missing something?
The absolute learning difference between control and experimental group could be due to their initial scores being different. That is why absolute differences are siginificant but not relative.
What is the definition of control? The experimental group was imparted with special coaching and control group being with normal coaching? Is it? The definition of experimental and control is critical here.
Did you use one tail test or two tailed test? I suggest that if the experimental group is imparted with special coaching when compared to control group, you have every reason to use one tail test. Please use one tail test and confirm the significance?
I agree with Norman, in this case. When scores are there why dont you use ANALYSIS OF COVARIANCE. Parametric test are always robust when compared to non-parametric.
If you are doing paired t-test then you dont need control group. The pre-test itself serve as a control for post test score.
In case of more doubt, you can send even the data to me, I will do the needful for you and sent you back the appropriate results with required details.
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis