Hi,
I would like to compare two sets of data using t-Student test in MATLAB environment.
The data are the values of parameter X calculated for n=88 pairs of signals where each pair contains the signals modified by two processing methods: Y1 and Y2.
There are very small difference of mean value of X parameter between these two processing methods (0.480(0.053) for Y1 and 0.478(0.053) for Y2, see figure below).
The question is: which MATLAB function I should use to obtain the correct p-value?
According to the documentation of ttest (https://www.mathworks.com/help/stats/ttest.html) and ttest2 (https://www.mathworks.com/help/stats/ttest2.html) I think I should use the first one, because it "returns a test decision for the null hypothesis that the data in x – y comes from a normal distribution with mean equal to zero and unknown variance, using the paired-sample t-test ", while the second function "returns a test decision for the null hypothesis that the data in vectors x and y comes from independent random samples from normal distributions ".
But using ttest function I obtain very small p-value (p
@ Mateusz Smoliński
You are right, the first test is the correct way for your problem, even thought the p-value is smaller. It is smaller comes from the fact, that the two methods to be compared are substantially different. As it is seen from your data, the result of the second method (Y2) is significantly higher then the resuls obtained by Y1 method. The difference between the tests shows, that the randomness in X's is caused mainly, by the source(s) of the signal(s), not by the methods of measurement; neither by the first, nor the second one. Summarizing:
Applying the first test you are comparing the different methods of measurement.
The second test would be appropriate when comparing randomness of two different sources of the signals.
Best regards, Joachim
As the previous answers have indicated a paired ttest is indeed done using ttest command, ttest2 allows for unequal size of groups and is not paired . To answer Lieven's confusion, for a paired ttest the key is how consistent the shift between the two element in the pair are, no matter how variable either elements of the pair are. I would argue both stats are important and give two different insights into your data.
It is critcal to connect your science with your statistics to understand EXACTLY what the stat is telling you. The key here is that majority of the variation within each of processing method (Y1 and Y2) derives from the same original data (Y0), so all points have the same underlying cause. You are then manipulating your data using two methods and looking to see if there is a consistent difference imputed by the two approaches.
The fact that there is a significant difference in the ttest result indicates there is a bias in the result obtained by the two methods (a p value in and of itself does not indicate one is desirable over the other, but that the two approaches will give different apparent results). The implication is that Y1 and Y2 are not interchangeable and use of one or the other must be justified by other means.
ttest2 is not significant because you are measuring the dataset variation resulting from Y1 against that arising in Y2. The fact is not significant suggests that the Y1 and Y2 methods are imputing a much smaller element of variability than exists in the original signal Y0. In your case
What ttest2 would detect is if there was a significant random effect in one method and not the other (in which case the variance would be enlarged in one but not the other).
If the goal of your analysis is to compare the two methods Y1 and Y2, I recommend an unpaired analysis, thus using the ttest2 function. The fact that you got a high p value means that the methods actually return very similar results, and this is also confirmed by the graphs and the mean values you reported. We are supposed to perform a paired analysis if we would like to test the effect of an event or an external action (for example, the effect of a drug in an analysis before-after), and I do not think it is your case. However, before proceeding with the comparison, check the distribution of your data. If data are not normally distributed, you should rather use the Matlab functions ranksum and signrank (instead of ttest2 and ttest). Good luck with your experiments.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Control Theory
- Control Systems