I have two datasets: "Model data" and "Measured Data". There is not significant difference between these two datasets. My question is how can I statistically justify that there is not significant change between two dataset. (I have drawn graph and both dataset are almost overlapping with each other).
@Tiziano Suran,
No. A t-test tells you whether the difference between two sample means is "statistically significant" - not whether the two means are statistically different. A t-score with a p-value larger than 0.05 just states that the difference found is not "statistically significant". It does not say that there is no difference that can be relevant in statistics.
It's a or b. a) there is a statistically significant difference between those two sample means or b) there is no statisticall significant difference between those two sample means.
but it's important to note that b) =/= no (statistical) difference
More on topic:
I put my answer simply, because there are many missconceptions on null hypothesis significance testing (NHST). One of them is, that a non-significant result means no difference.
My advice is still: use your graph and error bars for argumentation (and justificataion).
Dear Gazi
You can test the data through two samples independent t-test or paired t-test depending upon the situation, if your response variable is quantitative continuous data.
Thanks
@Rajiv Pandey,
No. non-significant t-test does not mean that there's no difference. The reason for this is simply put: t-test assumes with H0 (initial hypothesis) that there is no difference. If you can assume there are no differences between two samples, then you don't need to test them for that.
@Gazi Md Riasat Amin,
There is a small difference between the two datasets. We can see it. If you mean to say that both samples (datasets) are drawn from the same population you can argue by stating the dispersion values (Standard deviation, Standard Error of the Mean,.. ) and introducing Error Bars. They might show that both lines lie within a common range.
Of course all other factors should be held constant as good as possible, else there will still be enough room for speculation.
Article Failing to Reject the Null Hypothesis Does Not Mean that the...
@Alexander Steiger,
The t-test tells you whether two means are statistically significantly different or not, provided its assumptions are met (e.g., you first have to test the normality of the distribution). The point you make is valid for your general non-binary hypothesis, or your interpretation of the test. With a t-test you have only 2 alternatives: a) the two means are statistically different and b) the two means are not statistically different. It's either a) or b).
@Rajiv Pandey,
data are not normally distributed.
@Gazi Md Riasat Amin,
As Rajiv wrote, if your DV is continuous data, you can run a t-test. However, in this case, given the non-normality* of the distribution (as far as I can tell from the graph), you can run a Kolmogorov-Smirnov two sample test to check if the two distributions differ, and that's it.
*To test the (non)normality you can run a Shapiro-Wilks test, or the same Kolmogorov-Smirnov test, but this time comparing each line to a normal distribution (one-sample KS test).
@Tiziano Suran,
No. A t-test tells you whether the difference between two sample means is "statistically significant" - not whether the two means are statistically different. A t-score with a p-value larger than 0.05 just states that the difference found is not "statistically significant". It does not say that there is no difference that can be relevant in statistics.
It's a or b. a) there is a statistically significant difference between those two sample means or b) there is no statisticall significant difference between those two sample means.
but it's important to note that b) =/= no (statistical) difference
More on topic:
I put my answer simply, because there are many missconceptions on null hypothesis significance testing (NHST). One of them is, that a non-significant result means no difference.
My advice is still: use your graph and error bars for argumentation (and justificataion).
Dear Gazi
You must test your data for normal distribution, you can use T-test for normal distribution, and use non-parametric test ( Mann Whitney) for non normal distribution.
Good Luck
Dear Gazi,
I completely agree with colleague Khalid Hassan who has explained you very simply what you need to do.
Wish you success!
+1 for Tiziano Suran's answer. Kolmogorov Smirnov test is independent of distribution and would be suitable for your data.
Another approach would be to rather than trying to establish whether the two datasets are different or not, you can flip the question and ask if the two datasets are actually similar. One of the ways to do that would be to divide the data into separate bins and perform chi square test for association. Jaccard Similarity Index might also be useful here.
Dear Gazi,
I would suggest that you specify the question addressed in each of the datasets, that you state clearly the types of variables you want to compare. This will determine the choice of the appropriate statistical tests. I hope this is helpful.
I am not sure what you mean by measured and model data, but I am assuming measured is the observed data, and that there is a model and you can use this model to generate data. I assume the model includes stochastic processes and error terms, so you can generate lots of model data sets. A fairly standard approach is to decide on some small number of characteristics of these data sets (e.g., mean, variance) and see how many of the model data sets are more extreme on these values than the observed data. Some people like p=.05, so let's say 95% of your model data sets have a variance of 3 or less, and your observed variance is 4, you could claim that the observed data are unlikely to arise if processes that gave rise to your model data (i.e., the MODEL) generated them (i.e., it is statistically significant ... which of course has its own problems).
Thank you very much everyone for your kind co-operation and helpful suggestion. It helped me a lot to choose the right way for statistical analsysis
for doing Kolmogorov Smirnov test, you can install XLSTAT and in excel, go to XLSTAT / Nonparametric tests / Comparison of two distributions.
It may be too late. But here is the link which shows what test has to be used for data comparisons.
https://cyfar.org/types-statistical-tests
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Graphs