1) I would not trust data with such small sample sizes, one participant can turn the whole data and therefore interpretation around. Dont try to fix bad data with any fancy statistics. It wont work.

2) Just describe your data descriptively and tell your story.

Your last statement is utterly worrying: there is no need to "make data more significantly"!!! Even for large data sets, it is not about the p-values but the effect sizes. Please read more about the flaws of null hypotheses significance testing (NHST). The NHST procedure is flawed in the first place and your goal should not be to get anything significant, but to tell the truth to the audience!

Hello Parivash,

I concur with Rainer Duesing 's suggestions. The reality is, with such small sample sizes, no family of tests: parametric, non-parametric, or exact/resampling, will have very much statistical power to detect differences unless those differences are profound in magnitude.

Stick with descriptive statistics. You can, as well, estimate and report an effect size. However, your caveat should include a recommendation that additional data are needed in order to arrive at a (more) confident conclusion.

Good luck with your work!

Rainer Duesing and David Morse, I worry about the advice of show things descriptively. What do you recommend showing/reporting?

If your inferential statistics show too much uncertainty in the estimates to give much confidence in even the direction of the effect, report this uncertainty. Rather than report the point effect size, report the confidence interval of the endpoints.

Maybe I am missing something, but I would not want the questioner to come away thinking that if you have a small sample that you should ignore the uncertainty in estimates and just say which sample mean is higher.

Daniel Wright maybe I was not clear enough. I do not recommend to report effect sizes for such small samples, but if this sample is the only information available and you want to share it with others, just report the descriptive statistics, nothing more, nothing less. Effect sizes can be problematic even in larger samples, because they directly rely on the measurement error of the mean and the standard deviation (in case of any d-family ES of course). So, in case of n=4 and n=2, there is not much one can do, because you cannot rely on the precision of your estimate (and a CI with a very large size would be rather uninformative, wouldnt it?)

Rainer Duesing : "and a CI with a very large size would be rather uninformative, wouldn't it?"

Yes, and that is the point. Saying that the data do not convey much precise information may or may not be telling readers much, but it is not misleading. And it is telling them something about the study.

As an aside, I am confused about why you are okay reporting descriptives like sample mean and sd, but against reporting the ratio of these (or okay reporting two sample means, but not the difference between them). I think I am just missing something (and it is a different point than what the questioner is asking).

Dear Rainer Duesing Thank you for your reply

I understand what you mean. Actually It was my bad luck to have this small sample size. I was trying to figure out if there is another way to analyze the data.

Dear David Morse

Thank you for your suggestion.

I just thought maybe descriptive analysis is not enough for analysis data.

Dear Daniel Wright

As you mentioned, I did the descrptive statistics as basic analysis.

and I found out there is improvements in EX group. but I don't know how to show the improvement since it's not significant.

I was wondering if there is another analysis method to show the improvements.

Parivash Jamrasi , the issue IS exactly whether you "found out there is improvements in EX group." I would say no. Your estimate of the improvement has a wide confidence interval. From what I gather, you do not have confidence in the direction of the population effect (or whether this sample difference is attributable to things other than the experimental manipulation). If I am misunderstanding this please correct me!

Dear Daniel Wright

You are right... the problem is sample size which makes it complicated to see the result. the sample size is too small to make a conclusion out of it.

Thank you for sharing your opinion with me. I appreciate it

How many time points of observations are available? If you have observations at multiple time points, the variability might decrease a little in your estimates.

Akhil Goel

I measured for 2 times. Pre-test and Post-test (after 8 weeks)

Could you please explain more about what you mentioned?

Wilcoxon's test needs a minimum of n=6 in one group, not two. If you use Fisher's test it could work e.g. with 2 x2.