Hi Anuroop,

First off, how do you have a population where N=3? That is not much of a population, unless you are living on a deserted island or the North pole.

All kidding aside, your sample sizes are much too small to perform any parametric tests (those that assume a normal distribution, and usually require sample sizes >=30 (as a rule of thumb only).

My suggestion is that you perform Fisher's Exact test or more generally a permutation test, which will permute through all [n! / (n-k)!] = 720 possible outcomes of the 2 groups of 3.

Which test to use depends on what type of variable your outcome is. If it is a continuous variable you could perform a Two-sample Fisher-Pitman permutation test for equality of means, or an Exact Mann-Whitney ranksum test for equality of medians. If you have categorical values, you could run a Fisher's exact test. 

Or, you could use a permutation algorithm on any of the appropriate tests.

I hope this helps

Ariel

Hi Ariel.. Thanks for the reply. I might have used the wrong term probably.. Should have used Group1/Group2 or Treatment1/Treatment2. So just to make it clear, I was talking about qPCR data for control and cells treated with a drug and the number of replicates is 3. Hope this makes sense now, sorry for the confusion. However, I did try Mann-Whitney but since I have only 3 replicates in each group and the value of U and Ucric is both 0, I am not sure if Mann whitney would be the correct method. I didn't try Fisher's exact but will look into it.

Hi Anuroop,

I don't understand what you mean by "replicates", and that there are three of them and that the value is 0 for two different variables (U and Ucric)? This provides me less clarity than your original post.

What I need to know is:

(a) how many groups do you have? (I believe it is 2)

(b) what type of variable is the dependent variable (continuous, binary, nominal, ordinal)?

(c) what is the sample size for each group?

Ariel

Hi Ariel.. I am sorry for the confusion again.. I hope I can make it clear now!!

a) Two groups (as below)

b) Continuous data

c) Sample size is 3 for each group (as below).

The below one is just an example (not the real values)

Group A (Control) Expression      Group B (Treated with X at YuM) Expression

0.005                                             0.0521

0.006                                             0.0496

0.006                                             0.0538

Ahhh, much better. Yes you should consider either the Two-sample Fisher-Pitman permutation test for equality of means, or an Exact Mann-Whitney ranksum test for equality of medians. These are different than the standard version of the tests, because they test each permutation, rather than the group as a whole. Thus, you'll get the results based on 720 permutations as opposed to 2 groups comparing three outcomes.

Ah! Brilliant.. I will have a look and get back to you if I am still not clear. I haven't used them before but I hope they are easy to understand and incorporate! Thank you so much for patiently replying to my questions :)

If you have access to Stata, there are some great user-written commands that will do this for you. fptest, mwtest, or alignedranks. All these programs can be downloaded from within Stata by typing ssc install X (where X is replaced by the program name).

I was actually using XLStatistics but I guess I would try that but I have to check if I have access to that!! Hopefully I should.. Thanks for the suggestions..

I think you can get a trial copy from www.stata.com

I just downloaded it!! Thank you.. I am now trying to know how it works.. I hope I could contact you in case I have some doubts on using it!! Thank you..

Sure. As I wrote before you have to download those user-written programs into Stata (ssc install....) and then each program's help file will tell you how to run the command. There are examples in each one that make it easy to follow. I wrote alignedranks, so I can provide more info if needed.

I confime the notes of Ariel Linden mentioned prviously ,  

You can use Gpower software ( avaliable as free version in the web ) to calculate sample size for many types of tests . 

also, you can get information about sample size calculation from the attacments files

Good Luck

Hi Khalid.. Thanks for your reply.. I was able to get my answers from Stata software but it was a trial version.. I will download the Gpower and use it in the future.. Thanks for the suggestions and the documents on sample size calculation.. Thank you..

I apologize for coming to this thread so late--perhaps everyone has now moved on.  But I think there is an important issue that has not been discussed.  Any t-test (or z-test) has the same basic format:

t = (statistic - parameter | H0) / SEstatistic

When the statistic in the numerator has a sampling distribution that is approximately normal, the test will be reasonably valid.  So the question is what are the conditions under which that statistic has a sampling distribution that is normal? 

To keep things simple, let's take the single-sample t-test to illustrate.  The statistic is the sample mean (Xbar).  The shape of the sampling distribution of the mean depends on both the shape of the original population and the sample size.  Importantly, if the population from which you sample is normal, the sampling distribution of the mean will be normal, even if n = 1.  So when the population is normal, you do not have to be anywhere near n = 30 to get a valid t-test result.  The further the population is from normal, the greater the sample size you need to achieve a sampling distribution that is approximately normal.  The old guideline of n >= 30 is far too onerous, IMO, especially if the population is unimodal and not too severely skewed (e.g., see first link below).  Here is a short note I found on the stattrek website some time ago that expresses it nicely, I think.  (Unfortunately, the page as I copied it is no longer there.)

---- Excerpt from old Stattrek.com web-page -----

As a rough rule of thumb, many statisticians say that a sample size of 30 is large enough. If you know something about the shape of the sample distribution, you can refine that rule. The sample size is large enough if any of the following conditions apply.

• The population distribution is normal.
• The sampling distribution is symmetric, unimodal, without outliers, and the sample size is 15 or less.
• The sampling distribution is moderately skewed, unimodal, without outliers, and the sample size is between 16 and 40.
• The sample size is greater than 40, without outliers.

Source (no longer working): http://stattrek.com/Lesson3/SamplingDistributions.aspx

------ End of excerpt from old Stattrek page -----

For the independent groups t-test, which is the test most people would think to use for comparing two group means, the statistic in the numerator = Xbar1 - Xbar2. With n1 = n2 = 3, you would need to be sampling from populations that are pretty darn close to normal in order for Xbar1 - Xbar2 to have an approximately normal sampling distribution. 

One final thought:  People often behave as if the only source of information about the population they have sampled from is their own sample(s).  But often, that is not the case.  If your outcome variable is a well-known, much-studied variable, then draw upon the knowledge that is available from other studies.  Bland & Altman make this point in their nice note on analysis of continuous data from small samples (see link below).

HTH.  And again, sorry for being so late to the party!  ;-)

p.s. - In old textbooks, the cutoff of n >= 30 was usually more about when you could legitimately use the critical value of z rather than the critical value of t than it was about when you could legitimately do a t-test.  Bear in mind that tables of critical values would have gotten very large if such approximations had not been used.  Nowadays, there is no need to switch to using the critical value of z, because our software uses the t-distribution (regardless of sample size) to compute the p-value for a t-test.

http://www.bmj.com/content/338/bmj.a3166

Hello Weaver.. Thank you for the message... I feel there are some interesting statistical observations from what you have mentioned, particularly on the context of sample size and normality.. I haven't read the article you attached but I just wanted to ask a question regarding this.. Sorry if I the question is too basic.. 

I have read in quite a few articles, where they mention that they have used log-transformation (I think to normalize the data) and then carried out the t-test on the respective sample/population.. Do you think higher sample size would be important in such a scenario, where the normalization has been obtained by log-transformation? (Because I am not sure if any data obtained would always be a normal distribution)

I hope I have explained it clearly.. Sorry if its wrong!!

I found these responses very useful to me as well

One of the fundamental assumption of parametric statistical is normality of data. If your data violate this basic assumption, then you can then conduct data transformation test. However, the transformed data is no longer the original data you obtained from your sample/population. My suggestion is to use non-parametric statistical test when your data is not normally distributed and your sample size is relatively small.

I am responding to Anuroop's post on Oct 15, 2015.  Anuroop, if you are still following this thread, I apologize for having missed your question back in 2015.  I only noticed it now when Bakare revived this thread.

Regarding transformation of data, I think you may find the thread linked below useful and interesting.  Cheers, and apologies for my tardiness in replying. 

https://www.researchgate.net/post/Why_do_we_do_transformation_before_data_analysis

 Hello everyone,

This is a very relevant post for me. I have similar situation with very small sample size. However, my data is in percentage (continuous variable). I am comparing 3 groups with sample size 3 in each. Can anyone suggest which test will suit my data?

Dear Alka

You have a small sample size to be in normal distribution, so I guess that your data not normally distributed.

The non-parametric test (Kruskal Walis ) will be suitable for test the three groups.

Good Luck

The sample size is depend on the significance level. If your significance level is high, then your sample size should be large.

You can use two types of t-test formulas. First, for small samples (N < 30 or N < 50) and second for large samples. 

Thanks all. Can someone help. Currently I have problem of which test to use. I have 2 groups. In group 1 there are 3 subgroups with sample size of 32, 6 and 3 while group 2 has 36, 3 and 1 subgroups.

First, I am to compare parametric data....BP, height, hormonal values(FSH,LH) etc among each group.

Second, I also want to compare across subgroups.....32 vs 36; 6 vs3; 3vs 1...

What statistic is suitable

@ P.I. Ugwu

The first thing you would need to consider is if your data sets well normally distributed, even at that your sample sizes look small and probably you have been trying to ignore non-parametric tests, precisely Mann-Whitney and Kruskal Wallis, due to lesser power compared to the parametric types. But, you may face an external supervisor who could disagree with your parametric choice on the ground of your sample size.

Hi Kamoru,

Thanks much, I will go discuss with my supervisor

Hi,

Very good thread, thanks to everyone! One thing remains somewhat unclear to me though. Ariel Linden writes above that "These are different than the standard version of the tests, because they test each permutation, rather than the group as a whole", in reference to the Two-sample Fisher-Pitman permutation test for equality of means and Exact Mann-Whitney ranksum test. Does this mean, that it is possible to do a permutation test using the Mann-Whitney test with exact significance in SPSS? I have been told to do a permutation test because one of my two groups is so small and the data is skewed, but I can't really find any clear information on how to do this.

Thank you in advance!

Hi Linda,

Your question is very important in a real life situation or serious research. If you do not have normal population with small small in one group and highly skewed data, I will not recommend any test:). However for more than 5 observation on each group with low weakness(k

Sice poulation size is so small,therefore most of differences (if any at all) are likely to be confounded and making the results no-significant.

We need to add sufficient population so that stasticalanalysis could become viable,else it has no interpretation in terms of significance.

as a rule of thumb that I used for my experiment for my PhD (...but I could have been wrong!) to be able to test the assumption of parametric test I used a minumum of 5 sample (to test normality) and at least a possible variation of 30 results (to test that data are continuous).

Hi, what should we do when the data we have are binary, but we have multiple trials and the average for each is continuous?

For example, in the case of proportions of the sample that are positive for the binary factor:

Trial 1: Control: 3/50 Tx: 4/50

Trial 2: Control: 4/50 Tx: 3/50

Trial 3: Control: 2/50 Tx: 2/50

Trial 4: Control: 5/50 Tx: 2/50

I have been doing Fisher's exact test, but others have been using t-tests (and ANOVA when there's a second Tx group).

The population distribution is normal and significance level is low, then your sample size could be small...

T-tests are based on calculations of means and SD; these estimators or their derivatives (SE) are the correct display items --- as they directly correspond to the test distributions. Medians and percentiles show valid data distributions but not valid test distributions. If you use non-parametric stats it is the other way around; and now it is the means and SD (SE) that don't show valid/corresponding distributions to the test. We sometime show means and SE anyhow though for M-W tests, because many referees don't know statistics sufficiently well and start complaining because they don't understand the difference between data distributions and test statistic distributions and also they misread medians and percentile plots, and sadly this is also often the case with editors.

Are your data normally distributed? If yes, you should keep the t tests and show means and SE in the plot. Remember to state the distributions in the Statistics section of the supplement. If they are not normal then MW tests are fine but make sure to state that too in the supplement. I think you have sample sizes ranging 12-50, and I don't doubt that it can be valid to use the t-tests. But, point is you choose the test you can argue for in the supplement based on the properties of the data distribution/s. All the best!

First you have to do a normality test on your data (for both treated and untreated samples). for an example Anderson-darling test,etc.

if they are normally distributed you can perform independent sample t test. There is no minimum sample size for the t test to be valid other than it be large enough to calculate the test statistic.

if your data are not normally distributed, go for the Mann-Whitney U Test in non-parametric statistics to compare those two groups.

If you are going to compare the results GroupWise, I mean n1 and n2, you can use simply paired sample t test. For that you just need n=3 (the replicates. You can interpret the data by using P values. Hope you are fine with that. But here you have mentioned (20). If you are going to do it with a population, the size of the population should be 30 or more than 30. But do not confuse with sample size and the population size. These are two words and analysis are completely different from each other.

This is really an interesting debate. I also had such question when I was performing a secondary analysis. Today I coincidentally found an answer (err...may not be widely accepted?) to the question "Is it valid to compute a t test or ANOVA with only two replicates in each group?" on the website of GraphPad. It says "Sure. You get more power with more data. But n=2 is enough for the results to be valid. (But of course t tests and ANOVA cannot be done with n=1.)".

Hi my good researchmate!

Please do not use a sample of size 3 , if you have to use it the population must be normal.

At least do not make any inference and conclutions.

Again if you must do so, at least try some other appoaches such Bootstrap, or permutation test, but not traditional non- par test like Mann-Whithney etc.

Good luck

please fully sure that n=3 is eligible to use t-test, as i read statistical book mentioned this, n=3 can use t-test, but i forget the title of book name, i am fully sure that i have read this before, at that time, i am shocked and n=3 can use t-test, please patiently to search this reference to support this

de Winter, J.C.F. (2013) "Using the Student's t-test with extremely small sample sizes," Practical Assessment, Research, and Evaluation: Vol. 18 , Article 10. DOI: https://doi.org/10.7275/e4r6-dj05 Available at: https://scholarworks.umass.edu/pare/vol18/iss1/10

maybe this book, i am not quite sure

t-test can be used with a sample size of 3. You have to make sure it follows the basic assumptions for the t-test before performing the t-test on small sample size. See more here https://www.reneshbedre.com/blog/ttest.html

Please send