The sample size is a critical factor only when the assumption of the conditional normal distribution is clearly unreasonable. If that assumption is reasonable, small simple sizes are not problematic.

[EDIT: see Shanes comment below, from which the following correction is cited]

"It should read:

The sample size is a critical factor only when the assumption of the conditional normal distribution is not clearly reasonable. If that assumption is reasonable, small sample sizes are not problematic."

The sample size is a critical factor only when the assumption of the conditional normal distribution is clearly unreasonable. If that assumption is reasonable, small simple sizes are not problematic.

[EDIT: see Shanes comment below, from which the following correction is cited]

"It should read:

The sample size is a critical factor only when the assumption of the conditional normal distribution is not clearly reasonable. If that assumption is reasonable, small sample sizes are not problematic."

Taylor -

You might get a suggestion for hypothesis testing, and i have worked extensively in the past in that area.  However, the problem is that of doing a meaningful analysis of effect size.  The term "significance" is a misnomer, and p-values represent incomplete information which is very often misleading.  You change a p-value just by changing the sample size.

A confidence interval for a parameter (or prediction interval for regression "predicted" values) is also impacted by sample size, but in a far more practically interpretable manner.  Here you would apparently (if I read your question correctly) need to compare each of three populations, as represented by your three samples, one pair at a time.  That would be three comparisons.  (Say, A to B, A to C, B to C.)   In each case, you could find a confidence interval about the difference in the two estimated means.  With a small sample size you might assume the Central Limit Theorem might still let you assume a t-distribution when constructing the confidence interval.  Or you might consider Chebyshev's Inequality. 

With small samples, the confidence interval about the difference in your means in each case will be quite wide for any reasonable confidence level.  However, that is all you have, so this usefully points out the limitations of your data in making a decision. 

It sounds like you may have three independent data sets, and so covariance is zero and not needed in estimating a confidence interval about a difference in estimated means.  You might take a look at the part regarding means in the following:  

 https://onlinecourses.science.psu.edu/stat100/node/57

.....

It might be useful for you to report estimated mean, and estimated standard error of the estimated mean, in each of the three cases.  That would give you a very small, interpretable table, which says much, even at a glance. 

Cheers - Jim

 Although Jochen is right, the theme of the sample in Experimental Design is slightly different, the number of replicas per treatment depends on the number of factors present and other things.

The type of statistical analysis to be applied, will depend on the corresponding assumptions.

If the measurement process meets at least a quasi experimental design, I do not see why not use some kind of anova.

Thank you for your responses. I think I was not clear enough. I am using a randomized control trial with a pre-post measurement of PTSD symptoms. There are 2 intervention groups and 1 control group. I know how to calculate the changes between pre and post, however, I need to find the means of each group and see which had better or worse change in PTSD symptoms after the intervention. I was told by my advisor that 30 participants meets the bare minimum for ANOVA so he thinks I should do something else. 

If your data does not support Normality assumption you are better off using a Non Parametric one-way ANOVA test such as Kruskal-Wallis test. You can get most of the details by searching Wikipedia. Most of the statistical packages support this test.

Dear Taylor

There are many assumptions related with ANOVA one of them is the normal distribution, you must test your data for normality, and variance homogeneity.

In case of available of assumptions you can perform ANOVA, but in case of no assumptions available, then you can use nonparametric tests ( Kruskal-Walis test)

Good Luck

Taylor,

Is not a general value, it is a value per cell, and this number varies, depending on the number of factors.

What happens is the level of conclusions that you can reach with those levels,

Being a sample of that size, perhaps calculating the power of the test may also be an option to back up your first analysis

There is a small error in Jochen's statement:

The sample size is a critical factor only when the assumption of the conditional normal distribution is clearly unreasonable. If that assumption is reasonable, small simple sizes are not problematic.

It should read:

The sample size is a critical factor only when the assumption of the conditional normal distribution is not clearly reasonable. If that assumption is reasonable, small sample sizes are not problematic.

Let me give three examples to illustrate some possible scenarios. 

Example 1: We can reasonably assume the conditional normal distribution is reasonable. If we have a strong theoretical argument that the conditional distribution is expected to be normal, then small sample sizes do not pose a problem. Suppose the "experiment" was using a random number generator to generate normally distributed data, then we can reasonably assume that the conditional normal distribution is approximately normal. Here you use knowledge about where the data come from to justify the procedure. 

Example 2: We have large sample sizes but no strong a priori argument to justify the conditional normality assumption. If inspecting the data suggests the assumption is reasonable, then the anova results are approximately valid.

Example 3: The conditional normal distribution is clearly unreasonable. Suppose the distribution were actually cauchy. This is to motivating example for opening this comment. Here, the assumption of the conditional normal distribution is clearly unreasonable, but the sample size is not a critical factor. If the distribution were cauchy (or some other poorly behaved distribution) then no matter how much data are available, anova would not be appropriate. 

In example 2, I use "inspect" rather than "test" to be purposefully vague. It is possible for data sampled from a normal distribution to fail a formal test for normality. However, even if such a test were to fail, if the conditional distribution is known a priori to be normal, then anova is still valid.

Shane, yes, that makes sense. Thank you for pointing that out. I edited my comment above.

 There is nothing inherent in ANOVA that is rendered invalid by using a sample size of 10, or even 3 for that matter. As Jochen and Shane have said, the critical factor is the underlying distribution. Even given a normal distribution, the problems with small sample size are with power and repeatability. With a sample size of 3, it is likely that another researcher will come to substantially different conclusions if they repeat your experiment. If thousands of other scientists repeat your experiment, the average of all these trials will converge to the population mean and standard deviation. the distribution will be Gaussian by the Central Limit Theorem. As sample size increases, the error associated with taking a sample rapidly declines.

Defining "small" sample size depends a bit on your field. If this is 10 surveys from three populations, then it is probably too small to be meaningful. If this is 10 field plots for an insecticide trial comparing three insecticides, then this is plenty -- or maybe I should rephrase and say that there are many published research reports that have far fewer replicates.

If these are the results from three independent experiments, then there is another problem to consider. The "treatment" effect may be confounded with a "time" effect or a "location" effect. Treatment A was assessed at hospital 1, while treatment B was assessed at hospital 2. There is no way to know if you are seeing a treatment effect or a hospital effect or some interaction between the two.

@ Timothy, minimum sample size suitable for ANOVA and other tests depends on standard deviation of the data ( can be estimated from previous experiments) and the margin error that we adopted, so we can not give a certain number to be suitable for all case, in some case sample size = 3 will be okay but in other cases will be very small.

Regards

It is true that population (sample estimated) standard deviation, not some universally approved number, is key, in relation to n, to estimate a standard error.  When sample size gets too small, we also have to be concerned about having a passable estimate for standard deviation.

@Khalid, Agreed, and therefore it is not sample size per se that is the problem. A sample size of 3 may be fine for one experiment and a sample size of 100 may be too small in another context. As Shane pointed out there may be cases where ANOVA is not appropriate at any sample size. If anyone comes to me and says that they have a sample size of 10, I cannot justify claiming that the ANOVA model is invalid (this was part of the original question). We need information that the advisor to this student may possess, but that was not included as part of the background information. Things like: does the data conform to the assumptions of the ANOVA model? Is ANOVA appropriate to the experimental design? How did you decide that the sample accurately represents the population? Given what you know now, what was the power of the test? What sample size would be more appropriate?

To provide the best help, we need Taylor to provide more background. What I hope Taylor gets from all of these comments is a better understanding of what sort of information is needed to answer his question.

I agree with @ Timothy. Khalid you can run the ANOVA on small sample size. It's all about that you are checking the variance between the sets.  

The problem is more serious is not the size of the sample, the problem is to know if a previous experimental design was made, but, by much normality or not, the answer provided by the analysis is quite poor.

Thank you all for your help!