First, if you assume your data are completely randomised, your n is not 3 but 6: three of which have been exposed to condition 1, three of which are a control. Secondly, check the definition of a distribution, you will understand you can't honestly test normality for 6 data.

Besides, it is not the normality of your data which is important, it is the normality of your residuals.

Anyway, I guess you applied an analysis of variance, explaining your response "y" by the factor "A".

Your statistic model is y = mu + A + eps.

If you can't prove your residuals (eps) follows a normal distribution, you don't necessarily have to reject it neither. You can still assume eps (the residuals) follows a normal distribution based on your expertise for this kind of data.

The Fisher's statistic tells you if A is significant.

Then you use the student t-test to know the significance of the effect itself (control versus condition 1).

At this point, the most important condition for such a test in your specific case would not be the normality, but the homoscedasticity: the range of variation within your control must be comparable to the range of variation within your condition 1. Visually, you have a data high (309) in comparison to the other, which might reject the homoscedasticity hypothesis. But again, you are working with only three data in each group, and in this case this high value doesn't seem very alarming. If you still worries, the log transformation gives you the homoscedasticity between your groups (control / condition 1).

François COLLIN

NB: being pragmatic, the result is: your condition 1 is a significant effect, with or without the log transformation. Myself, I would use the results without log-transformation.

If your variable is something that is known to be usually log-normal, and your audience will understand this, you might transform it, unless you have reason not to.

I work in water quality, and concentrations of things that are thought of as pollutants (nitrate, bacteria) are often log-normal.  I recently presented on bacteria counts in a lake, and presented geometric means.  The audience didn't bat an eye because they are familiar with the fact that bacteria counts are often log-normal and so are often log transformed. 

As a matter of fact, regulations in the U.S. concerning bacteria counts are expressed in geometric means.

*     *     *

Be sure you are thinking of _conditional_ normality.  That is, if you constructed a model for anova, it would be the _residuals_ that are supposed to be normal (in the general linear model interpretation of anova).  So you don't need to look at n = 3, you can look at N = 3 x number-of-treatments.

A formal test of assumptions of other tests is nonsensical. It also gives you no useful information, as it does not provide you any meaningful effect size that you might try to interpret. In small samples the power to detect a relevant disagreement with the assumptions is likely too low, so "non-significant" can by no means considered "non-relevant".  With larger samples, such tests will detect also non-relevant disagreements, so "significant" is not neccesarily "relevant". Further, the logic behind these tests is to interpret the effect only when the tested hypothesis is rejected (i.e. only "significant" results are "interpreted"), but here would obviousely interpret "non-significant" results as "in [sufficient] agreements with the assumptions", what is a logical no-go.

You should think if the assumptions are reasonable/sensible for your kind of data. You will need to understand (at least vaguely) how your data is generated or you conduct a well-planned experiment especially devised to give you data that is able to discriminate between possible distribution models.

Often, biological data is some kind of concentration (and I will just assume that your data is some concentration), and the gamma-model  and log-normal models are likely a good models to describe our knowledge about concentrations. A t-test using the log-transformed data may be sufficient for your purpose. 

PS: The expected increase is 4.4-fold. The gamma model gives me a 95% confidence interval for this estimate of (2.7...7.4)-fold. Calculated via the log-transformed the data:  (2.1...9.6)-fold. Thus, both methods give the same point estimate, but the gamma-model allows a slightly more precise estimation. This is also reflected by the p-values: 0.0046 for the gamma-model and 0.0051 for the t-test using the log-data.

I found this explanation here, i thought it was also pretty reasonable: 

"Why must you test for normality?

The standard assumption in linear regression is that the theoretical residuals are independent and normally distributed. The observed residuals are an estimate of the theoretical residuls, but are not independent (there are transforms on the residuals that remove some of the dependence, but still give only an approximation of the true residuals). So a test on the observed residuals does not guarentee that the theoretical residuals match.

If the theoretical residuals are not exactly normally distributed, but the sample size is large enough then the Central Limit Theorem says that the usual inference (tests and confidence intervals, but not neccessarily prediction intervals) based on the assumption of normality will still be approximately correct.

Also note that the tests of normality are rule out tests, they can tell you that the data is unlikely to have come from a normal distribution. But if the test is not significant that does not mean that the data came from a normal distribution, it could also mean that you just don't have enough power to see the difference. Larger sample sizes give more power to detect the non-normality, but larger samples and the CLT mean that the non-normality is least important. So for small sample sizes the assumption of normality is important but the tests are meaningless, for large sample sizes the tests may be more accurate, but the question of exact normality becomes meaningless.

So combining all the above, what is more important than a test of exact normality is an understanding of the science behind the data to see if the population is close enough to normal. Graphs like qqplots can be good diagnostics, but understanding of the science is needed as well. If there is concern that there is too much skewness or potential for outliers, then non-parametric methods are available that do not require the normality assumption."

But again, residual normality, based on n=6 data can't be checked although It can be assumed regarding your expertise on your data. You can use all the tests you want or graphical representations, you just can't use them to reasonably conclude for Normality whatsoever.

However, homoscedasticity has to to be confirmed for your analysis. Beside your knowledge of your data, it is the only justification for your data transformation (log or gamma or etc. ...).

Look for some nonparametric permutation test. Normality tests start with at least n=12.

@ Leo Vohandu : with n=3, non parametric tests are much likely to be unable to detect anything, whatever the values are (Mann-Whitney test for instance will simply never ever reach the 0.05 level, even with so obviously different data than the one presented here).

Who says that o,05 level is needed...

TRiviallty  3!3!/6! is eexactly 1/20= 0,05... Leo

How you   decide  is zero-hypothesis   yes or no  significant..

Normality assumptions are not working with such small data sets. Therefore, a simple rerandomisation test can and will do that:

From all 20 rerandomisations of the attribution to treatment and control group, the observed is the smallest. That means, the p-value of the data is exactly 1/20 = 5%.

In this way your are just at the edge of accepting and rejecting.

@Manfred, why show assumptions not work? Why should the sensibility of assumptions depend on the size of a data set? Small data sets only make it impossible to check assumptions - but checking is not always neccessary. For instance, I may reasonable assume blood glucose levels in healthy 20yr-old female humans normal distributed. This does not change with the number of actual data I have.

@Jochen, why bother about assumptions if it is not possible to check them. Of course, from the design and pre-knowledge about the context, one might be able to justify a normal model.

The question about assumptions (apart from a random sample) is obsolete here insofar as the observed mean difference of 182 is significant anyway (at the 5% level).

Furthermore, a confidence interval based on Chebyshev intervals (which are valid for any distribution) shows that values between 0 and 363 have an observed confidence level of 14%; while this is an unusual application of confidence intervals with fixing the confidence level with hindsight, the method indicates that there are substantial differences. It might, therefore, pay the effort to obtain a few more data to check whether this difference can be "confirmed".

Of course, if one invests more assumptions into the problem, one gets more precise answers about the difference between the two groups.

@Manfred: agreed. Without making assumptions we can not infere anything. Assumptions are the basis of any understanding. More assumptions do not only allow us to get more precise answers (as you said) but also to get answers at all. The key is that the assumptions (and thus the questions we ask) must make sense for the problem at hand. It is absolutely legitimate to argue about the senisiblity of assumptions only based on theoretical considerations (the assumptions must be plausible). The results of an analysis based on such assumptions are valid (because they always refer to the assumptions*). However, if the assumptions will one day turn out to be non-sensical, then we will recognize that the (valid) results are non-sensical, too. Therefore it is a good strategy to check if our assumptions are supported by our experience. A bad practice, however, is to try to get this experience from a small set of data that was generated for a different purpose. It is rather illogical to perform an experiment and then decide on the inferential analysis based on features of the observed data. It would be appropriate to first generate data to get an idea about which assumptions are (in)acceptable, then decide for an inferential model, then generate new data and analyze this using the model we already have decided for.

---

this might be a point where many say: no! - so here is a short explanation:

I have some data and I use a t-test to get a p-value. This p-value is, say, 0.01. I say that this result is valid, irrespective of the actual distribution of the observed values. How can I say so? Because this p-value is the probability to observe more extreme test statistics that the one calculated from my observed data under the assumptions of the test. There is not more to say. If I did not a calculation mistake, than the result is valid. If it turns out that the assumptions are clearly inappropriate for the variable (e.g. the variable are survival times), the p-value is still the probability to observe more extreme test statistics that the one calculated from my observed data under the assumptions of the test. Still valid, but now known to be not very sensible. We asked a non-sensible question, and the math gave the valid answer (e.g. asking for a mean difference in survival times is not very sensible; it would make more sense to ask for the hazard ratio instead).

@Manfred: so what do you mean by rerandomisation? As in I need more data point? so for each condition (control or treated), there are 3 technical replicates (those 3 numbers you see above). Each technical replicate was calculated from at least 7 random data sets. So what you suggest is for me to take 20 random data sets? Or its not needed? 

@Anh, you ask for rerandomisation. First the task with the 6 data, then general remarks.

With 6 data, 3 in treatment and 3 in control group, everything that is possible can be denoted easily. To test the null hypothesis that no difference in mean between the two groups exists (with no specified alternative), the following scheme is appropriate:

If there is no difference in mean, both data sets can be pooled and then re-attributed to treatment and control. We have 6 choose 3 = 20 different possibilities to select three treatment units, the remaining are the controls. For this selection, we can calculate the mean difference between treatment and control. We have 20 values for the statistic difference of means. All values have the same probability (any selection has "equal right"). For the data set given in the initial question, we have

-181.7, -78.3, -96.3, -13.0, -87.7, -105.7, -22.3, -2.3, 81.0, 63.0, -63.0, -81.0, 2.3, 22.3, 105.7, 87.7, 13.0, 96.3, 78.3, 181.7.

The observed difference (in the initial sample) is the smallest of these and thus has a p-value of 5%. You might orientate more easily in the attached Excel file.

General remarks:

If you have a group comparison (related to a target variable): classical, the control group and the treatment group. Let us make specific that there are 25 in each group. We have observed x1, ..., x25 in the treatment group and y1, ... y25 in the control group, with an observed difference in mean.

Rather than modelling the distribution under the two conditions (e.g., by a normal distribution with the same variance but with possibly different means), we could argue as follows:

Under the null hypothesis of no mean difference between the two groups, it does not matter where the data comes from, any treatment unit could evenly be a control unit. Thus, any selection of the treatment group from all persons in the study has the same probability.

If it is possible (as numbers are small) to find all different selections for both groups with the related means and mean difference, then we have a finite population of differences (all of equal probability) and can rank the observed value within this distribution.

If the size of the groups is larger, we cannot establish all cases: for the 25/25 design we would have to  follow up more than 10^14 (more than 14 zeros after the leading 1) different re-attributions of control group and treatment group labesl to the units.

In this case,  we sample from the whole test units a subset that constitutes the treatment group and the non-selected are the control group. We RE-sample from the data, therefore, the method is called resampling. If we resample enough times, we get a reliable estimate of the distribution of all possible selections of "treatment" groups and the related mean difference. Therefore, we can rank the observed mean difference in the original sample. If it is within the smallest 2.5% or in the largest 2.5%, we would reject the null hypothesis of no difference in the mean between treatment and control at the 5% level..

The method is assumption free as refers to any parent distribution. However, the basic assumption that the first data is obtained from a random sample (of any distribution) has to be fulfilled.

The name rerandomisation comes from the fact that in many group designs the units are randomly attributed to treatment and control group. In a way, we re-randomise the first random attribution to the groups.

@Manfred: Thank you so much for the very detailed explanation. So if I understand this right, going back to my example of 6 data points, if the difference of my original data wasn't 181.7 but 105.7, the P value will still be 0.05, am I correct? Because ±181, ±105.7 are still within the 2.5% extreme. 

@Anh

yes, at least if it were a little larger than105.7. The small data set does not allow to do it better.

However, if the observed mean were 106 (let us agree about that), this would change the confidence level considerations of the Chebyshev interval. It would turn out as non convincing about the size of difference between the means.

There are two considerations:

• to assure that there are differences (by p value);
• to estimate the difference by confidence intervals (or - distribution free, by Chebyshev intervals).

@ Leo: you're right for the 0.05 level, it was not mentioned anywhere, I made an unsupported assumption. Sorry.

@Leo & Manfred: I think you're p-value computation assumes an unilateral hypothesis testing, which is not suggested in the original post (« different  »), and, as described, seems to be in the direction suggested by the data, hence not a true unilateral hypothesis. For the bilateral test p-value, you should consider both extremes, and the p-value becomes 0.10.

@Anh Hoang Le: if your 3 values (3 technical replicates) come each from 7 random data sets, there's something in the structure of your data that was absent from your original post and that may well completely change the possible statistical methods to handle them and answer your question (like hierarchical model). But I may well have misunderstood what you meant.

@Emmanuel: embedded in the setting of which I thought, a one-sided test is appropriate. I would not reject the null hyhpothesis of no difference in favour of the possibility that treatment is worse than control. I would reject in only in favour of treatment is better. Pilot studies, stage II studies would already have indicated that treatment may be beneficiary.

However, you are right, in a general setting, one would test two-sided, which doubles p-values then.

@Emmanuel: Ok, so the way that i obtained the data usually go like this: so for example: for the first biological replicate (1st set): I have 3 technical replicates, let say 3 coverslips; from each of these coverslips, I select 7 random representative points to obtain the data (much like a simple random sampling). So for 3 technical replicates, i will have in total 21 points. Then I repeat this whole experiment again in a different day (2nd set), then again in another day for a total of 3 days (3rd set) => 3 biological replicates..

So in total, 3 for the control and 3 for the treated set.  

@ Manfred: right, unfortunatly the original description does not give enough details about what the control and conditions are to determine if a one-sided ot two-sided test is best.  Beside, I've heard that often, even in pilot studies or stage II studies, two-sided tests (or 2.5 % one-sided tests) are often performed, to avoid missing a deletary (or other direction at least), unexpected effect. But that's another debate.

@ Anh Hoang Le: definitly, you do NOT have n = 3. You have a complex design with n = 7 values nested in n' = 3 samples (or whatever wording you like). Just using the average (or any other method you use to resume your 7 values in a single one) of the 7 values and then compare them between the n' samples loses many information, including the variability between the n=7 values that may well be higher than between the n' = 3 samples.

A more correct way to proceed would be to use a hierarchical model/a two-ways analysis of variance  with nested factors/a mixed effects model (don't know which vocabulary you're most familiar with).

As a general rule, never ever ask for statistic advice without giving full details on the experimental design, or you have a high risk to obtain unappropriate advices...

@Emmanuel: In pilot studies test two-sided, of course; in the main study, it could be tested one-sided if pilot study results are indicating the direction strongly - and else there would not be a main study.

Still, after you insisting about the design, the details are not openly clear to me. Final analysis will depend on how the three units bear the single data. Maybe Anh gives us the full data set and a documentation how they are linked and how the 3 scores are obtained in the treatment and control groups.

@Emmanuel: I see, i'll have a look into the two-way ANOVA and see. I have not done much statistics before so I didn't know the scope of the problem or how serious it is. That is why I'm learning it now. 

@Manfred: Yes, so let me describe how I obtained these data better: 

Treated group vs Control group: Each group has 3 technical replicates (so means that they are identical in treatment and condition). From each technical replicate, at least 7 data points were obtained, these will then be pooled together to form 1 single number (this is just purely adding them all together). So in total, 3 technical replicates gave 3 data points. Then from these 3 data points, they were averaged out to give 1 single number. This number is one of the number you see in the question. 

So I repeat this whole process again 2 more times, and thus have 6 data numbers in the question. 

@Anh: then I would check by ANOVA whether the number of replication has any influence. I the replication has no influence, I would pool the data (not add) and test for difference between treatment and control group.

@Manfred: So let say there's no influence fro the replication, then I can just use t-test to test for the difference between the two groups, right? For nested ANOVA, this example will be 3-factor nested ANOVA and neither Prism nor SPSS have the power to do this. I wouldn't go for R as this requires too much programming for me.  

If there is no difference for replication number in the ANOVA, I  would simply pool the data and apply a simple t test for group differences reassuring beforehand whether the variances are equal.

I'm quite surprised that SPSS does not allow a 3-way nested ANOVA, it can do much more complex analysis... May be it's called "hierarchical model" or "mixed effects model2.

Beside, you say that for each replicate, you take "at least" 7 points, then add the values. But if the number of points is not the same for all replicates, you'll have trouble to distinguish if a change in the sum comes from a change in the value or from a change in the number of points (the more points, the higher the sum). Why not use an average, which does not have this problem, and is basically the same thing that the sum if the number of points is the same in each replicate? Or use the individual values in the model if you manage to get it work in SPSS...

(note also that by using individual values, you'll be more confortable to check your distributionnal assumptions if needed... Here, you have at least 6 * 3 * 7 values if I understood correctly...)

@Emmanuel: Thank you very much, that's a good point. I can't remember exactly how many data points did I take, but I can go back and double check. I did average out, but again as you said It's better to have equal number of points.