The concept you are asking for is the (statistical) "power" (of a hypothesis test).  You may start reading here:

http://en.wikipedia.org/wiki/Statistical_power

or here:

http://www.biostathandbook.com/power.html

or here:

http://www.ats.ucla.edu/stat/seminars/Intro_power/

Practically (if you know the background, your design, your test, the expected variance, the desired effect size and all this), you can then practically calculate the required sample size with G*Power (it's free for academic use):

http://www.gpower.hhu.de/

Many thanks I will definetly use it

To be honest I dont know if i have to espect a normal distribution. The main question is to make a previson of the number of animals necessary to get significant data. I could then ask to the ethic commetee  to host in the animal facility just that precise number of mice. Generally we were asking 5-10 mice per condition and these numbers were fine to get statistical significance. However now the control is getting stricter and they are asking statistical calculations to get a previson of the  required mice number

Fausto,

I think we agree that almost all tests in reality are only approximative (some exceptions are the "exact tests"). The approximation will be better for larger sample sizes, and less good approximations are not critical when relatively large effect sizes are to be tested.

Now there are two scenarios possible:

- a small effect should be tested. Any exact or approximate test will require a large sample size to achieve a reasonable power. Buth then the approximation will anyway be good enough. (Example: binomial test, to distinguish p1=10% from p2=20% with a power of 90% and alpha of 5% would require n=266 in each group, what will make the results of the binomial test and the approximative test for proportions (based on the normal approximation of the binomial) really negigible for all practical concerns. A similar argument applied for other pairs like MW-test and t-test)

- a large effect should be tested. The required sample size will be smaller. However, for large effects the power of exact tests won't be better anyway, so there is again no practical concern.

And there is yet another aspect: Usually, the determination of the "minimal relevant effect size" (that should be tested) and the variance, even the underlying model (what is always assumed to be "correct"!) are guesses at best. Also, the power as well as the level of significance for the test are generally chosen completely arbitrary. So if the fundament is a conglomerate of arbitraryness and guessing, why should one make a major issue out of possible (and very often practically negigible) differences between "exact" and approximate methods?

Since I know that you are working with failure-time analysis (survival analysis), I will add:

Yes, the power for not all kinds of analyses can be obtained with GPower (at least as far as I understand), and there are other sofware and alternatives. An overview is given in the following review:

http://www.creem.st-and.ac.uk/len/papers/ThomasBESA1997.pdf

where the author state that:

"In some areas of statistics, appropriate methods of analysis are still being developed (e.g.,survival analysis, trend analysis) or are controversial (e.g., contingency table analysis: Zar 1996 p.489)."

On Page 10 they write:

"We also reviewed one commercial program (NSURV) that specializes in power analysis for survival tests (also called failure-time analysis). It is similar to the survival modules in NQUERY ADVISOR, PASS and STPLAN (Table 3). As with trend analysis, appropriate methods are still being developed for survival analysis and especially for calculating power (G. Fox and G. White, pers. comms.). Our informal test runs showed large differences in results between packages for what appeared to be the same problem. We therefore advise caution when interpreting the results of power analysis for survival tests."

Elena,

what is the response/readout? How is the experiment being performed? Is there experience from other, similar experiments available? Is there an accepted standard procedure how the data has been analyzed in the past?

Be aware also that in practice you may loose a number of animals due to other causes that the conditions tested in the experiment (unrelated health problems, say) . I think typically 10% more animals are therefore planned than the power calculation indicated.

Dear All,

First of all many thanks for your help. Here are some more experimental details to help you understanding my statistical needs. The experiments I am performing are of anticancer drug administration in mouse developing tumors. We do not perform time to death experiments but rather measure the dimensions of the tumors. We have performed some preliminary experiments and used the T test to asses significance. The number f animals tested ranged the 8-12 per condition. However now I need to make a prediction of the total number of animals I will need in the next five years and this off course is difficult because if I overestimate I risk not to get the ethic committee ok but if I down estimate I risk not to have the authorization to perform a sufficient number of tests. In addition, I would like to apply a scientifically recognized method also to improve the quality of my experimental plan. Berlinda point has to be taken into consideration too as mice often dye for reasons not related to the experimental procedure

So you have preleminary experiments, and they were analyzed using the t-test. I assume that these experiments have been evaluated by the ethical committe as well, so I further assume that the t-test is acceptable for you and for the committee.

These preliminary experiments give you some hint about a miniimum relevant effect size you might be looking for. Further, you get a rough estimate of the variance of the data.

Now you set some level of significance (5% is typically accepted) and a desired power (should be larger than 80%). You put all thin information into the program and calculate the required sample size. Then, according to Berlindas advice, increase the sample size by the proportion of expected "losses".

To the ethics committe you can then write: Preliminary experiments revealed an expected effect size of ... and a variance of ... . In order to detect this effect with a power of ... and at a level of significance of ... an effective sample size of ... is required. From past experinece, the expected loss of animals due to .... is about ...%. Considering this the number of animals required for the experiment is ... .

Perfect many many thanks Jochen. That was exactly what I was looking for

Dear Fausto,

the numbers are much higher than 8.  Referring to the preliminary exeriments  were i got an effect size of 0,65 and a power around 0.85 I obtained an extimate of 20 animal per condition that I have to admit is high in my experience

Elena, when I correctly assume that you go for a two-sided t-test with alpha=0.05 and that the standard deviation is about 0.67 (in this case I get n=20 per group for the power of 0.85), and the effect size is the difference in means (not Cohen's d), then you will get the desired power when you will look for an effect of 1.0 with 10 animals. If you select alpha=0.1 and power=0.9, 9 animals will be enough for an effect of 1.0.

Playing around with these values thus allows you to "tune" some of the parameters to get the number of animals you can handle and the committee will accept. This seems arbirtary, and it is arbitrary, but the whole process is arbitrary. Noone in research ever has a sensible reason to give and justify particular values for alpha, power, and the effect size, so they are anyway made-up; and additionally the required estimate of the variance is hardly of any useful precision, so this typically is nothing but a wild guess anyway. Because of this "wild guess" on a very imprecise estimate for the variance, the effective power is lower than the nominal power. For instance, if you determined the standard deviation from n=5, the nominal power of 0.85 will give an effective power of only about 0.73. If you have R, you can find this by simulation. The R-script is attached.

Many thanks Jochen and Fausto for the intresting discussion. What I have got at the end is that the experiments we were doing choosing an empiric value of around 6-10 animals per condition will give a good power to our data. I also got some hints on how to justify the numbers  in a statistically correct form to get the approval of the ethic comitee. Moreover, 

I am not trying to answer this question. I do have a highly related question and I think it might be appropriate to ask here. So let's say for a given experiment to be done, we know we need 20 mice per group to achieve the power we need. If there is not possible to do 20 mice per group in one experiment, could we do 10 mice per group and do it two times?  Furthermore, do we still need to do the whole experiment 3 times (i.e. total 60 mice per group)? Or we don't need repeats and instead we just need enough number of mice to get enough power? Some told me that if we have enough power, we don't need repeat experiment. But I'd like to see a documented reference. Thanks.

What does your power refer to? To 20 mice per group or to 60 mice per group?

If splitting the experiment into parts is a "problem" depends on many things. If results differ relevantly and systematically between the parts, then this should be addressed in an appropriate analysis (e.g. by modelling the differences between parts as fixed or random effects).

All these things (all experimental details and properties of the data) would need to be known for the calculation of the required sample size anyway, so something is wrong here. If you really had calculated the sample size to achieve a certain power, then the experimental design and the entire analysis must be clear and would not be questioned by you.

I meant 20 mice per group. Let's simplify the question without the experiment splitting issue. Say we need 20 mice per group based on the power analysis. And then we got results from one experiment with 20 mice per group. Then do we still need to repeat the experiment? Thanks.

When the power was calculated for an experiment with 20 mice, then do the experiment with 20 mice. This experiment was then designed to either accept HA with confidence (1-beta) or to accept HB with confidence (1-alpha), based on all assumptions of this analysis. If this was the aim, why do you consider repeating the experiment?

If, on the other hand, you think or know that results will be considerably different in replicate experiments and therefore the experiment should be repeated several times, then I wonder where your power or your sample size come from (i.e. if the assumptions of the power-analysis were reasonable).

Thanks Jochen. I agree with you. However some reviewers are difficult to argue. I'd need some well-documented proof for this. Otherwise, some bench biologists would just simply ask for more repeats.

A well-documented proof is given by common sense. People (especially reviewers) lacking this sense will always be a problem.

43

Two possible additions to this interesting discussion.

1. We often find that experimenters at the outset of a research programme simply don't know enough to be able to do power calculations. So why the standard design of groups of 6 animals? The answer is that in early phase research, nothing short of a more-or-less complete non-overlap between the scores of the groups suggests a lead that is worth pursuing. So although sample sizes are small, it does allow researchers to focus on really big effect sizes.

2. When there is absolutely nothing known, you can always fall back on the resource equation method (mead, 1998, but better to read the article attached to this response).

Festing describes the method thus:

An alternative method of sample size determination is the so-called “resource equation method,” which depends on the law of diminishing returns. This method is useful for small and complex biological experiments that involve several treatment groups for which the results are to be analyzed using the analysis of variance. In such a situation, it is difficult to use a power analysis. The experiment should be of an appropriate size if the error degrees of freedom in an analysis of variance are somewhere between 10 and 20 (Festing et al. 2002; Mead 1988). This case reduces to the very simple equation:

X = N – T – B + 1,

where N is the total number of observations, T is the number of treatments, B is the number of blocks (litters for a within-litter experiment), and X should be between approximately 10 and 20.

For a within-litter experiment with three treatments, an average litter size of six, and a proposal to use five litters,

X = (6 × 5) – 3 – 5 + 1 =23.

The limits of X being between 10 and 20 can be liberally interpreted, so this proposed experiment would be of an appropriate size, although just beyond the suggested upper limit.

See: http://ilarjournal.oxfordjournals.org/content/47/1/5.full

Mead, Nicola, et al. "What predicts patients' interest in the Internet as a health resource in primary care in England?." Journal of health services research & policy 8.1 (2003).

http://ilarjournal.oxfordjournals.org/content/47/1/5.full

In long-term testing of new drugs for carcinogenicity and/or geroprotective effects "gold standard" is 50 animals per group , 2 sexes, 2-3 doses, 2 routs 2of administration. and 2 species eg, mice and rats)

With increased attention to animal use, people are trying to use the least number of animals to report significant results. There are papers on that, like what Dr Eckelmen wrote in Nuclear Medicine and Biology. Theoretically, it really depends on the % CV (% coefficient of variation, i.e. the % standard deviation) and the sensitivity you aim at (what significant difference you want to tell). In biology, it seems a 20% difference between two values is a reasonable number that does not require a large number of animals.

The hard question is that you do not know the %CV with a new construct that has not been tested in animals yet, so the perspective analysis is not as good as retrospective analysis. Using a pilot study to get a %CV and then to perform a study with calculated number of animals probably is worse than using all those animal one time to provide a more significant result.

In addition, % CV is also depends on the study design, I ever tried to inject both agents to the same animals (one of my papers hanging on the first page of my publications), and using the accumulation ratios to signify the difference, the number of animals was reduced significantly for the same significance level.

So, the question is still open and I would say there is no formula to follow as along as you got significant results. Interestingly, once discussing with a colleague with expertise in statistics, I argued it is OK to measure my height for only once. He said yes and they have a term on that which I forgot what it was. Really thinking deep in that, if we do not know the error in the measurement is very small and we do not think a difference at that level is meaningful, once would not be OK.

The 'resource equation method' could be the solution for sample size calculation in animal experiments.

Standard protocol of long-term study geroprotective potential of one  new compound includes at least 50 animals per group, If you need to test a combination of  2 drugs you will need 4 x 50 animals (control; drug 1; drug 2; drug-1 + drug-2) of each sex

Dear Elena Adinolfi

dear Jochen Wilhelm

dear All,

do you maybe already published something where your calculations for the histology are included and I could cite it for my research pruposal for the ethical comitee? I have also to calculate sample size for histological screenings of ECM during HCC development and really don´t know where to start. I have no numbers which I can use to do the power analysis for the histology part in my experiment. It will be a longitudinal study with several timepoints (4) for measurements or taking histological samples. For the measurements we already done a power analysis and it comes 9 animals considering 10% of loses we want to take 10 per groups. The statisticans who I´m in contact with told me that I can´t just use this power calculation and need to do another one for the histology. I couldn´t find any paper which explained it well. I hope you can help me.

Planning such animal experiments is a difficult process. One really has to understand the topic, what is to be measured, if you suspect a change over over time that is smooth, step-like or a pulse-like. And much more (from the scientific context to the given local limitations). It is impossible to give you any effective help through RG except the suggestion (again) to cooperate with a local statistician or another scientist who is experienced with planning animal studies (these people may still turn out to be noobs, but this is the only chance to get effective help). It's a lot of hard work. You will have to sit and think and discuss. This is nothing that can be done "on the fly", in an online post or in a 1h consultation in a statistical office with a low-motivated statistician.

Thanks Jochen,

I'm already realized that it is not a 5 minutes job. I spent a couple of months planning, but I get stuck and hoped for an idea/suggestion which will push me on.

It is really disputable question. I believe that standard protocol should be

50 males and 50 females of 2 species (mice and rats), 2-3 doses and 2 routes of administration of a drug tested. And animals must be allowed survive until natural death

Please try this one. Kindly have a look on the attachment

It depends on the magnitude of the expected effect. The bigger it is, the fewer animals you need to get a reliable result. And vice versa. Formulas for the calculation are in all manuals on the methods of experiment planning and statistical processing of the results obtained.

That is, the bigger is the difference between means and the smaller is the dispersions, the fewer animals you need to get statistically reliable result.

I agree with Jochen, planning any new animal experiment with a never before tested agent really depend on experiment. I believe one way to evaluate the correct sample size in such cases is to begin with minimum numbers 3- 5 and looking at the variation or precision of values without any treatment.

Suppose, there are small (again its subjective but a 30% or lesser change can be taken as small) differences in values between individual mouse values on a particular day then if even 5 animals of treatment group are giving mean value which is giving more than say 40% difference than untreated group mean, and if the t test gives p value of 0.08, Can this be acceptable difference in this case ??

Hi, do anyone have the exact formula or link to a online sample size calculator (for animal studies) ? please provide. I have gone through the links provided, they do talk about a lot of things including all the parameters but no formula provided.

Thanks

https://lmgtfy.com/?q=online+sample+size+calculation

Six mice is sufficient for both the statistical significance and adherence to the rule of investing mice in experimentation.

agree with Jochen Wilhelm

I think that sample size calculations are unfortunately almost exclusively misused and misinterpreted in the realm of basic science - by two different groups of people who, regardless of standing on the opposite side of the spectrum, do equal injustice to the field, or to be more dramatic, science in general. In my opinion these are 1) basic scientists who cultivate their ignorance and refuse to learn anything about the problem of determining the right sample size - let's call them N=10-ers; and 2) scientists who learned how to do power calculations and not only personally use them as a shield against provoking questioning of the scientific soundness of the experiments, but also expect others to use power calculations equally uncritically - the G*Powers.

N=10ers

On one side, as some of you accurately point out, are scientists who don't really understand the concept of "power" and usually decide on the number of animals (subjects) "empirically". Unfortunately, in this context, "empirically" does not refer to the decision-making process backed up by preliminary data, experience, and observations (with varying levels of uncertainty), but by the fact that they quite randomly decided to use 6 or 10 animals previously (because everyone does it that way) ... and nobody stopped them ... and nothing bad happened (if we decide to ignore the fact that more likely than not the animals were inhumanely sacrificed for no purpose, and the results most likely contributed nothing to the general scientific knowledge). So one group of researchers from the field of basic science is in my opinion comprised of people simply illiterate in the field of basic statistics who don't know what exactly power calculations are or how one would go if one decides to determine the number of animals for the experiment they would like to perform. I feel that these people also usually have a deep admiration of the "statistically significant" stars proudly accentuated in their plots as if they are not aware of the fact that i) they could have been admiring many more if they accidentally decided on using 2 more animals per group; or ii) their starry plots mean literally nothing in terms of biological relevance. In threads like this one you can easily recognize people from this particular group by rather short and direct answers launched from the position of blissful ignorance who are willing to answer the extremely complex question that has been bothering scientists for ages by something like: 5! You always need exactly 5 animals! (https://www.youtube.com/watch?v=aboZctrHfK8)

G*Powers

On the other side of the spectrum is the group of people that I feel superficially grasp the general idea of power analysis, but blinded by the vast knowledge of how to type in several numbers in G*P (or any other software) remain completely ignorant to the fact that this type of analysis does not mean anything at all if you don't fulfill other assumptions that are a prerequisite of scientific soundness. It is particularly unfortunate if such people find themselves in institutional review boards or seated in the bodies conducting grant application reviews as they will usually insist on power calculations even though they don't understand their limited use in the context of a quite specific animal research.

For example, unlike in some clinical settings where sampling strategies are extremely complex and have to be done with a clear goal in mind, exploratory analyses are a fundamental part of animal research where some rather general and simple (e.g. 2x2 factorial) experimental designs permit the attempt to answer several different biological questions (at once, but with varying levels of certainty). Although explanatory models are indispensable, a substantial number of experimental results from animal studies are of exploratory nature (whether their authors realize this or not). This is not something to frown upon (regardless of the fact that not being realistic and aware of this limitation more often than not leads people to follow meaningless scientific leads). In contrast, it should be considered highly ethical and positive to conduct exploratory analyses in animal experiments as long as the experimental design permits it (in a way that the results, no matter how "underpowered", still reflect biological truth from a meaningful logical construct). In fact, researchers who comply with standards of conducting and reporting animal research are aware of the fact that the results from animal experiments should be reported as effect sizes (e.g. point estimates + confidence intervals) as some effects that may be biologically relevant and worth studying turned out to be "statistically insignificant" e.g. due to power analysis being conducted with another (primary) variable of interest in mind. E.g. see the elaboration of ARRIVE 2.0 - 10b: Article Reporting animal research: Explanation and elaboration for t...

That being said - it may be necessary to have a clear goal in mind, and it may be useful to conduct power analysis with this particular single aim (variable) in mind, however, in my opinion, this should by no means limit the experiment or the experimenter from exploring.

Secondly, and more importantly, unlike in clinical research, where we can benefit significantly by pre-defining most analyses (from biochemical to statistical ones), basic research is oftentimes "blessed" with not only the opportunity, but by the need of using the adaptive experimental design, not only at the level of the in vivo experiment, but also at the post-experimental bioanalytical level. In more simple words, a basic scientist has to "work on the go" and more often than not adapt the analytical methods before applying them (to make them suitable for answering a specific scientific question). The consequence is that what you have used to obtain preliminary results (used for power calculations) will usually not be directly relevant to what you will use in the end to obtain the final result on the variable of interest. We have to acknowledge that the experimental results from a pilot experiment in animals will not only reflect the biological variance of the variable we are interested in (e.g. activity of catalase), but also the biological sampling error (e.g. how well I drew the blood from the animal, did I stress the rat, was it during the particular time of the day or did I do it randomly, were there 5 people in the facility or I was alone), the analytical error (e.g. what is my batch-to-batch variability in the purity/potency of chemicals, when and how did I calibrate the instrument, what is the temperature, how fresh are my buffers and how carefully I prepared them, ...), and our personal error (e.g. how precise was I when I was pipetting, did I recalibrate my pipette, do I know the CV of my pipetting at the volumes used). There are of course many more, but we have to be aware of the fact that, regardless of how well the pilot experiment is conducted, a mindful researcher can and should adapt all of the above (and more) to obtain an insight into the problem being studied. Imagine doing power analysis based on data obtained from western blots or quantitative immunohistochemistry - this would be almost meaningless, as it is expected that you will adapt the method (e.g. the retrieval, blocking, the choice and concentrations of primary and secondary antibodies, the incubation times and temperatures, the choice of signal detection and image acquisition methods, the choice of sampling any biostatistical analysis [e.g. how many images per how many slides per how many levels of the organ of interest + how I will model the hierarchical structure of the data analytically to obtain meaningful estimates of reality]) during the experiment. This is even more important in the (common) scenario where people conduct power calculations using preliminary data from the control/wild-type animals not taking into account the possibility that the pathological condition being studied will most likely affect their experimental method at more than one level of pre- and post-experimental design (e.g. the animals are being stressed out more by the sampling procedure causing larger variations in blood parameter due to variable quality of the obtained samples, but also the pathological condition being studies affects blood flow to extremities what makes tail vein sampling "unexpectedly" extremely variable). To make things worse, small pilot experiments are sometimes additionally un-informative due to the fact that scaling up in case of biological experiments and bioanalytical methods is not without its caveats. For example, as Cheng-Yuan hinted, you cannot simply increase the number of animals as you like. Take for example a well-known circadian variability of many biological parameters and try to think how you would sample 30 animals in the same time frame you used for 10 animals in the pilot experiment used for calculating power.

In the case of the experiment inside the experiment inside the experiment - i.e. experimentiception (what all biological experiments are) deciding on whether something is meaningful or meaningless based on very vague estimates typed into the G*P only at a single level of design (taking into account only the number of animals, but ignoring all of the abovementioned "post- and intra-experimental" options) probably means nothing and it should not be a factor in deciding whether or not something makes sense (or at least it should be taken with a quite large grain of salt) ... especially by somebody not familiar with all the downstream processes that have to be done before determining the outcome.

Also, many thanks to Jochen Wilhelm for giving back to the community through his restless presence on RG... I'm quite limited cognitively and don't understand half of what you say sometimes, but with what I do understand I almost always agree completely, and I admire the energy. We need more people selflessly helping others especially by contributing to discussions like this (-:

tl;dr - there's definitely something to power analyses in animal research, but use it with caution - you should acknowledge it, but don't let it draw your focus from the experimental design and regularly exercising analytical flexibility (your analyses can always be much more sensitive and specific, but you have to spend some time nurturing the methods you are using). Spiderman said that with great power comes great responsibility, but for animal researchers, I think, great responsibility also comes with low power ... a responsibility to come up with a way to increase it to obtain the most reliable results while saving the lives of as many animals as possible (-:

Jan Homolak , wow!!