It is a difficult question. The replication is needed to get an idea of how variable the results are. Typically we want to assure (up to some level of confidence) that our conclusions from our data will hold for a more general case. How we can reasonably generalize from our data depends on two things: (1) understanding of possible sources of bias and variability and (2) inclusion of the variance of the "pool" or "superset" to what we wish to generalize.

Example: I want to analyze a drug effect.

(a) I have a single mouse and I make several measurements on this mouse. From the limited set of data I have I can generalize to measurements on this particular mouse. I have only information about the variability of my measurements - not of mice; another mouse might behave differently, I don't know, so I can't make this generalization to "mice in general".

(b) I have measure the effect in several different mice of a litter. With this data I can generalise to all offspring mice of these parents. Mice from other parents might react differently...

(c) Same as (b) but from different litters of the same inbreed strain. Now I can generalize to all mice of this strain.

Actually, both experiments might be used to make wider generalizations, for example to "mice in general" (all strains), "rhodents", "mouse-like species", particularily including h.sapiens. Clearly, this is less easy to legitimate for (a) than for (b) than for (c). It depends on the quality of the particular model, the comparability of the involved physiological aspects.

On the other side, the actual generalization might be limited when there are lab-effects to be considered. If the results will be different when a different scientist is performing the assay or the assay is performed in a different lab. If this is a reasonable assumption, I would only be able to generalize to "mice" when I had data from different labs. Also, the vendor from which the mice are bought might have an effect (although this is less plausible). After all, it is a matter of the plausibility, what is considered to be different and what has to be included in the experiment to assure the possibility of the desired generalization.

In your case, I think, the generalization should be that others, repeating your experiment, would come to similar conclusions. When you only have one stock than you are in a situation like (b) above. At least all steps starting from the stock should be performed independently. So you get as close as possible to the scenario of using several different stocks. The other groups will have a different stock, but it might be reasonable to assume that the different stocks of the same microbe will not be very different. Also, the other groups will set-up new flasks, so this at least you have to include in your experiment. Further they will do it in a different lab. A possibly considerable lab-to-lab difference is always hard to judge and often implausible, so it is common practice to perform an experiment in one and the same lab (multicentric studies are more relevant in medicine).

A note on the high variability of different experiments: The resulting variability is a combination of the variability between flasks and between measurements. The influences of both can be reduced by averaging several readings. It is worth to check what the greater source of variation is: the flasks or the measurement. The variability of the measurements is easy to detemin just by replicate measures of the same flask. If you compare the averages for different flasks you get an impression of the variability between flasks. If this is considerably larger then the pure measurement error then forget replicate measurements of a single flask and take more different flasks.

Hi Daniel, my gut feeling would be that these are pseudo-replicates. In other words, the 3 containers represent one replicate. Maybe this paper can help you:

Pseudoreplication and the Design of Ecological Field Experiments

Stuart H. Hurlbert

Ecological Monographs,Vol. 54, No. 2. (Jun., 1984), pp. 187-211.

It is not directly related to microbial ecology, but the statistical principles should be the same.

It is a difficult question. The replication is needed to get an idea of how variable the results are. Typically we want to assure (up to some level of confidence) that our conclusions from our data will hold for a more general case. How we can reasonably generalize from our data depends on two things: (1) understanding of possible sources of bias and variability and (2) inclusion of the variance of the "pool" or "superset" to what we wish to generalize.

Example: I want to analyze a drug effect.

(a) I have a single mouse and I make several measurements on this mouse. From the limited set of data I have I can generalize to measurements on this particular mouse. I have only information about the variability of my measurements - not of mice; another mouse might behave differently, I don't know, so I can't make this generalization to "mice in general".

(b) I have measure the effect in several different mice of a litter. With this data I can generalise to all offspring mice of these parents. Mice from other parents might react differently...

(c) Same as (b) but from different litters of the same inbreed strain. Now I can generalize to all mice of this strain.

Actually, both experiments might be used to make wider generalizations, for example to "mice in general" (all strains), "rhodents", "mouse-like species", particularily including h.sapiens. Clearly, this is less easy to legitimate for (a) than for (b) than for (c). It depends on the quality of the particular model, the comparability of the involved physiological aspects.

On the other side, the actual generalization might be limited when there are lab-effects to be considered. If the results will be different when a different scientist is performing the assay or the assay is performed in a different lab. If this is a reasonable assumption, I would only be able to generalize to "mice" when I had data from different labs. Also, the vendor from which the mice are bought might have an effect (although this is less plausible). After all, it is a matter of the plausibility, what is considered to be different and what has to be included in the experiment to assure the possibility of the desired generalization.

In your case, I think, the generalization should be that others, repeating your experiment, would come to similar conclusions. When you only have one stock than you are in a situation like (b) above. At least all steps starting from the stock should be performed independently. So you get as close as possible to the scenario of using several different stocks. The other groups will have a different stock, but it might be reasonable to assume that the different stocks of the same microbe will not be very different. Also, the other groups will set-up new flasks, so this at least you have to include in your experiment. Further they will do it in a different lab. A possibly considerable lab-to-lab difference is always hard to judge and often implausible, so it is common practice to perform an experiment in one and the same lab (multicentric studies are more relevant in medicine).

A note on the high variability of different experiments: The resulting variability is a combination of the variability between flasks and between measurements. The influences of both can be reduced by averaging several readings. It is worth to check what the greater source of variation is: the flasks or the measurement. The variability of the measurements is easy to detemin just by replicate measures of the same flask. If you compare the averages for different flasks you get an impression of the variability between flasks. If this is considerably larger then the pure measurement error then forget replicate measurements of a single flask and take more different flasks.

We use replicates and biological replicates to check and also control the variability at different Levels. For instance, a PCR with a triplicate: checks the variability of your amplification process - to be dependable they should be similar to each other and that's why you repeat the experiment when they are not. When you do another PCR with triplicate from the same cDNA: that would be a pseudoreplicate; when you do another cDNA from the RNA stock and make other PCRs from it: they would all be pseudoreplicates. However, it is not bad at all, but it only helps you to find out the Situation in this one experiment (say your experiment is to treat a flask of cells with a chemical). For statistical Analysis it does not help you much because you should take mean of all of These results and put as one single experimental replicate.

Now, when you have similar PCRs (or other readouts, does not matter) from another flask of cells which are treated separately, then this would be your second experimental replicate. Make a third, fourth, fifth of it: Then in final statistical Analysis you can take your n=5.

Now, coming to your question:

All of the flasks above might be coming from the same one flask by Splitting the cells - so as your bacteria from same strain.

Is it totally meaningless? Is it totally useless, pseudo, biased? I don't agree.

If you have differences between passages of cells - a variance which does not come from your experiment-; they cannot be counted as biological replicates based on that variance (Or equivalently saying from the opposite: One cannot tell that your experiments are pseudoreplicates because your cells are two splitted descendant from same stock (or flask) of cells - That would be simply skipping the idea what an "experiment" means). Neither the cells from different Providers can be. That I would rather call "incomplete standardization". In such a Situation the best would be to use similar passages of cells to have a standardized Setting (Or again equivalently saying: you are actually supposed to use (not to avoid) same stock of cells or bacteria to make your experimentation standardized). Because that is the only way you can assess whether the response you get from a given treatment is significant or not. The most crucial Point over there is "You are experimentating a given Standard material with a given Standard Treatment with an anticipated Response (hypothesis). Then you calculate the probability by which such Response can happen just by Chance (this you answer by statistical testing)."

If you see consistent differences between the passages of cells, then you discuss it (unless you take it as another hypothesis and go for experimentation for that issue).

The differences between different cell (or animal providers) - which should not be - is to be addressed by the scientist who is using the Information you provide. This does not tell that what you publish is false or biased, but it tells that the Situation Needs a careful Interpretation. If other Groups can replicate what you have done, then "the scientific Information that you contribute to becomes more mature", which goes along with reliability testing and self-check of scientific Information by science itself (a beautiful fact which was frequently emphasized by Carl Sagan). (This is achieved to some extent by a multicentral Trial - that's one reason they are valuable: Another Group is able to replicate what you have got).

Your question:

"On the other hand, if I grow the same bacteria in 2 different experiments, each has 3 containers of cultures, how many biological replicates do I have two or six?"

My answer for this would be two. This is like you have:

Two flasks of cells, treated separately and RNAs were extracted. Then you synthesize cDNA from These RNAs. And then you put a PCR with These cDNAs with triplicates.

There I would take mean of triplicates and finally I would have n=2, where you can use for testing the mean of them against a control (e.g. untreated or vehicle treated). (Of course you would Need more than two to make a reliable testing).

Regards

Hi,

I think you want to discriminate between biological and technical replicates.

In your first case - 1 stock in 3 flasks - you are talking about 3 biological replicates. These are usually run in parallel and are important to confirm your results. Generally, variation should be rather small (same stock, same batch of growth media, parallel handling and incubation of cultures, etc.) and you can calculate mean and standard error out of it.

If you repeat the experiment as before again, there you have a technical replicate of your first experiment. Optimally you get same results, but things might get tricky (e.g. same stock but different cell handling in comparison to the first technical replicate). So I would expect more variations between technical than between biological replicates.

Robust results are generated with 3 technical replicates each of them with 3 biological ones. If you make a statistical analysis ot of it, first make it for biological, then for technical replicates (you'll get same average for the 9 independent samples, but different - generally smaller - errors).

For sensitive downstream applicatoins - e.g. RNA isolation and microarrays - always run biological replicates. For simple experiments - e.g. generation of growth curves - if you don't discriminate between tehcnical and biological replicates (e.g. you run 3 experiments in parallel, or make it in 3 different days) I think it's fine.

Hope this helps !

Cheers

If you perform the same experiment in parallel in different flasks you are doing technical repeats of the same experiment. Theoretically, if you standardize the experimental conditions, you must to be able to repeat the same experiment with similar results at any other moment. In this case you will do a biological repeat.

If you need to standardize bacterial cultures, try to initiate liquid cultures from colonies with identical age, always using the same volume of liquid medium and identical flasks or tubes. Depending on the nature of your experiments it will be necessary being so accurate with your initial cell numbers, do you adjust cell densities to a particular O.D600?

@Ricardo: I do not agree with your first sentence. I do not see why the same experiment becomes biological replicate when you do it tomorrow while it is considered as technical replicate when you do it parallel. I simply do not see the point over there.

Anyway - "technical repeats of the same experiment" sounds a little bit weird, because it means clearly "experimental replicates", which means you repeat an experiment in two similar biological systems, twice.

The key point is to do the "experiment" separately (not necessarily on separate days).

An experiment is "to treat the cells".

If you treat two flasks separately, that is your experimental replicate, because there you repeat an experiment (also biological).

If you treat a flask and

a-divide it into two

b-measure some property twice

c-divide it into two and isolate RNAs separately

d-measure one of the RNAs in (c) three times

e-PCR in triplicate with one of your cDNAs above

f-blah blah blah

they are all "non biological/ non experimental replicates".

Every time you separate it, you measure a variability of what you are doing:

b. variability of your measurement (technical replicate for that measurement)

c. variability in your RNA isolation (technical for your RNA isolation)

d. variability in your RNA measurement (technical rep. for RNA measurement)

e. variability in your amplification process (technical for PCR)

as you see, depending on your step numbers you can generate technical replicates at tens of steps, each is to check and control variability of the consistent step. But none of them is useful for your final statistics - the statistics where you test whether "treatment of X raises the response Y".

Because all of them are coming from the single experiment.

Two separately done treatments are simply two different experiments, and that counts for the final statistical testing.

Using a similar material (like same passage of cells) is in your responsibilty so that you provide a standardized experimentation.

and to add:

when you use technical replicates I describe above as single datapoints in your final statistics, you end up with the same mean of course, as Corrado mentioned.

However, the trick over there is that you "incorrectly" increase your group size. This is dangerous, because even if you have same mean value, a larger group size brings you closer to a significance level and "incorrectly" reduces the standard error mean.

I say incorrectly, because when you make those values as one group, then the variance you detect over there is not anymore the "variance in response Y to treatment X" but rather a mixture of technical and experimental variances. There it becomes dangerous because it is a common source of unwanted(?) bias in reported results with a cheap trick, which has nothing to do with a qualified scientific work.

Regards

With bacteria, the initial experiment with three different flasks with samples drawn from same stock would represent technical replicates and not biological replicates because you have to take into account that bacteria is an organism and it has ability to change when kept in culture for long duration of time. To develop a biological replicate you have to first culture the bacteria in broth and then use this broth culture as stock for taking samples. As for same stock with two different experiments you again would have two different sets of technical replicates. If the samples are taken from a broth culture then the two experiments would be your two biological replicates.

For your question regarding the variation in your experiments remember that the bacteria have a tendency to change their behavior.

Good discussion. I am following up with another question. How do you count replicates when growing bacteria in 96-well plates? For a single experiment I have one bacteria stock and then from the stock I grow bacteria in two plates and each plate with 4 replicated wells, how many replicates do I have for statistics?

Read my first post again. You have generated data that can (hopefully) be reproduced by someone who has access to the same stock, when he is doing the same experiment. For this, you replicated two plates. Well-to-well differences are averaged over 4 wells. This is just some kind of hierarchical error structure. By measuring each plate in several wells you just get more precise per-plate estimates.

As real and honest biological replicate in the strict meaning of the word you have n=1, no matter how often you grow and plate and measure. For more replicates you would need more original bacterial isolates (different stocks from different isolates). However, as I pointed out above, it has to be asked how different such isolated *can* behave w.r.t. your experimental conditions and measurements. If there can not be much variability between different isolates and the variability introduced by plating and culturing is considerably larger, then each plat (n=2 in your case) can be considered a meaningful ("biological") replicate.

I think in the answers two items are being mixed up. As I understand it, the question goes to replicates and pseudoreplicates and not to repeatibility and reproducibility. The following is my view on bacterial cultures and the replicate/pseudoreplicate issue. Compare the situation to fish in aquaria in a toxicity experiment. All fish are from the same genetical lineage. You have 12 fish in every aquarium and 3 aquaria per exposure concentration. The 3 aquaria are the replicates, if you were to analyze the 12 fish of one aquarium in groeps of 4 pooled together, these groups are pseudoreplicates because the come from one treatment. Now look at bacterial cultures: 3 culture flaks per treatment. These are the replicates. If you take several samples out of one flask they are pseudoreplicates for thetreatment. The will show the variabilty in your final assay but they are not independent.

@ Freddy: If it was possible to keep just one bacterial cell in each flask then your explanation with the fishes would have been perfect but as is the case with bacteria there are millions of bacterial cells in each flask and therefore your explanation will not hold ground for many reasons. And one has to take into account that there will be genetic changes in bacterial cells of subsequent generations which would account for the variations in the experiments. This changes would also bring changes in behaviour of individual cells and also in the whole community.

@prashant: I don't completely agree. Unless there is selective pressure in the experiment, mutations will have random effects and no mutant will become predominant. The fact that indeed there are millions of bacteria in one treatment (and much more) assures averaging out these effects, if any. That's exactly why, if you keep your stock cultures in a proper manner and you keep experimental conditions well under control, experiments on bacteria are quite reproducible (and repeatable). The variability of bacterial experiments are mostly and mainly due to poorly controlled conditions (incubator temperature, media, pipetting variability, operator, culturing conditions, lab temperature, ...).

I think that you are working with a (millions?) bacterial cell that duplicates themselves by bacterial fission and all the cells are genetically the "first" cell. When you work with a bacterium this fist cells "disappears" during fissation and 2 new cells start to live? (It is a philosophical question). It is a difficult question. Another point is the possibility of a mutation of DNA of the exchange of DNA by other bacteria if the time for the experiment is too long of if there are other bacteria in the medium. In my opinion you are working with the same cell because the DNA is the same.

Good work

@ Freddy: I completely agree with you when you say that experiments with bacteria are repeatable and reproducible and also that random mutations will not affect the experiments in general. But the problem faced here is that the experiments are showing variations that means there is some selective pressure in the experiment. But even then there are cases where bacteria cultures kept in labs have shown changes in serotypes etc which may lead to variation in experiments.

I still think that the first answer by Jochen Wilhelm is the best one. You cannot define what is a true replicate and a pseudoreplicate without first defining the "universe" or "population" of possible experiments to which you want the conclusions from your study to be applicable. Going back to Freddy's fish example, if you want to test whether the fish perform differently in the two tanks (question: is there any difference between tanks?), then the individual fish are true replicates. If say, you have the two tanks at a different temperature, and you want to test if the temperature affects fish performance (question: does performance of fish differ in low and high temperature?), then the fish can no longer be considered as replicates. You need several replicate tanks at each of the two temperatures to be compared. This is so because, you need to assess the variability in fish performance among tanks at the same temperature, to be able what is the uncontrolled variation in performance among tanks, caused by any other differences in conditions unrelated to temperature. In addition, the fish to be used in the experiments should be a random sample from the population of fish to which you expect results to be applicable.

In the case of bacteria, you need to first decide whether your aim is that the results from your experiment will apply to a single culture batch, any batch of a given strain within a certain time-frame, the whole local population of the bacterial species in a certain habitat, or the bacterial species as a whole. With respect as to whether the flasks are replicates with respect to treatments, say if you are interested only in batch of bacterial culture, it depends on how you apply the treatment. Say if the treatment is again temperature, and you use one incubator for each temperature, then the flask are not strictly true replicates, because if you consider them replicates when concluding about temperature you need to ASSUME that the only difference between the two incubators is temperature, rather than obtaining strong evidence that the difference in response was caused by temperature.

In conclusion, you need to think about the design of the whole experiment, and if you cannot avoid pseudoreplication you need to report very clearly the additional assumptions involved (or step based on "faith") that is needed for reaching your conclusions.

Hi Daniel,

It is interesting question. In this case, one experiment = one experimental units. Any units within experimental units are evaluation units. (see "Experimental versus evaluation unit ...in cell biology study")

http://link.springer.com/article/10.1007%2Fs12013-012-9464-9#page-1

In this context, the experimental units must be physically independent.

Best regards,

Oleg

An excellent question, Daniel! I am joining in late but would like to add the simple view of our lab since this topic has been discussed many times in our lab.

I would agree with several previous people that Jochen Willhelm has summed up the topic the best. In addition, I would like to say that our understanding independent experiments i.e. biological replicate are cases where only the step of starting the cultures from stocks is the same. Hence, independent experiments should be performed at different days (even at an interval of two weeks if one believes in the effect of phases of the moon) and experimental setup (media, preparation of fermenters for example) by different people since the real meaning of biological variance in my understanding is to show the variability of the results performed at different times using the same experimental methods. We have always been concerned that this is not often the case as this definition would obviously decrease throughput.

This article might help understanding this point. Hence, it will help to better design your experiment.

http://labstats.net/articles/cell_culture_n.html

Hello

Replication in biology was clearly described by Stuart Hulbert in 1984.

A long list of publications about the problem of pseudoreplication/replication was presented in literature( see http://scholar.google.co.il/  about 1,450 results  )Example from  cell culture experiment is paper entitled  "Experimental versus evaluation unit and multiplicity problem in cell biology study"  which was publised in  Cell biochemistry and biophysics, 2013.

the real question to ask is "how likely is this effect reproducible"? therefore there are no simple answers. i have been going round and round in circles on this topic for a while and have been pretty exhasperated by the diversity of opinons out there!