Dear Barry

You have to fix some treatment as control, then you can estimate degradation value for each treatment with control. The data thus obtained may be analysed similar as you analysed the rest. Pl see.

Your question is not clear.  What kind of model or test are you using to calculate significance?

If the degradation rates are constant over time, the log-transformed values should decrease linearily over time. In this case you could do a multiple linear regression analysis and test the differences between slopes.

Otherwise you might consider a survival analysis (e.g. cox proportional hazards regression or some other model).

I'm unsure of the details of your experiment, but you might consider Kaplan-Meier analysis.  This would plot the degradation over time and produce "survival" curves for each treatment (i.e., degradation curves) and you can determine significance by log-rank tests. 

The null hypothesis is that there is no difference. IF there is no difference in degradation rates, then it does not matter which treatment any given observations originated from. Take your data and randomize it. Calculate the absolute value of the difference in degradation rate between two groups that have the same number of observations as the original. Do this 20,000 to 50,000 times. Sort the differences from small to large. Find where the original data fit in the sorted data. Is the result where 95% of the observations are smaller than the observed difference?         This is a randomization test. It will not work if you only have 3 or 4 replicates in each treatment. I am ok if you have about 20, but some would strongly recommend 100 or more. There are no assumptions about the distribution of the data. However, there is an assumption that the distribution of the data is the same in all treatments. Like with many statistical test, minor violations of the assumption are usually not a problem. I know this test can be done in R and in SAS. I am not familiar enough with other software to answer the question.

Timothy, the randomization test you describe is about the (expected) difference. This test works "ok" for about 20 reps and "good" for about 100 reps. Now there is an elegant test available for testing the exprected difference ("t-test") that does not need big calculations - at the cost of the additional assumption of a particular shape of the likelihood function. But this assumption is almost always resonable for 20 or more reps. So I don't see any advantage using a randomization test in this case (I would see it if one wishes to test some statistic or which no designed test is available). Do I miss something?

Dear Jochen, You did not miss anything.

1) some people would prefer not to have to assume a specific distribution.

2) The numbers sound big. In R on my laptop, I can run 1,000,000 randomizations in 1 to 5 minutes. I expect the described test to take something like this. However, if a non-linear regression is being done, then this might take a bit longer. I suppose if there are thousands of data points and a complex equation it might take a few hours. Yet that is not the typical case. Thus, I don't see this as "big calculations."

3) A sample size of 20 is insufficient to identify the underlying distribution with any accuracy. Even if you assume a specific distribution, a sample size of 20 is still subject to considerable error (though much less than a sample size of 5).

Well, a t-test takes about a fraction of a microsecond, so 1-5 min is considerably more computer intensive, and that is a comparably "big calculation" (think of all the power plants we need to run just to let some 1000s of scientists do their beloved tests all using randomization strategies!).

I doubt that the differences are relevant. If the sample sizes are small, all tests and estimates are "difficult", and if the exceed 20 or 50 or so, knowing the "exact" distribution does not matter. Here the bigger problem is that scientists tend to use entirely unsuited models for the problem at hand. To give an example from my area: often, the expected difference for gene expression is "tested", what simply makes not much sense because the expression changes are proportional to the gene activity. A randomization test would allow to technically correctly asses the significance of the data under an expected difference of zero, but the biologically sensible question would be about the relative change, which is assesed by the difference on the log-scale. The failure to see these things is the main problem. If this is recognized, the assumption that the likelihood function of the difference of a sample of 20 or more log expression values is very well justified. And in this case the direct test and the randomization test would give very very similar answers again.

I hope you got the point.

Another note: I think you actually meant "bootstrap tests" rather trhan "randomization tests". A randomization test is a test on actually all permutations. When the sample size gets larger, the number of permutations quickly becomes extremely huge, therefore the "permutation space" is sampled using a bootstrap algorithm. The permutation test is called an "exact" test, whereas the bootstrap test is an "approximate" test.

I guess it all depends on how you use the computer. If I go get a drink of water do I power down the computer? If it is just sitting idle (consuming power), it might be better to get it to do something. Of course if you turn the computer off, then you have to add in the time (and power) lost in having the computer boot up. The computer still uses power when it is "asleep." It might be more productive to have the computer doing something.

One small issue: I did mean randomization. However, you make a good point.

A permutation test looks at all possible arrangements of the data. This is good if there are a small number of permutations (two or three million). It is unproductive when there are a large number of permutations.

A randomization test randomly samples all possible arrangements of the data, but without replacement. All observations are used once and only once for each randomization. The number of randomizations should be a small fraction of the number of permutations. Thus 50,000 randomizations with 109 permutations is 0.005 %. A small fraction by most reckoning.

A bootstrap is similar but resamples with replacement. I am not a big fan of this approach because it allows for the possibility that a treatment in one randomization will consist entirely of the results from one observation. It also (fairly or unfairly) reduces the perceived impact of an unusual observation because the unusual observation gets swamped by the bulk of data about the mean value. Of course, this might be a good thing if the unusual observation was caused by some undiscovered error.

These definitions were set forth in Bryan Manly's book "Randomization, Bootstrap and Monte Carlo Methods in Biology." This is the book that I use, so those are my definitions.

Thank you for all your help :)

It was very much appreciated :)