Rose, Your question may be a little more complicated than the above suggests. I would try to take advantage of the fact that you are at Harvard. There are consulting services for statistics. I have listed a link to the one I think is most likely to be of help to you. If this isn't the one for you they should be able to direct you. Contact me if I can be of help. Best wishes, David

http://hedwig.mgh.harvard.edu/biostatistics/support/consultation

Dear Rose,

you can run both tests for all the "factors" and very likely you'll find out that for most of the factors, the tests will give similar results ( either both tests or none will show statistical significant differences). For the few cases that they will provide different results, a normality test will indicate which one is more likely (which test is more appropriate) , but these cases are probably borderline significant anyway.

If you have many factors, from a practical point of view, I think that the most "rewarding" thing you can do is to use Benjamini-Hochberg FDR (instead of the more traditional multiple comparison corrections, such as Bonferroni). The statistical analysis you described is pretty straightforward (not messy :); one concern might be that if you are testing simultaneously many factors, the sample sizes are probably too small.

Best regards,

Mickey

Please use paired t-test, t-test and ANCOVA.

Rose, Your question may be a little more complicated than the above suggests. I would try to take advantage of the fact that you are at Harvard. There are consulting services for statistics. I have listed a link to the one I think is most likely to be of help to you. If this isn't the one for you they should be able to direct you. Contact me if I can be of help. Best wishes, David

http://hedwig.mgh.harvard.edu/biostatistics/support/consultation

Dear Rose

about your first question, when you have more than two samples you should use ANOVA instead of 2 sample t-test, so using multiple 2 sample t-tests is not efficient. you can use ANOVA plus pairwise comparisons  tukey or fisher for evaluating multiple samples mean. Now about your second question regarding normality of some samples. As you know, when samples are normal you should use parametric methods which is ANOVA while for non parametric case you should use Kruskal Walis test. 

now if all of your samples are not normally distributed, what should you use?

This may be a surprise but parametric tests can perform well with continuous data that are nonnormal if you satisfy the sample size. While nonparametric tests don’t assume that your data follow a normal distribution, they do have other assumptions that can be hard to meet. For nonparametric tests that compare groups, a common assumption is that the data for all groups must have the same spread (dispersion). If your groups have a different spread, the nonparametric tests might not provide valid results.

so i think if you have more than 20 experiments in each sample, you better use parametric tests which is ANOVA plus pairwise comparisons in your case.

I hope you find this useful.

To the extent possible, you want to analyze the data in such a way that there are the fewest statistical tests possible. Say that you have ten independent t-tests and you decide that you will declare significance at the 0.05 level. If each test is independent then you have a 95% chance of correctly deciding that there is no significant difference. This probability is then multiplied by the number of tests that you make. Given no treatment differences your probability of having ten tests that all agree is (0.95^10)=0.6. The chance of having a significant result when none exists is therefore 0.4. So it is fairly likely that you will find at least one "significant" difference when none really exists. Multiple comparison procedures correct for this effect. There are other approaches to correct for this effect in other situations. However, there is no correction when doing multiple t-tests.

David's advice is excellent. Before you go into that meeting try running some of the analysis yourself. You can then discuss why some methods are better than others, and what may be more appropriate. If you can, make sure that you look at some test (graphical or formal) to assess if your data fit the assumptions of the model that you tried first. Can you get a q-q- plot, or even just a plot of residuals versus predicted?

Also think about your goals. Do you want to learn more statistics, or just enough to get by? If you read literature, do you only find t-tests, or do other methods show up? What is the Mahalanobis distance between the two groups, and does it change between 2h and 20h? If nine out of the ten variables show a statistically significant difference, but the variables are 90% correlated, then all you have done is measure the same thing over and over. If the correlation is 4%, then you have a more complete picture of the treatment effect. There is a great deal that you can do with these data if you ask the right questions, or ask enough questions.

After the consultation you may have some more ideas. Even if you do not use SAS, I think that the SAS/Stat user manual could still be useful.

http://support.sas.com/documentation/cdl/en/statug/63033/HTML/default/viewer.htm#statug_introclus_sect001.htm

It discusses different analyses, and gives examples.

Another useful resource is Wikipedia. It is not organized in a way that it could be used as a textbook to learn statistics. However, if you have a key word you can often find very informative articles.

@Rudolf,

 I thought the entire point of the note in the question about using different animals at 2h versus 20h was to avoid a repeated measures design.

If repeated measurements had been taken this is a good point. However, the question is worded to indicate that this was not done.

Followed by "(Note: The animals analyzed at the 2 hour time point are different from those at the 20 hour time point.)" The key word is "different." I hope that you are not arguing that repeated measures designs are necessary just because there were two time points. I agree that it is more typical to measure the same animals at both time points, but the quote indicates that this was not done.

    People see two time points and instantly figure that repeated measures is appropriate. However, repeated measures assumes that there is a correlation. If I measure an individual today, it is likely that there is a good correlation with tomorrow's measurement. However, the correlation becomes undetectable as days become years become decades. Ideally, you have a repeated measures in the control and can then check to see if (in fact) the repeated measures are correlated. If no correlation exists then a repeated measures design is not necessary nor appropriate. The best you can hope for in this case is that the repeated measures design converges to a non-repeated measures design as the correlation becomes zero. I don't know if this is the case, so I tend to avoid repeated measures designs in cases where I cannot detect the correlation. If the correlation is not significant (p-value 0.06) I would still do a repeated measures design. I have a harder time convincing myself that a repeated measures analysis is a good idea when the p-value is >0.2. The cut-off is flexible, and I also consider what others might think. So if I measure the same animals two days apart, I might stick with the repeated measures analysis rather than trying to fight the system. Is it really worth three pages of the manuscript to show that a repeated measures design was not appropriate when that should not be the focus of the paper? Fortunately, this is not an issue for Rose because her observations were on different animals.

@Rose,

     This discussion with Rudolf is very important. It is possible that you will have the same issue with the statistics consultant. Remember that they are trying to help many people. They see two time points and instantly you have a repeated measures design. Do not accept what they say blindly. You will still have to take their advice and then spend some time trying to figure out if their advice is really appropriate to your particular problem. 95% of the time the statistics consultant will provide good advice. That leaves 5% (maybe less) where they did not quite catch the idiosyncrasies of your particular data set. You may need to see the consultant a few times to work everything out.

I prefer the way that Rose conducted this experiment. However, in this sort of research you will end up being forced to use repeated measures designs because you do not have sufficient space to use this many animals. You will also have ethical issues if the treatments are fatal after 4 days because now you have to kill twice the number of animals using the non-repeated measures design. If you like this sort of research, you need to learn about repeated measures analysis, but it is not appropriate for this particular experiment.

Dear Rose,

you may wish to consider instead of a hypothesis test a confidence interval for the difference. This informs not only whether there is a (sigificant) effect but also how large the effect is. With Mersku Alkio, I have a paper and a free software (2SAMPLES) on that.

Cheers, Manfred

Mudelsee M, Alkio M (2007) Quantifying effects in two-sample environmental experiments using bootstrap confidence intervals. Environmental Modelling & Software 22:84–96.

http://www.manfredmudelsee.com/soft/2samples/index.htm

http://www.manfredmudelsee.com/soft/2samples/index.htm

A combination of non-parametric signs and Man wite in SPSS

If I understood correctly you have serum factor (LDH) from four different groups (age group1 2hr, age group1 20hr, age group2 2hr and age group2 20hr). In such situation it is better go with ANOVA when samples are normal if not you should use non parametric Kruskal Walis test followed by post hoc test.