Dear K. Hopton ,

If you are looking to compare a scale variable for your two groups then you should first test the assumptions of a parametric test (independent t-test). If they are not satisfied you can consider the robust exceptions. As your groups sizes are unequal and small they will not apply. The main issue to consider is therefore whether you can assume that the distribution of data in your two groups is approximately normal. The most appropriate test for this is Shapiro-Wilk. If this comes back negative you can run the independent t-test. If it comes back positive you should run the Mann-Whitney U test. Please see my study guide for more information.

I disagree with Peter's advice about testing for normality.

The normality assumption for the unpaired t-test is that the populations from which the scores are sampled are normal. Unless the outcome variable is something that has never been studied before, there is likely lots of information beyond that which is available in the two samples in hand that can be used to judge whether the populations are approximately normal. This is one of the main points in the following BMJ Stats Note by Bland & Altman:

• https://www.bmj.com/content/338/bmj.a3166.abstract

Another major problem with testing for normality prior to conducting a t-test is that tests of normality lack power when sample sizes are small, when the normality assumption is most crucial; and they have too much power as n increases--and as n increases, normality becomes less important (because the sampling distribution of the difference between the two sample means approaches the normal distribution). Here is a short conference presentation I gave on this topic a few years ago.

Presentation Silly or Pointless Things People Do When Analyzing Data: 1. ...

K. Hopton, I think it would help if you gave much more detail about what you are trying to do. Bland-Altman agreement analysis requires paired measurements, but you appear to have two independent groups. What are you measuring, how are you measuring it, and what do you hope to find? HTH.

Bruce Weaver Thanks for the advice! I will have a look at your presentation.

I am measuring the Microbial biomass Nitrogen. I have used one method where I collect 12 samples and another with 25 samples. I am looking to see if there is any statistical difference between the two methods.

So all the samples are from the same area.

Hope this helps! Really appreciate your help

Sorry, but I still don't understand.

• Where do the 12 and 25 samples come from? Are they all sampled from the same "population"?
• How are you defining statistical difference?

At the moment, the only way I can make sense of what you're saying is to assume that the 12 and 25 samples are all from the same population, and that "no statistical difference between the methods" would be indicated by equality of the means and equality of the variances for the two independent samples. Is that what you're talking about?

Bruce Weaver

" At the moment, the only way I can make sense of what you're saying is to assume that the 12 and 25 samples are all from the same population, and that "no statistical difference between the methods" would be indicated by equality of the means and equality of the variances for the two independent samples. Is that what you're talking about? "

Yes what I am talking about, I want to see if collecting the additional samples has a statistical significance or if they are the same (In which case there would be no use in collecting additional samples). Sorry hope this makes sense?

Thank you for your help by the way, you presentation was also very easy to follow and understand !

I'm still not really understanding what you are doing. Do you hope to find a significant difference between the sample means, or to not find a significant difference? If the latter, perhaps you are really interested in equivalence testing. Here's a nice overview I just found:

• http://homepage.divms.uiowa.edu/~rdecook/stat6220/Class_notes/equivalence_testing.pdf

For conventional testing (with H0 specifying no difference between population means), increasing the number of observations will increase power (all other things being equal), so you'll get statistical significance with smaller mean differences.

And since you've already looked at the results from the 12 and 25 samples, you should not use those in your new analyses.

Daniel Wright I havent analysed them yet, I've looked at the raw data that has been collected from each method, not sure what you mean

Since you've looked at them and making decisions about what you saw, you should not just add more data and pretend that you hadn't looked at them. See the discussion of stopping rules, often discussed in the Bayesian literature on an issue with frequentism. Zoltan Dienes has a cartoon on this here: http://www.lifesci.sussex.ac.uk/home/Zoltan_Dienes/inference/index.htm. (and his book provide good discussion of this).

Daniel Wright I haven't checked for significance yet. I am trying to, so any suggestions on how would be gr8. ta x