I don't see your point: in my opinion it doesn't make sense to firstly group the responses according to their values and secondly test the count difference between the groups .  Why did you collect these data ? Was it to assess the effect of some factor ? If yes you should consider using statistical methods to analyse count data. Poisson regression is a good start.

Hei Renaud,

The original idea was just to know what is the count of microbes in each samples. I did that with FCM, where you measure in triplicates. This is why for each sample I have 1 average and 1 stdev taken from the triplicates.

Afterwards I realized that the samples were clustering together in 3 groups according to their counts: a group with high counts, a medium one, and a low one. These groups do not contain the same amount of samples.

So I want to know how can I show that there is a significant difference between these 3 groups.

As a matter of fact these 3 groups are the exactly the same than the "depth category" groups of the samples: Shallow waters, intermediate waters, and deep waters. But I am not sure if that is important for the test. Originally I was not willing to test the effect "of a factor", just see if there is a difference in the counts between the groups, but in the end it could be that I am testing the "depth category" on the "counts". I am not sure if this is correct statistically.

Thanks a lot.

"So I want to know how can I show that there is a significant difference between these 3 groups."

That is a test "after the fact". You have your data and you have observed in your data that there are obvious differences. This lead you to hypothesize that there might be particular classes of distingt counts. It makes no sense to test this hypothesis on the same set of data that was used to generate the hypothesis.

Two steps should follw:

You should more specifically put forward what your statistical hypothesis is. For instance: "the counts do not follow a Poisson (or negative binomial) distribution", or "the counts follow a a:b:c mixture distribution of three Poissons with rates ra, rb, and rc" or something similar.

Then you can collect new data and test your specific hypothesis.

"As a matter of fact these 3 groups are the exactly the same than the "depth category" groups of the samples: Shallow waters, intermediate waters, and deep waters."

So it looks like the depth category is your factor of interest. I would start assessing this effect on your data, using a Poisson regression model, or a negative-binomial regression model (as suggested by Jocelyn) if there is evidence of over-dispersion w.r.t Poisson distribution.

Hey,

thanks both of you for your answers. It is terrible how it is easy as a biologist to go completely wrong in statistics...

After doing more research on poison and negative binomial distributions and regression models i think I understood the point of it, and also why a Kruskal-Wallis test is not suited here. I still need to figure out how I ll apply a poisson regretion model on my data but I ll figure out I guess.

Still, one thing that you guys are saying opposit is  if I can test on the data I have or if I should take new data to test my hypothesis.  Jochen, why can I not test it straight now? what is the problem with saying that there is a difference in the different groups I observe?

Thanks.

That's the fundamental logic of statistical tests.

On my way to work I pass 8 traffic lights. If I happen to stand at 7 red traffic lights this morning, I may come up with the hypothesis that these traffic light are usually red. The statistical model is a binomial model with P(red)=a, and my research hypothesis is p > 0.5, say, so I will test how unexpected my observation is of a

Ok, I get  your point. Nice explanation.

However I am not sure if my situation is the same. In your different examples you always want to have produce model, in order to generalize to a global situation. This is not what I want to do. I just want to speak about the data I have, saying that in this specific case we observe this and that, with this group showing a difference in counts compared to that other group. In no case I want to make a model. Is there really no way to do that and be correct statistically? Almost all papers in microbiology do some comparison of means in some similar situations. Are they all wrong?

It is not wrong to say: I found counts that cluster around the three distinct values (a, b and c - whatever).This statement is even "statistically correct". For 21 samples, b.t.w., I would recommend to plot the observed counts in a 1D scatterplot to visualize these clusters (that's more comprehensable that reporting means and SDs*).

What makes no sense is to calculate a statistical significance of this finding.

---

*SD is not a useful statistic for counts; it's interpretation in this context is usually non-sensical, especially when the mean counts are low. Generally, if the Poisson model is appropriate, the distribution is defined by the mean completely and the variance will equal the mean. Here it would thus be better to report the variance so that one can better compare if the observed means and the observed variances are more or less similar. But again, it makes no sense to claculate a statistical significance of the data under whatever model (Poisson or negative biniomial).

Yes, the scatter plot is already done

Ok, then I will avoid doing any test for these counts.

A last question: As a part of this study I also looked at the microbial community of each sample (454 sequencing of the 16S gene). If I plot an NMDS analysis of the samples based on the different OTUs, they cluster in 3 groups again (the same groups than for the counts btw). I did a PERMANOVA to see if these groups are statistically different from each other (very low P value for testing if everything is same, so the groups have different microbial distributions). After your messages I am not that sure that this is correct anymore... should I just not use that test?

Thanks.

To my understanding of what a statistical test is : no. It is the same question you like to address with your count-clusters. It is a test "after the fact" and it thus does not provide the insight a reader (or you) would think. It is abusing the test, provoking avoidable misinterpretations. Again, reporting what the actual differences in you sample are is absolutely ok and valuable.

Great. Thanks for your help Jochen and Renaud! It has been really useful to me.

The problem now is that some reviewers are really eager to get p values everywhere... I just hope to not get one like that :)

Yes, I know this problem, and I don't have a good solution.

I suggest to try promote a better quality of science and argue appropriately. But this may delay and in the worst case even hinder the publication (at least in the envisaged journal). In case of a bad response one can still ask the editor if he would be willing to involve a statistical reviewer (although, unfortunately, this is also by far no guarantee to "win the battle"). Good luck anyway!