This is just my idea, not sure if this is correct and some expert comes up with a better or more correct solution:

Average the observations per animal.

If the counts are about >30:

Calculate the mean difference and its standard error, so that you can calculate the t-value. Retrieve the p-value for that t from a t-distribution with 3 degrees of freedom (you have 5 animals and the mean difference needs to estimate two coefficients: 5-2 = 3).

If the counts are considerably less than 30:

Better use a quasi-Poisson model using the averaged counts. You would need a software that can do that.

In both cases you might consider using weights proportional to the number of observations per animal you averaged.

I wonder, if Pre group has 5 animals and Post group has only 2 animals (these were included in the Pre- also), what happened to 3 animals out of 5?

In this problem/experiment, it has been found that there are two groups of animals namely Pre Group and Post Group.

The Pre Group consists of five animals, each with varied number of observations ( 8 ,11, 6 , 4 , 3).

Similarly, the Post Group consists of two animals, each with varied number of observations ( 50 , 57).

The two animals in the Post Group were included in the Pre Group also with 6 and 4 observations respectively.

If we denote the two animals in the Post Group by A & B,

then we are to denote the five animals in the Pre Group by A , B , C , D & E.

Thus the two animals A & B correspond to 6 & 4 observations respectively in the Pre Group and correspond to 50 & 57 observations respectively in the Post Group.

Now, in order to test the significance of the difference between the effects of the two groups, arrange the

6 + 4 + 50 + 57 = 117

number of observations in a two-way table (with the two groups namely Pre Group & Post Group as two rows and the two animals namely A & B as two columns) and then apply Analysis of Variance Technique to achieve the purpose.

The suggestion provided by Jochen Wilhelm, is not statistically accurate.

This is due to the reason that the animals in the two groups are not identical.

In order to test the significance of difference between the mean effects of the two groups, the animals in the two groups must be identical. Otherwise, t test cannot be applied.

The question raised by Roberto Molteni is justified.

It is difficult to answer without more knowledge about the kind of observations you have on the animal, and if animals « pre » and « post » are indeed the same, as suggested by the name and Roberto Molteni, or not. Jochen Willhelm assumes observations are counts, but it's not clear to me when reading your post, and after averaging they are not count anymore anyway.

Basically, the framework would one of a model including at least two effects, the animal and the group. Nested if animals are different. With as few as 2 or 5 animals, a true random effects model will be difficult to fit. Averaging for animal considerably lowers the sample size and you should wonder what kind of variability you are interested in, between animals or in animals ; in the second case, averaging remove this information. Usual models has a lot of assumptions; checking them may be difficult here, but if they are not fulfilled, much more complicated technics may be needed. Unless results are very "clear cut" (but with as low as 2 animals, generalization to a population is difficult anyway...)

The main message is: give more details on what you have exactly as data, what is your question... And eventually consult a local statistician for optimal assistance.

2 out of 5 are identical. I did not recommend a paired test, I just suggested to adjust the degrees of freedom (for a usual independent-samples test) because not all values are independent. What's wrong with that?

There is no need that the animals in both groups must be identical. The independent sample t-test (aka two-sample t-test aka t-test) assumes independent samples in the two groups, and these are assumed to be from different animals. It is not even required that the sample sizes of the two groups is equal. In contrast, the paired-sample t-test operated on the differences between pairs of (dependent) values. Such differences can be calculated only for defined pairs, what would be possible here only for the two animals that were mesaured in both groups, whereas the data from the other 3 animals would have to be disregarded (although that would be possible, I think that's a bad idea because a considerably proportion of the data is thrown away).

What happens if the data is dependent but this is not considered in the analysis (e.g. do a twa-sample t-test for pairwise dependent data, where one would actually do a paired-sample t-test)?

The dependency information tells the model that pairs of values shar a common source of variance, so the pairs may scatter around entirely different means, and that these "betwee-pairs" differences are unrelated to the "within-pairs" differences. The latter is the thing we want to analyze, as it reflects (only) the systematic difference between the groups. So if the model looks only at the "within-pairs" difference, the possibly present variance between pairs is not relevant and does not disturb. This comes at a cost, because looking at the "within-pairs" differense cuts the degrees of freedom into half. If the disturbance from the "between-pairs" variance is large, the benefit is higher than the cost, so there is a net win. If the "between-pairs" variance is small, it can be that the benefit does not outweigh the cost.

If the the model does not consider the common sorce of variance, this variance completely "hits through" into the data analyzed, and if there is a considerable possible "between-pairs" variance, a smaller systematic difference between the groups may be undetectable. So if the result is not significant, we don't know if we could do better using a paired-sample test. If is is significant, the "between-pairs" variance was obviousely not too disturbing.

@ Jochen: yep, I missed the inclusion of the two animals in both groups. I agree with your analysis of the consequenes of (in)dependance, no problem with that, even if the problem of partial pairing may be more difficult to handle than the usual dichotomy between paired or unpaired T-test (the problem is not only of having the correct degrees of freedom, it's also an incorrect variance estimate, and it's not clear if the distribution remains a Student one, I think... and with such a small number of individuals, that's may be problematic).

Note that the remark on your contribution was not on this, but on the fact that you assume count data (an information I may have missed also, but I still don't see it in the original post. Or I may have misunderstood something in your answer).

Thank you all for your kind answers.

Jochen Wilhelm, It's not counts but a measurement made at different instances.

Roberto Molteni, There was an intervention done on the two animals (only)

Dhritikesh Chakrabarty, Thank you for the correct rephrasing of the experiment and a possible way out, though it disregards other data in the Pre- condition.

Emmanuel Curis, There was an intervention performed on animals A and B, after data acquisition in Pre-, continued with data acquisition of the same kind in Post-.

The question is- Is there a significant change in the measured metric when you compare Pre- to Post- in the two animals A & B considered separately?

I agree n=2 is way too less for a generalization to populations, but nevertheless this investigation enquires about the effect of a specific intervention.

Jochen Wilhelm " What happens if the data is dependent but this is not considered in the analysis (e.g. do a twa-sample t-test for pairwise dependent data, where one would actually do a paired-sample t-test)? ":- I had similar query too. Thank you for the explanation.

So, The premise of the problem goes like: I measure a metric from the animals under certain conditions. I had 5 animals from which I did this acquisition over 8, 11, 6, 4, and 3 sessions respectively. I did an intervention on two of these animals and further measured the same metric over multiple sessions (50, 57).

John Thomas : as previously said, giving a precise answer on the « right » method to use would need much more informations about your question, experimental setup, data and so on. And you are faced to two technical questions : how to handle the sessions-in-animals setup, and how to handle a partially paired design.

To answer the sessions-in-animals setup, you may average the results in each sessions for each animal, leading to a single value for each animal, then do the comparison on these single values. Note however that since the number of sessions changes, the hypothesis that the sample (of the means here) comes from the same population (ie, all from the same probability distribution) is obviously wrong, so most usual tests (including Student's test in all variants) will be hard to interpret.

As a general tool, if you want both to respect the sessions-in-animal setup, and use data for all available animals, then mixed-effects models would be the road, but with only five animals, and only two having the post-intervention value, do not expect much, and be ready to face convergence issues and all other kinds of technical problems.

So, consulting a local statistician still remains the best option.

Best regards,

Emmanuel Curis ,

" Note however that since the number of sessions changes, the hypothesis that the sample (of the means here) comes from the same population (ie, all from the same probability distribution) is obviously wrong, "

Interesting comment. Are you sure? Why?

The individual values were sampled from the same populations (of animals!). Having several readings from a single animal just gives an average observation with a smaller measurement error (the biological variance is unaffected, and particularily the population mean is unaffected). The total variance would be lower (depending on the relative contribution of the measurement variance), but that can be accounted for using appropriate weights. At least I think so.

The story is different if the readings represent a time series and the response changes over time. But then the whole experimental design is, well, strange.

There is also a problem if the number of readings is somehow related to the response (e.g. if animals with higher response values are somewhat more likely to be sampled more often, or something like that). I don't know how such data could be analyzed without introducing bias. Particularily using weights would introduce bias.

Again: I think so. Happy to learn if I am wrong :)

“The story is different if the readings represent a time series and the response changes over time. But then the whole experimental design is, well, strange”.

Yes, but I am only guessing, it looks like a time series. I have the same comment of Jochen Wilhelm.

Jochen Wilhelm Let note Xij the value of the j-th session for animal i, and ni the number of sessions for this animal. Then averaging replaces (Xi0, ..., Xini) by its arithmetic mean, Xbari. If the Xij are iid, then Xbari has a distribution that depends on ni, at least because Var( Xbari ) = Var( Xij ) / ni (and if the Xij are not Gaussian, the shape of the distribution also changes with ni).

So, if the ni are different for each animals, the Xbari are not iid anymore since they each their own variance. Hence, all theory on usual tests, that assume iid values, do not apply.

You are certainly correct. But I thought the iid assumption is attenuated by using appropriate weights...