What would you like to be able to say about your data? If you are interested in describing just this sample, say, the differences between the session average of each animal in one group vs. the session average of the single animal in the control group, then you can use whatever summaries make sense to you. But if you want to make inferences about a population, you need more data.

Well, I wanted to see if there are any significant differences between and within the group. In the figures it is quite obvious that there are differences, but I wanted to provide further proof by statistical analysis. But yeah, I was worried that this would be the case. Is there any way I can at least do some trend analysis or something ? 

Thank you 

If your intent is to describe what happened to just these five animals, then it's no problem to plot score by session for each animal, for example. That might be enough to justify a new study.

If you want to infer to the population, that is, conduct significance tests, you'll need bigger samples. Think about it this way, would you take a new pharmacological agent tested on only four patients? And if their response was compared to only one control? There are exceptions of course, but generally more information produces more confidence in the inferences.

Yes I knew that those sample sizes were to small but I hoped that there might be at least some statistical tweak I could use.

Thank you for your insights. 

Statistic is always based on assumptions. The less data we have, the more will the conclusions depend on the assumptions (and their sensibility).

In light of the very limited data I'd think that a Bayesian treatment would be in place, what would again require *explicit* assumptions about prior beliefs.

But even under a purely frequentistic setting on can make a conservative estimate of the probability to get "more extreme data under the null hypothesis" (a.k.a "p-value"). This does not require more data, but having lesser data requires a careful consideration if the underlying assumptions will make sense.

There does not seem to be a trend over the sessions, so one may average the replicate measurements per animal to get a more precise value per animal. The average for the 1 control animal is 21.03, the average for the 4 treated animals is 23,08, with a standard deviation of 0.47. The difference is 2,06.

Now one may assume that the values / averages are approximately normal distributed, and one may further assume that the variance (or standard deviation) in the control group would be of similar size as in the teated group.

Under these assumptions the pooled standard error for the difference is sqrt(0.47²/1 + 0.47²/4) = 0.53.

The t-value is 2.06 / 0.53 = 3,9.

From a t-distribution with 3 degrees of freedom follows a p-value of 0.03 (two-sided) and a 95% confidence interval of (0.38, 3.73). 

The critical point is not the maths, this works, also if there is only n=1 in one group. The critical part is to judge if the assumptions made are sensible - since there is no way to confirm this by looking at the data.

My solution is simple and possibly a bit crude. There may be more sophisticated ways using mixed models etc., but I think it is appropriate to keep things as simple as possible.

I'd try bootstraping.

I find bootstrapping very problematic with small samples. You won't generate information that is not in the sample (-> too low power) and the method critically relies on the assumption that the samples are (highly) representative (what is more than doubtful for n=1...). The second point is also the case in other analyses (like e.g. the one I suggested). It should be ok as long as this possible problem is considered and seriousely discussed. This is an assumption of bootstrapping, and it is often forgotten (I read wrong sentences like "boostapping is an asumption-free method" etc.). It becomes relevant for small samples.

Hi Jochen, hi all,

@Jochen: Totally agreed, but I was making one of the assumptions you made, namely that there is no temporal trend in the data.

@Maximilian: Why not run some more control subjects? Whatever your experimental condition is, it should be easier to run control- than experimental- group subjects, and that would improve (somehow) the generality of the conclusions you would get by analyzing your data. Just a thought, not sure whether it's feasible.