The data are as follows (overview of the design attached): One parameter was measured in the blood of the same patients at various time points and also in a healthy control group at one time point (as a baseline value). The first step is to compare the parameter at the various time points of the patient group. Here, I would have to use a repeated-measures ANOVA (or Friedman's test if assumption of normality is violated), if I'm not mistaken. However, I also want to compare each individual time point to the one control group. Should I test each time point in the patient group with the control group individually using a t-Test (or Mann-Whitney)? If so, do I have to adjust the p-value because of multiple testing?
I know that this is a rather basic statistics question, but the combination of dependent and independent tests in this experimental design confuses me a little bit.
Thanks in advance!
You could, for each time point, run the 2-way between subjects ANOVA; and yes, do adjust the p-values.
If you have an equal number of subjects in the control as in the experimental group, you could pair the subjects from the experimental groups with those from the control group. To do so, rank order the subjects on the 'parameter in the blood', in the control group and in each experimental group. You can then pair the subjects with the same rank and run a within subjects (repeated measures) ANOVA.
Check the distributions of the 'parameter in the blood' with histograms before you decide to use either parametric or nonparametric tests. (The Shapiro test is known to be biased by size, i.e. the larger the size, the more prone to (falsely?) yield significant deviations from normality of the distribution.) If the histograms do look quite normal, use parametric tests. T-tests and ANOVAs are considered to be quite robust.
Fabian,
you ask the wrong questions. A much better questions to answer would be:
How can one characterize/formulate the time-course of the response?
You could then investigate interesting features of the time-course (e.g. is there a long-term change over time (linear or exponential)? is there a peak (and if so: at what time, how big)? is there a time at which the response changes rather fast ("step"; and if so: at what time, how big?)...
Based on your subject knowledge you should have some idea about the general form of the time-course. Plot the data to see if this makes sense or to get an idea what form might be appropriate to describe the data. Then fit an appropriate (possibly non-linear) regression model. The coefficients in that model will have particular meanings (related to the questions I gave above) and you can test meaningful hypotheses.
The data of the control group is not needed for that. However, you can use the fitted models to predict the response for "T0" and test the hypothesis that the mean response at T0 equals the mean response of the controls.
Note that the whole experiment is NOT ABLE to show a treatment effect (without additional assumptions, which might not be justifiable)!
The response may change over time even in untreated patients. Any change you may see in your data may thus be due to a time-effect or to a treatment-effect or a combination thereof. If you have solid external data demonstrating that the response is constant over the period of time you used in your experiment, you may carefully assume that changes might be causedby the treatment. Otherwise you actually can't say what the treatment effect is. You would need data from similar timepointsof the untreated controls.
I agree with Jochen, plotting the data is a good starting point and it is true that the response may change over time even in untreated patients.
Unluckily, there is not the time trend for the control group, to be able to answer more experimental questions.
Use
Repeated measures ANOVA and do a post hoc test to compare means at each time point and also with control group.
Thanks for the answers.
Perhaps I should clarify that the control group does not serve as a placebo group but simply as a means to obtain a baseline value for that parameter since said parameter is, shall we say, rather experimental. That's also the reason why the control group was only measured once. Also, this was not done to show a treatment effect but rather examine that parameter in a certain "disease" (severely injured patients).
I have plotted the data and there is a substantial difference between the various time points and the "baseline" values obtained from the healthy control group. Now I want to test whether said differences are significant, and, as Mercedes suggested, I would have used a t-Test or Mann-Whitney-Test for pairwise comparisons between each time point individually and the control group, without adjusting for p-values.
I don't think that this is a good approach (and you still silently assume that the control values would not change over time - that should be justified).
It would be better, IMO, to only compare T1 and control, and to test a trend over time for the treated samples. These are two tests asking two independent questions that are (to some extend and given the limitations of the design) meaningful (at leat more meaningful than comparing T3 vs controls etc.).
PS: your approach (testing each timepoint individually) is a screening-like approach and required an adjustment for multiple testing.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA