Repeated measures analysis of variance makes horrible assumptions about your data (compound sphericity) that are not usually observed under the current laws of physics.

I would go lightly on the statistical analysis. This is a pilot study, designed to inform the next steps in the development of the intervention. 

A couple of points:

1. Use the baseline scores as a covariate. 

2. Use robust variance estimation to adjust for clustering of scores within participants, rather than repeated measures. This also allows you to include observations with missing values, whereas in repeated measures they have to be thrown out.

3. Do not use MANOVA. This bundles all the scores into a single hypothesis test. If you are running a small pilot study, it is important to approach the analysis as scoping the effects of the intervention, and to see which measures change and which do not. Since no-one is going to make a final decision about the intervention based on your analysis, you can be exploratory. 

Remember, and this is the important point, that this does not constitute a valid test of the intervention. People who suffer from conditions where the symptoms fluctuate in  intensity, such as depression, will enrol in a trial at a point where the symptoms are at their worst. Subsequent measures will show an improvement even in the absence of treatment, simply because of naturally-occurring fluctuations. So even if scores improve, this does not show the effectiveness of the intervention. It could also simply be the passing of time, that healer of wounds. 

Check out the MRC guidelines in evaluating complex interventions. These are very helpful, and steer you away from the heavily statistical approaches advocated above. 

http://www.mrc.ac.uk/documents/pdf/complex-interventions-guidance/

The analysis depends on your hypothesis to be tested. What will be your hypothesis?

By looking at your variables different scenarios come to one's mind. One of them is stated by Paul which is valid for considering effect of time on those variables together. But there are others too, like considering effect of time on them separately, or mainly for primary endpoint or on a composite endpoint.

There is also mixed effects model you can consider for repeated measures like yours.

Repeated measures analysis of variance makes horrible assumptions about your data (compound sphericity) that are not usually observed under the current laws of physics.

I would go lightly on the statistical analysis. This is a pilot study, designed to inform the next steps in the development of the intervention. 

A couple of points:

1. Use the baseline scores as a covariate. 

2. Use robust variance estimation to adjust for clustering of scores within participants, rather than repeated measures. This also allows you to include observations with missing values, whereas in repeated measures they have to be thrown out.

3. Do not use MANOVA. This bundles all the scores into a single hypothesis test. If you are running a small pilot study, it is important to approach the analysis as scoping the effects of the intervention, and to see which measures change and which do not. Since no-one is going to make a final decision about the intervention based on your analysis, you can be exploratory. 

Remember, and this is the important point, that this does not constitute a valid test of the intervention. People who suffer from conditions where the symptoms fluctuate in  intensity, such as depression, will enrol in a trial at a point where the symptoms are at their worst. Subsequent measures will show an improvement even in the absence of treatment, simply because of naturally-occurring fluctuations. So even if scores improve, this does not show the effectiveness of the intervention. It could also simply be the passing of time, that healer of wounds. 

Check out the MRC guidelines in evaluating complex interventions. These are very helpful, and steer you away from the heavily statistical approaches advocated above. 

http://www.mrc.ac.uk/documents/pdf/complex-interventions-guidance/

Hello everyone,

Deeply appreciate the answers thus far. As a new researcher, it can feel you're thrown in the deep end and they've all been a great help.

Regarding lumping variables together, i do not think it is appropriate but i could be mistaken.

This is due to the differing nature of our scales. Whilst i am very new at this, i don't think it would be appropriate to lump depression scales with quality of life or cognition scales.

It seems a folly to assume that because our drugs may affect one scale or dimension, that it may *equally* and similarly improve all dimensions. Indeed, there is value itself in seeing how these individual items change over time with treatment.

---

Dear Ronán,

Could it please be elaborated on regarding what is meant by "robust variance estimation", located under list item #2? Is there a particular test or method to consider?

If the DV is "change in Y" you should not, as Martínpoints out, adjust for baseline. However, change in Y is a problematic variable, since higher values at baseline have higher potential for change. Where you adjust for baseline is when comparing mean values of Y at follow up.

J Kirk: I am referring to methods such as Huber White variance estimation, implemented in most statistical software nowadays. 

You can also use Generalized estimating equations or Latent growth curves, as both help you managing missing data, however it depends heavily on the study hipothesis

Dear J Kirk

As you have within subject factor  and  variety of psychological scales (between subject factor)it seems mixed model of  repeated measures is suitable .you can compare these scales perfectly by this model.

question: how can i adjust mean difference (after mines before treatment) using repeated measure ANOVA?