To check I have understood it correctly - you have a 3 level model, where your treatment is effectively a level-3 variable, but those not recieving the treatment aren't assigned a level 3 unit.

You could presumably put all of the waiting list people into their own level 3 'group'. You are presumably including a variable for each of the 2 treatments in the fixed part of the model, so the constant will represent the average effect for that group (waiting list) and the effects of the two treatments will be differentials from that group for everyone else.

Hope that helps - I don't know much about psychology or HLM (I use MLwiN) but I assume the logic can apply across.

That is helpful. We are considering that as an option (WL in a their own group), but worried about the implications of that (i.e., WL client data are not dependent on others in their "group" in the same way that the two treatment condition clients are dependent). Thanks.

I think that in this situation it should be fine. Because WL is one of the categories of the treatment variable, the WL group is effectively being dummied out of the distribution of L3 units by that variable. So although it may not have the same dependency as the other groups, it's residual will be in the middle of the distribution of L3 residuals and it therefore wont bias the estimated dependence in the other units.

If you didn't have your treatments (and thus WL) in the fixed part of your model as you do, you would need to add that dummy variable (1 if WL, 0 of not) to the fixed part of your model to avoid the problem you describe.

Very helpful. Thanks!

Dear Nathaniel,

Is this a longitudinal study with lets say 4 measurement occasions? If not I think this is a level 2 model with patients (level 1) nested in group (level 2).

Coming to your question. (I consider a two level model.) If you use Mlwin you could consider separate dummies to estimate the error variances at level 1: one for the waiting list and one for the other groups at level 1. In this way you would estimate different within group variance (error) for the treatment group and the waiting list . If the estimates are statistically different you keep them otherwise you join together again. Moreover I would use the dummy of the treatment group to estimate the between group variance at level 2. The dummy for the waiting list is left out at level two. (This is fully in line with Andrews answer)

Best wishes Jan

Ok,

I suggest you to analyze your data in SPSS software by GLM and with following model.

Inter observations in dependent Variable;

Inter time, clients, and groups in random factors;

Build model as:

Group

Client*Group

Time

group*Time

Software will be don ANOVA using mean expected squares. consider that Client*Group is Client within Group.

I hope that answer can be helpful.

Jan, thanks for your answer. However, it is a three-level model as we have 4 time points over the course of treatment (pre, mid, post, follow-up). However, I am guessing your analytic strategy will still hold in the three level model. It seems that you are suggesting an exploration of within group variance in two groups (really, two different conditions, those in wait-list condition as one group and those in treatments conditions as another group). Does this approach then ignore the individual treatment groups that clients were in (they were treated in small groups)? Or were you thinking the wait list condition would be one group and then all the individual treatment groups that made up the two treatment conditions would be separate groups? For this initial analysis of exploring comparability (variance equality) among the groups, we could just look at post-treatment and make it a two-level model. Thoughts? Thanks.

I think there are two separate issues here.

The first is that the WL 'group' is not part of the distribution of the (level 3) treatment groups (this is equivalent to saying that the WL group has a different level of group dependency to the treatment groups). This is solved by including WL as a dummy variable in the fixed part of the model (or in your case, including the treatments and so including WL by implication).

Jan's idea is about something slightly different, that is to allow you to model varying variances - ie letting the variance be different for different treatments. You could do this at any, or all, levels, and it would still be within the 3-level multilevel framework so no, you would not be ignoring treatment groups. I assume this can be done in HLM (it certainly can in MLwiN).

This is not a bad idea and may be very substantively interesting. A treatment that has very variable outcomes between individuals (level 2) or very volatile outcomes over time (level 1) may not be a treatment you would choose even if its outcomes were better than another treatment on average.

Excellent. Thanks so much Andrew and Jan. These comments are really helpful and will give us a great start as we move forward with the analyses.

Dear Nathniel,

I agree with the three level structure. I also agree with Andrew's answer. However there is a small not very important difference. I suggest to use a dummy with 1's for non waiting list at level three (and no variable for WL) to esitimate the random betweeen group differences (level 3 variance).

At levels two and one the same dummy is used together with a dummy with WL code 1 and other 0.

The major benefit of this proposal is that you do not estimate the (random) mean of WL as part of the random variance at level 3. You only estimate the fixed WL mean in the fixed part of the model, which may be time dependent (spontaneous improvement).

Best wishes Jan

Sorry to add a monkey wrench to this, but how were people randomized? Were they randomized in pairs? Was there any matching that occurred? I think this might change the sources of variation in ways that might suggest that creating pseudo-groups in the waitlist arm should be considered. I would have to think more about that, and you might want to find a (bio)statistician to give it a ponder.

So long as randomization occurred not in blocks (or I suppose sufficiently large blocks) independent of group assignment then I think what has been said already clearly applies. While I also agree that modelling the variance separately for the waitlist and the control is likely the wisest move (especially using Jan's suggestions), I think it's also important to consider the question being asked here. Are you intending to add variables at that level to try to explain some of the between group differences? If so Jan's point is especially critical about how you construct the dummy variables since you will not have such explanatory variables for the waitlist, and perhaps then excluding that arm completely makes more sense and considering this a secondary analysis to describe factors which contribute to better or worse group outcomes.

Dear Nathaniel,

I suggest you to read the seminal paper of Prof. Daniel Baeur and colleagues.

Bauer, Daniel J. , Sterba, Sonya K. and Hallfors, Denise Dion(2008) 'Evaluating Group-Based Interventions When Control Participants Are Ungrouped', Multivariate Behavioral Research, 43: 2, 210 — 236.

Best regards

Gian, Thanks so much for this reference. I had not seen this one yet. Very helpful.