Dear Vegard,

I understand your point. I am not sure if you can let the participants write their names and then you number each while analyzing. Can this help?

Or perhaps you can give each participant a number to put in his questionnaire and make sure he keeps it and uses it again. I am not sure if I help you....

Hello Vegard,

What you can do is treat the data set so that schools are considered the experimental unit (e.g., you assigned a school to a condition, but not individual classes or students). This multi-level (or hierarchical linear) model means that students are nested within classes, and classes are nested within schools, and schools are nested within treatment conditions. Note that, if you have the opportunity to assign individual classes at random to treatment, that would be an improvement.

How to analyze? The HLM approach would seem most suitable, but a simpler option that would still allow for direct comparisons to be made would be to use school means as the data points (so that you'd have two sets of 3 schools; grade level could be used as a stratification/blocking variable, since that appears to be common across schools). While you might be concerned that such small sample sizes (3 per cell) would yield very poor power, you'll find that school means do not vary as much as do class means or individual student means. Hence, the error terms tend to be smaller.

This situation is not uncommon in educational research, which is why it is helpful to get good statistical advice before initiating a study.

Good luck with your work!

I have had similar problems when doing financial audits. To get around it, I typically use codes in lieu of names. In your case it could be student numbers (without having a link to student names) or dates of birth (some of which will not be unique) or a concatenated field such as date of birth + initials (which hopefully are unique). In each case student names have been avoided yet the research will link back to individual students.

Thank you all for the answers! I highly appreciate it.

Konstantinos Karampelas and Chris Kelly

Norway has very strict privacy laws. It seems that as long as there are codes involved, even if the scientists are blind to them, the data is not considered "anonymous", but rather "de-identified". This means that I would need consent from parents and child to gather research data. Having a child bring a note home to its parents, then to bring the note back to school fully signed is a surprisingly difficult process that would probably severely reduce the number of participants. I am still hoping to find a way to code things that would still fulfill privacy regulations. That would also make it possible to compare these test results with other measures.

David Morse

I have been thinking that HLM would be useful, but I do not yet have enough understanding to know how best to implement that (or other statistical methods, for that matter). How are the two time-points best included in the analysis? Is it not possible to include the individual level in any analyses without the data being paired on the individual level? Also, I think the eight grade classes should be compared across treatment conditions, and the same with ninth and tenth grade classes.

since your intervention will probably be applied to whole classes rather than individuals, and because pupils from the same class (even the same schools, for that matter...) will talk to each other, the individual pupils are not independent. therefore, i don't think that you should even consider pairing on the level of individuals. the data from the pupils will inevitably be clustered, and your study design should take that into account. coding or anonymizing individual data won't properly address your issue.

"How are the two time-points best included in the analysis?" you could work with cluster-aggregates (means over all pupils from same classes) of individual post-pre-differences, and compare those cluster-aggregates across your study arms. however, a problem with such differences might be some regression to the mean effect, so maybe better try to include the timepoint as a factor in your hierarchical (= mixed) model.

"Also, I think the eight grade classes should be compared across treatment conditions, and the same with ninth and tenth grade classes." That's what david morse suggested when he wrote "grade level could be used as a stratification/blocking variable".

i'm also with david when he suggests to consult a good statistician (which i am not ;-))!