Dear colleagues,
I am currently teaching myself in multilevel analysis using educational data. I found few to zero literature on basic analysis methods. In my context we frequently use level-1 pre-post data and level-2 (pre) predictors, e. g.: I have student achievement in the beginning of the school year (pre) and at the end (post). On level 2 I have teacher knowledge assessed at the beginning of the school year (pre).
"Question": I want to regress student achievement on teacher knowledge, controlling for prior student achievement.
What would your multilevel approach be?
(a) Do a multilevel panel data analysis with a long data set? I.e. using time on level 1.
(b) Do an multilevel ANOVA approach with a wide data set? I.e. controlling for prior knowldge.
I know it is simple, but self-teaching is really hard. I hope you understand!
Best wishes!
Alexander
I'm assuming your data for students (level 1) is clustered for each classroom taught by teachers (level 2) and you only have 2 timepoints (pre and post). In this case, ANOVA would not be desired since it won't be adjusting for the nested data unless you aggregate the student scores at to the teacher level (but then you're losing out on data and it's just odd given the topic of interest).
So MLM would be the better alternative of the two. Time wouldn't be covariate because you're only examining pre and post. In this case, I believe the best method would be to:
Outcome variable (Y2): Post-student achievement
Covariate 1 (Level 1; Y1): Pre-student achievement
Covariate 2 (Level 2; X1): Pre-Teacher knowledge
With this, you're examining the effect of teacher knowledge at Time 1 on students' achievement scores at Time 2 controlling for their base achievement scores at Time 1. Then, you can argue for partial Granger causality of the effect of X1 on Y2 above and beyond the effect of Y1 on Y2.
Whether the data should be wide or long depends on what software you use.
This is what I would recommend based on the information given but open to hearing others' inputs.
If the level 2 is the classroom taught by teachers (asindependent variable),
I think an ANCOVA could be used:
post = pre + Teachers/Class (in that order)
The need for a multilevel approach can be tested
https://www.researchgate.net/post/Level_1_pre-post_multilevel_regression_Panel_data_or_ANOVA_approach?isAnswerFieldFocused=true
Andy Field describes this test for the need of a multilevel approach very well in his Dicovering Statistics using SPSS/SAS/R books
https://www.discoveringstatistics.com/books/discovering-statistics-using-r/
First, you could calculate the IntraClass correlation (ICC). This represents the proportion of the total variability in the outcome of the students that is attributed to the teachers classes.
Secondly, you can test the need for multilevel modeling by comparing 2 models.
The first model is the baseline model in which we include only the intercept:
[In R:]
baseline
Hello Alexander F. Koch. When I first wanted to learn about multilevel models, I found the book by Jos Twisk very helpful. (The chapter that shows code examples may be a bit dated now, given changes in some of the software packages, but it should still be helpful.)
- https://www.cambridge.org/core/books/applied-multilevel-analysis/7755169EDD3B2304B041773D48F2BA6F
If you happen to be a Stata user, here's another good introductory book:
- https://www.stata.com/bookstore/multilevel-modeling-in-plain-language/
HTH.
Check https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5645294/
ANOVA, Check https://www.theanalysisfactor.com/repeated-measures-approaches/
- Badges
- Science topic