Dear Dirk,

    Models are supposed to serve your interests, not the other way around. So don't feel compelled to use a multi-level model. Psychologists, especially, often take repeated measures and derive an average. They then analyse differences between averages. It is particularly suitable if you are actually interested in average performance and no curiosity about the actual distributions involved. 

   Two things to reflect on, though. One is that your between person effects can often be modelled more precisely and accurately using a multi-level model then can be done using an aggregated approach. The second is that the size of the difference between the two approaches can be easily verified if you use a MLM or HLM first... so assuming that the aggregation effects are irrelevant seems a little silly once you become accustomed to MLM/HLMs. The literature on the aggregation bias and ecological fallacy give reasonable pointers on what sorts of mistakes in interpretation are likely to be made by ignoring these points - however, as I wrote, in some fields aggregation is still the standard approach. 

   WRT to what you wrote I am not so sure that these points should dominate your decision. Aggregation doesn't assume that within level/person variance is zero. It is done precisely because people are deciding that the variance is large enough to matter and that they think it is irrelevant to what they wish to test. There is no compelling reason to worry about the weights given to participants as a function of the number of measurements if the pattern of missingness is not informative (MCAR) and you are disinterested in that level of variation. But participant averages can be weighted by their number of constituent observations - and Bayesians would likely insist on this - without much effort on your part.

   So, to summarize, it is not always wrong and not always a problem; in your field aggregation may even make more sense ( I can picture how sources of variation like temperature, feed humidity, sunlight might be less interesting than the average milk output). The things to watch for are biases in interpretation and a rather unnecessary loss of information.

Hope this helps

Dear Bruce, thank you for your elaborate answer. You are right, of course, that you would actually decide for a multilevel model if you expect that there is variance at both levels and you want to take them apart.

I am asking because I ran a series of multilevel models, only including between effects. Effectively, a between effect is just a mean level difference (over the days). When I compare the coefficients from these models with my results when I aggregate my data and use OLS techniques, they are almost exactly the same.

So in regards to your comment "The second is that the size of the difference between the two approaches can be easily verified if you use a MLM or HLM first...": in my case there appear to be little if any difference at all.

The second thing is that I am comparing the relative influence of different between person variables: R2 in multilevel models can be tricky, and are more straightforward in OLS techniques.

Still, based on the possible pitfalls of aggregation, like the ones you name, it makes me inclined to in fact use multilevel models: the fact that aggregation does not appear to be a problem in my data should not influence the decision per se, it should be a theoretical decision. Would you agree?

Again thanks, Dirk

Aitken and Longford (1986) - " the means on means analysis is meaningless".

Seriously though - it really depends on what you want to find out; and for what I do the within person variation can be important.

Theory is overwhelmingly concerned with averages - we have not usually thought about heterogeneity and a full random coefficient approach allows this in quite subtle ways. Thus men and women may have the same average but differ in their variability - to me this would be interesting. In a diary study the mean response at the weekend is the same as the weekday but the variability is less; again this is interesting. The ability to structure the variance by predictors (categorical and continuous) at any level is a powerful aspect of the random coefficient approach - for me multilevel modelling allows you to model explicitly the mean and the variance simultaneously and when I teach I stress this variance function approach:

https://www.researchgate.net/publication/23538539_Modelling_Complexity_Analysing_between_Individual_and_between_Place_Variation.

THis paper looks at the aggregation and atomistic fallacy using some classic data

https://www.researchgate.net/publication/23959447_Revisiting_Robinson_The_Perils_of_Individualistic_and_Ecologic_Fallacy

Article Modelling Complexity: Analysing Between-Individual and Betwe...

Article Revisiting Robinson: The Perils of Individualistic and Ecolo...

Dear Dirk,

    When you say that it should be a theoretical decision I have to say that I am a little more on the fence about this. It should be a theoretical decision if your theory (see Kelvyn's resources) is precise enough. In your case you have already run the MLM and seen the result. So your basis to run a second OLS model is based on the evidence at hand and your desire to report r squared and some other easier to interpret statistics. In practice I see nothing wrong with reporting the finding from the MLM model and then following up in the way that you think makes the most sense.

   In theory, of course, those results are already in the MLM and you have the opportunity to address other questions as they arrive. 

   I tend to be pragmatic about what I report. In this case I think you should report the initial MLM model and then follow it with what you believe to be the best description of your data. If you believe that readers are more likely to make correct judgements based on an OLS table then put it in. 

   In the original question you posed it was universal - is it always wrong. I think that many would answer MLM is in principle better. When you can show an MLM table that assures me that the interesting effects can be captured in an OLS table then I really can't see a problem with it! I have more trouble understanding the people who don't check in the first place!

Take care