I agree with Adrian that it depends on what you exactly want to do, but not so much upon the statistical technique you want to apply. I would say that it depends on what you want to generalize to: countries or populations?

If you want to generalize to countries, treat each country-aggregate as a single observation - and thus do not apply weights. This would allow you to test hypotheses such as: countries with policy X have a population that on average is more liberal on attitude Y. (note, that although I use the word population here, the unit of interest here are the countries).

If you want to generalize to populations, do apply weights. This would allow you to test hypotheses such as: the world population has become more liberal on attitude Y, because more countries have implemented policy X.

Note that in the latter example, treating Andorra and China with equal importance would not make sense: even if Andorra would implement policy X and become more liberal on Y, this hardly effects the worlds' population regarding attitude Y if China does not change.

I hope I explained this distinction clearly

It depends on your analysis. If it is a regression and you want to test if a relationship applies for all countries, then do not weight. If you do, results for Andorra will not have any relevance, and this is not something that you want to have.

If you want to do tables, descriptives etc. then definitely you need to weight.

If you want to account for a size effect, e.g. big countries behave differently, you can add logs of population as regressors.

I believe there is a Stata manual on how to handle survey data. You may look for it as it explains survey data issues in detail.

Hope this helps

I agree with Adrian that it depends on what you exactly want to do, but not so much upon the statistical technique you want to apply. I would say that it depends on what you want to generalize to: countries or populations?

If you want to generalize to countries, treat each country-aggregate as a single observation - and thus do not apply weights. This would allow you to test hypotheses such as: countries with policy X have a population that on average is more liberal on attitude Y. (note, that although I use the word population here, the unit of interest here are the countries).

If you want to generalize to populations, do apply weights. This would allow you to test hypotheses such as: the world population has become more liberal on attitude Y, because more countries have implemented policy X.

Note that in the latter example, treating Andorra and China with equal importance would not make sense: even if Andorra would implement policy X and become more liberal on Y, this hardly effects the worlds' population regarding attitude Y if China does not change.

I hope I explained this distinction clearly

Hi,

I think my answer would benefit for some clarification. World Values Survey was conducted in several waves.For example, If the analysis is being done using several years of data using a mixed panel model, the way data is considered is highly relevant! Doing country averages would reduce the size of the dataset.

Generalization to countries does not really need averaging across countries, as in most cases the sample is limited and more or less comparable in size across countries. We should keep in mind that the sample as is is deemed to be reliable for each and every country in the survey. A full data set is much better in capturing the relationship between variables, and this is what is desired. Mathematical logic says we should average, but common econometric sense ( a la James MacKinnon:) is to keep most observations and not alter the data.

With the rest I agree.

I am using a fixed-effect model with 245 country-wave observations. I would like to see how cultural values change with economic development measured as GDP per capita. The idea is to test the modernization theory so I would like to make conclusions to the whole world population. This is why I thought that weights would be appropriate.

I like your idea in general, Adrian, but in order to use the fixed-effect model I need panel data (the same units of analysis measured over time). WVS is a repeated cross-sectional data set unfortunately.

You can do a panel data set by stacking up observations from several waves. Thus you can model the different waves as another set of fixed effects.

Try also a random-effect model, even a pooled model; in some cases they yield better results. You will need to run some tests to guide you through.

Random-effect model produces higher coefficients in almost all cases, including mine, but this is not the point. It doesn't control for unobserved heterogeneity as the fixed-effect does. Moreover, I am interested in change over time, not in cross-sectional differences.

Perhaps I need to change my question to "Would you use weights if you were doing this kind of analysis and if so, how?"

If I understand you correctly, Plamen, you are testing the (interesting) hypothesis whether the world population has changed their cultural values, because they were living in countries with increasing GDPs (?).

- I would then indeed suggest to apply population (and design) weights;

- However, this may also result in influential cases (e.g. any change in China may determine all the regression outcomes), so I would strongly suggest to evaluate for influential data. Here is a paper in this subject, relating to WVS:

Van der Meer, T., Te Grotenhuis, M., & Pelzer, B. (2010). Influential Cases in Multilevel Modeling: A Methodological Comment. American Sociological Review, 75(1), 173–178. doi:10.1177/0003122409359166

- Personally, I greatly appreciate the multilevel model for change, as formulated by Singer and Willet. Particularly, since you first aggregate the person-level data into (some sort of) a country-level panel, this model seems applicable. They also discuss latent growth models, but I'm not sure if those fit your data:

Singer, J. D., & Willet, J. B. (2003). Applied Longitudinal Data Analysis. Oxford University Press.

Hope this is helpful.

Rense, I would like to see the picture through different angles. I already found that overall there is significant change in values with increase in GDP per capita. Further consideration is whether this conclusion applies to the whole world population or primarily to the Western democratic societies which are over-represented in WVS.

Thank you very much for your comments and suggested literature. I thought that multilevel model analysis would be far too complicated for me and I am writing only a seminar paper. May I ask you though, what are the key advantages of the multilevel model compared to the conventional random and fixed effects model?

Plamen, hi. As I understand it, a multilevel model is appropriate since it is able to partition variance at different levels in the same model. So the variance at each level will be different in a MLM since it takes account of variation between individuals and between groups, whereas ANOVA will average values within groups at the level you are analyzing, i.e. person-level variation would be ignored at the country-level in ANOVA. MLM also copes with heteroscedasticity much better than ANOVA, and as such is more robust at departures from equal sample sizes for each group within a level. One of the dangers with ANOVA and simple regression is the generation of too narrow confidence limits, hence you might find significance when really there is none. MLM also allows for random intercepts and random slopes in the same model. That's my understanding, and I hope I have understood what you are wanting to achieve.