I am interested in testing whether any of three different versions of a survey question (respondents randomly assigned) results in significant differences in self-reported height and weight. My hypothesis is that version 3 will result in lower reported height and higher reported weight as compared with version 2, which will in turn result in lower reported height and higher reported weight as compared with version 1. Am I correct that I would conduct a two-way ANOVA with an ad hoc test (Least Significant Difference?) if the f-statistics are significant?
Based on the literature, I also hypothesise that the impact of the three different question wordings will be greater on reported weight for those respondents with higher body mass index (a statistic based on height and weight), higher SES, and older women, and greater on reported height for older people. Suggestions for what test to use to examine these hypotheses?
Dear Nina,
what would be the second factor? If I understand it correctly, you have 3 different independent groups and ask them for their height and weight? Height and weight are 2 dependent variables and can't be added as a second factor (if you had that in mind), since they are not on the same scale and the comparison of height in cm and weight in kg is meaningless. Therefore a oneway ANOVA would be appropriate (assuming that all assumptions are met of course).
I am also not sure about the second part of your question. Do you want to assess the impact of the grouping variable (your factor) on other dependent variables (e.g. SES whatever that is) or do you want to incorporate that into the main analysis as covariables?
Rainer Duesing, "SES" presumably is social economic status.
Nina Van Dyke, your first part has one thing that does not fit. You can analyze the reported weights and heights using Fisher's LSD, this is ok. However, Fisher's LSD is based on (or: protected by) a one-way ANOVA (not two-way). There is only one factor: the "version/wording of survey questions". This factor has three levels (-> 3 groups). You would do one analysis for weights and another for heights.
If you want to analyze weights and height together as two dimensions of the same underlying variable, you might consider a multivariate ANOVA (MANOVA).
To the second paragraph: "that the impact of the three different question wordings will be greater [...] for those respondents with [...]". So the assumption is that the effect of the wording itself depends on BMI, age, SES and sex. Here you have additional factors in your model (besides wording), and you need to test the interactions between these additional factors and the wording.
The general approach to analyze this is called a "general linear model". I suggest to seek a cooperation with a local statistician to help you get this analysis right. You will learn a lot and you will get the maximum out of your data.
Hello Nina,
For the basic design (does wording affect self-reported height and weight), one plausible approach is that of a one-way manova (as suggested by Jochen Wilhelm ). If you wanted to run height/weight as a repeated measures (a.k.a. within-subjects factor), you'd have to convert the values to a common scale (such as percent bias: e.g., 100*(Self-report height - Actual height) / Actual height to overcome the valid concern about scale raised by Rainer Duesing .
For the second part, where you'd like to bring in a lot of potential moderating variables, a factorial design could work, but do recognize that the more variables you add to a model, the bigger the sample size needs to be (because you are necessarily estimating more parameters). I agree that consultation with a local expert could be very helpful in sorting out the best tactics for your data set.
Good luck with your work.
Thank you Rainer Jochen, and David for your advice.
So if I understand correctly, for my first question, I should run two one-way ANOVAs with Height and Weight as the DVs, and then run Fisher's LSD if the F-statistics are significant, to distinguish which groups were different.
I'm still not quite clear how to go about analysing my second set of questions around moderation effects. To perhaps clarity with one example: I expect that respondents who receive wording 3 will report on average higher weights than those who receive wording 2, who will report on average higher weights than those who receive wording 1. However, these differences will depend on BMI -- those will higher BMIs will show greater differences between versions 1,2, and 3 than those with lower BMIs (who indeed may show no differences in reported weight among the three versions). I appreciate any additional thoughts. I will also take the above advice and seek out a statistician to assist.
This is then an interaction of the reported weight and the BMI. This can be modelled. I think the legacy name for such models are "ANCOVA" models.
There seem to be a bunch of complexities emerging in your project (e.g. trend-analysis vs. group comparisons, within- or between subjects designs, possible multivariable and multivariate analysis, additional covariables and their possible interactions with the treatment factor, and who knows what else). So this is becoming too complicated to be adviced via RG-posts. I suggest you seek the collaboration with a local statistician.
PS: Also note that SES is a very indirect and vague compound measure. It would be better to model the underlying and actually measurable variables directly. See for instance here: https://www.youtube.com/watch?v=PwCyvSDkUCY&feature=youtu.be&t=740
- Badges
- Science topic
- Similar topics
- Correction