Hello. My question is about manipulation checks in experimental studies. I am well aware of their benefits and I can see that proponents of survey experiments widely recommend using them. Although I cannot come up with one for my research design. Could someone give me an example of MCs in the political science which have been successfully used, just to have an idea? In psyhchology, self-report of a mood, etc. could be used, but in my research where I aim to measure the willingness to strike a foreign country, I do not know what kind of MCs I could use, relevant to my vignette. Thank you.
In your case, I don't think you can find an established scale to measure "willingness to strike a foreign country". Instead, you can develop a new scale to measure it with 3 or more items....and then check with several area experts for the content validity. then..you can do a pilot study with 30 or more participant whether the scale reflects your manipulations well.
Thank you, Dr Tae-Yeol Kim. Let us say, if my IV is democracy or non-democracy, then just after the experiment I could ask the participants to recall what kind of political regime they dealt with in the vignette, I could offer a scale of democracy-non-democracy, to see if they "took" the treatment of either the control group (democracy), or the treatment group (non-democracy), right? I find it too straightforward though. What do you think?
What you describe in your reply would certainly be a manipulation check, and if democracy versus non-democracy is what you are manipulating in your vignettes, then that is what you should check.
However, it seems to me that you might want to evaluate this in a pre-test, so that if it is straight forward, then you would not necessarily need the manipulation check. More importantly, if people in the pre-test do miss the distinction, then you will need to improve the vignettes.
Dear David L Morgan, Thank you so much for your reply. Yes, in the end, I constructed a battery of questions on my treatment, and it seemed to work, as almost 97% of respondents seemed to "take" the treatment, by correctly answering the questions. Thank you.
what if I checked the manipulation check with an item? is a cross table sufficient as a check? - besides, I have the first hypothesis that asks exactly this: is there a difference between Manipulation 1 and 2. that is actually the overriding assumption of all the work. Do I have the hypothesis 1 wrong? Thanks and regards
- Badges
- Science method
- Similar topics
- Mathematical Sciences
- Statistics
- Applied Statistics
- Survey Design