I have data in four dependent variables, each representing the relative location of one of four words in a participant's text response, calculated as word location / word length, resulting in a range of .01 to .99. For example, if Word 1 appears as the first word in a response, then DV_Word1 = .01. If Word 2 appears in the middle of the response, then DV_Word2 = .50. Each DV measures a different word, and some words do not appear in all responses (and hence do not have a value for that DV). I can calculate the means of each DV, and can see that they are different (some words appear later in responses on average). What statistical test do I need to run to determine if the difference is statistically significant? There is no IV, and I want to include values from all participants, even if they only used one or two of the words.
Ette - can you be a little more specific? If I run a repeated measures ANOVA, I will only get results from participants who have values in all 4 DVs. I'm not trying to test for differences between DVs for each individual participant, but for differences between the overall means of the variables.
Your text responses are your observations. Your DVs are relative positions in your observations, ranging from 0 to 1, in principle, continuous, quantitative variables, for they may assume any value in (0,1].
To check whether means of quantitative variables are different, you may use anova or ttests, though usually you do that when you have an indep variable. Anova does not tell you which means are larger and which are smaller, nor how larger or smaller they are. But this is not where I would focus the attention now.
Now the ugly part (this is where I'd focus the attention). Any analysis involving different samples is tricky. This is true for comparison, regression, whatever. Differences may owe to the different samples used. To illustrate with a different example, instead of texts responses, imagine you compare means of two variables (electoral participation and participation in demonstrations) in two samples, one, 80% developed countries, the other, 80% undeveloped ones. Ugly, ugly.
Can you restrict to observations (text responses) that include the four words? Then the sampling problem fades. But you may end up with a sample that does not fit your research interests. In your case, it may even condition the placing of words.
I cannot point to a solution, but I hope that pointing at the issues you should bear in mind helps you find the way to answer your question.
- Badges
- Science topic
- Similar topics
- Biological Science
- Kinesiology
- Running