Hi everyone, i am new to statistics and would like to consult you about some statistics issues.
I have a judgement task with multiple-choice questions. The design is a 5x2 factorial design, in this case i have 10 conditions, each condition had 10 questions, we had a total of 100 questions. Each participant Chooses one choice out of 6 options which correspond to 6 categories (i.e., Category A-F).
In this case, i will collect frequency data, and i think i could use chi square test for the frequency data. I want to know (1) whether any two of the 6 categoies in each condition significantly differ from each other and (2) whether the option A of any 2 conditions differ from each other (in this logic, option B of any 2 conditions and so on so forth).
I have a few questions, they are:
1. Since i have 10 items for each condition for each participant, how should i manage the frequency data, like in a tabulated table or in SPSS? Should i just add up the frequency such that if i have 50 participants i would have a total of 50people x 10items = 500 counts for each condition? Would it be inappropriate because each count represents not one person? I wonder whether there are any better way to handle the case.
2. i think the frequency data in fact can be turned into percentage data. In this case, we will have ratio scale data. then we may be able to use ANOVA to handle the data. But i wonder whether i should do in this way.
Thanks a lot for comments and advice.
This seems like a large project for someone new to statistics. I assume that you have at least one college level statistics class, but I think you should consult a statistician at your university. It sounds like the questions were using a Likert scale, so you will want to read about the analysis of Likert scales. Make sure your data conform to the assumptions of the statistical models you are using. Plotting the data often helps. You will need to run univariate statistics for each condition, and then (maybe) some multivariate analysis. Consulting a local statistician will be more efficient than RG for this problem.
Converting frequencies in % would not work. Focus on you variables which you are measuring. Your Data(and the variable you are measuring) must be on Interval or Ratio Scale.
With so many obervations clustered within subjects, you would violate the independence assumptions if you would just aggregrate and loose power. Calculating percentages is fine to get a feel for the data, but it is quite unlikely that these 10 observations for each subject in each condition will result in a percentage score that follows a normal distribution.
Your design requires sophisticated statistical analysis methods based on multilevel models. As you have choice data, it seems that a conditional logit in Stata would be a good option: that models a participants choice of one out of several options, so you actually model the likelihood of choosing one category over another between conditions as well. Stata allows for robust standard errors based on the clustering in your data. However, you should find a statistical expert to help you with this, these models are very complex and hard to understand.
A simpler approach would be to code every answer as correct or incorrect and do a multilevel logistic regression, something even SPSS might be able to handle... or use R with lmer.
What do the A-F mean? Could these be coded as right v wrong? This could make IRT useful (and the lme4 package, in particular the glmer function, could also be used ... the lmer function Martijn mentioned would be if the A-F were like a scale from low to high, but not if the values are right v wrong where a generalized linear mixed model is more appropriate ... old versions of lme4 allowed glmer to be called from lmer, but not for about a year).
What is your research question? This is needed before anyone can give a more specific response (other than Timothy's which I agree with).
Hi, thanks a lot for the replies. And sorry that i could not manage to come back and reply earlier.
Yes, i also think converting the court to percentage could let us have a feel of the pattern. and doing it in inferential statistics may result from a violation of sth.
A-F represent 6 different categories.
Thanks, i will try to consult more people about this. Really appreciate your help.
I agree with Dr Timothy. He has provided right suggestion.
I have found it difficult to understand how the design, mentioned here, is a 5x2 factorial design. There are 10 conditions, each condition had 10 questions, with a total of 100 questions. Accordingly it is to be a Completely randomized where each of the 10 conditions (which can be treated as 10 treatments) has been replicated 10 times. Also, each participant chooses one choice out of 6 options which correspond to 6 categories (i.e., Category A-F). Thus the observations obtained from this design are not of measurement type. Accordingly, the analysis of data as per the method used in design of experiment will not b applicable in this form of data. For this, data are to be converted in to some measurement-type of data.
It is also not a case where, as I know, the ready-made chi-square test is available. A test statistic will have to be formulated first to deal with this situation.
I figured there were two treatment variables (V1, V2) that were applied or observational treatments. V1 has 5 states and V2 has two states to give a total of 10 conditions. Each participant was given 10 Likert-type questions with six levels.
I hope the part about 100 questions is not relevant because I would interpret it to mean that the questions changed based on treatment group. If that is the case then I don't see how these data can be analyzed except as ten separate experiments (group=experiment).
An alternative interpretation is that there were a total of 100 questions. We asked each person ten questions, but the phrasing of the questions could be one of ten different forms. One of the key issues was how phrasing affects survey results.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software