I’m conducting a content analysis with three main categories (A,B,C) for type of comment received. These three main categories are also split into their two underlying facets. I wish to test for group (2) differences in the proportion of comments received for each facet (6).
What is the most valid way for me to do this, given that the facets are related (2 for A, 2 for B, 2 for C), and that the group sizes are very unequal (approx. 85%-15%)?
I can only think of doing a 2x2 Chi-square (one facet v all other facets x group) repeated for each facet. Is that valid in the context of my study?
Any advice would be much appreciated.
You can do "hierarchical" chi -square tests that assess whether adding new variables increases your explanatory power (i.e., significantly reduces your chi-sq). More formally, you can use log linear analysis for the same purpose.
There are 3 ways you can do this. First is creating dummy variables and running a chi square test for each level. Since you only have a 2x3 table you shouldn't run into any issues (which usually happens when you have a large number of categories that increase the risk of type I error). However, this is only a descriptive technique that describes the relationship but does not allow you to model or predict the outcome
Second, as suggested by David, the best way to test more than 2 categorical levels is log-linear analysis (a modeling technique and extension of chi-square). this allows you to make predictions and to define the dependant and independent variables.
The 3rd is to use Bonferroni correction, which is a form of a "post-hoc" chi-square testing that counteracts multiple comparisons in a chi-square table by testing every hypothesis in your table at an a/m significance level (a is your desired p-value, m is the total number of observations). However, this method is largely debatable and only decreases type 1 error by increasing the risk of type 2 error.
In summary, log-linear analysis is the best test for the purpose of your analysis.
- Badges
- Science topic
- Similar topics
- Comment