As long as your DVs are the seven categories, GLM sounds fine. You might consider Generalized Mixed Models as you can account for within participant variance (i.e. between texts).

I do not know why you would want to employ ANOVA for post-hoc comparison since ANOVA is unable to detect which group out of 3 differs from the others.

I also wondered whether you wanted to estimate 7 different models or if a composite score, summing up the outcomes of your classification would be more suitable. This however depends on your research question and you stated, that you were interested in the singular categories

As long as your DVs are the seven categories, GLM sounds fine. You might consider Generalized Mixed Models as you can account for within participant variance (i.e. between texts).

I do not know why you would want to employ ANOVA for post-hoc comparison since ANOVA is unable to detect which group out of 3 differs from the others.

I also wondered whether you wanted to estimate 7 different models or if a composite score, summing up the outcomes of your classification would be more suitable. This however depends on your research question and you stated, that you were interested in the singular categories

As Sven stated you do not need post-hoc but concentrate on language and age effects as such and clearly to their interaction. If you deal with categorical variables you can supplement GLM with multiway contingency tables based on the generalization of chi-square that are less dependent on both linearity and continuous assumprions with respect to GLM:

Article On Chi-squared Tests For Multiway Contigency Tables with Pro...

Hugo Vilar Weber I am sorry but i must disagree with my colleagues. Your DV has 7 groups but is apparently not ordinal. Thus GLM is not appropriate because The DV requirement for GLM is continuous. What you need is multinomial logistic regression. I will give you a few references that should clear this up. First is the requirement for GLM DVs. The standard work in this area is this: The Linear Model and Hypothesis: A General Unifying Theory | George Seber (auth.) | download (1lib.us) Please notice the continuity requirement. NOW, What you need for your analysis - DV=a 7 group categorical variable is a special case of the generalized linear model or glm. see this reference: Categorical Dependent Variables and Multinomial logistic Regression - Bing . A good reference to glms can also be found in the z-lib. the categorical GLM is discriminant analysis and the glm is in the usual case logistic regression. I am attaching a paper of ours that covers the 2-group case, and is in press. Yours is, of course, 7-group.We are in the process of attacking a three group similar problem where 3-group multinomial logistic regression will be used. I recommend the R software.: Categorical Dependent Variables and Multinomial logistic Regression with R - Bing This is more complicated than the GLM would be but it is the CORRECT analysis. Since we are embarking on a very similar study, please feel free to contact us for further information. I wish you the best of luck with your study. David Booth PS you may find this link useful as well: R for everyone: advanced analytics and graphics | Lander, Jared P | download (1lib.us)

Though it is not exactly clear how the classification into seven categories is done, I assume that the categories have a hierarchy associated to them based on the strength and complexity of structuring the text. Ordinal regression can be considered.

Sven Greving thank you very much for your reply. My mistake, I did Bonferroni post-hocs, not ANOVAs. Regarding your second comment, I actually conducted both: I estimated a model with a composite score and 7 different for each variable, as I was interested in both options.

Alessandro Giuliani thank you very much for the information. I will examine the article provided as I'm indeed using categorical variables.

As a matter of fact multiway contingency tables correspond to the multinomial logistic regression advocated by David...good luck !

I think you can go for RCB design, analysis of variance.

regards.

Hugo Vilar Weber what exactly are the possible values in each DV? I agree with David Eugene Booth that in case of non-ordinal values you should employ a different approach.

David Eugene Booth thank you very much for your answer. Actually the 7 groups hold an ordinal relation as Viswa Chandu mentions, as they are ordered according to structural quality. Then, if I used a model that embraced the 7 groups, would that be ok? And then, if I want to explore each group separately, I should use multinominal logistic regression, right? I will examine in detail those references that you provide, thanks!

Alessandro Giuliani good to know! I'm not familiar with this terminology.

Itrat Naqvi thank you for your answer. Is that randomized control block design? Honestly, it's the first time I hear about it. Any reference that you suggest examining to apply it?

Sven Greving I have one variable with each value corresponding to a different structure (from 1 to 7). This variable is actually ordinal in terms of "quality. I also have 7 other binari variables, each one corresponding to a specific structure.

In my first response, I assumed something like this, so it still stands

If you have 7 groups corresponding to to structure how is that ordinal? If they were buildings they might be house, church , split level, and so on. Where is the ordering in that? You use what you have with the with the data prior to processing. You can't regroup them after you calculate the square footage after the regression. Be serious. D. Booth

David Eugene Booth the structure of the text is coded according to qualitative criterions, thus there is an intrinsic order in terms of quality between each group; you can say that group 1 is better in terms of structural quality than 2, that group 2 is better than group 3 and so on. This order was decided when the coding criteria was developed, not after the texts were analyzed. Wouldn't you, then, consider this variable as ordinal?

I think Glm

@HUGO, oh I see you know the numbers are correct because you have qualitative information that can be put in quantitative order so that your information is ordinal now. Please when you send your paper to a journal be sure to ask that I be excluded from the reviewing pool. Good luck, David Booth

I would use multinomial logistic regression, however if the text quality assessment based on the text structure was done with methodological rigor and you want to use it on ordinal scale I suggest you to apply a non parametric Kruskal-Wallis test rather than an ANOVA.

Hi Hugo Vilar Weber ,

Can you define/explain what the concept "structure" (of text) exactly is that you used for classification - that could help to decide on the right/objective approach to adopt. I am extremely concerned about the qualitative criteria you applied in making the respective variable ordinal.