Dear Judicael,

I'm not sure if I understood your design correctly. But for binary variables, log-linear analysis could be a possible method. If you could say a little more about your variables and number of participants, I might be able to help you better.

(Do you have repeated measures?)

Thank you for your answer. I realized that the ANOVA test wasn't suitable for data such as mine which are categorical and have hence edited the question.

My data set comprises 63 psychometric variables assessing for the presence or absence of a given trouble, made on 25 subjects. I'm trying to find hints about a potential overlap between these variables. Consequently, I opted for a Chi square test over all my data set, but does it bear too much bias risks given the sample size, as well as regarding the multiple comparisons ? Which method or correction would then be the most suitable ?

Or to rephrase : I have 63 factors with two levels each, and I opted for pairwise chi square test for each of those. Does it require any correction due to the multiple comparisons (knowing that I already implement the Yates continuity correction in my test) ?

Thank you in advance,

Judicaël

If binary, no.

There are proportional corrections s for ordinal > levels in X2.

See SEM texts.

Hope this helps

Matt

Theoretically you could run a principal components analysis on binary data. See:

https://cran.r-project.org/web/packages/logisticPCA/vignettes/logisticPCA.html

or a factor analysis on binary data: https://it.unt.edu/sites/default/files/binaryfa_l_jds_sep2014_0.pdf

This would reveal how much 'types' of different categories there are.

Yet, you do not have enough data points for such an analysis. In the example above there are 1000 observations.

I would suggest you run the chi-square on all 63 categories and inspect the plot of the residuals to see which categories are similar and which are different. See R script in attachment on fabricated data. In this example the first and last categories are completely different. The first 20 are not 'troublesome', but the last 10 are. The middle pack is rather neutral, it does not contribute much to the chi-square. The chi-square is very significant here, indicating there is an association between category and presence or absence of a given trouble .

You ran the chi-square test for independence (for the differences between categories). I think it would be better to run the McNemar test for change, because the samples (subjects) are related. In attachment the addapted script with the Cochran Q test on all 63 categories and the McNemar tests. I tried to implement the pairwise comparison list as illustrated in the R Companion (see: https://rcompanion.org/handbook/H_07.html), but it did not work. Yet, I could run the parwise tests themselves with several adjustment options (Holm, fdr, etc.).

I should add the fabricated data I used. You should place these in the correct directory. Set the working directory before reading in.

Before you continue to draw conclusions about your results, let me alert you that there is a rule of thumb that for meaningful analyses the number of variables should not exceed the number of subjects (unless you have repeated measures).

As I understand, the63 presens/absence assesments of trouble of the categories were repeated measures.

Thank you a lot dear Jos Feys, Wolfang Shoppek and others who took time to answer,

I took time to dig into the links you provided and they ended up being extremely useful.

Also, thank you a lot for the attached R code and example you provided, I modified it in order to plot a matrix of the resulting p-values obtained after each pairwise Chi-square test.

To answer, the presence of all troubles constituting the categories were assessed on the same subjects, these are however different troubles whose presence where assessed. Consequently, these are not repeated measures, but a set of different categorical assessments that have been conducted one after another, for each subject.

Given my small sample size (25 subjects for 63 categorical variables + another set of data comprising 25 subjects for 44 observations), is chi square still worth being implemented ?