Not sure that I understand. Do you have two data sets with the same two variables (both binary) and want to see if, for example, the association between these variables differs between the two samples? Or do you want to compare specific cell frequencies (or many proportions) for one cell across the two. If it is one cell versus the other three for the two sets this is just a 2x2. If you want to compare, say the associations, then combine the data and depending on what you are interested in either a log-linear model or logistic regression.

Maybe if you show the contingency tables this will help my understanding.

Hello Corinne,

Like Daniel Wright , I'm not fully clear on your intentions. The attached figure is my best guess as to the type of data you have (values are fictitious, of course).

If that represents your situation, and you'd like to know whether groups are equally prevalent in some specific cell (relative to their respective total N), you could:

1. Run a logistic regression using group as the IV and whether a given case showed in the target cell (e.g., 1 if "yes," and 0 if "no").

2. Use one group as the reference group, and compute the proportion of cases in the target cell. Then, multiply that proportion by the N of cases in the second group, to get an expected cell frequency for the second group. Compute chi-square as (fo - fe)^2/fe to determine if the second group's value differed from the rate observed for the first/reference group.

Using the attached figure's values, and the target cell of "positive/negative" as the one of interest, I got these results for the two methods:

1. Group is a significant explanatory variable for relative prevalence of "positive/negative" pattern memberships: Chi-square(1 df) = 15.792, p < .001. Specifically, B (regression coeff) = 1.792, so group 2 cases have 6 times higher odds of being in pos/neg cell than do group 1 cases.

2. Observed freq for group 2 in pos/neg cell = 25. Proportion of group 1 cases = 5 / 95 = .052632, so Expected freq ~ 5.2632. Chi ~ 74.0, p < .001.

While both methods suggest that the groups are not equal in their representation for the target cell, I would prefer method 1 (logistic regression).

Good luck with your work; sorry if this answer completely misses your aims.

You can compare sets of 2x2 tables with the Mantel-Haenszel Test. The idea is to evaluate the association between the row variable (e.g treatment/control) and the response variable, while adjusting, or controlling, for the effects of a stratification variable.

David Morse Daniel Wright thanks for your suggestions! In the future I will add the contingency tables. My data looks as David suggested. I will try the suggestion of Jos Feys first.

As might be pointed out above, this seems to be related to checking interaction in a logistic or log-linear model. Mantel Haenszel may/may not be appropriate. To understand, some more explanations needed.

The experimental design is really important here as it determines the best analysis (Chi-Squared, G-Test, McNemar Test, one or two sample Binomial Test, Cochran-Mantel-Haenzel, Breslow-Day, Goodness-of-Fit test, Logistic Regression, etc...) Therefore you must clarify several things including is the data paired? (same sample at both time points or different samples)? Are the sample sizes in each table the same? Is is randomized or observational data? If random, is it random sampling of the two groups (tables) or is it random treatment assignment to each? If observational is it binomial or multinomial sampling? Is one table the gold standard/ reference? Is this two-dimensional or three dimensional data ( Cochran-Mantel-Haelszel and Simpsons Paradox) etc. Please elaborate. My best guess as to what your situation be is each table is [Status (pos/neg)] vs. [Time (interim/final)] in two random samples with unpaired data (ex. not McNemars test). And time is structural difference so it is binomial sampling and not multinomial sampling (i.e. not a chi-squared). If that is the case, within a given time point (in a given table), the status (pos/neg) is binomial distributed (ex. % positive). Therefore, to compare treatments (table A vs. table B) at a single given time point would be a two-sample test for equal proportions (ex. is % pos in table A larger than in table B at a given time). You would have to do two of these tests, one for each time point. You could also test for differnece in Odds Ratios between tables (Mantel-Haenszel/ Breslow-Day test). Also, these tests tend to need large sample sizes. See an online calculator here https://www.medcalc.org/calc/comparison_of_proportions.php

Cochran-Mantel-Haenszel using R:

https://rcompanion.org/rcompanion/b_10.html

using SAS:

https://v8doc.sas.com/sashtml/stat/chap28/sect27.htm

With the data in David Morse 's answer, you could test the association of sample and time 1 emotion on the time 2 emotion in R (I'll use R since it is free and popular and on my machine) using a logistic regression with:

datax

If your outcome variable is categorical then you can use chisquare test of association,If conditions of chisquare not fulfilled use Fisher's exact test.

An important question .. worth follower.