The attached file contains the information you want. The last link has your information in it. R Programs are in the next to last link. I would skim all of this stuff before you compute anything. Please send my Bonus points to:

David Booth

C/O St Peter

Heaven

Thank you

Hi David Eugene Booth

Thank you for the answer. I am adding it here as I am also looking for an answer to the same question. I think what Nicholas S. Holtzman is referring to is a hypothesis test for comparing two correlation coefficients say r1 and r2 from two different or overlapping data. This is explained for Pearson's correlation coefficients using Fisher's transformation on page 13 of the documentation of cocor (https://cran.r-project.org/web/packages/cocor/cocor.pdf) package in R. Can we apply the same procedure for Phi correlations?

Apologies if I missed it in the links you provided.

Many Thanks

Usman R.

Usman Rashid I understand. Look at the last link, The Section on PHI. It is all THERE if you read it. The answer to the question is Trivial.

D. Booth

PS In case you can't find it:

Similar to the t-test/correlation equivalence, the relationship between two dichotomous variables is the same as the difference between two groups when the dependent variable is dichotmous. The appropriate test to compare group differences with a dichotmous outcome is the chi-square statistic. And, we can also show that the test of the phi coefficient is equivalent to the chi-square test.

Dear Nick,

try here: https://www.psychometrica.de/correlation.html

Best wishes,

Serge

@Serge Brand While that program appears to be very good it does not contain Phi as far as I can tell. Have I missed something? Thanks, David Booth

Thanks everyone [Usman Rashid, David Eugene Booth, Serge Brand] for the ideas, references, and discussion. I understand how to compute a single phi correlation and run the Chi-square test and get the p-value for this (single) phi coefficient. Usman Rashid is correct, however, in that I am still struggling with how to compare two phi correlations. I think a major barrier is that, unlike Pearson correlations, phi coefficients have a theoretical maximum that is sometimes lower than one----indeed the max possible value depends on the data obtained. People have used something called phi/phimax, but I have yet to find a way to compare two of these magnitudes. The idea that phi is the same as Pearson (and therefore the Steiger [1980] procedures can be used) appears to be false, but of course I'm open to being convinced otherwise. Thanks for your help in this puzzle.

Nicholas S. Holtzman

One last time, is this not the answer to what you asked? The appropriate test to compare group differences with a dichotmous outcome is the chi-square statistic. If you mean that you want to test a hypothesis about phi1-phi2 or phi1/ph2 all I can suggest is subtraction and division. There is no known sampling distribution for these quantities that I can find... Further even if there was such a distribution(s), what does it mean to compute a mean of Phi1-Phi2 from a sample of such differences.? As far as I can see this makes absolutely no sense. I await your explanation. May the force be with you. D. Booth

Dear Nicholas,

Perhaps this helps: Although the text is in German, statistical indices are universal; briefly, Leonhardt (see attached file) suggests to z-transform the correlation coefficients, and to compare these two z-values by means of the further z-value; the latter z-value can be compared with z-values from tables, i.e., is a z-value below the p-value of .05 or not. BW serge

Serge Brand I haven't read German in over 40 years but The Z transform applies to Pearson correlation coefficients and with a correction factor to Spearman coefficients but as far as I can find to nothing else. See: https://en.wikipedia.org/wiki/Fisher_transformation

and

https://bookdown.org/burak2358/SARP-EN/analysis-of-variance-anova.html

Best, D. Booth

One idea is to compute 95% confidence intervals for each of the phi statistics you want to compare, with the decision rule being that if the two confidence intervals don't overlap, you can conclude that the statistics are significantly different at that level.... At least when compared to comparing means with a t-test, this method is typically conservative. That is, it's possible that two means are different by t-test, but their CI's overlap.... Below is some R code for the approach. To keep it simple, the data for the tables start as two binomial vectors. Some inspiration is taken from the last section of http://rcompanion.org/handbook/G_10.html , of which I'm the author. In the example below, because the sample size is so small, the boot function is likely to throw an error. If so, it can be re-run. EDIT: the rcompanion package in R now has a function to calculate phi and confidence interval. Caveat: I am the author of this package.

if(!require(psych)){install.packages("psych")}

if(!require(boot)){install.packages("boot")}

Type1 = c(0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1)

Success1 = c(0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1)

Table1 = xtabs(~ Success1 + Type1)

Table1

Type2 = c(0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1)

Success2 = c(0,0,0,0,0,1,0,1,1,1,1,1,0,1,0,1)

Table2 = xtabs(~ Success2 + Type2)

Table2

library(psych)

phi1 = phi(Table1, digits=3)

phi1

phi2 = phi(Table2, digits=3)

phi2

Data1 = data.frame(Type1, Success1)

Data2 = data.frame(Type2, Success2)

library(psych)

library(boot)

Function = function(input, index){

Input = input[index,]

Tab = xtabs(~ Input$Success + Input$Type)

Stat = phi(Tab)

return(Stat)}

### Test Function

n = length(Data1[,1])

Function(Data1, 1:n)

### [1] 0

### Bootstrap

Boot1 = boot(Data1,

Function,

R=5000)

boot.ci(Boot1,

conf = 0.95,

type = "perc")

hist(Boot1$t[,1], col="firebrick3")

Boot2 = boot(Data2,

Function,

R=5000)

boot.ci(Boot2,

conf = 0.95,

type = "perc")

hist(Boot2$t[,1], col="dodgerblue3")

Sal Mangiafico Thank you for the code. I got the following error:

> Boot2 = boot(Data2, + Function, + R=5000)

Error in phi(Tab) : prod(dim(t)) == 4 || length(t) == 4 is not TRUE

Can you please tell me how to resolve it?

Prajwal Mani Pradhan , I'm not sure. The code as written should work. It may help if you put all of the Boot2 definition on one line: Boot2 = boot(Data2, Function, R=5000)