To gauge the significance on a single correlation coefficient you can Fisher transform the estimated mean correlation and its standard error, and then use the cumulative distribution function for the Normal to estimate either the two-tailed or one-tailed P-value. The reason for the transformation is that the correlation is not bounded by minus, plus infinity and so the variance is not homoscedastic. (See http://en.wikipedia.org/wiki/Fisher_transformation)

To account for multiple comparisons (four tests in your case), assuming the tests are independent and are from similar families of questions (e.g., coin flips) then Bonferroni correction should suffice (0.05/4). Bonferroni is conservative and a more relaxed condition may be appropriate, for example if the hypotheses are correlated. Of course, there are many other solutions to this problem including Bayesian. As you may know, solving the MCP problem is an active area of research.

Do you mean compare several correlation coefficients with each other? The answer may depend on what you mean and even more what purpose is that you are wanting to test these hypotheses. So, more detail might help get more responses.

In general, the p.adjust function (http://127.0.0.1:16648/library/stats/html/p.adjust.html) function is often useful. If you have say four p values, say .01,.02,.05,.10, you can do (in R)

p.adjust(c(.01,.02,.05,.10))

and it adjusts those p values for different adjustment methods (Holm's is the default). But as said above, more details will help people to know what answers might be appropriate.

I do not know the tests you mentioned, but if you want to compare the coefficients between them, considering that the tests are independent, and by definition the values of the correlation coefficient ranges from 0-1 , you can try with nonparametric statistic such as Kruskal Wallis test because it not required normality and homogeneity of variance, comparing with the value of the chi-square distribution, this will reveal whether they are statistically identical or different.

Thanks you all for the help.

what I have is four independent correlations, and I need to test if they are statistically equal? The situation here exactly like T-test and ANOVA. But instead of comparing means, I want to compare correlations.

Hi Mahmoud

I think this is what you are looking for, an online calculator that will do exactly what you want - - http://vassarstats.net/rdiff.html

Simon

i recommend this paper:

http://psycnet.apa.org/journals/met/12/4/399/

though you might have to contact the authors for a copy of it.

Hi Simon:

Thanks, but my questions was what if I have 4 correlation from for different samples and I want compare them?

Thanks kate:

I will contact the author?

I think you should do a multivariate analysis, like EFA, CFA, SEM... Try to find out something about "omega" function in 'psych' R package, written by Willian Ravelle.

Could'nt your problem be rewrote in an "analysis of covariance" way, with additional variable the sample identifier (1 to 4, categorical of course). Since correlation coefficient are related to slopes, it should be equivalent to a test of interaction. And if you need pairwise comparisons, use subsequent suited contrasts.

Of course, it assumes the variables X and Y are the same for all 4 samples, and that you have a preferential Y = f(X) direction. But since you did not described exactly your experiment/context, one can not say very much and have to guess...

What I am trying to explore if correlation coefficient between students’ academic achievement (GPA) and students engagement in school activities (questionnaire) differ according to family income (4 levels).

So, I have four correlation coefficients between students’ academic achievement (GPA) and students engagement in school activities. One for each level of income. R1, R2, R3, R4, and I want test if they are statistically equal?

Definitely, analysis of covariance with test for equality of slopes should answer your problem.

(Note I am assuming you want to stick with comparing correlations. If not, then other solutions such as ANCOVA, mixed effects modeling, etc. may be useful on the original data as Emmanuel suggests if they don't violate the assumptions for these models.) 

Comparing correlations is a common problem. There is a lot of literature on this. Please note that the major issue with comparing correlations using standard tests such as t-test, ANOVA, ANCOVA, etc. is that correlations naturally violate the Normality assumption for these models/tests due to the -1,1 bound of the correlation measure. A common solution to this is to transform the correlations. This was first proposed by Fisher in 1921, who developed ANOVA.

If you look online, you will find that this violation of the Normal assumption is what most approaches deal with. Second to this they deal with dependence between correlations. A common approach to compare independent correlations is to transform them using the Fisher transformation and use the degrees of freedom (n-3; due to loss of degrees from estimating the correlations) to estimate the standard errors for the t-test (they assume the respective variances for the correlations are one). I believe the calculator that Simon Hunter pointed out earlier on this thread does this.

There is also recent research on this problem (comparing correlations) that use different approaches from Fisher's. One of these papers was recommended by Kate Levin earlier on this thread. I have reposted links and others that I found through a quick search on the net.

Here are some links that may be useful:

from Simon Hunter:

http://vassarstats.net/rdiff.html

similar to Simon's link but gives the formulation:

http://www.danielsoper.com/statcalc3/calc.aspx?id=104

calculators that deal with dependence and common sampling:

http://quantpsy.org/corrtest/corrtest2.htm

http://www.psychmike.com/dependent_correlations.php

more about these methods (talk about independent and dependent cases):

http://core.ecu.edu/psyc/wuenschk/docs30/CompareCorrCoeff.pdf

http://ww.w.statpower.net/Steiger%20Biblio/Steiger80.pdf

http://stats.stackexchange.com/questions/114001/how-to-test-for-a-significant-difference-between-several-dependent-correlation-c

some alternatives to Fisher's approach:

recommended by Kate Levin:

http://www.ncbi.nlm.nih.gov/pubmed/18179351

http://psycnet.apa.org/journals/met/12/4/399/

(if this is not free then you may be able to get a copy from the author: http://www.uwo.ca/epidem/people/Faculty/BaseComplement/zou.html)

http://dornsife.usc.edu/assets/sites/239/docs/pcorcom.pdf

Hello. The design is: comparison of several independent correlations (> 2 correlations). There ready such technique

Comparison of independend (and dependend) correlation coefficients): Several calculators are available in the internet - free of charge. To control for the alpha-error inflation, just use the Bonferroni adjustment or the less conservative sequentially rejective Bonferroni test by Holm.

Hello. the formula of V statistic in the attachment...

Thanks Mr Ghassan

You are welcome... [df=k-1]

I think the formula added by Mr Ghassan from:

Hays, W. (1981). Statistics, New York Holt-Rinchor and Winston.

Would you provide the link for the original literature of the formula please?

You can find the formula in this book:

Hays, W. (1981). Statistics, New York Holt-Rinchor and Winston.

Mahmoud Alquraan Hello, thank you for your reply but I can't find the publisher you mentioned. The one I found is

Statistics

Hays, William Lee

Published by Holt, Rinehart and Winston (1981)

ISBN 10: 0030567068ISBN 13: 9780030567063

Is this one correct?

I might be wrong, please check page#467 in the book. I will double check that for you.

As shown in the image:

In the 1981 version this is on p. 467.

This will be far too late to help Mahmoud Alquraan with the original problem, but Karl Wuensch and I discussed this situation in our 2013 article. One approach is to use the standard Q-test for heterogeneity that meta-analysts use. We provided code for both SPSS and SAS. Use the appropriate link on this page, and look for syntax file #5.

https://core.ecu.edu/wuenschk/W&W/W&W.htm

If you use some other software, you can probably translate the code fairly easily.

Here is the article.

Article Erratum to: SPSS and SAS programs for comparing Pearson corr...

HTH.

Bruce, thanks for posting that. Nice paper. Using Hays' formula, implemented in R as:

rtoz

Daniel Wright It always amuses me that everyone writes their own r to z function rather than using the in built hyperbolic tan functions (and yes I've done it myself):

atanh(r) and its inverse tanh(r)

In practice it won't matter much but it could save time and be more efficient if you are doing simulations etc. Its also useful to remind people that the connection between r and Fisher z is a hyperbolic slope. So r is the Euclidean slope of y =rx and fitting a hyperbola to the graph rather than a straight line gives a hyperbolic increase per unit increase in standardised y that is Fisher z.

Bond, C.F., Richardson, K. Seeing the FisherZ-transformation. Psychometrika 69, 291–303 (2004). https://doi.org/10.1007/BF02295945

Good point Thom Baguley ! Particularly for missing the teaching opportunity!

Here is the time saved:

> #with microbenchmark from github installed

> library(microbenchmark)

> r2z microbenchmark(atanh,r2z,times=10000)

Unit: nanoseconds

expr min lq mean median uq max neval cld

atanh 100 100 145.24 100 200 17700 10000 a

r2z 100 200 193.40 200 200 20800 10000 b

> x max(abs(atanh(x)-r2z(x)))

[1] 4.440892e-16

>

Here are some comment lines from one of my old files that may be of interest to folks who use SPSS. ;-)

* SPSS has no tanh and arctanh functions.

* HOWEVER, one can use IDF.LOGISTIC and CDF.LOGISTIC as follows:

* arctanh[r] = .5*ln((1+r)/(1-r)) = .5*idf.logistic((1+r)/2,0,1)

* tanh[d] = (exp(2*d)-1)/(exp(2*d)+1) = 2*cdf.logistic(2*d,0,1)-1

I learned this from my co-author Ray Koopman.

See also: https://www.researchgate.net/post/I_want_to_compare_correlation_coefficients_of_unequal_three_groups_how_can_I_do_it/