Dear Researchers,
I am doing bivariate Pearson's correlational analysis between 12 variables wherein correlations coefficients are varying from .2 to .7 but all correlations are coming significant at alpha = .001 level.
is it okay or is there some issue that I need to resolve?
If you have large amounts of observational data, possibly collected over a longer period of time, it is expected that all the correlations will have low p-values.
http://www.tylervigen.com/spurious-correlations
Hi @Jochen I collected data within span of 3 months so I dont think that's long period. Next, my problem is not low corrleation but all correlations being significant at alpha = .001 level. Though as per nature of study I expect strong correlation among variables but do not know how come all p-values are less than .001.
Significance of correlation coefficient depends on sample size. With large samples very small correlation coefficient may be significant or vice Versa. With small samples high correlation coefficient may not be significant. In a sample of 11 participant you have to correlation coefficient at least .735 to be significant at .01 level. However with 1000 participants .081 or above significant at .01 level. It may be related to your sample size.
Second posibility may be that ıf you are using strucural equation modeling, this softwares report attenuated corrected correlation coefficient (latent correlations) typically larger than observed correlation (Pearson).
Last reason may be that ıf you have modest sample size, your variables theoritically related to each other.
Within 3 months, summer turns to winter, flowers blossom and fade, entire insect states are built up or go down, sunspeckles emerge and disappear, a human female has about 3 ovulations, ... if 3 months are short depends on what you analyze.
Getting low p-values is always a matter of sample size. How large is yours?
Please let me know if the reference/site below is useful to you:
The American Statistical Association on misuse of P values – a ...
https://www.mq.edu.au/__data/assets/pdf_file/.../ASA-on-misuse-of-p-values-DP.pdfCached
Success rate: about one third to one half observed in the replication study. .... P- values conflate effect size, variability and sample size: you can get low p-values.
Dennis
Dennis Mazur
@Ertuğrul Şahin @Jochen Wilhelm @Dennis Mazur Thanks for comments. My sample size is 343 and I have 12 variables. I am measuring the influence of negative attitude toward women on violent attitude toward women and variables are 4 dimensions of negative attitude toward women scale, total score on these dimensions, 5 dimensions of violent attitude toward women scale, total score on violent attitude toward women scale, and ,egalitarian attitude toward women (another scale by Spence & Helmerich, 1973). All items are Likert scale type.
A highly significant correlation is expected as per theory between these dimensions/variables, but what I want to understand is why all correlations are significant with p
As Jochen and Ertugrul already pointed out: the significance does not only depend on the magnitude of the correlation coefficient, but also on your sample size!! The same coefficient of r=.40 could be either non-significant with a small sample size (maybe N=10 --> p=.25) or highly significant for a larger sample size (e.g. N=100 --> p=0.000037). The coefficient is the same, but the significance changes with sample size.
That is because the t-statistics for the significance test includes the sample size, t=r*sqrt((N-2)/(1-r^2)). As you can see, with growing sample size, t increases (p-value of the specific t-distribution in turn depends on df=N-2).
So, in your case, with quite a large sample size it is not surprising that approx all r> .105 are significant.
But this does not mean that some of your correlations are not nonsense correlations driven by other variables which you already partialed out, but this is with regard to content and not statistics per se. The same logic of p-values also holds for partial correlations, so if the inclusion of a third variable does not decrease the partial correlation below a specific level, it will still remain significant.
In my opinion you should think of r as an effect size and do not care so much about the significance. Is a very small, but significant correlation, e.g. r=.15 really of practical interest for you and your research (question)?? Does it explain variance of interest?
I think the problem here is more that it is not clear why the analysis is performed in the first place. It reads as if something is done or calculated because such analyses are frequently seen in other studies, and because this is something that seems to be "doable" with the data at hand.*
Worth a look: http://meehl.umn.edu/sites/g/files/pua1696/f/113theoreticalrisks.pdf
---
* "seems doable" does not imply "makes sense to do". Likert data is special and allows only very limited analyses that would give reliable and interpretable results. Pearson correlation and linear models are likely no very useful tools.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Techniques
- Structural Equation Modeling