Hello,
I have a dataset of over 1.2m people, 40 potential predictive variables and a binary target variable to indicate if a person claimed or not. There are 2,500 people who claimed. When I tried to run a simple Logistic regression against variables that are clearly indicative of outcome, it came back with nothing meaningful.
I then checked for correlation and the outcome says that no independent variable is correlated with the binary dependent variable (|r|
Hello Gunnar,
While a correlation of |.02| is easily statistically different from zero with a total N as large as what you have (actually, you could go as low as |.002|), the real question is, is such a (small) correlation noteworthy from a practical, clinical, or policy-making perspective? Your query makes it sound as if the answer is "No."
Unfortunately, if none of the variables you have, either individually or collectively, can account for more information about correct classification of cases, then the issue isn't that of sample size as you describe it. It's rather that, in all likelihood, you haven't yet identified the variables which could/would be useful in such classification.
Good luck with your work.
As usual prof.morse is quite correct in what he says. There are a few other minor things that could be happening as well. However I will assume you wish to predict values of your DV.
.under these conditions you may find the attached of some interest In particular you can select an optimal subset of predictors that yield an excellent optimal prediction function that doesn't contain collinear predictors such as an oracle property.should you wish to try this approach I can easily supply R code and the dataset we used in the paper. It is currently coded in R which is perhaps the best way to go at present.
Continuation,:
BTW files and data are available if you wish to see our implementation. Best wishes, David Booth
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size