Hi Team,
I am creating my first Logistic regression on R Studio. I am working on a C-SAT data where rating (score) 0-8 is a dis-sat whereas 9-10 are SAT. As these were in numeric form so i had as below created 2 classes:
new$survey[new$score>=0 & new$score
I have not read the replies above, but I can say that the error message is due to separation. To solve it, Firth's method helps. There are many packages in R I guess where Firth's method is implemented including logistf package.
It is always safer to code factors as such but it won't change anything in this case (because you have defined 2 categories coded as 0 an 1). Moreover it's a common practice to code the response as 0/1, and even better to use probabilities or success/failure (as a matrix for the latter) when possible.
In short, I think you are confronted to an over-fitting problem: too many variables in your model, leading to a perfect separation of cases. The consequences is that model likelihood is not defined, and thus you can't get the model to converge.
I would recommend to do a preliminary exploratory data analysis to exclude correlated variables.Moreover, in this case, I would progressively build the model, starting with a very simple model and adding variables one by one.
Thanks Renaud. i have completed the exploratory analysis and yes there are few correlated variables.
Can you please whether i should drop the lesser significant from those or stepwise regression help here.
I would not drop variables according to p values in bivariate analyses: the reason why you want to do a multivariate analysis is to mutually adjust the coefs and p-values of the different dependent variables. Regarding the stepwise procedure, you may want to use one based on AIC. However, even this can be misleading. I prefer to use the selection method described in
Burnham, K. P. & Anderson, D. R. Model selection and multimodel inference: a practical information-theoretic approach Springer-Verlag, 2002, 496 p.
Thanks Renaud, the book you suggested is way too costly.
Another question can you please assist in performing a correlation for independent variables as in my case i have 21. Performing a multicolllinearity is not feasible. Moreover on the linear regression side i am very aware that we can use the VIF(Variation Inflation Factor) to remove the highly correlated variables and simply looking at their impact on the model output we can remove those.
But in case of logistic, i am not sure what is the statistical method and specially in R.
Please suggest.
The book is really a valuable investment, quite illuminating on many aspects. In addition, you will very easily find many application papers by the same authors, as well as course notes and other materials.
Regarding multicollinearity, it is quite easy to assess bivariate correlations, eg with function pairs() in R (look at the examples in the help page). You can also use multivariate techniques like principal component analysis, or multiple correspondence analysis (see package ade4) - followed by cluster analysis on the factor scores, to assess and map multivariate correlations.
At the end of this process, you should be able to draw a list of biologically plausible models, fit them and compare the fitted models with AIC or related indicators, as proposed by Burnham and Anderson.
I have not read the replies above, but I can say that the error message is due to separation. To solve it, Firth's method helps. There are many packages in R I guess where Firth's method is implemented including logistf package.
Hi, I found the book mentioned above in the www
http://ecologia.ib.usp.br/bie5782/lib/exe/fetch.php?media=bie5782:pdfs:burnham_anderson2002.pdf
I had the same problem and my proffesor told me that it is because of the excess of variables but now I am not sure
If anyone is still interested in this question, see the attached notes. I think they will be helpful. Best, David Booth
- Badges
- Science topic