Hi Tomasz,

When I encountered separation problems I found Tree based models to be very helpful. The packages rpart and party can handle such models nicely.

Here is the rpart vignette: https://cran.r-project.org/web/packages/rpart/vignettes/longintro.pdf

and for party: https://cran.r-project.org/web/packages/party/vignettes/party.pdf

One big advantage of tree based models is they do feature selection automatically and are immune to highly correlated predictors. Second advantage is they provide a sense of variable interactions.

For FLR I would recommend checking out the brglm package: https://cran.r-project.org/web/packages/brglm/brglm.pdf. I haven't used it but it seems similar to the glm function from MASS.

For variable selection in logistic regression I prefer the lasso vs AIC and BIC. While AIC and BIC put a penalty on the number of variables, the lasso penalizes the coefficients associated with the variables. Lasso is also quite immune to correlated variables.

More about the lasso: 

https://www.youtube.com/watch?v=A5I1G1MfUmA

https://www.youtube.com/watch?v=xMKVUstjXBE

https://www.youtube.com/watch?v=1REe3qSotx8

great free book: http://www-bcf.usc.edu/~gareth/ISL/

I can also recommend: Applied Predictive Modeling 2013th Edition

by Max Kuhn  and Kjell Johnson  especially chapters 12.1, 12.2 and 12.7

as well as: https://topepo.github.io/caret/

The main point of these chapters is that logistic regression performs much better when high-correlation filter is used on the predictor set to remove  highly redundant predictors from the data. Additionally very sparse and unbalanced predictors should also be removed.

Ways to do it in R:

https://topepo.github.io/caret/pre-processing.html#zero--and-near-zero-variance-predictors

https://topepo.github.io/caret/pre-processing.html#identifying-correlated-predictors

https://topepo.github.io/caret/pre-processing.html#linear-dependencies

good luck,

Milan

Thank you Milan,

I've already run 'rpart' for my dataset of 177 municipalities in Central Poland investigating the impact of municipal investment expenditures per capita in electoral cycle 2003-2006 on becoming re-elected as a mayor. The equation of the first estimated model was RELE2006 (1 when re-elected, 0 when not, in mayor elections in 2006) ~ INV2003 (investment municipa expenditures in the first year of electoral cycle) + INV2004 (2nd) + INV2005 (3rd) + INV2006 (investment expenditures in the year of elections) + LOCAL2006 (1 when candidate represented local committee, 0 when political party). You can see the plot with the results. Investment municipal expenditures in the year before elections were diagnosed as the most important factor determining probability of re-election. This is in line with results of previous research. Now I can go further with more detailed data and other electoral cycles. So, thank's again Milan for your recommendations.

Best,

Tomasz

Tomasz,

I have several additional sugestions

First of all prior to running rpart you should properly code your variables,  your Y variable RELE2006  should be as.factor. Also LOCAL2006 should be as.factor also.

After rerunning the model I suggest you plot a nicer looking tree

install.packages("rpart.plot")

rpart.plot::prp(model, type=2, extra=106, nn=F, fallen.leaves=TRUE, yesno=1,

faclen=0, varlen=0, branch.lty=5, cex=0.7)

Second the most important variable is not the variable in the first split, you can get the variable importance by:

model$variable.importance

Third you should check for the appropriate amount of pruning by:

printcp(model)

plotcp(model)

You can specify the amount of pruning by rerunning the model with the following addition

control=rpart.control(cp="some number as suggested by plotcp(model)")

As for your original question on FLR and AIC/BIC, the FLR model obtained by the brglm package contains the AIC: model$aic is how you can get to it. The stats functions BIC and AIC should also work: AIC(model), BIC(model) so you can do comparisons on where to stop adding/removing variables.

Here's a starting point: http://pages.pomona.edu/~jsh04747/courses/math158/multlin4.pdf

good luck,

Milan