The issue with quasi-complete separation is that the dispersion matrix is unbounded meaning that estimates for parameters that are affected by the quasi-separation will not have a maximum likelihood solution and their magnitude will grow indefinitely towards positive or negative infinity until the estimation algorithm stops iterating (which is why your OR estimates are so strange).

There is a correction for separation problems by Heinze and Schemper (2002, Statistics in Medicine) that is available in SAS via Proc Logistic although I don't know if there is an analog in R. Alternatively, using a Bayesian framework with informative priors might help as well. As a last option, you could as remove the predictor as you originally suggested, but this is often undesirable because the chance that the model is misspecified will be much greater. 

I just would like to say that

> round(8.739675e-01,3)

[1] 0.874

is not a strange value for an OR.

Thank you all for your hints! I ran a Firth penalized logistic regression which allowed me not to obtain the warning of quasi-complete separation anymore. Furthermore, I found Boris's suggestion very useful as I finally managed to get the rounded OR values.

Now I'd like to ask you one more thing. In fact, one independent variable (say, IV1), which was the one responsible for quasi-complete separation, makes another independent variable (IV2) to change the sign of its coefficient. In other words, when IV2 is taken alone, it has a positive relationship with outcome; when IV1 is added, then IV2 acquires a negative relationship with outcome. Their correlation coefficient is -0.5, thus multicollinearity should not be an issue. Any suggestions? How can I deal with this, do I have to exclude IV2 (or IV1) from the model?

Again, many thanks!!

Hi Leonardo,

you can check variance inflation factors (VIF) to infer if multicollinearity causes the problem.

Do you have an interaction term in the model? If not, add it in and see how it impacts both the overall model fit and the marginal effects for IV1 and IV2. In any case, Boris is correct that the most likely culprit is probably multicollinearity, which can be tested for using VIF, but it is still worth looking directly at the interaction term.

In addition, I'm not sure you are correctly interpreting the correlation coefficient. Simply because the coefficient is negative does not imply that multicollinearity is not an issue. In fact, a correlation of -1 introduces the same exact multicollinearity problems as a positive correlation of 1. The sign of the correlation coefficient only indicates the relative direction of the correlation effect, not the magnitude of the effect; that is, a correlation close to 0 would imply that there is no multicollinearity, but a large negative correlation is just as "bad" as a large positive correlation.

If you DO have a multicollinearity problem, you don't necessarily need to remove one of the predictors from the model. You simply need to explore the application of some slightly more advanced techniques. For example, penalized regression (especially ridge regression) is often used in this situation. Another possibility is to use an orthogonalization or residualization process.

Thank you Ryan for your hints! About correlation coefficient, I intended that -0.5 was not really close to -1, so I thought that correlation was not an issue. Following your suggestion, I analysed VIFs among independent variables and they were all small and < 10 (I read that VIF > 10 should pose the problem of multicollinearity, and this was not my case). 

Anyway, I will try ridge regression as you suggested. Can you please explain to me orthogonalization or residualization?

Thank you again!!

Correlation magnitudes can be difficult to interpret. It is hard to judge based only on the -0.5 whether or not it will be a problem. It depends a lot on the context. I didn't mean to imply that 0.5 was necessarily a strong correlation, but only that it isn't vanishingly small, so it can't be discounted completely. It is helpful to think about correlation and the issue of multicollinearity from a geometric perspective. Remember that the purpose of regression is to project a vector of outcomes (Y) into the column space of a predictor (X). See the attached file "regress1.jpg". That is, the various X's (in this simple case, let's just look at X1 and X2) exist on a plane, onto which we are projecting the vector Y; this projection (Y') are the model predictions, and the errors (e) is the difference between Y and Y'. In fact, the angle between the vectors Y and Y' is actually equivalent to the R squared of the model!

The correlation between X1 and X2 from a geometric perspective determines the angle between the two vectors on the plane. So you will see in regress1.jpg that X1 and X2 are almost orthogonal (i.e. independent) because the angle between them is almost 90 degrees. A 90 degree angle would correspond to a correlation of 0; a positive correlation would decrease that angle, while a negative correlation would increase that angle. Now, see "regress2.jpg" for an illustration of the same regression, but now with X1 and X2 as perfectly collinear (i.e. correlation~1). You see that in a perfectly collinear case, X1 and X2 provide the same information; that is, the plane of X is UNDEFINED. So there is nothing to project Y onto, your Y' will just be numerically equivalent to your predictors! The regression will fail. Note that a correlation of -1 poses the same problem, the only difference being the orientation of X2 with respect to X1, but the plane is similarly undefined.

Now, with "regress3.jpg", you see the more realistic situation where the predictors are only partially collinear (that is, correlation~0.8 or so). This is a lot more difficult to interpret. Since an angle>0 exists between X1 and X2, the plane of X is defined, so it is possible to project Y onto it. However, these results will be numerically unstable. It is difficult to tell from the picture, so let me try to explain. The entire point of a regression is to generate predictions from a sample to generalize to a wider population. If you pull multiple samples from that population, you will always get slightly different results. So, imagine you resample from X1 and X2, our nearly collinear predictors. See the grey line in the figure? That represents a "new" X1 sample; due to expected random variation, it is very slightly offset from the "original" X1. Even though the difference is so small as to be almost unnoticeable, it causes a huge "swing" in the plane X. That is, a COMPLETELY different plane is generated by only very slightly different data, so you will get a completely different set of predictions Y'. This isn't a problem with X1 and X2 are less correlated, because their vectors are further apart in the column space. Imagine balancing a pencil on your two index fingers; if your fingers are very close together, only a slight movement of one finger may cause the pencil to fall, but if your fingers are further apart, the pencil is more stable, and will stay in place with greater degrees of movement in your fingers.

All of this to say that a correlation of -0.5 might not be a problem, but it is worth thinking about the data geometrically.

As for VIF, you are correct that VIF less than 10 are traditionally considered to be "safe" in terms of not posing a multicollinearity problem. Most likely, you don't have an issue if the VIFs are all small.

As for orthogonalization, look up a process called "Gram-Schmidt orthogonalization". Most statistical software should be able to do it automatically. Basically, you have to realize that a regression with multiple predictors is equivalent to a sequential regression of each individual predictor. Orthogonalization is a method of enforcing independence at each step of this sequential process; it is essentially a transformation that forces the vectors to be orthogonal, generating a stable column space. Of course, the problem with this (like any transformation) is that the results of the regression will only be meaningful on the scale of that transformation, and may be difficult to interpret in terms of the original variable values.

(Images courtesy of @whuber at stats.stackexchange.com)

Wow, you were able to explain very difficult concepts in a simple and straight way, that's great! It is no doubt the best explanation of multicollinearity I got up to now.

I will try with orthogonalization and see what happens. 

Many, many thanks!