1-It is impossible (or very hard) to obtain a significant p value and OR including 1.

2-Yes, use logistic regression.

Mr. Mehmet, thanks a lot. Yes, I thought the same, but in one dissertation, OR was 2.56 (95%CI 0.89-7.34) with p

confidence intervals and p-values are closely related which is why it is rare to have a significant p value and OR crossing 1, but in a very very small number of cases it can happen. I would recommend going with the pessimistic answer. If you expected a difference, that is the OR include 1 and so the test has failed. If you expected no difference that is that the p value is significant. Either way the data cannot be used to confirm any hypothesis and the result is inconclusive.

These decisions (use p value or CI as pass/fail criterion) should be made before even collecting data, not examined after analysis, otherwise we run the risk of biasing the outcome by choosing the one we want. I assume the p value is barely below 0.05?

as for whether to use logistic regression or Chi square, it depends on what you want. I always go with the simpler approach that meets my needs. If all you are interested in is a test of association then chi square will do, but if you need to create a model for predictions then regression makes sense. a good discussion at http://www.theanalysisfactor.com/chi-square-test-vs-logistic-regression-is-a-fancier-test-better/

Aleksandar it might even be a typo.

Dr Beattie, I am pessimistic by nature, but in this border situation, should I be a bit optimistic to use this variable to create multivariable logistic model? Somehow, I have more confidence in OR. Ok, I will use logistic model to identify variables for multivariable model. It seems thet ther is no reason for both.

1) It is possible when the p value and the test are derived from different procedures. For example a Wald test is generally more liberal than the Likelihood ratio test.

2) In the 2 x 2 case the logistic regression and liklihood ratio ratio from 2x2 table are I think identical (but can differ from the Pearson chi square for the 2x2 table, but usually not by much)

Dear Aleksandar P. Medarevic,

I think there is one think you did not mentioned in your question. Are you planning to investigate a hypothesis for your data or you want to do a prediction?

in case one you need to use Chi-square test to find the association for the hypothesis. you can not have any conclusion about logistic p-value. By the way if you are talking about the P-value of Wald test I should mentioned that the test is showing some thing else. in case the p-value for Wald test is significant it means that if you eliminate that specific predictor from your model it would significantly effect the Nagelkerke's R-square for your model. to simplify my sentence it does not show any association at all. 

Maybe this papers can be useful

Regarding first question, it is impossible to have significant P value along with 95% CI value containing 1. It may be due to error in input while giving the command; or assumptions were not met while carrying out logistic regression. For example: (i) selection of independent variables is crucial and one has to assess the fit of the model. Independent variables with P value less than 0.25 in simple regression model are included in the final model. (ii) There should be large sample size, (iii) Multicollinearity and outliers must be absent in the data set. 

Regarding 2nd question, logistic is always superior than chi-square test, because one can adjust the effect of other confounders. 

For the first question, it is likely that you have a very limited sample.  A co-variate pattern is a subset of the data all of whose independent variables are the same.  When this is small or, only 1 or 2, or EMPTY then you can get  fallacious results.  Examine how well the logistic model fits.  Incidentally it is called the logistic regression model no the log regression model.  Log reg is a different model.  

    For the second question, if the expected value in each cell is at least 5 then you are assured that the chi square test is appropriate and it is easier to defend the results.  If there are small or empty cells then that is a problem of fit for the two by two table, which is the same as logistic model.  Good Luck,  LL    

Let's be clear : the confidence interval and the Chi-squared test are calculated differently. Here is a marginal example :

Getting a significant p-value makes no sense if the CI covers 1.

Getting a significant P value depends on the test. The likelihood ratio and Pearson Chi-squareds will give you different P-values, and can be significant when the OR crosses one. The OR crossing 1 is testing a different hypothesis. 

Thanks for the clarification Ronán. I assumed that both p-value and CI were for the odds ratio.

If that had been the case then my statement that if the OR crosses 1, then the p-value for the OR would be not be statistically significant. 

If that was the question, then a Fisher's exact test for the OR p-value would be better than the chi-sq test which is an approximation.

@Ramtin Nazerian,

prediction, mainly.

@Aleksandar P. Medarevic,

then the regression is your tool. But pay attention to the Negelkerke’s R-square of your model. Don't make conclusion unless you get more than 0.5. 

Wish you the best on your research :)

The chi-sq distribution is perfectly useful for modeling when the data is chi-square distributed. This would be the case if for the sum of  series of random variables that are the square of normally distributed random variables. And these may be tested using likelihood methods as well as a number of the optimality tests.

In the limit a chi-square distribution is also a very nice approximation for test random variable fit contingency tables: log-linear models (the response and explanatory  variables are categorical) or logistic regression models (the response is categorical but the explanatory variables are continuous). In this case the p-value is an approximation - often very, very good, But the likelihood ratio tests or exact tests are more desirable with all sorts of nice statistical properties. 

Nagelkerke’s R-square is one of many tests for some "chi-sq tests" and is used for prediction studies. But prediction is by no means the sole goal of such test.

It is not possible to obtain a significant p value and having confidence interval of Odds ratio including 1.

Check your claculation once again.

Chi square just check the association between the variables.

Go for logistic regression.

Thanks

Greetings Alek, 

I have a couple thoughts that may be helpful.

Q1) How to Interpret output in situation when p is statistically significant, but 95% CI OR includes 1 (0,97-...)? Article ''Confidence Interval or P-Value?'' did not give me an answer about it.

A1) Depending upon how you calculated what you perceive to be the odds ratio, you may have calculated or examined the coefficient. The coefficient(s) for your predictor term(s) is/are the unexponentiated odds ratio.  In some programs, like Stata, there are two possible commands. In stata, the command "logit" provides the true coefficients. You could in theory have coefficients sometimes called "Betas" or Greek letter B that range from 0.97 - 1.4 for example and see a p-value less than 0.05.  The reason for this situation would be that you are looking at the actual coefficients from the logistic regression. The odds ratio is determined by calculating the exponentiated coefficients.  In the statistics program Stata, the command to have this done automatically is "logistic" rather than "logit".  

So... if your beta values ranged from 0.97 - 1.4, then you would use the mathematical expression e^(0.97) to figure out the lower bounds for the odds ratio which would be 2.638. To get the upper range, you would do as follows: e^(1.4) which is 4.055. Therefore the 95% CI for the coefficients could be 0.97 - 1.4; however, the 95% CI of the OR values would be (2.64 - 4.1). The p-value and the Wald statistics are unchanged. It is still the same logistic regression (also called logit) model. The only difference is the software can exponentiate or not depending upon the command used in the program.

If the software clearly states "Odds Ratio" or "Odd Ratio", then ignore everything I just indicated; however, if it says "Coeff" or "Coefficient" or "B" or "Beta", then this was possibly your problem.

I would be curious to know your actual 95% CI and the coefficent along with the p-value provided.

As stated by others here already, it is not mathematically possible to have an OR 95% C.I. that includes 1.0 and with p-value less than 0.05. 

If you had a 95% CI of 0.97 - 1.00 for an ODDS RATIO, then the p-value would be approx p = 0.05, which is not less than 0.05 and usually deemed not significant by statisticians using alpha = 0.05.

Q2) For binary independent value, is there any reason why I should use 2x2 square test? Logistic regression gives me enough evidence whether association exists and to measure it, especially if I plant to create model?Add your answer

A2) Logistic regression is by far more commonly used for modeling binary outcome variables, especially when you have multiple predictors or factors to adjust for as potentially confounding variables. *My perspectives are from environmental health, epidemiology, and biostatistics. Statisticians or scientists in other disciplines may have other points of view.

In the field of economics and political science, linear binomial regression is widely used in favor of logistic regression. In some cases, OR may not be the best way to present your association.

Linear binomial regression is also recommended if you are testing additive interaction. Logistic regression is not.

Linear binomial regression is easy to apply in any statistical software - simply change the distribution and link function. 

By linear binomial regression, what do you mean? How is that different from logistic regression (binomial response, continuous explanatory) or log-linear models (categorical response and explanatory variables).

The interpretation will be different for all three analytic approaches. For instance, odds ratio will be used for logistic (binomial) regression; relative risk will be used for log-binomial regression; and risk difference will be used for linear binomial regression. I personally prefer to use 'log-binomial' or 'linear-binomial' than 'logistic-binomial' regression. Of course, the choice will depend on one's research question. Reference: 'Regression Modeling Strategies' by Frank E. Harrell, Jr

There not so much information about the setting. It would be important to know why the tests were conducted (e.g. to answer what kind of research question, and what was the design) and what was the actual data and was missing data involved somehow. Generally randomized trials follow a strict protocol (and you would not need to ask these questions, because the answers are preset by the design), while observational studies proceed in a more exploratory fashion.

1) I assume (and from the data you supplied) that you refer to the p-value of the regression coefficient calculated from logistic regression by the usual method utilizing the central limit theorem and symptotic normality of the estimator (if likelihood ratio test is used then this is the same p-value for the OR, too). You did not say which method of calculation the author of the dissertation used for the confidence interval, but if it is this default method used in most software, then the following enables you to check the result.

OR is a transformation that is computed as OR = exp(b), where b is the regression coefficient from the logistic model on the binary outcome. In a similar manner statistical programmes approximate the confidence interval by computing the lower limit as: exp(b - z*se(b)), where z is the quantile from standard normal distribution for a preset level of significance (usually researchers use 0.05) and se(b) is the standard error of b obtained from the model. Now, the p-value from the test is obtained from the test statistic computed on the ratio of the regression coefficient to its standard error, as w = b/se(b) which is assumed to be normally distributed (or if you square it, chi-squared distributed with 1 degree of freedom). Now, you can see that you are able to trace the calculations back to the original b and se(b) and compute the significane of the test statistic w. For example in R you can compute the ratios b/selo and b/sehi:

b = log(2.56); lo = log(0.89); hi = log(7.34)

z = qnorm(1-0.05/2,0,1)

selo = (b-lo)/z; sehi = (hi-b)/z

b/selo

b/sehi

After computation you find that b/selo gives 1.743785 and b/sehi gives 1.749098 (note that we need to compute these two approximations because of the rounded numbers used in report). Now, by normal approximation you would call the result statistically significant at the 0.05 level of significance, if the test statisic exceeded 1.959964, which is not very close in either case. So based on this I suspect something went wrong in the calculations performed in the dissertation OR that the calculations were performed some other way (you could dig the paper's method section to find more information about this or perhaps contact the author). For example, if the b/selo and b/sehi are very different from each other, it might be the case that the CI was computed e.g. by bootstrap, and hence the CI is not necessarily symmetric. In this case it is not possible to check the result, because the results of the bootstrap draws are not usually reported.

Now, you see from equations that the same standard error estimate is used in calculating the test statistic that is used in calculating the confidence interval. If the same quantile is used as the critical value in the test statistic and the multiplier for the se(b) in CI, it is impossible to get the p-value and CI to disagree (unless calculation is conducted on rounded values, of course). You originally mentioned a CI extending from 0.97 to some value, perhaps you can trace the steps similarly in that case. If you need further assistance, it would be great if you let us see the cross-table of the situation that caused the original problem. I would be very interested to see a table where these two actually do differ.

For example, I pasted below R-code for simulated data, where you can see that p-value is close to 0.050 level (p-value = 0.0539), while the 95 % CI includes one (0.97, 30.60). Could it be that the p-value you quoted from the dissertation was rounded downwards? Anyway, p-value close to the 0.05 limit (if that is the set alpha level) are not very strong evidence, but they may merit a second look in future studies (on this issue see more in: https://www.nature.com/news/scientific-method-statistical-errors-1.14700).

> f d summary(fit |z|) 

(Intercept)  0.09531    0.43693   0.218   0.8273 

d[, 2]       1.69645    0.87991   1.928   0.0539 .

---

Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 45.004  on 34  degrees of freedom

Residual deviance: 40.548  on 33  degrees of freedom

AIC: 44.548

Number of Fisher Scoring iterations: 4

> b se c(exp(b[2]-q*se), exp(b[2]+q*se))

   d[, 2]    d[, 2]

 0.972249 30.601283

2) This of course depends entirely on your research question and the study design. If you have a clear predictor-outcome relationship, then regression model may be the preferred way to go on. For example, binary logistic regression may be useful, if you have to condsider also other predictors in the same model (i.e. adjustment) and especially if you have a specific series of models in mind. In this case you would probably benefit from using logistic regression to begin with, because then you have a basis for comparing significance of single or groups of parameters between models. If you are just interested to just show that there may be an association, then you could choose an appropriate measure of association, but more information is needed on the particular research setting.

Hi Question for you all related here-

If I run a chi square on my study groups and binary outcome and it isn't significant, but there is a percentage point difference... then I run my logistic regression models with multiple vars and my study var is significant with increased odds, is this a spurrious relationship? I want to make sure there is not something I'm missing which would mean it is false. I ran 3 models starting with control covariates, added var 1 study model, then added var 2 study model - it was my last model that showed significance in my 2nd study var.

Thanks for your assistance.

Hi Lisa,

how many positive outcomes and negative outcomes do you have in your binary outcome variable? If you have less than 30 in each, you may be violating the rule of 10s. Peduzz paper in J Am Stat Assoc. it can be possible that your results in the simpler analyses are confounded or it could be you are overfitting your model. It would be interesting to see your 95% CI values on your odds ratios or coefficients. You also may need to assess model discrimination and model fit. Both of which are easy commands to enter for you to look at in stata. A lot more could be going on too.

Lisa, Depending on how you ran the logistic regression, the analysis could have more power to detect differences than a simple chi-sq approximation which is only accurate in the limit. Without knowing the details - did you use a likelihood ratio test (G-statistic) - i would tend to trust the logistic regression (assuming that the logistic regression was the best model) more than the chi-sq approximation.

Well, I dont find it necesary to interprete odds ratios whose confidence intervals include 1. Obviously Logistic regresion has much better power power than chi square in detecting differences; atleast in my view.

If you seriously want to interprete the logistic regressions' odds ratios whose confidence interval includes 1, then you have to specify this was not significant at 5% level of precison (95% confidence level).