Confidence intervals can be overlapping and the difference still statistically significant, so I would not use overlapping or non-overlapping confidence intervals as a criteria for statistically significant differences.  That does not give you a guide to how different or similar the ORs are. Use standard statistical tests on the logarithms of the ORs, using the sum of the variances of the logarithms of the ORs as the variance of the difference in logarithms of the ORs.

Read this: http://www.stats.org/faq_odds_ratios.htm

do one big analysis with predictors intervention, 'after', and intervention*after.  the p-value for intervention*after is the p-value for the difference between the odds ratios you seek.

Hi !

If you just wanna get a crude ORs, a cross-table at follow-up would be sufficient (with a Odds = Risk/1-Risk). However, such a estimate totally ignore the difference at baseline, and does not take into account the influence of other factors associated with the "intervention effect". 

As a consequence, a complex longitudinal analyse would be required to obtain your p-value at follow-up. I think I have the SAS synthaxe to perform such an analysis...let me look at this...

Sincerely,

A statistical test for the difference between the two odds ratios is not necessarily the best or most informative approach. Unless you have a reviewer of your manuscript requesting a statistical test, a more informative approach would be to describe the relationships between the two odds ratios, with respect to the direction and magnitude of the effect size and also whether or not the confidence intervals overlap. For the example that you gave, the odds of having the outcome at follow-up compared to baseline were greater than null (increased odds at follow-up) in control group and less than null (decreased odds at follow-up) in the intervention group. So the intervention changed the direction of the effect. The confidence intervals are non-overlapping, and this can easily be shown graphically in Excel using the high low graphing function. These descriptions, plus showing the odds ratios with confidence intervals as you have done here, give more information than a statistical test.

Thank you all for your usuful comments.

Ellen, what do you mean by predictors intervention? I can not clearly understand 'after', and intervention*after. I would appreciate you if you could guide me about how to analyse data in more concrete form.

Kim, thank you for your encouraging comment. However, in case that the odds ratios are overlapping, how can I prove the magnitude of difference?

Perhpas I am reading your problem incorrectly, but why don't you just run a logistic regression analysis with intervention group as a predictor variable and baseline values as covariates? 

it looks as if these were just 2 places, not randomized (and even if randomized, a small number of places would not be enough to allow the assumption that randomization cures all imbalances), and indeed you have an imbalance at baseline.

in this situation, you should not use baseline variables as covariates.  rather you need to use location and 'after' as the main effects, and location*'after' (i.e. intervention place, after the intervention started as an interaction effect, that is really the effect you are looking for).

to be a bit clearer, some notation:

location:  0=non-intervention location, 1=intervention location

time:  0=before intervention, 1=after intervention

location*time=1 if in intervention location and after intervention;  0 otherwise.

model  (trying to avoid any package-specific notation) is

outcome is a function of location, time and location*time.

the p-value for the last variable tells you whether your intervention is statistically significant.

It is unclear to me, Ellen, how your approach handles the baseline difference ("imbalance"--I usually think of imbalance as a difference in sample size rather than a difference in some attribute or measure).  Could you say more about that?

in this case, the imbalance we are worrying about is the fact that the prevalence of the 'outcome' differs between the 2 sites before the intervention starts.  from the phrasing of the question, it sounds as if all the individual observations are independent, conditional on their location, but otherwise we don't know much about the type of outcome we are talking about.

for example, suppose the intervention is trying to get hospitals to make sure that patients discharged after a heart attack are prescribed beta blockers (unless contraindicated).  what we have is 4 series, one in each location before the intervention and one in each location after the intervention.  information about beta blockers is 'in the air,' so a straight before-after design in one hospital is not ok.  so we get 2 comparable hospitals with (presumably) the same access to information about beta blockers and do our intervention in 1.  the 2 hospitals started out with different rates of beta blocker prescription.  in both places, beta blocker prescription goes up with time.  so the question is whether it goes up more in the intervention hospital than in the control hospital.  that is what the interaction, location*time is getting at.

Yes, I understand that you are trying to control for baseline differences, but as you have described it, I don't understand how the interaction term you propose controls for that difference.

It is not necessary to do other test.

Sadatoshi, in the case where the confidence intervals are overlapping, then that alone tells you that the outcomes in the control and intervention areas are not statistically different. A graph of the odds ratios and confidence intervals will show you how similar they are by how much they overlap. 

If you have the 95% CIs for all of your odds ratios, you can do a quick check for statistical difference by looking for overlap in the odds ratios. If 95% CI's for two OR overlap, they aren't significantly different.

Confidence intervals can be overlapping and the difference still statistically significant, so I would not use overlapping or non-overlapping confidence intervals as a criteria for statistically significant differences.  That does not give you a guide to how different or similar the ORs are. Use standard statistical tests on the logarithms of the ORs, using the sum of the variances of the logarithms of the ORs as the variance of the difference in logarithms of the ORs.

As I see your problem the solution is quite simple. What your want is 95% CI for the odds ration between the "odds ratios for the intervention area" relative to the "odds ratio for the control area". To do that you simply include the interaction term between the two independent variables you have two describe your data.

You must have an outcome variable (outcome_no=0 & outcome_yes=1), and you must have a follow-up variable (Baseline=0, follow-up=1) and an intervention variable (control area=0 & intervention=1).

In your logistic regression model you simple include the two indenendent variables AND the interaction term (follow-up*intervention) as explanatory variables.

Make sure to check that the odds ration for the interaction term is what you expect (0,61/1.48=0.41).

To get the interaction term you can generate it simply by creating a new varable by multiplying your follow-up and intervantion variable, or you can construct it like this:

Interact = 1 if (outcome=1 & intervention=1); othervise interaction = 0.

In this case I would suggest to use Mc Nemar Test as they are not independent groups.

be careful with the overlapping (or not) of CI - that can be misleading.

attached a test with R - your p-value is about 0.004

Dr Maccario, thank you for your response.

Since I can not open the attached file, could you send it again, please?