This question relates to one I asked previously (https://www.researchgate.net/post/How_to_deal_with_large_numbers_of_replicate_data_points_per_subject). I have performed a generalized estimating equations model (binary logistic regression) with SPSS on categorical repeated measures data. As factors my model has condition (3 levels) and cell diameter (3 levels, cells grouped by diameter), plus their interaction. In my results, the main effect of diameter and the interaction term are significant, and looking at pairwise comparisons I can see exactly which groups differ from which as the effect only concerns some pairs, which is the expected result.
My question concerns reporting: I would like to use odds ratio in addition to p-values since it gives a better idea of which differences are really notable, but I can’t find a way to get the odds ratio for only some specific pairwise comparisons. The parameter estimates includes B for each main effect and interaction, which when exponentiated gives the odds ratio, but I’m not sure if it’s possible and how to extrapolate the odds ratio for specific pairs (since each of my factors has 3 levels)? Or can it be calculated using any of the information in the pairwise comparisons EM means output (it provides mean difference and standard error for each contrast, but since I have categorical data these seem to be inapplicable anyway)?
Thank you very much for any help!
Hello Mikaela. The topics for your post include SPSS and Generalized Estimating Equations. Is it safe to assume, therefore, that you are using the GENLIN procedure to estimate your model? If so, I think you could make any comparisons of fitted values you want via the !MLEcomp macro I wrote. (I've never tried it with a model using GEE, but it should work just as well as without GEE.) It requires as input a dataset containing the vector of model coefficients and the covariance matrix for the coefficients. You can get GENLIN to produce the needed covariance matrix by including COVB on the /PRINT sub-command, and you can grab the vector of coefficients from the table of parameter estimates. You can download the macro (along with examples) at the page linked below.
HTH.
https://sites.google.com/a/lakeheadu.ca/bweaver/Home/statistics/spss/olscomp_mlecomp
Hello Bruce,
Thank you for your response! I am indeed using GENLIN, and after a quick look at the macro and the article it seems like it would be exactly what I need. I’m not very familiar with using SPSS syntax (I usually run all my tests from the Data Editor), but since SPSS kindly writes the syntax before the output I assume I can run this macro by copy-pasting the automatically generated GEE syntax and combining it with the OMS commands to produce the output file in the correct format?
Hello Mikaela. See if this helps.
* How to use MLEcomp macro.
* First, run the macro definition file:
https://sites.google.com/a/lakeheadu.ca/bweaver/Home/statistics/files/OLScomp_and_MLEcomp_macros.sps.
* Next, turn on OMS as shown below, and estimate your model via GENLIN.
* OMS.
DATASET DECLARE B.
OMS
/SELECT TABLES
/IF COMMANDS=['Generalized Linear Models'] SUBTYPES=['Parameter Estimates']
/DESTINATION FORMAT=SAV NUMBERED=tnum
OUTFILE='B'.
DATASET DECLARE covB.
OMS
/SELECT TABLES
/IF COMMANDS=['Generalized Linear Models'] SUBTYPES=['Covariances of Parameter Estimates']
/DESTINATION FORMAT=SAV NUMBERED=tnum
OUTFILE='covB'.
* Estimate your model via GENLIN.
* Replace the following syntax with your own GENLIN command.
GENLIN default (REFERENCE=FIRST) BY educ (order=descending) WITH employ
/MODEL educ employ educ*employ INTERCEPT=YES
DISTRIBUTION=BINOMIAL LINK=LOGIT
/MISSING CLASSMISSING=EXCLUDE
/PRINT CPS DESCRIPTIVES MODELINFO FIT SUMMARY SOLUTION (EXPONENTIATED) COVB.
OMSEND.
* Now prepare the B and COVB datasets for merging.
* Manual steps to prepare the file of coefficients:
* 1. Bring dataset B to the foreground.
* 2. Delete any cases (rows) where Std.Error to Sig are all missing.
* 3. Delete all variables except for Var1 and B.
* Manual steps to prepare file holding the covariance matrix:
* 1. Bring dataset covB to the foreground.
* 2. Delete any variables where all values = 0.
* 3. Delete any cases (rows) where all values in the covariance matrix = 0.
* 4. Keep only variables Var1 and Intercept to the last variable in the covariance matrix.
* Now merge the B and COVB datasets, then close
* the B and COVB datasets.
* Now you are ready to use the macro.
* Here are the first two examples from my syntax file found at
https://sites.google.com/a/lakeheadu.ca/bweaver/Home/statistics/files/Example_2.sps.
* Your syntax will look a bit different, because you're estimating
* a different model using your own data. But this should give
* you the general idea.
* Analysis 1.
* Set1: Did not complete HS, 10 years with current employer .
* Set2: Did not complete HS, 0 years with current employer .
!MLEcomp
FirstCovB = Intercept /
LastCovB = educ2employ /
set1 = { 1; 0; 0; 0; 10; 0*10; 0*10; 0*10 } /
set2 = { 1; 0; 0; 0; 0; 0*0; 0*0; 0*0 } /
expB = 1 /
title = "FV1: EDUC=1, EMPLOY=10; FV2: EDUC=1, EMPLOY=0".
* Analysis 2.
* Set1: High school grad, 10 years with current employer .
* Set2: Did not complete HS, 0 years with current employer .
!MLEcomp
FirstCovB = Intercept /
LastCovB = educ2employ /
set1 = { 1; 0; 0; 1; 10; 0*10; 0*10; 1*10 } /
set2 = { 1; 0; 0; 0; 0; 0*0; 0*0; 0*0 } /
expB = 1 /
title = "FV1: EDUC=2, EMPLOY=10; FV2: EDUC=1, EMPLOY=0".
Hello Bruce,
This is very helpful, thank you! I’ve compiled the merged file with B and the covariance matrix and everything looks mostly fine. I ran the GENLIN command with my two factors (each with 3 levels) and their interaction terms, and removed columns in the COVB output which only had 0’s as values as per your instructions. The end result is that the third levels of each factor and the interaction are not included anymore. Is this just a feature of how SPSS builds these matrices? I noticed that the example data file in your article also only has educ levels 4-2 listed; I’m assuming there is a level 1 which just doesn’t have values in the matrix just like in my case. How is the last level included into pairwise comparisons here?
I also have a question about the selection of comparisons (i.e. the set1 = and set2 = commands). I take it each value within the {} refers to a variable, but are the numbers 0 and 1 used to just indicate a binary selection or the row number of the variable you want? I guess what throws me off is the “0*10” which seems to be used for the interaction terms in your example, what does this mean?
Hello Mikaela. If you post your GENLIN syntax and say what contrasts you wish to make, I may be able to offer more specific advice. But generally, for the set1 and set2 macro arguments, the first value = 1 to include the intercept, or 0 to exclude it. The remaining values are for the explanatory variables in the model. For a categorical explanatory variable with k categories, there are k-1 dichotomous indicator variables (1=Yes, 0=No). Only k-1 are needed, because setting all of the k-1 indicators to 0 represents the kth omitted category. For continuous (or scaled) variables, just put the value of interest. For product terms, you can enter the products, as in the 0*10 you asked about.
HTH.
Here is the syntax for my GEE:
* Generalized Estimating Equations.
GENLIN UCM (REFERENCE=LAST) BY Phenotype Size (ORDER=ASCENDING)
/MODEL Phenotype Size Phenotype*Size INTERCEPT=YES
DISTRIBUTION=BINOMIAL LINK=LOGIT
/CRITERIA METHOD=FISHER(1) SCALE=1 MAXITERATIONS=100 MAXSTEPHALVING=5 PCONVERGE=1E-006(ABSOLUTE)
SINGULAR=1E-012 ANALYSISTYPE=3(WALD) CILEVEL=95 LIKELIHOOD=FULL
/EMMEANS TABLES=Phenotype*Size SCALE=ORIGINAL COMPARE=Phenotype*Size CONTRAST=PAIRWISE PADJUST=LSD
/REPEATED SUBJECT=Animal SORT=YES CORRTYPE=INDEPENDENT ADJUSTCORR=YES COVB=ROBUST
MAXITERATIONS=100 PCONVERGE=1e-006(ABSOLUTE) UPDATECORR=1
/MISSING CLASSMISSING=EXCLUDE
/PRINT CPS DESCRIPTIVES MODELINFO FIT SUMMARY SOLUTION COVB.
The /EMMEANS command is likely unnecessary specifically because the MLEComp will allow me to do selective pairwise comparisons, but it shouldn’t affect the covariance matrix or parameter estimates I would assume.
The kind of contrasts I’m interested in are the differences in my outcome variable (UCM) between different phenotypes (Phenotypes) within the same cell size group (Size), so comparing e.g. Phenotype1*Size2 with Phenotype3*Size2, and separately Phenotype1*Size2 with Phenotype2*Size2. Perhaps I need to change the interaction term from Phenotype*Size to Size*Phenotype. But because the third levels are omitted as you explained, how do I indicate in the macro that I wish to compare groups with these levels?
With the EMMEANS command in the regular output I of course get the p-values and CIs for these pairs, but I’m not able to get the odds ratio (or some comparable estimate of effect size) which is what the MLEComp seems to be able to produce. Since EMMEANS also does each and every possible comparison, most of which I have no interest in, any adjustments for multiple comparison get hopelessly thwarted, so my intention is to run each of the specific contrasts independently and then setting an adjusted alpha level for each independent “family” of comparisons.
Ah, okay. Given that you want pair-wise contrasts of the different phenotypes within each size, the problem is much easier than I first thought, and you don't need to use my macro. You need to change COMPARE=Phenotype*Size to COMPARE=Phenotype on your EMMEANS sub-command, and you need to change SCALE=ORIGINAL to SCALE=TRANSFORMED. Then you need to exponentiate the mean pair-wise differences (and their CIs) to show the results as odds ratios. (You are currently seeing differences in log-odds.)
Re SCALE=TRANSFORMED, note the following from the Command Syntax Reference manual entry for GENLIN > EMMEANS:
- TRANSFORMED. Estimated marginal means are based on the link function transformation. Estimated marginal means are computed for the linear predictor.
If you use TRANSFORMED, you are using the logit link function. Thus, the estimated marginal means will show the log-odds, and the pair-wise contrasts will show differences in log-odds. When you exponentiate those differences, you get odds ratios.
You can use OMS to write the table of pair-wise contrasts to a new dataset, and then use a few simple COMPUTE commands to do all of this. Your syntax will look something like the following.
****** START OF SYNTAX *****.
* Use OMS to write the table of pair-wise contrasts from
* the EMMEANS sub-command to a new dataset called 'pwcon'.
DATASET DECLARE pwcon.
OMS
/SELECT TABLES
/IF COMMANDS=['Generalized Linear Models'] SUBTYPES=['Pairwise Comparisons']
/DESTINATION FORMAT=SAV NUMBERED=TableNumber_
OUTFILE='pwcon' VIEWER=YES
/TAG = "pwcon".
* Use EMMEANS to get pair-wise contrasts between different phenotypes
* within the same cell size group (Size).
*******************************************************************.
* NOTICE that I changed COMPARE=Phenotype*Size to COMPARE=Phenotype.
*******************************************************************.
GENLIN UCM (REFERENCE=LAST) BY Phenotype Size (ORDER=ASCENDING)
/MODEL Phenotype Size Phenotype*Size INTERCEPT=YES
DISTRIBUTION=BINOMIAL LINK=LOGIT
/CRITERIA METHOD=FISHER(1) SCALE=1 MAXITERATIONS=100 MAXSTEPHALVING=5 PCONVERGE=1E-006(ABSOLUTE)
SINGULAR=1E-012 ANALYSISTYPE=3(WALD) CILEVEL=95 LIKELIHOOD=FULL
/EMMEANS TABLES=Phenotype*Size SCALE=TRANSFORMED COMPARE=Phenotype CONTRAST=PAIRWISE PADJUST=LSD
/REPEATED SUBJECT=Animal SORT=YES CORRTYPE=INDEPENDENT ADJUSTCORR=YES COVB=ROBUST
MAXITERATIONS=100 PCONVERGE=1e-006(ABSOLUTE) UPDATECORR=1
/MISSING CLASSMISSING=EXCLUDE
/PRINT CPS DESCRIPTIVES MODELINFO FIT SUMMARY SOLUTION COVB.
OMSEND TAG ="pwcon".
DATASET ACTIVATE pwcon.
COMPUTE OddsRatio = EXP(MeanDifferenceIJ).
COMPUTE ORlower = EXP(Lower).
COMPUTE ORupper = EXP(Upper).
FORMATS OddsRatio to ORupper (F8.3).
LIST MeanDifferenceIJ to ORupper.
****** END OF SYNTAX *****.
HTH.
Great! I tried the above syntax (only changed REFERENCE=LAST to FIRST because in my data 0 = absence and 1 = presence of my feature) and it produces exactly the kind of output I was looking for. This was definitely a good lesson in the importance of knowing your syntax and using it to optimize tests :)
Thank you again for all your help! I will keep the macro in mind since I'm sure I will also have some data later on that requires more crossed pairwise comparisons.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS