You can allow the curve to be more flexible, and then based on cross-validation or some other method decide how flexible the curve should be. See generalized additive models or GAMs.

You can allow the curve to be more flexible, and then based on cross-validation or some other method decide how flexible the curve should be. See generalized additive models or GAMs.

Hello Sarah,

As an alternative to the viable option outlined by Daniel Wright , you could try transformation of your IV(s). This might reduce the magnitude of the interaction terms to ignorable levels. A polynomial approach might work as well (e.g., include X, and X-squared--after centering--as model IVs.

Yet another approach would be bootstrap model estimation. Here's a simple link showing how to elicit in r: https://www.r-bloggers.com/2015/08/bootstrap-with-logistic-regression/

Good luck with your work.

Just to link David Morse 's and my previous answers. GAMs often use piecewise polynomials (connected at knots, called splines) to achieve this flexibility. They often add things like being linear at the ends to prevent some of the odd things that polynomials do.

I have a question on the second part of David's answer. How will bootstrapping address the non-linear (with the logit) relationship? You'd still be re-sampling from data with the same non-linear (wtl) relationship.

Hello Daniel Wright and David Morse,

Thank you a lot for your ideas of alternative approaches and the fruitful discussion! This really helps me a lot (although it’ll take me a while to read and get into GAMs and polynomial models).

Regarding David Morse's suggestion of transforming the IVs (which for a first try, seems like the easiest option to me), I have a (maybe stupid) questions (sorry, I’m a med student): How do I approach this?

I just found a paper stating that: “If linearity of the logit is violated for logistic regression, the supplementation of the natural log of the continuous variable in place of the original independent variable should solve the problem.”

(if of interest: https://www.researchgate.net/publication/341354759_Logistic_and_Linear_Regression_Assumptions_Violation_Recognition_and_Control, pdf page 20)

Do you agree? So instead of variable 2, e.g. I create a new variable being the natural log of variable 2 and use this new variable in SPSS to perform binary logistic regression? Are there other mathematical operations commonly used/suggested to transform independent variables?

Hi Sarah M. Volkers , it depends on 1) the empirical relation of the data, and 2) whatever theories you have about the relationship between the variables. Ideally your statistical models represent your scientific models, and so are about the same thing. The log is just lots of people's first thoughts for a lot of variables. For example, I am looking at response times at the moment, and almost always just work with the logs of response times. But, that is because response times often approximate these. Transforming the variables is using a different function for mapping between the variables, so picking a few transformations and trying these may get a good fit. The GAM approach is little the data/computer decide. This might sound like an advantage, and sometimes is, but if you have a theory about the relationship that should be considered first.

ps. I looked through that section of that paper. The idea that a single transformation will always making a relationship linear is silly. I would not advise following that paper's guidance.

pps. Not sure how to in SPSS, but to include a spline in a logistic regression in R it is just

library(splines)

glm(y ~ bs(x), family=binomial)

There are other options and things that can be done, but that is a start. bs stands for B-splines, and the default is two cubic curves joined smoothly (in a quadratic curve, so the eye can't really see where the join is) at a single spot (the median of x). That said, as said above, often people have specific (non-linear) models that they believe describe the relationship between things, so those should be considered.

Dear Sarah,

Another option might be to simply change the offending variable into categories? High/low, high-medium-low etc.? That would remove the issue completely.

Hope this helps.

Roxy

Daniel Wright: Thanks a lot for these explanations and sharing your expertise!

Seems like SPSS does not have a (regular) procedure to handle GAMs, so on the long term I'd should probabely switch to R (maybe better anyways)

(For anyone else interested: https://www.ibm.com/support/pages/does-ibm-spss-statistics-offer-generalized-additive-models-gams)

Roxanna Short : That's indeed a feasible and good option that I'll try! Thanks a lot!

I'm really amazed by all these suggestions I got here!

Hello Roxanna Short & Sarah M. Volkers. It is true that carving the problematic variable into categories might solve one problem. But it would like introduce others. Here is a list of problems caused by carving quantitative variables into categories:

• http://biostat.mc.vanderbilt.edu/wiki/Main/CatContinuous

You can also find many articles on this subject. Try a Google Scholar search on , for example.

Having said that, I was going to suggest that you could, as an exploratory analysis, use a categorical variable to get a sense of the functional relationship between X and Y. E.g., if a plot of the fitted values for the categorical X-variable looks U-shaped, that would suggest including X and X2 in the model when using the original quantitative X-variable. Or it might suggest that some kind of spline would work fairly well. Bear in mind, though, that your particular sample includes noise that is unique to that sample, so you need to be wary of over-fitting your model.

HTH.

Too bad. I get the point, thanks Bruce Weaver !

Yes it's not ideal, but it is something that's frequently done in health research - age is the prime example here. However, it may also be appropriate to categorise variables that have clinically-relavent/diagnostic cut-offs - especially if your only other option is to fit a complicated non-linear model that you may not fully understand!

Roxanna Short Is the researchers job to help someone in biomedicine? If so then look at the DATA not your ease in using a particular model, in science DATA RULES ALL. D. Booth

Splitting into categories is, as David Eugene Booth and Bruce Weaver write, wrong. I'll add a couple of thoughts. One issue is where to cut the categories (or binning). This can be tricking, and adds researcher degrees of freedom in (this is one of the reasons why physicists talk about the need for the 5sigma rule for discoveries and why some psychologists talk about navigating the multiverse of possible analyses). So, Roxanna Short saying it is simpler is only true if done without concern. There is also the issue of information loss. The amount depends on how the binning it done. COncern #3 in https://www.frontiersin.org/articles/10.3389/feduc.2019.00147/full relates to this.

That said. If you have a theory that says something qualitatively different happens to your stimuli at some point (like water changes form around 0 and 100 C, or the passing point on a teenager's driving test when researching parent stress), then putting in dummy variables for these. Or if you can say that all in each bin are the same, then it makes sense to bin. But these require knowledge of the variables.

Just thinking about the discussion again from a clinical point of view:

I think Roxanna Short is right in saying that splitting continuous variables into clinical categories is frequently done in healthcare research (probably different from other fields). And it’s present in everyday clinical life.

E.g. basically many clinical decision are based on scores predicting the probability of a certain disease being present or not, the probability of an event (e.g. death, stroke etc..) happening, or similar. Lots of these scores contain categorial variables (e.g. lab values, age) and they are really always split into categories. Just as an example, the C(U)RB-65 score predicts mortality risk in pneumonia thereby aiding a clinician in the decision if a patient needs in-hospital (or even ICU) treatment or not. 4 of the 5 parameters (respiratory rate, blood pressure, age and urea) are continuous variables split into binary categories. It’s simple and easy and a physician can calculate it in his/her head.

Of course, there’s more behind those “final” scores used in the clinic than a simple logistic regression model performed in one study. I also see that a lot of information is lost and (from what Bruce Weaver and Daniel Wright have written) that there might be other problems. But for now (still have to read more and are always open for more thoughts) using diagnostically relevant cut-offs to categorize variables for models in medical research doesn’t seem like a general and absolute no-go to me and might also have advantages (applicability at the cost of the quality of the model). Still haven’t made up my mind and of course I don't claim to be an expert :) Just my (maybe false) thoughts, which - before any publication - will of course be checked by a biostatistican of our institution

Sarah M. Volkers Having seen someone do it and it being a good idea idea are NOT the same thing. You can kill someone with a poor choice in biomedicine. Here are some possible examples to assist you: https://b-ok.cc/book/5243562/0e8c4b and: https://www.google.com/search?client=firefox-b-1-d&q=Is+splitting+continuous+variables+a+good+idea+with+clinical+predictive+models

David Booth

PS what on earth is linearity of the logit?. If you look at a logistic function it is always S-shaped. Linearity of the logistic regression model means the RHS is linear in the regression coefficients. If you got something else please advise me.. Thank you. Hierarchical regression in the SPSS literature usually involves some form of stepwise regression which is unstable (not reproducible)-see Steyerberg above and the references in the attached. The take home message is rethink this whole thing.

D. B.

David Eugene Booth wrote: "PS what on earth is linearity of the logit?. If you look at a logistic function it is always S-shaped. Linearity of the logistic regression model means the RHS is linear in the regression coefficients. If you got something else please advise me.. Thank you."

The expression "linear in the logit" can be found in many resources on logistic regression. E.g., see slides 18 and following here:

• http://christinemauro.com/downloads/cmauro_LogRegres_SAS.pdf

--- start of excerpt from slide 18 ---

An underlying assumption: log odds of outcome (e.g., disease) increases by the same fixed amount anywhere on the X scale

• That is, the effect of [continuous variable] X is linear on the logit scale

--- end of excerpt from slide 18

See also point 3 here: http://www.statisticalassociates.com/logistic10.htm

David Eugene Booth also wrote: "Hierarchical regression in the SPSS literature usually involves some form of stepwise regression which is unstable (not reproducible)-see Steyerberg above and the references in the attached."

David, I don't know what "SPSS literature" you have been reading, but most of the textbooks I am familiar with that include SPSS examples make a very clear distinction between hierarchical regression and algorithmic variable selection methods such as stepwise (and advise one to avoid the latter for all the reasons people like Harrell and Steyerberg discuss). In hierarchical regression, according to the "SPSS literature" I know, the researcher (or data analyst) specifies in advance and controls which variables are entered on each step. Two common approaches are:

The first approach, it seems to me, is more common in psychology, and the second in epidemiology and medicine.

HTH.

Dear Bruce Weaver I realize that we have vastly different opinions on SPSS. HERE is where I got my information. NOTE the IBM AND SPSS part of the link: https://www.ibm.com/support/pages/hierarchical-regression-spss

BTW I do assume these folKs know what is in their software though perhaps not much about how well their advice works. For more pertinent details, esp in logistic regression, I suggest the Austin and Tu reference in our paper attached above and available on RG. Well the approach may be common in psychology but i haven't seen a lot of psychologists recommending stepwise in these posts. If you read the Austin and Tu paper you will see why. BTW Austin and TU are both quite well known epidemiologists which I assume is pertinent to the issue. NOW the field here is unimportant to whether a particular statistical method is used only whether or not the method works which stepwise clearly doesn't. BTW you might comment on your dubious claim that, "... the change in R^2 is statistically significant". I am not aware of any such test because the sampling distribution of R^2 is unknown. Or perhaps I have missed the relevant paper. Wishing you the best, David Booth

PS some other information you may find interesting:

https://www.google.com/search?client=firefox-b-1-d&q=Hierarchical+regression+SPSS

Now this: http://www2.econ.osaka-u.ac.jp/~tanizaki/cv/papers/dist_r2.pdf

may be the paper you were discussing with respect to R^2. Note that this doesn't apply to OLS or logistic regression.

Further R^2 doesn't apply to logistic regression. The classic paper is:

Article Logistic Regresson Diagnostics

Further a summary on stepwise and its relatives( Austin and Tu results included):

Article Performance of using multiple stepwise algorithms for variab...

Good morning, David Eugene Booth. Yes, you and I do appear to have very different opinions about SPSS. Ironically, as someone who has actually used it for many years, I suspect I may be more aware of its deficiencies and its capabilities than you are. As one example of a deficiency, I do not for one moment believe that SPSS is nearly as user-friendly as claimed. See my 4-May-2018 post in this thread, for example:

https://www.researchgate.net/post/Setting-aside-pricing-issues-is-STATA-really-that-much-different-better-than-SPSS

At the end of the day, I do not think it is wise to assume that all SPSS users are statistical newbies who are very likely doing everything incorrectly. Nor do I think it wise to assume that there is no such thing as a mis-useR of statistics. I hope we can agree that we should hold neither IBM nor the R development team responsible when individual users engage in statistical malpractice. :-)

By the way, that IBM support page you pointed to makes no mention of algorithmic variable selection methods such as stepwise regression. It describes use of the ENTER sub-command to carry out hierarchical regression in the manner I described in my last post: I.e., "the researcher (or data analyst) specifies in advance and controls which variables are entered on each step".

The LOGISTIC REGRESSION command, which estimates binary logit models, has the same ENTER command, but also has options for using the following methods: forward conditional, forward LR, forward Wald, backward conditional, backward LR, or backward Wald. None of these methods involve tests on changes in any of the available psuedo-R2 values, nor do I recall seeing anyone (in this thread) suggest such comparisons. See the following page for more info:

• https://www.ibm.com/support/knowledgecenter/en/SSLVMB_26.0.0/statistics_reference_project_ddita/spss/regression/syn_logistic_regression_method.html

I hope this clarifies the options available in the SPSS REGRESSION and LOGISTIC REGRESSION commands.

Oh, I nearly forgot to address this bit of your last post:

BTW you might comment on your dubious claim that, "... the change in R^2 is statistically significant". I am not aware of any such test because the sampling distribution of R^2 is unknown. Or perhaps I have missed the relevant paper.

If you use R to estimate two OLS models where the first model is nested within the second, can you not request an F-test to compare the fits of the two models? If you can, is that F-test not equivalent to the usual F-test on the change in R2, as shown here?

• https://tien89.wordpress.com/2010/03/18/testing-multiple-linear-restrictions-the-f-test/

Various pages I have found suggest that you would use the aov command to compare those nested models, and that it yields F-tests to compare nested models. I suspect very strongly it is the usual F-test on the change in R2. But I am happy to be enlightened if it is not. :-)

@Bruce Weaver apologise but a winter storm has ruined my week.. I intend to reply as soon as possible. By the way had my first SPSS course in 1977 last time I taught it was a multivariate course in 2005 I believe. Best wishes,. David Booth

Hello David Eugene Booth. Sorry to hear that. I hope the damage was not too severe and that you and yours are safe and well. I also apologize for suggesting in one of my posts that you might not be terribly familiar with SPSS. (I was still in high school in 1977, and had never heard of SPSS.) FWIW, I now prefer Stata (over SPSS) for any kind of regression modeling, largely because the command syntax is usually much simpler and far more consistent across different types of models.

All the best,

Bruce

Dump it and use Generalized Additive Modeling (GAM). It is a non-parametric approach that handles the curve of the data without complexity. Hence, it will produce a well defined spline of the model.