I'm running a logistic regression test. On my first test, I've identified 6/7 significant explanatory variables. I then ran another logistic regression test with all the interaction terms. The test resulted (as per the delta deviance' p-value) to only 2 significant variables and 1 significant interaction term. The other 4 which I previously identified significant became insignificant. Does this mean that I should just include the 2 significant variable and 1 significant interaction term to my model? I'm kind of confuse because in doing so, wouldn't I commit a type II error? I hope you can help me, guys. I'm new on this field. Thanks!
It seems that type-II errors are not defined in your setting because you have not sepcified any substantial alternative hypothesis (your p-values refer to significance tests, not to hypothesis test, and theer is no concept of type-I and II errors in the logic of significance tests!).
You models with and without interaction are just different models, in which the coefficients also have different meanings. So you can not compare the p-values of any of your coefficients across the models.
P-values do not help you finding a good model. Good models can only be defined based on your aims and substantial knowledge. If interactions are important to you, then include them. If the model is not fitting well (enough) to your data, consider looking for some other model. How well the data can fit your model may be inspected by diagnosic plots. What is ok and what not is your judgement call and dependt on the aim of the model and your research context.
Dear Rob,
Just to add briefly to Jochen's answer. I presume you are carrying out binary logistic regression with an automated technique and have then added interaction terms. You should not think that you under the control of your software - you need to demonstrate that you are in control of the process. For example, there are different automated techniques - we recommend backwards. There are different cut off values for eliminating variables. There are different measures of the quality of a model (e.g. the percentage of correct predictions or r square equivalents). You also need to balance the quality of a model against its complexity (this is known as parsimony).
Apart from considering interaction terms you should also check the assumptions of binary logistic regression with your scale variables before you start, check your residuals and consider running bootstrapping if they are non-normal.
The sources I have used on this subject are:
Field, A. (2013) Discovering Statistics Using IBM SPSS Statistics: And sex and drugs and rock ‘n roll. 4th edn. Chapter 19.
University of Sheffield (n.d.) Logistic Regression. http://www.sheffield.ac.uk/polopoly_fs/1.233565!/file/logistic_regression_using_SPSS_level1_MASH.pdf.
Hi. In addition to the above, I would suggest doing variable selection on your full model using the lasso method. I have attached a paper showing that stepwise methods are unstable and that lasso is a much better choice. The paper also gives a method for performing lasso with references to R software. Please contact me if I can be of help. Best wishes.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Applied Statistics