I have a dataset from a questionnaire that consists of a dependent variable called skill level. This is an ordinal variable that has dummy coded values 1-5. I also have multiple (up to 10) independent variables that have dummy coded binary values 0 or 1. The independent variables are not ordinal, meaning that 0 does not mean "less" or "worse" than 1. The variables were simply statements in the questionnaire that the respondents chose to be true or not for them. It's also not inherently "better" that a respondent chose all or none of the statements to be true for them.
I would like to study which independent variables are the most important and affect the dependent variable the most. So I'd like to find a model or a set of analysis techniques that would find correlations for IVs and skill level. If possible, I would like to get the significance values also for each independent variable. My dataset has around 350 data points, but the data points are not equally distributed per skill level. There are less data points for the highest and lowest skill levels than for the middle skill levels. The data is also not normally distributed.
I tried ordinal regression but it did not perform well. I think it might be because the variance in the data values is large (only 0 or 1). In some cases, it even showed p-values near 1 for some variables that had Spearman correlation p-value 0.00, in addition to having the sign of the coefficient (+/-) being the opposite one.
Could you help me find a technique to study the data and find the most "important" variables in it? As for software, I'm most familiar with Python but I can get help with SPSS and R if there are packages or techniques that are limited to those. Thank you!
ps. I was thinking of doing a logistic regression for each independent variable and skill level separately, but I'm not sure if it's a good idea. I don't know if there is a statistically satisfying way to compare the results of the logistic regression models. Please tell me if you have any idea on how that could be done and interpreted.
You can run a random forest or boosting and then use the variable importance measure to identify the most important variables. Here is a short Youtube video on running these models and making the plot: https://www.youtube.com/watch?v=igBfnECna_8&ab_channel=EhsanKarim. This could also be helpful: https://topepo.github.io/caret/variable-importance.html.
Throwing away any predictor will change your research design, all variables contribution must be known. Variance inflation factor can be obtained, especially when the IVs are continuous.
You can make box plots of your ordinal responses, that will be quite informative.
Ordinal logistic regression is appropriate for your research, and it's possible to compared saturated and various levels of restricted models comparing both the deviance information creterias(AICs, and P-values)see example here https://www.google.com/url?sa=t&source=web&rct=j&url=https://rpubs.com/PandulaP/logisticregression_model_compare&ved=2ahUKEwiI5v2drdXuAhUM8hoKHSFQBGMQFjABegQIAhAB&usg=AOvVaw1VT5AcMdQ5VGmpAKHFPuEB.
Check for information, "Ordinal Logistic Regression" in the SAS LOGISTIC documentation which fits a logistic model to rating response measured on an ordinal scale, with example and Interpretation of the parameters and odds ratios of a categorical predictor in the fitted model. http://support.sas.com/kb/22930.html
Note pls: a test of the proportional odds assumption is very critical your analysis. So also power calculation, to a level of effects(low or medium).
Goodluck
It seems to me that a binary ordinal dependent variable could also be treated as a classification variable and dealt with using binary logistic regression. Boosted regression trees is a good choice too. You might look at the attached paper now now in the final step for publication in scientific reports. Good luck David Booth
I will go with David Eugene Booth . you should consider doing Boosted regression trees. Still it is not clear for me why your 'ordinal logistic regression model' failed.
I think your dependent variable violate proportional odds assumption(you could use Brant test to identify the variable that violate this assumption), which is the basic one for doing ordinal logistic. In this case youd could use partial proportional odds model.This is one family of ordinal logistic regression.
- Badges
- Science topic