Hello,

To the extent that the demographic variables are theoretically linked to the dependent variable and/or linked to the dependent variable in past research, they should be included in your model. More simply, if they matter, and you can justify their inclusion, include them.

What is your reasearch question? What is your DV?. That determines the kind of regression you do. see https://www.bing.com/search?FORM=U528DF&PC=U528&q=what+regression+method+do+I+use Is your goal prediction or explanation? Answer these questions and we will be ready to talk. Best, David Booth

David Eugene Booth I want to predict a specific behavior of internet users (my DV) with some variables. My DV is a seven point liker scale. There's not any moderator or mediator involved.

Hi,

why do you want to perform regression analyses for each predictor separately? It is clear that a model with lots of predictors requires a proper sample size, but if you do it for each variable separately, you totally miss the relationships between the predictor variables. I think this is one of the main strength of this analysis. Otherwise, you could stick to simple bivariate correlations, which are equal to the bivariate regression.

So option 3: do a hierarchical regression analysis with all 11 variables, demographic variables in the first block and then blockwise the other variables. You could structure the blocks depending on the content of the predictors, e.g. similar constructs entering together and not each variable separately. Despite that, if you have highly correlating predictors, i.e. they measure similar things, you could combine them to one score (e.g. via component analysis) and enter this new variable to reduce complexity.

Hooman Jafari Thank you very much for the reply. If your DV is Likert then you want ordinal logistic regression see my comments here:

My tome on Likert DV’s and ordinal logistic regression can be found here:

https://www.researchgate.net/post/Can_you_help_with_statistical_analysis_advice

https://statisticalanalysisconsulting.com/how-to-analyze-likert-type-dependent-variables/

https://www.researchgate.net/post/Possibility_of_using_point-scale_data_in_a_mixed-model_ANOVA

https://www.researchgate.net/post/Possibility_of_using_point-scale_data_in_a_mixed-model_ANOVA

I pretty much agree with Rainer Duesing on the RHS of your model-you might want to look at this book as well:

https://www.bing.com/search?q=Andrew+Gelman+Sue+Hierarchical+Models&pc=U528&form=QBLH&sp=-1&pq=andrew+gelman+sue+hierarchical+models&sc=1-37&qs=n&sk=&cvid=55284B5EDBB14B41866E00CFD2E9AB2C

In addition, if you have correlation among your predictors you might try this which doesn't allow that for predictive model selection see my comment here:

https://www.researchgate.net/post/In_the_ordinal_regression_model_should_estimates_coefficient_slopes_and_thresholds_modified_to_accommodate_significant_explanatory_variables_only

Good luck, David Booth

David Eugene Booth Thank you very much for your informative response. About the kind of regression, there's a paper which my paper is based on and in that paper DV is likert and an hierarchical ordinary least square regression is employed. That's why I'm going to employ hierarchical OLS regression.

Rainer Duesing Good tips, thanks.

Let me attach one more reference. I hope you look at all the links before you make you make your choice. Remember that publications live forever. There is nothing wrong with a hierarchical RHS here. IT IS THE LIKERT DV THAT MAKES THIS AN ORDINAL LOGISTIC REGRESSION. I suggest that you look at my LINKS and then note that an ordinal logistic regression would have the SAME RHS as an OLS regression. Please also reply to this post and tell me how a LIKERT variable could be normal as is required by ols. Please do respond because I really need to know this. Best wishes, David Booth BTW Notice that I recommended Rainer Duesing post. Everything he says is concerning the RHS of a regression model, as I said in my last post. AGAIN PLEASE LOOK AT THE LINKS IN MY PREVIOUS POST. Good luck, David Booth

David Eugene Booth I would agree that an ordinal logistic regression is more appropriate, if the DV is only a seven-point likert scale (although most publications do not recognize this and is seldomly done...). But you said that for OLS the DV has to be normally distributed? This is not true, it is the residuals which have to be normal for proper inference. Imagine conducting a t-test within the regression framework, if the groups differ then the DV should be bimodal not normal, but the residuals will be normal.

And you recommended the Gelman and Hill book, but I think Hooman Jafari does not want to conduct a hierarchical linear model/linear mixed model, but a blockwise simple OLS regression, which is termed sometimes "hierarchical", since blocks are entered in a hierarchical manner.

I went through links David provided which gave me great insights into different kinds of regression. I think this link was very helpful:(https://statisticalanalysisconsulting.com/how-to-analyze-likert-type-dependent-variables/)

It seems it's a controversial issue when it comes to use what regression method but I'll attach some papers to make my intention more clear.

In response to David, please see paper "resisting". In response to Rainer, please take a look at these paper which used hierarchical regression.

Hooman Jafari I've looked through all of your attached papers and none(!) of them used hierarchical linear models (other names for this type of analyses:, random effect models, linear mixed models or multilevel models), all of the papers used OLS regression or some form of logistic regression and incorporated the predictors blockwise, so step by step and not all together (one article used the term "hierarchical", which is confusing but not(!) the same has hierarchical linear models!).

Rainer Duesing Actually two papers I attached used the term "hierarchical" as you can see in attached files (fb and resisting). However, I'm not that experienced with regression analysis but I've been told, in SPSS, OLS regression is the same as simple linear regression and that's why I assumed those paper employed hierarchical regression.

So, are you implying those regressions termed as "hierarchical" in those two papers are not actually "hierarchical" !? Please tell me what kind of regression exactly is this?(see "regression")

Please take a look at this:

https://stats.stackexchange.com/a/66772/276057

Rainer Duesing Thank you for what I missed, that Hooman Jafari was probably referring to some sort of stepwise regression. In that case, I doubly agree with you and very strongly recommend this paper:

Article Regularized Ordinal Regression and the ordinalNet R Package

for variable selection. Lasso does not allow correlated predictors. See references in the attached paper.

BTW my comment on the DV in OLS being normal follows from the ordinary ols assumption normal errors as you pointed out and the theorem on mean and variance of a linear function. Then this allows the usual coefficient stats to have t-distributions by the precursor theorem to the CLT . I believe this is in Freund, Mathematical Statistics always assuming that my memory holds. Mea Culpa if I am mistaken in my memory both to you and the ghost of Prof. Douglas Bey who introduced me to those theorems. I must admit that if I am mistaken it would not be the first time. LOL Best wishes to all, David PS I now think that it also follows immediately from the theorem that says OLS and MLE are equivalent. again, please set me straight if I am mistaken as well as off-topic.

David Eugene Booth thanks for the papers on lasso regression, I am not very familiar with this topic, so something new to learn ;-)

Aren't OLS and MLE identical when normality holds? Otherwise they will differ, when I got it correctly....

Hooman Jafari I can believe that this terminology is confusing and itself a little bit misleading. I hope I can make the differences clearer to you:

1) If you take an OLS regression, with more than one predictor, you have a multiple regression. Typically, you would incorporate all of the predictors at once into the model and see which one is significant and which one not. But there is also another method, which some authors refer to as "hierarchical regression". This means that not all predictors are entered simultaniously, but consecutively. This can be done on basis of statistical inferences (“stepwise regression”, typically not recommended!) or on basis of theoretical considerations (“blockwise regression”, e.g demographic variables first etc). This is all in the framework of OLS and can be extended to other regressions, e.g. logistic regressions. “Hierachical” here means an order of entering the predictors, which is presented in your papers. So you can stick to this approach.

2) There is another conceptually different method called “hierarchical linear model” (HLM), which also has the names “linear mixed model”, “multilevel model” or “random coefficient model” (often they are used synonymously). Here “hierarchical” means an order or nesting of the data structure, not the predictors. For example, pupils are nested within classes, while classes are nested within schools and schools are nested in districts. To incorporate these nesting structure (3 levels in my example) and therefore correlated data within each cluster, you need hierarchical linear models. Now the twist: you can also incorporate predictors sequentially in a HLM ;-)

Rainer Duesing Sure but if the MLE is normal which it is, gives Xbar and the unbiased var estimator, Note Xbar is regn coeff for Y=a+0x +eps and both are equal.constants are equal and the only thing left is the DV. I think all that's left is the formalism since the only dist's on the other side are normal. normals combine to normals and so forth. The formalism should be a proof by induction on the no. of regn coeffs. and as Fermat said, "but the margin is too small to contain it." LOL

If there's a slip in my thinking please tell me. I am definitely not a theoretician. Best, David

Rainer Duesing Thank you for the clarification but I was wondering about the rationale behind the OLS regression with ordinal DV in this paper. Shouldn't we employ ordinal regression with ordinal DV or is it just preferred and not strictly needed?

David Eugene Booth I am not sure what you mean with "if the MLE is normal". The MLE estimates are asymptotically normal (good for inference), but this is not relevant for us I think. What is relevant is the distribution of the data (or residuals). Only in case of normally distributed data OLS and MLE will come to the same estimates, since it can be shown that MLE estimates for mu and sigma are the mean and standard deviation, as in OLS. Is that what you meant?

And why do normals combine to normal? A mixed normal distribution is not necessarily normal itself.

I am not a theoretician myself, just a “normal” psychologist (as normal as they can be….;-)

Hooman Jafari I think the authors simply did not think about their data, because a lot of people use OLS regression, even if they have strictly speaking only ordinal data. This very common, but as David pointed out, it would be more appropriate to use ordinal regression (under certain circumstances they will have similar results, but this cant be guaranteed).

Rainer Duesing Not quite check out the method of MLE invented by R A Fisher in about 1920 here: https://www.bing.com/search?q=method+of+maximum+likelihood&qs=SC&pq=method+of+maximum+liklihood&sc=3-27&cvid=0BF49D8246A04C3E906CE9D648298103&FORM=QBLH&sp=1

I think if you look at that and the theorems on normal distributions you will see what I mean. If something is not clear Please ask and I will try to assist. Though I admit this is long in my past. Best wishes, David Booth

Thank you all for your help.

Hi, is this a statistical exercise to predict a dependent variable. If it is, then the number of variables is too large and you lose degrees of freedom. In practice, we try to limit the number of independent variables based on the theory (or stepwise regression which is a statistical technique to drop off the variables). You need a relatively large sample to have 11 independent variables and statistically it is a problem with multicollinearity, an issue. Another way is to amalgamate the variables using factor analysis, which is a method of combining them and determining which of 11 variables belong to major factors (determined by the factor analysis program).