I am working with two sets of colleagues on research in which we have conducted multiple regression analyses. In one project, there are two independent variables (IVs), one of which is categorical (it's gender) and the other IV is continuous (age). In the other project, there are 11 IVs (they are all significant after backward elimination of several other IVs), most of which are categorical.
I am not sure whether we should be reporting Cohen's f-square as an indication of effect size or something else when describing our results. My main concern is whether the presence of categorical variables would render f-square inappropriate.
Can anyone help, please?
G'day Robert
I can tell you both from my own experience as well as from other studies that I have never used or seen Cohen's f-square in connection with multiple regressions. To the best of my knowledge, either the unstandardised regression coefficient (in case of binary IV - b) or standardised regressions coefficient (in case of quasi-metric/metric IV - ß) is reported alongside its p-value and the explained variance (R²) for every step (and total).
As you know, I am not a statistician, so it would not go amiss to have a second opinion on the matter.
Cheers!
Marcel
Marcel Grieger, thanks for responding, my friend.
Believe it or not, there's some reassurance in what you wrote. It's the same kind of feedback I'm getting from around my "local traps". However, there are some confident-sounding, but also somewhat conflicting, sites on the web, so I thought I'd put my question onto RG.
I hope some light can be shed on the issue - for me as well as for others.
Robert
Hello Robert Trevethan You talked about IVs, but you didn't say anything about DVs. What is that? What is its Measure? What regression did you use? Linear? Nominal? Ordinal?....
Respectfully
Abolfazl Ghoodjani, sorry for my lack of information. In both cases, the DV was a continuous variable, so we were using multiple linear regression. Might that help you to respond to my question - please?
In addition to the topics that Marcel Grieger mentioned, in a multiple linear regression model, I draw graphs such as
Pareto Chart of the Standardized Effects
Incremental Impact of X Variables
I'm more interested.
Abolfazl Ghoodjani, thank you for getting back and attempting to help me. I have to confess that I don't understand what you have provided (that could be put down entirely to my ignorance), but I also can't see how it addresses my original question, which was concerned with reporting effect size in multiple regression and whether or not we should be reporting Cohen's f-square.
Robert Trevethan Thank you for your comment.
Honestly, in my opinion, since f2 is made by R2, I do not think it can be considered a new parameter. That is, the same information and finding that R2 contains is sufficient. However, for the beauty of the report, it might be interesting to mention f2 as well.
Abolfazl Ghoodjani, thanks for persisting with me. I fully appreciate what you've written. Here's a problem, however: In our study with 11 IVs, the R-square value is only .19, which means that only 19% of scores on the dependent variable is explained by the predictors. That strikes me as a bit (disappointingly) low.
However, the f-square calculation for that R-square comes out at .24, which is midway between a Cohen's effect sizes of moderate and high.
There seems to be a discrepancy there, with the R-square value being unimpressive and the f-square value being quite impressive. I'm having trouble resolving that and wonder, therefore, whether f square should be used/reported with caution.
At the back of this is my concern that many journals now want researchers to report effect sizes in relation to their results. However, I'm sure that those effect sizes should not be misleading.
Robert Trevethan This is obvious, because 0.24 = 0.19 / 0.81. Of course, you know better than I do, and you are my teacher.
However, Regardless of anything, my idea is to improve your R2.
This can be done by identifying and discovering Large Residuals and Unusual X Values.
Next, set them aside and fit the regression model again.
Repeat this until we no longer have data named Large Residuals and Unusual X Values. in our regression model.
Look at the red dots on the graph. They ruin our model and bring down R2. So it's good to find them and get them out of the model.
In the end, you will definitely have a better R2.
Abolfazl Ghoodjani, thanks for keeping on trying to help. As it happens, we had attended to the issues you have mentioned in your post immediately above here - though we didn't delete the outlying data; we winsorized them.
We checked all of the other assumptions for running multiple linear regression, and there didn't seem to be any problems.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Regression