Hello,
I have a kind of productivity measure that was plotted for American and Japanese companies through the seven years. This plot shows the American companies have slightly higher value, less productive, than Japanese. However, I estimate this measure based on 10 different independent variables, that were statistically significant, in which ownership is one of those, ownership could be American or Japanese companies. It was found that the related coefficient for this categorical variable in the statistical model (it’s a combination of regularized method and robust regression) is showing an opposition result, indicating American companies are more productive (the coefficient value is 1.213, meaning that on average this measure is higher for Japanese by 1.213). Any thought and insight concerning to this contradiction would be appreciated. Thank you.
Amir -
I am not familiar with your topic, but it sounds like you are measuring productivity as a function of another kind of productivity, and other "independent" variables. Your results would seem to indicate that one kind of productivity is antithetical to the other. If that makes no sense, then you might consider that your independent variables actually have some kind if interaction, which I think is to be expected. It is the degree of interaction which can be a problem. Looking at "significance" is often misleading, as effect size and sample size matter, and here it is a problem in that, for example, collinearity can greatly impact coefficients, such that a positive regression coefficient can even appear negative.
Perhaps you should research "model selection," and "model validation."
The following might give you some ideas, such as graphing y on the y-axis, and predicted-y on the x-axis for different models, to compare them. However, you might overfit, and it is therefore important to save out some test data. Also, I am not clear on how the categorical variable would be considered. Not my area. But maybe you could come up with some ideas from this:
https://www.researchgate.net/publication/313925533_Comparing_Model_Performances_Graphically
Cheers - Jim
Method Comparing Model Performances Graphically
Jim,
Thank you for the comments and paper.
In fact, the productivity measure that was used is a function of some other independent variables that are not productivity measures. However, I used lasso regression, a regularized technique, to take care of any possible multicollinearity and overfitting. Moreover, I used cross-validation with 10 folds to avoid any possible overfitting also.
Amir
Thank you Jim. I meant the coefficient value that I got from the statistical model is in opposition to the plot and I don't have any answer for that. I modified the question's explanations though.
The total number of observations is about 132 plants of which 22 are Japanese companies and the rest are American. I am thinking the low number of observations for Japanese plants could be a reason, am I right? Any feedback would be appreciated.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling