I have a dilemma when doing bootstrapping method to examine my model:
I identified predictor A & B in my regression model as significant estimates.
However, it can be rational only when (A & B) are the same direction ( both positive or both negative). Otherwise, it is not consistent with previous literature.
Unfortunately, in my model, A & B are in the different direction (One is positive while the other is negative) in the regression model.
But bootstrapping method proved A & B are statistically significant factors (CI 95% in the same direction, with 0 excluded).
I don't know how to appropriately explain and interpret the result.
Please help.
As a scientist you can only report and use what your data tells you. However, it may be tell you something about your data collection, processing or analysis rather than the underlying hypothesis you are testing.
Go back and go through your methodology with a critical eye. Is there any steps done differently? Would the difference be able to explain what you see? Have you sampled a different population? Have you used the same equipment and procedures? Has your experimental regulation been adequate (especially seeking to eliminate any biases and minimise variability)? Have you tried something different in the analysis?
I find it useful to try and put myself in the position of a peer reviewer. For better effect, pretend it is your rival's paper and try and see where the holes are.
“I don’t see the logic of rejecting data just because they seem incredible.”- Fred Hoyle, Scientist and thinker
It really depends on the roughness of the previous literature. Maybe you can prove it wrong ?
If the literature can not be challenged, then follow @James Renwick Beattie advice. "Pretend it is your rival's paper and try and see where the holes are." This is an excellent advice !
Well, while I agree with what was said before, I would like to ask if you have used also other criteria when testing for variable importance (other than the bootstrap) like for instance simple analysis of p-values, best subset selection with the various Cp, AIC, BIC or adjusted R2 criteria, validation and cross validation, etc.
If the statistical model is inconsistent with the physical or expected model, there can also be a risk that the data you are using are not well representing the population under study.
So what I would do is:
1. Question your data
2. Question your model
3. And finally, quoting Sherlock Holmes :
"when you have eliminated the impossible, whatever remains, however improbable, must be the truth"
Good luck!
Have you tested the correlation between the predictors A & B?
From my experience with regression models, sometimes if two variables are strongly correlated it may happen that the regression model will give a positive weight for one of the predictor and a negative weight to the other, because of the strong correlation between them. I would suggest you test the two predictors one at a time (univariate models) to see if the effect without the "masking" of the other predictor is still significative and with different signs (one positive and one negative).
If 2 variables are correlated, then you should include only 1 of them in the model. Otherwise their collinearity will generate a paradox like you've described.
I have found it to be a fairly common situation for researchers to want all of their variables in the model - only for me to find that some of those variables were highly correlated with each other, causing one to have a positive coefficient and the other to have a negative coefficient. After performing a correlation analysis on the variables in question, if the correlation coefficients were very high for the variables in question, say .9 or higher, I would recommend that the researchers pick just one or the other.
lurking factor? https://en.wikipedia.org/wiki/Spurious_relationship
I'm not sure to understand what you mean by « A and B in the same direction » or « different directions », but what is certain is that if you assume a linear relationship, by definition it holds on all ranges of A and B. So, the simple fact that you have a rational relation only for some combinations of A and B rejects the linear model* as a possible model.
Based on that, fitting a linear model on your data can give any result, including by luck some meaningful one in some circumstances.
You do not specify which kind of regression you did, but if it was non-linear regression, nobody can answer without knowing which model you fitted...
*Note that "linear model" should be understood here as on A and B directly [that is, z = a * A + b * B + c + epsilon], but not on polynomials of A and B — here again, if you used such complex models, including interaction, you should give use details.
- Badges
- Science topic
- Similar topics
- Philosophy
- Esthetics
- Interpretation