In a simple linear regression sometimes the intercept is not significant, and the question is whether you would keep the intercept term or not. What would influence your decision? The sample size? The p-value of the t test for the intercept? Additional information about the experiment that produced the data?
The first reaction I have is to ask you the question you asked us!
Because the answer will first of all depend on the model. There are some (very few!) processes where we expect that the predicted variable will be zero when all the predictors are zero. So unless there is a a priori argument that the intercept is redundant, I cannot imagine dropping it.
However, remember that having a theoretical intercept at (0,0) is not the same as having a sample intercept at zero zero. For this reason, I would be reluctant to get rid of the intercept even when theory was on my side.
Usually, with intercept, you mean b0 from the y = b0 + b1x, where the main analytic interest is if the term b1 is significant (is x associated with a significant change in y). Now, bx is the gradient of the slope, unstandardised, beta = a 1 unit change in x = a 1 unit change in y. The intercept indicates where the slope of X starts on Y. So, if direct interpretation of the effect size is important, on the scale of X and Y, the intercept should be included, but its always better to visualise the relationship on a graph. If it starts at 0, maybe it doesn't add much.
You should never remove terms from a model only because they're not significant. Only remove a term if you have a specific reason to do so (e.g., if you don't have sufficient data to fit a model with that term, or if you're trying to fit a "parsimonious" model that makes decent predictions with as few predictors as possible). If you don't have a specific reason to remove the intercept (there rarely is a reason to do so), then don't remove it.
Why would you ask whether or not to keep the intercept is you modeled it?
Non-significance just means the data is inconclusive: the estimate from the data should not be interpreted - not even its sign, because the data was insufficient to provide enough information about that coefficient. Nevertheless, the term can play an important role in the model, it can have a huge impact on the estimate and interpretation of other coefficients as well as on prediction intervals, so removing the intercept just because of a "not so low P-value" is an extremely dangerous, if not stupid thing to do.
Having nothig else but the data and trying to find a simple model means that one needs to judge if the removal of the intercept has some relevant effect. That could be precision and accuracy (or bias) of predictions or precision and accuracy (bias) in the estimates of the other coefficients. This should be checked. If even this is not possible, one may have a look on the confidence interval or posterior predictive interval of the intercept. If it does not include any values that might be relevant, the size of the intercept might be assumed 0 without being "too wrong".* The problem remains that one has to say what "relevant" means in the particular case.
The golden solution is thus clearly to acknowledge the "additional information about the experiment that produced the data". If the model with without intercept would make sense, scientifically, then the model should not include an intercept. If the intercept has a scientific justification, the model should include the intercept (irrespective of an estimate and a P-value of that estimate). What leads me back my initial (rhetoric) question...
------
* interestingly, this might be seen by a highly significant P-value, in conjuction with an irrelevantly small estimate. The tiny P-value indicates that the confidence interval is "relatively far away from 0", and if the centre of this interval is close to 0, the whole interval will be close to zero and very tight, likely excluding values that are relevantly different to zero. Thus, it is a high significance (not a low one) that should make one think of the possibility to remove the intercept (or any other coefficient) from the model!
The first reaction I have is to ask you the question you asked us!
Because the answer will first of all depend on the model. There are some (very few!) processes where we expect that the predicted variable will be zero when all the predictors are zero. So unless there is a a priori argument that the intercept is redundant, I cannot imagine dropping it.
However, remember that having a theoretical intercept at (0,0) is not the same as having a sample intercept at zero zero. For this reason, I would be reluctant to get rid of the intercept even when theory was on my side.
Thank you for your answers. I was recently going over some reviews of a manuscript that I submitted for publication on some clinical application. It was not an experiment in which x=0 must result in y=0. I decided in the end to keep the intercept. Just as Jochen said above, a p-value that barely was greater than 0.05 did not convince me to use the no-intercept model.
Sometimes it makes sense to remove the interaction term. If the dose of insecticide is zero then the mortality is also zero. This simple experiment should not have an intercept, except that the experiment doesn't really work this way. There are two problems.
1) There is some control mortality. It wasn't caused by the insecticide. So one subtracts the control mortality from all the treatments.
2) I have an interest in sublethal effects. So I try doses that are smaller and smaller. I know that 1 mg/kg body weight will cause 5% mortality. So now I apply a 0.1 mg/kg body weight dose. I have only 5000 test subjects at this dose, but the mortality is so low that there is considerable error in this estimate even with a large sample size. I bias the model (residuals at low doses are highly skewed) if I force the model through zero.
So eventhough it makes sense that if x=0 then y=0, it may not make sense to force the model through this point.
I thought of another case: I am looking at a dose response model. I have doses 1 mg/kg through 1 gram/kg. This represents mortality of 30% at the low end and 73% at the high end. While dose=0 should equate to response=0, there is such a large gap without any data between zero and the next nearest dose that forcing the model through 0 is a really bad idea.
No. I work with pesticides for either weed or insect control. In general we lag behind medical approaches. There is a wonderful herbicide called glyphosate (Roundup) that has a fun dose-response curve. At very low doses there is no effect. At low doses it stimulates plant growth. As dose increases further, it acts as a very powerful herbicide. At high doses it burns the plant causing localized necrosis. However, the herbicide is then trapped on the localized lesions and you get reduced plant mortality with higher doses. The plant may drop all of its leaves, but it recovers. The part of this response curve that you see depends on the range of doses, and how you measure efficacy: mortality, seed biomass, total plant biomass, etc... I have only ever worked with glyphosate in the optimal herbicide range, and not tried to model the entire curve in one pass.
One common problem that we have is an insect that weighs 0.0001 grams and accuratly measuring that weight takes specialized equipment.
It is unusual to tamper with results of models. In a regression analysis for example, the intercept value is where the model crosses the dependent variable axis.
It is basically the value of the dependent variable when the independent variable is zero. So intercepts should not be tampered with.
Raid, the issue is that non-retention of the intercept would imply that the model passes through the Cartesian coordinates origin. In effect. both the dependent and independent variables would have zero values at that point in time. I will recommend retention of the intercept value, however small it may be
Dear Raid, a reviewer normally has a right to decide on an article he reviews. However, he or she need not for the sake of simplifying a model change the mathematical meaning of the model..
Many other reviewers will definitely didagree with him or her. Models should be interpreted exactly the way they come.out as outputs of honest analysis.
Dear Raid,
The intercepts is a way to catch the relation between the DV and what is not matched by the IVs. Thus, I am always very reluctant to drop it as long as there are no very good reasons to argue that the IVs catch the largest majority of the aspects that affect the DV. If you are asked to do it by the reviewer, my suggestion is to include both models or at least to comment on any difference you face when you run the model with ad without the constant term.
Hope this helps
Andrea
Dear Raid, every data set gives rise to one and only one model. Any adjustments are just regarded as comments to guide the reader. So the model without the intercept should only be added as a comment.
.
Raid -
"When would you keep the intercept?" also brings up the question "When would you not keep the intercept?" and my decades of experience with establishment surveys for official statistics is consistent with what is found in Brewer, KRW (2002), Combined survey sampling inference: Weighing Basu's elephants, Arnold: London and Oxford University Press, on page 110. There, Ken Brewer, the internationally renowned 2013 winner of the Waksberg Award, says that often the best auxiliary variable available in a survey sampling application is proportional to the variable of interest, and that the relationship is a straight line through the origin.
In Cochran, W.G(1977), Sampling Techniques, 3rd ed., John Wiley & Sons, on the top of page 160, he also has strong words of support for such ratio estimation, but his comments are with regard to cases where a graph would indicate such a relationship.
Ken says this is an especially important consideration when the range of that auxiliary/supplementary/regressor variable is relatively limited, which he notes often happens when we use stratification by size. (Again, that is on page 110 in that book.)
Ratio estimation is an often useful technique for survey sampling, especially for highly skewed establishment sample surveys. Such uses are the basis of the following:
https://www.researchgate.net/publication/319434665_Comparison_of_Model-Based_to_Design-Based_Ratio_Estimators
Also, examples of such simple prediction (usually one regressor and no intercept term) are found at the US Energy Information Administration (EIA):
https://www.researchgate.net/publication/319914742_Quasi-Cutoff_Sampling_and_the_Classical_Ratio_Estimator_-_Application_to_Establishment_Surveys_for_Official_Statistics_at_the_US_Energy_Information_Administration_-_Historical_Development
Cheers - Jim
There is one case where you never have an intercept because zero does not exist. A mixture experiment. Say I have an insecticide. In its pure form it is essentially non-toxic because it cannot get into the insect. I add a solvent, and a soap. What mix of these products will make for the best product? In a 100 ml volume I could have 100% toxin, 100% soap, or 100% solvent. I can also make blends. In this experiment the "intercept" is used as error.
This model can be used elsewhere. I have a trebuchet. The throwing arm is 2 meters. I can locate the pivot point anywhere along this length I want. What is the best place to locate the pivot point?
"There is one case where you never have an intercept because zero does not exist." - ?
The intercept is, I think, relating to a model. It does not matter if a predictor value of zero practically exists.
You may see the Linewaver-Burk plot (https://en.wikipedia.org/wiki/Lineweaver%E2%80%93Burk_plot) used to analyze enzyme kinetics as one example: the x-axis is the resiprocal of the substrate concentration. The intercept of the model is the reciprocal of the reaction velocity for an infinite substrate concentration. This is impossible, yet does the intercept (of the model) refer to a meaningful characteristic of the kinetic (the reciprocal of the maximum reaction velocity). It is also the case here that the model intercept cannot be zero, as this would indicate an infinite reaction velocity, what is impossible.
I believe you should keep or not the intercept depends on the inference that you may wish. Data driven p-values are dangerous stuff, as many people said here. If you wish a model parametrization for inferring deviation of some factor in a population based way, you should keep the intercept, if not, remove it, simple like that. I have conducting a project (Bayesian cancer incidence meta-analysis) that we preferred to remove the intercept, and let the spatial and casual factors to fight to each other to explain the response, and the density of the data generated by the models showed nice properties in relation to the density of the observed data (Figure attached).
If it was useful, please recommend for others,
In both statistical model and in the experiment there is no zero. I can have a mixture-amount experiment. So my trebuchet throwing arm the length can be 0 meters to hundreds of meters. For any fixed length, 100% of the throwing arm can be on the left side of the pivot point, or 100% of the length can be on the right side of the pivot point. The pivot point can also be in the middle (50%-50%), but there is no location where all of the throwing arm is on both the left and right side of the pivot point. Likewise there is no point where none of the throwing arm is on the left side and none of the throwing arm is on the right side.
Here is another example: http://reliawiki.org/index.php/Mixture_Design
In the simplest sense, keep the intercept. A line has an intercept and slope for a reason. Without the intercept your line can float anywhere. How predictive is that?
Of course if there are theoretical reasons that are interesting and testable regarding the intercept, such as it must be zero or some other number, then forcing this might be necessary.
But I like the data to tell their story, not mine.
If y=a + bx, and a is far away from zero for all observations in the experiment, and there is no theoretical foundation for having y=0 when x=0, then I will most likely keep the intercept term in the model. I referred above somewhere to a "reviewer" who asked about removing the intercept term from the model. That manuscript was accepted for publication a couple of days ago, and I seem to have provided a good enough argument for keeping the intercept. I had dose of some drug as Y, and trough level in the blood as X. Trough levels were mostly between 12-18.
The arguments are very educative and can be summarized as follows:
1. Never remove the intercept term unless there is theoretical appeal to do so.
2. Do not remove variables only on the basis of their p-value contribution to a model.
3. Where there is the instinct to remove any term, particularly the intercept, the implication of doing so should be well understood.
However, I will like us to explain the the implication of retaining the intercept, apart from the theory of where it crosses the y-axis (0,0). That way the removal or not will make more meaning.
Note that some time series regression favors removal of the intercept.
In the mixture model you have y=ax1+bx2, where x1+x2 equals a constant. In typical models x1 and x2 are independent. In this model, if you know x1 and x2 you know the constant, or if you know the constant and x2, you can calculate x1 exactly. This is a very specialized experimental design, and it has no intercept term in the model. It does not apply to the particular problem in the original question. However, it is a specialized case where the intercept is never present in the model.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size