Leanne, I suppose that the question of a "significant slope" here is irrelevant at all. Looking at your picture there are two questions coming into my mind:

1) what is the *difference* in slopes between control and treated sides? and

2) probably what is the *difference of slope differences* between June and August.

These differences should have been estimated. The point estimates are simply the differences as they might be calculated from the given individual slopes, but the confidence intervals (or p-values) for these differences cannot be directly determined from the given results. This can all be calculated as interactions in in two-(or three-)factorial models.

I absolutely agree with you! This was a strange request. Not showing the regression line because the slope-test is not significant would be equivalent to omit all the bars (shwoing the group menas) in a barplot because the differences were not significant.

However, I would suggest to show the confidence bands (equivalent to error bars in a barchart showing the confidence intervals).

I agree the confidence bands are the best way to communicate the strength of the relationship even when non-significatn at a particular alpha level.

That said multiple confidence bands tend to get a bit messy, so at times I have presented a separate graphic plotting a group identifier on the horizontal axis and the regression coefficient estimate on the vertical with confidence limits shown as error bars. By also including the horizontal line at 1.0 the reader can judge for themselves the degree to which the estimated slope differs from zero. Also it is frequently interesting to test Beta=( some specified constant) and this line can also be plotted on the same graphic.

Not 100% sure if this is your case, but some reviewers are very strict and do not tolerate any ambiguity. Plus it may be that the reviewers and the potential readers are not that statistically savvy. Have you explained a bit the graphs in the text?

I suggest a flexible workaround: split the graph into two plots, one with the significant line and the other with the non-significant lines.

About confidence bands, it's a good idea but what if a part of the audience is not that familiar with it? Some may interpret the existence of confidence bands as an implicit statement that these regressions are significant; this is not uncommon in Northern cultures, e.g. Canada.

Usually in the review process you need to convince the reviewers and editor that your approach is better, and, if not, make a reasonable compromise that suits both parties.

You might also used dashed lines for the ns regressions, which would emphasize their nonsignificance and their difference from the significant lines. I have to admit, I just made this up; but it might be communicative.

Please, can someone kindly explain me why we present mean values as estimates for groups (i.e. for a categorical predictor) -independent of any significant differences between the groups!- but if the predictor is continuous we should not anymore indicate the mean estimate (i.e. the regression line) or distinguish the line styles depending on some significance criterium?

Any why are so many people willing to adopt good science (with statistics being a vital part of it) to bad knowledge or understanding of potential readers? Won't new scientists being forced to read such papers over and over again become again such statistically under-educated reviewers making such funny requests...

Sorry.

I'm not sure if there is some "rule" regarding this. However, I completely agree with you that it makes sense to show all of the lines irrespective of whether they are significant or not. Unless of course, the figure gets too messy and crowded, as someone rightly pointed out in this discussion. Otherwise, I feel it's best to show all the lines.

There is no *rule* about this, but I can imagine that many journal editors have invented rules for themselves (especially journal editors who don't know much about statistics themselves, but do publish many papers with a lot of statistics in them). Anyone can do what they like. The important thing in a scientific publication is that you explain what you have done so as to make your research maximally reproducible and that you take care that you do not "tempt" readers to draw conclusions which are not justified. This last could be the "thinking" (if there was any) behind the editor's rules. An even less statistically skilled reader than the editor might imagine that the regression lines you draw on a graph actually mean something rather precise. If the text makes clear what the status of those lines was, I cannot imagine why such "protection" of incompetent readers is needed. After all, a scientific publication in a reputable journal has to be intelligible to the intended readers, who are themselves scientists - not to the whole world.

I see no reason why not to present nonsignificant regression, for example, if the visual relationship is taking place, and the variance is large, so the significance is not obtained. Strictly speaking, it would be necessary to compare the slope angles.

Generally, it may be good to mention the failed or non significant attempts but beyond that, non significant, in essence means non informative and cannot be used for decision making.

Of curse you can put regression line even regression is not statistically significant, but is important to say that in this case is only for descriptive purpose not to make inference

In general, I suggest to represent a linear/nonlinear, parametric/nonparametric regression together with their uncertainty. That is with the appropriate confidence or credibility bands.

Using modern graphic tools this can be done by endearing shadows and colors.

If a linear regression is non significant this is clearly seen by its confidence bands.

It sounds like you showed a lot to convince the reader that soem of the regressionsa re not significant. Perhaps an alternative is to show y = ybar heavy lines with dotted lines for the regression in those cases where the slope was not significant.

Sorry, I didn't think to include the graphs here to help illustrate the question. Here is a screen cap of the graphs. The grey line on b) of the graph is the only significant relationship. I have not yet added confidence intervals, as has been suggested.

The caption of the graph is as follows:

Figure 4.5. Linear regressions of the emergence of all Chironomidae and the cover of emergent macrophytes for a) after treatment-late June 2009, n=22,22, α=0.058 and b) after-treatment-August 2009, n=13,13, α=0.145. Macrophyte survey data was not available for the spring of 2009 or 2010, to correspond to the first and last sampling of Chironomidae, as the plant community was too immature for identification. The critical effect size for all tests was R2=0.40.

Sergio, as you can see, my alpha is set at 0.058, and that I have explicitly stated a critical effect size based on the methodology of Mudge et al 2012. See attached publication for more on the alpha optimization methods. As such, this method incorporates practical significance (a meaningful effect size) into statistical significance. An R2=0.400 is equivalent to detecting an effect that would fall outside of 90% of the variation around the mean of the control data.

Article Setting an Optimal α That Minimizes Errors in Null Hypothesi...

Dear Researcher,

When you use simple regression or multiple regression and found no factor have a significant effect you can ignore the scatter plot. The aim of regression equation is to predict of dependent variable values depending on independent values. Moreover when R square is low this mean that the variable or variables in the model did not describe the dependent variable correctly.

Leanne, I suppose that the question of a "significant slope" here is irrelevant at all. Looking at your picture there are two questions coming into my mind:

1) what is the *difference* in slopes between control and treated sides? and

2) probably what is the *difference of slope differences* between June and August.

These differences should have been estimated. The point estimates are simply the differences as they might be calculated from the given individual slopes, but the confidence intervals (or p-values) for these differences cannot be directly determined from the given results. This can all be calculated as interactions in in two-(or three-)factorial models.

Jochen, those are very good suggestions! Thanks! I think all of the suggestions here will make these graphs much better than they are. And all from a simple question about whether the lines were useful or not!

Firas, I have the graphs in the manuscript because they are concerning one of the main hypotheses of the study. In particular, they are in contradiction to popular theory. I have fairly reasonable statistical power to say that the lack of effect, in these wetlands, under these conditions is a real thing. In my opinion, any answer, significant or not, to a stated hypothesis, is important to address. I am wary of publication bias, and I think non-significance is just as important as significance. They are both interesting to me.

I entirely agree with Richard Gill's answer. Well put.

significant or non-significant is related to parameters inference. Reallity say us information is usefull in many ways. I remember experiments with animals where regression lines have not statistic signification but when you see data you can note some relation or functionality between variables. So I think we are comfortable with significant regression lines but whe often have to deal with reallity.

When I read your question then I read the answers (1, 2 and 3) in your explanation

regards

Leanne I share the comments of Jochen, which arises after you showed the scatter plots. This is a case-control study and testing the difference of the slopes is the point, which can be done easily with standard piece wise linear regression .

Dear Leanne, I agree that you should leave only the significant results (if there are any) and (as some previous answers, mostly James) I think you should test your regression parameters and the regression itself. If you nead help on that, I can provide Excel sheet with example (but to me it seems it's quite straightforward and well explained in Help), since my students also have to such testing.

As I understand it the regression line depicts the best fit line for prediction and if the regression is insignificant my understanding will be that there is no best fit line to be shown and that could be the reason the Editor is asking them to be removed.

You have decided on a significance level for whether or not your regression is significant. By this decision rule, three of your regressions are insignificant, or in other terms, the regression lines don't exist, at least not with your data. Following this reasoning, you could draw any lines on the graphs and they are just as relevant as the insignificant ones. This, I believe, is why you have been asked to remove them. These insignifcant regression lines by your rules do not exist!

A best fit remains a best fit - independent of a p-value. What if the tested null was different to "slope=0" ? Then suddenly other lines wouldn't exist? This is nonsense.

Not the regression is significant or insignificant. Just the difference of a parameter value to the null value (whatever it may be) can be (in)significant.

The only thing(!) that should not exist for an insignificant test result is a decision to reject the null hypothesis.

I will again give an example to clarify the point: Look at the average blood pressures of males and females, what is considered to be significantly different. All fine. But what about the average blood pressure in american and european people? Assuming they are not significantly different, you say that these averages simply do not exist and one could take any arbitrary value for the average blood pressures. (Note that the regression line is nothing but an average that is not depending on a group (categorical predictor) but on a continuous predictor)

If you take the population of the US and the population of Europe and find a difference, then it exists. If you take a sample of each population and rely on statistical significance to say whether a difference exits, and the difference is insignifcant according to the decision rule you have set, then you retain the null hypothesis; i.e, there is no difference.

We are getting off track here. In the very least, there is a danger of misleading the reader by showing an insignificant regression line. I agree with you that it is paramount to explain what you did and why. Beyond that, it is a judgment call as to how you present your results. If I were editor, I would want a very strong argument as to why you need to show an insignificant regression line. It is not a clear yes/no situation.

Best of luck with the R&R!

James, "then you retain the null hypothesis; i.e, there is no difference." is typically WRONG (unless you made a power analysis, and even then you can only state that the effect is unlikely to be relevant)! This misconception arises because Fisher's significance testing (requiring only to state H0) and Neymans hypothesis testing (requiring to state *both* H0 and HA) are confused and meshed up in too many statistics text books (kind of "null hypothesis significance testing", NHST). Rejecting H0 based on some a priori set alpha is *not* Fisher (because there is no reasonable alpha to be set a priori and generally), and it is not *yet* Neyman (because HA is not defined and the power is not specified), but it clearly aims to control a type-I error rate, what is clearly Neyman (Fisher is about inference, Neyman is about controlling error rates). But without knowing the power there can't be made any confident conclusion at all when H0 can't be rejected.

And i still wonder why it is NOT misleading the reader to show mean values of groups that are not significantly different.

I absolutely agree with Richard's Gill comments. However, if there is such a suggestion of referee or editor, you have to agree with it.

I would agree with not drawing a regression line if the regression coefficients are statistically insignificant. This is simply because the data set and the regression model you have do not have sufficient explanatory power to argue what drives changes in the variable being explained. This may be because there is simply no statistically significant causal relationship between the variables under investigation (and then there is no point in drawing a regression line if it does not exist / does not pass significance tests) or perhaps there is a problem with the data set or the regression model.

This is clearly a standard analysis of covariance with interest in the difference in slopes--interaction between groups. This cannot be illustrated without plotting lines and providing test of significance for the interaction.

V. Ivanov.....hoping you are being sarcastic?

I agree not to put insignificant regression lines from the same graph and you can create another graph with sufficient explanation in the text to show the difference between the two graphs and not to confuse researchers with no background in statistics.

No!

What is the essence of doing regression analysis? To develop a predictive model and until the model is validated both internally and externally it is not a model. So what will be the essence of including data sets which do not satisfy the test of goodness of fit in the graph.Afterall they are not predictive models and have no theoretical and practical siginificance.

Abbas: "not to confuse researchers with no background in statistics" - very funny! You made my day!

Dear Leanne,

From the background information you have given it looks like you have at least 4 subgroups which you are comparing.

If this is the case, I agree with Jochen Wilhelm and John Kern that the difference in slopes between the groups is the question that can be directly answered with regression analysis. For this, you should perform one regression model taking the four subgroups together, instead of one regression for each subgroup separately. Include an interaction term between the group variable indicating the different groups and your explanatory variable. If this interaction term is significant, there are differences in slopes between the subgroups. As you mention that one group has a significant slope in its individual regression model and the others not, one can expect this will be the case. If there are differences between the groups, then have a look at the estimated slopes for each group.

Conclusions on the significance of estimated regression model terms are only valid if the regression model assumptions are fulfilled. James Nzokah has listed these assumptions. If you are not familiar with this kind of analysis, talk to a colleague with experience or a statistician.

If the regression model is valid, you can present a graph with the data and regression lines for all groups even if not all groups have a slope significantly different from zero. Clearly indicate for which groups the slope is significant and mention that these results are obtained on one single regression model.

I find your motivation for presenting the graph OK and would keep the not-significant lines (as long as there is at least one significant).

Best regards, Geert.

Just a nother note: In your example (as Geert again put nicely) the key question is to compare the slopes of two lines. It is NOT the question whether or not the slope of any of tehse two lines is nignificant but rather if the *difference* of the slopes of these two lines is significant. Given that, it would be stupid to omit drawing one of these lines just because its slope is non-significantly different to zero.

I think that the the opinion given by Jochen Wilheim is the best option for your problem. You should analyze if the difference of slopes are significative or if the slopes are or not equal. A similar situation was analyzed by Ruberg & Stegeman (Biometrics 1991, 47, 1059-1069) and Oliva et al., Drug Development & Industrial Pharmacy, 29, 513-521, 2003. In this last paper,you can find the statistical model in the page 515 .

Graphs of regressions are not meant for only regression lines that are statistically significant. The important thing is to present the results and discuss them as they are. The graphs are mere statistical tools for illustrations. Therefore, I do not see anything wrong in giving graphical representations of non-statistically significant regression lines.

As much as their are statistical reasons to include or exclude lines and parameters in a model, you also should think about your hypotheses and the story that you'd like to tell about the data. I'm working on a model with a whole lot of non-significant parameters that I'm still including. I'm doing it because they are all relevant to my biological hypotheses about the data.

As a clinical researcher not extremely good in statistics I would like to add that to me, the figure with the regression line is the most interesting information. I have seen a lot of “significant” regression lines that are almost flat and without clinical significance, but are statistically significant because of the high number of observations.