For some reason people tend to do this when looking at long-term data, especially for climate studies.  The point is to attempt to illustrate global warming.  Based on the top graph you have shown, it is hard to deduce a linear trend in the data with strong correlation.  Often the region being investigated, or in some cases the entire globe, will have fluctuations that exceed the rate of the trend (see attached).  Though I believe there should be a better way to illustrate this point, it is somewhat of an excepted approach in the scientific community when evaluating climate trends.  However, applying a linear regression to this kind of bar graph is a bit crude in my opinion.

Strange, two different people with the same answer!?

Addendum: Unfortunately, both answers (including the first brilliant answer) were deleted by RG because the second answer was a simple copy of the first answer.

For some reason people tend to do this when looking at long-term data, especially for climate studies.  The point is to attempt to illustrate global warming.  Based on the top graph you have shown, it is hard to deduce a linear trend in the data with strong correlation.  Often the region being investigated, or in some cases the entire globe, will have fluctuations that exceed the rate of the trend (see attached).  Though I believe there should be a better way to illustrate this point, it is somewhat of an excepted approach in the scientific community when evaluating climate trends.  However, applying a linear regression to this kind of bar graph is a bit crude in my opinion.

Dear Klaus,

I had wrote that data on Murmansk region are suspect or doubtful because of two resons.

1. If we use methods like Kriging and we want have a good interpolatin result in some point x, then we must have we need to have at least a few points inside the area surrounding our point and having a size of no more than 1-2 of the correlation radius (I suppose exponential or gaussian correlation function for the field) . If there is no points in this area, then we can not expect good results because of absence of significant statistical relationships between the values of the field at the desired point, and the remote points which are used for interpolation. If we have only one point then results may be good if and only if the measurement at this point is sufficiently accurate.

2. As shown in the second picture, the amount of data at different times very much changed. In the period from the early 1960s until the mid-1990s it had a lot of measurements, from the second half of 1990 - a little. If you will process a whole time serie, the weight of the first data will be substantially greater than the second.Therefore, the obtained results will mainly reflect the variability of the characteristics of the first period and distort the processes that took place in the second periiod observations.

Best regards,

Yuriy

Klaus -

Your question referred to comparing linear regression slopes, but the details seem a great deal more involved.  Climatology is not an area of expertise for me, but your description appears to me to paint a picture of circumstances in your discipline where there might be substantial danger of comparing 'apples to oranges,' which would provide statistical bias, and that even if it is possible to make direct comparisons, the statistical variance is quite high, measurement errors may be substantial, and determining a reasonable length of time over which one might approximate by applying a linear regression seems problematic.

I looked for the paper to which you refer, but neither author is on ResearchGate, and I did not try to obtain a copy, so can you tell us if linear regressions such as the one Edward showed would be similar in form to the first graph you showed?  How do the brown and red bars differ?  Are these continuous data, as I assume?

It sounds like Yuriy has a good point about data weighting, but I'm not certain of the way in which you constitute your data.

This looks like an application for time series analyses and forecasting such as ARIMA, exponential smoothing, or some other time series methodology.  However, even though perhaps "crude," as noted by Edward, it does appear that linear regression might possibly be used for some discussion.  A useful measure might be the estimated variance of the prediction error, as seen in econometrics.  Its square root in SAS PROC REG,  for example, is provided as STDI.  You mentioned standard errors. Were you talking about this, for individual predictions?  You could be looking at the standard errors for the slopes, b.  It depends on what your goals may be.

I'll assume that you mainly want to compare slopes, and the standard error of b is important in each case.  You can put a standard error about the difference in two slopes (assuming you aren't comparing 'apples to oranges,' and assuming intercepts don't enter into this).  If zero is included in a reasonable confidence interval around such a point estimate of the difference between slopes, then you do not have evidence that there is such a difference between those two slopes. (That does not mean that the slopes are not substantially different, as you may just not have enough data. A huge advantage of looking at such a confidence interval, however, over say a p-value, is that you get a more complete picture as to what your data may support. Without a power analysis or other sensitivity analysis, a p-value will tend to reject a null hypothesis, just by having a larger sample size, and 'accept' the same null hypothesis,  simply by having a smaller sample size.)

You might also study the sensitivity of your results by randomly removing parts - or here maybe even sections - of your data, to see how much the results change.  (You might want to research "jackknife and bootstrap resampling methods.") 

I hope some of this relates to your question. It was fun thinking about it anyway.  If you provide any follow-up later, that could be interesting.

Cheers - Jim 

Hi James,

thank you for your answer. To clarify, these data and slopes are not my work, but I read this paper and found the results not convincing. For the figure with the red and brown bars, the authors used data which were regionalized (only a few stations in the whole region, the number changed considerably over the time period 1951-2012) with Universal Kriging on a high resolution grid (0.25 °). They averaged these regionalized temperatures and made linear regression for the whole region and 9 smaller subregions. But they did not show any errors of the Universal Kriging (which are part of the E-OBS-data) nor did they compute confidence intervals for the slopes. But, at least, they gave p-values for the H0: trend is zero; H1: trend is not zero (but not for the comparisons of the different trends). - You can find more details in my Open Review for this paper (if you are able to find the button to get to this review; after the change of the Web layout on RG the reviews can be found only with difficulties). There I digitized the data of the figure with the red and brown bars and made some tests with these data.

Cheers,

Klaus

P. S.: Here is the link to the review:

https://www.researchgate.net/publication/265054773_Climatic_warming_above_the_Arctic_Circle_are_there_trends_in_timing_and_length_of_the_thermal_growing_season_in_Murmansk_Region_(Russia)_between_1951_and_2012/reviews

Article Climatic warming above the Arctic Circle: are there trends i...

Klaus -

I found and read your review. (I did not even know these reviews were out there.  Is there a place to go to see all reviews?)  I completely agree that information on uncertainties is vital, and metadata (information about the data and how it was collected) would all be crucial to effective science. - Also, we cannot rely on isolated p-values, as noted in the RG abstract and my letter to The American Statistician, found through the attached link.  In work I did with continuous data and naturally occurring heteroscedasticity,  I included the heteroscedasticity in the error structure,  and I think confidence intervals are far more meaningful in such cases.

As I stated, Climatology is not a subject area for me, but it sounds like your concern for the uncertainty due to the Kriging step is well justified.  In statistics, unfortunately, we often concentrate on one kind of error, and ignore others. This can be a big mistake, especially when the type of error being ignored may be substantial. (Analogously, in survey statistics,  people often look at sampling error, and ignore nonsampling error, such as measurement error, which can be a bigger problem - and is a major reason, in addition to cost, to sample rather than take a census all of the time, in the first place!  -  One needs to look at TSE, total survey error.)  

I see in your review that you have the following: "Even if the trends in the article are real, they could not be compared because of the very large confidence intervals, which are much greater than the differences in the trends."  This is an important comment.  You also mention large discrepancies in (between?) various studies. It seems to me that your's is a discipline in which one may expect a low signal-to-noise ratio, so-to-speak, and that preponderance of study results and attention to uncertainties is very important - as in all disciplines, but apparently very strongly so in Climatology.  Therefore, studies should be as open to constructive feedback as possible, if optimal progress through cooperation may be approached.  (Note that we need to be cognizant of each other's egos here.) 

Very interesting. Thank you.  -  Jim

Article Practical Interpretation of Hypothesis Tests - letter to the...

Hi Jim,

I hope that the majority of the authors which are using hypothesis tests are aware of the facts described in short article which you have attached above! But, unfortunately, I know many scientists which erroneously believe that (1-p) is the probability that the H1 is true.

Concerning your question "(I did not even know these reviews were out there. Is there a place to go to see all reviews?)..": No, I do not know such a place. But I have written an e-mail today to the RG support with a request for better highlighting the Open Reviews which are now almost invisible (after the Web layout update).

Cheers,

Klaus

Hello Klaus -

It is interesting that your attempt to confirm results from a scientific study/paper, should come at this time.  I just received my hardcopy of the latest issue of "Significance" (June 2015, vol.12, no. 3), a joint publication of the American Statistical Association  (ASA), and the Royal Statistical Society  (RSS).  I receive this magazine/journal as a part of my  membership in the ASA, but I find its very name part of the problem we both noted with regard to p-values.  Many words used as jargon in a discipline can be misleading, but "significance," as used in statistics, may, I believe, be the most egregious offender of all.  It is commonly misinterpreted to mean that a p-value contains all the information needed to make a decision, when in fact a single, isolated p-value is basically useless.  When trying to show information useful in decision-making, I have long used the word "substantial," since the word "significant" has been usurped.

This magazine may purport some level of outreach, but it is essentially written by statisticians, and for statisticians, so it is no wonder that an article in the recently published issue should point to the need for better understanding of statistical science as the key to improving the quality of studies such as the one being discussed by your RG question.  I believe that is true to an extent, but I think that the state of statistical science is partially to blame, notably when statisticians argue over the best way to estimate a p-value, when what is really important to implementation in various scientific disciplines is the practical interpretation of statistics. A little common sense could 'go a long way.' 

Here is a citation for the article in Significance to which I refer: 

"The reproducibility crisis in science: A statistical counterattack,"

Roger Peng

Article first published online: 15 JUN 2015

DOI: 10.1111/j.1740-9713.2015.00827.x

© 2015 The Royal Statistical Society

Significance

Volume 12, Issue 3, pages 30–32, June 2015

A Google Internet search shows this:

"The reproducibility crisis in science: A statistical counterattack ...

onlinelibrary.wiley.com › doi › abstract

Jun 15, 2015 - ... are neither replicable nor reproducible. It is time to invest in statistics education, says Roger Peng."

In that article, it says that "...data from publications have not always been available for inspection and reanalysis," which is a huge problem, but that even when a study is reproducible, he gives two important examples where this led to the identification of errors that had serious implications for the results that, if followed, would lead to erroneous decisions. Peng writes that "Ultimately, the problem was the poor or questionable quality of the original analysis."  The two examples he gives are very influential examples involving researchers and/or institutions of substantial repute.

Among Peng's conclusions, he states that "It is also critical that statisticians bring to bear their history of developing rigorous methods to the area of data science."  However, as I said, Significance is a magazine written by statisticians, for statisticians, and I find this statement somewhat self-serving.  The historical development of statistics is as riddled with haphazard order of discoveries, and faux pas as any other, with many  concepts (such as 'significance!') which could have been better - in hindsight - developed in other ways. There has been plenty of debate and healthy, as well as unhealthy, disagreement between those developing the field. 

There are at least a couple of other comments in this issue of Significance, germane to your question and/or related RG discussions which may make it worthwhile to obtain a copy of the entire issue. (I never thought I would say that, as I have generally virtually ignored this magazine with the ridiculous title, "Significance," as the concept of 'significance' is so commonly misused by almost everyone,  including statisticians.) 

Best wishes - Jim 

http://onlinelibrary.wiley.com/doi/10.1111/j.1740-9713.2015.00827.x/abstract

Hi Jim,

very interesting contribution to this problem! - I find that one of the most serious problems in statistics is the use of statistical software packages by people who know only the basics of statistics if any. These packages enable them to apply very sophisticated tests but they do not know what are the preconditions and how to interpret the results. Thus I suggest to get a "statistics license" (like a driving license) before people are allowed to buy statistics software.

But to be serious: Could the use of "Bayesian statistics" be a remedy to overcome the problems with the p-values?

Cheers - Klaus

Actually, in the Peng article, he mentioned the need for statistical software that could not be so easily misused. 

Back in 1990 or 1991, I attended a workshop/seminar of sorts, from which the (US) National Academies Press published these proceedings:

The Future of Statistical Software: Proceedings of a Forum

http://www.nap.edu/catalog/1910/the-future-of-statistical-software-proceedings-of-a-forum

I contributed a thought on page 82 with regard to hypothesis tests, and how statistical software should be improved to reduce misuse.  I am not aware of any "canned" software today that has yet used my suggestion.

As for Bayesian statistics, I could not provide a completely fair assessment, as I only have a superficial acquaintance with it, at best. There are many who love it, and I think many more who do not.  My impression, in light of the context here is that it would be even easier to misuse, accidentally or otherwise.  That is, I think there would be the same attendant problems, and more. 

In the world of Official Statistics, from which I retired last August, I think that justifying results using Bayesian statistics would be very problematic.  I used sampling and estimation techniques for establishment surveys that I proved were very useful, yet because other sample surveys, particularly household surveys, used something more conventional - though I showed them not to be as effective in my applications - there is still resistance to methodology others and I proved superior.   For Bayesian estimation to be accepted. I would guess, would be even harder to 'sell.' 

Worse, it is my limited understanding - a Bayesian statistician may want to respond to this if any of them see this - that by using different 'priors,' different Bayesian analysts may arrive at different results, using the same test data.  I imagine that would be a source of confusion and mistrust, if anything else.  But perhaps a Bayesian may respond to enlighten us, should they be able to argue the case. 

But to answer your question, "Could the use of 'Bayesian statistics' be a remedy to overcome the problems with the p-values?" I think it safe to say "No."   That would just be another way to estimate a p-value.  The basic problem is that a p-value is a function of sample size, and by not specifying another specific hypothesis against which you compare, to see which is better supported by the data, you cannot make an informed decision. 

Hi Jim,

I have just read your contribution to the proceedings which you mentioned above. As far as I know one needs the "real, unknown mean value "my"" to derive the type II error. Do you assume "my" = (sample mean)? And how does the knowledge of this error, resp. of the power of the test ( = 1 - type II error ), helps me to decide if the H0 is wrong?

Best wishes - Klaus

Analogous to a confidence interval, your point estimate  (what I think you were referring to as the sample mean) would lie between the null hypothesis and an alternative hypothesis.  The power and type II error are based on the  alternative hypothesis.

Though I long ago originated and developed a nonparametric hypothesis test for a special purpose, with programming help from a few others, and did use hypothesis tests extensively for awhile, I have not used an hypothesis test in many years.  However, I found that a paper I wrote in the 1980s has been saved in a proceedings and I have supplied a link to find the link to that entire proceedings, where I have another paper - the two papers coming from different years. 

So, if you want a detailed example of how to understand hypothesis test use, you could follow the attached link. The paper is from a conference and predates modern word processors, so please pardon the crudeness of the figures. 

Conference Paper On the Lehmann power analysis for the Wilcoxon rank sum test

You might also want to research the term, "effect size" on the Internet.   I just now checked to see what you might find, and this looked like it might be helpful:

 http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3444174/