The problem is not "trust", the problem is interpretation. We can certainly use observational data to find correlations that seem interesting. Although we cannot conclude that there is a functional link between these variables just from the data, we can still use this as providing a hint that there might be some kind of link (that may be further investigated in well-designed experiments, if possible), and there is usually a theory in which the finding is to be embedded. How plausible and consistent is all that? And finally, it may not always be interesting to think about functional relationship. A correlation may be used for prediction, and as long as the correlation-based predictions are better than mere guessing, it may be worth it, even if the correlation is actually spurious.

I have to admit that I don't fully understand the language in the question (maybe it is because I do not belong in the engineering/pure statistics/data science field), but I think that your question is, at the very least, similar to dealing with confounding in epidemiological research.

I am going to try framing the question and my response with a public example. Let's say that we are interested to determine whether smoking is truly associated with lung cancer, and we still do not know at this point that fact. In the resulting dataset, information on smoking status at the start of the study and of lung cancer status at the end of the follow-up period, together with other demographic and clinical information, are present for 1000 people - half of which were ever-smokers and the other half were truly never-smokers. Value 0 will refer to no lung cancer at the end of the follow-up period, and value 1 will refer to having lung cancer at the same point. Among the 500 non-smokers, N0 of them had lung cancer and M0 of them did not have lung cancer. Among the 500 smokers, N1 of them had lung cancer and M1 of them did not have lung cancer. The risk ratio or relative risk (RR) of lung cancer among smokers compared to non-smokers should be RR = (N1/[N1+M1])/(N0/[N0+M0]). The equation in your question, I think, refers to odds ratio rather than RR.

Anyway, going back to your question, dealing with confounding variable(s) that may lead to finding spurious correlation can be generally performed in the planning and/or analysis phases of the study. With the dataset considered, analysis phase-level assessment for confounding can be done through stratification, standardization, beyond-simple regression analyses or propensity scoring - all of which will depend on "secondary" data that has been collected together with the binary input data and the binary output data (in my example, such "secondary" data might include age, sex, body mass index, exposure to second-hand smoke, etc.). In all techniques, the suspected origins of the spurious correlation (basically the confounding variables) are controlled for in the analyses. For instance, if the crude/original RR is 3.0 (95% CI 2.5-3.5), and if the RR when stratification based on a suspect entity is performed becomes 1.0 (95% CI 0.5-1.5), then the entity might really be the cause for the spurious correlation (i.e., the RR = 3.0 that you initially found is due to spurious correlation).

Ultimately, however, identification or suspicion of confounding variables as the cause for the spurious correlation should be based on a priori/a posteriori knowledge & evidence and, perhaps, post hoc confirmation through another investigation. I think that statistics or statistical (non-)significance alone is not a consistent tool to detect relevant confounding - I won't be surprised that there will be instances wherein the attempt to detect spurious correlation would lead to spurious findings by itself as well.

Hello Sergey Porotsky. I'm not entirely sure I understand what you're asking, but perhaps this example of confounding will help. (IIRC, it comes from PDQ Epidemiology, or some other book by Norman & Streiner or Streiner & Norman.)

Imagine a 2x2 table with cells labelled a-d:

a=17 b=83

c= 8 d=92

Row variable = bifocal use (Yes vs No)

Column variable = nocturnal enuresis (Yes/No)

OR = (17*92)/(8*83)= 2.36, which suggests that bifocal use is associated with nocturnal enuresis.

But now stratify by age, making a cut at 60 years.

For age < 60

a=1 b=19

c=4 d=76

OR = (1*76)/(19*4)=1

For age >=60

a=16 b=64

c= 4 d=16

OR = (16*16)/(4*64)=1

The issue is that age confounds the association between bifocal use and nocturnal enuresis.

Is this the sort of thing you had in mind?

Well since the variables are binary you are not dealing with with least square s and pearson correlation any more. But rather logistic regression.. do you have any reason to believe that your model of spurious correlation even exists in this space. Looks like some experimental work is required to know if your question even makes sense in

this space.

Dear Aedrian Abrilla , hello and thanks a lot for quickly and detailed answer! First, about RR and Odds. Both your and my definitions of RR are right -:). But in my definition M0 is not “…of them did not have lung cancer”, but rather full amount of the non-smokers (500). Same for the M1 - in my definition it is full amount of the smokers (500). Please, tell me, what do you mean for “stratification”? If I understand well, you means separation of initial cohort for two sub-cohorts, according some third factor C – one sub-cohort with C=0 and other sub-cohort with C=1 (for example, C=0 is women and C=1 is men). And after this to calculate RR independently for these two sub-cohorts. Is it right or do you mean some else? Thanks for your reply beforehand. Regards, Sergey.

Dr. Sergey Porotsky, yes, you are right. I have to admit though that among the techniques that I mentioned, I only have experience in using stratification - and I believe that my said experience is quite basic. The example of Bruce Weaver is more comprehensive and illustrative, although I would add that the confidence intervals of the 3 point estimates should also be considered.

Thank you for pointing out the meanings of the N0/1 and M0/1 nomenclature in your question! They are duly noted and will perhaps be useful when I take advanced courses in pure statistics and computer/data science in the future.

Dear David Eugene Booth , hello! In my opinion, usually we don’t know a prior, exist “spurious correlation” or not. Let’s consider example from first answer (Aedrian Abrilla). Suppose, for factor “smoking” we get RR = 1.5. But after this we insert additional factor “gender” and have got RR = 0.7 both for men and for women. So, perhaps, it is “spurious correlation” due to gender. About “But rather logistic regression…” Please, explain me more detaily, how to use LR to check – is some factor really reason for “spurious correlation” or not ? Thanks a lot for your answer beforehand. Regards, Sergey.

This is a subject-matter question, not a statistical question. Having observational data, you can never exclude the possibility that the correlation you see is spurious. Even if you have data about a third (or forth, fifth...) variable that is a suspected reason for the spurious correlation, and you don't find both original variables can neatly be explained by this (these) other variable(s), there can always be another variableor a combination of variables that would explain the correlation (or some part of it).

Dear Bruce Weaver , thank you very much for your answer! Yes, I mean exactly that you have written. But how to detect, is “spurious correlation” really exists due to factor “age” or not? May be, this non-symmetrical influence of factor “bifocal use” is due to some other factor? Let’s cоnsider your example. Possibly, large difference of RR between “age < 60” and “age >= 60” isn’t due to age, but rather due to some third factor (e.g., “gender” ) which simultaneously correlated both with age, “ bifocal use ” and “nocturnal enuresis”. In this case “nocturnal enuresis” really depends of “bifocal use”, although RR are different for “age < 60” and “age >= 60”, isn’t it ?

Dear Jochen Wilhelm , I fully agree with you. But then it turns out that we cannot trust any conclusions about the influence of one factor or another based on passive collection of statistics. Can we only trust a Randomized Controlled Trial ?

The problem is not "trust", the problem is interpretation. We can certainly use observational data to find correlations that seem interesting. Although we cannot conclude that there is a functional link between these variables just from the data, we can still use this as providing a hint that there might be some kind of link (that may be further investigated in well-designed experiments, if possible), and there is usually a theory in which the finding is to be embedded. How plausible and consistent is all that? And finally, it may not always be interesting to think about functional relationship. A correlation may be used for prediction, and as long as the correlation-based predictions are better than mere guessing, it may be worth it, even if the correlation is actually spurious.

As Nancy Cartwright says, causes in causes out. You need to make some assumptions in order to reach conclusions. The data can inform the value of those assumptions and guide you to or away from different conclusions. You might have a look at Pearl (2009, Causality).

correlation in it self a spurious

Jochen Wilhelm said, "A correlation may be used for prediction, and as long as the correlation-based predictions are better than mere guessing, it may be worth it, even if the correlation is actually spurious."

A spurious correlation may be used for prediction as long as the spurious predictions are better than guessing. How long might that be? Surely, you do not mean this.

Joseph L Alvarez , what do you mean with "how long might that be?" ?

Let me give an example:

Check out http://tylervigen.com/spurious-correlations to get examples for spurious correlations. Take "US spending on science, space, and technology" and "Suicides by hanging, strangulation, and suffocation" with data from U.S. Office of Management and Budget and Centers for Disease Control & Prevention, respectively, which is listed quite at the top (if not first) as it has r = 0.9979.

Both variables raise over time from 1999 to 2009, and the correlation of the variables is due to developments over time that affect both these variables. That should be quite obvious.

Now if you did not know the year but only these two variables, knowing a value of one (say the US spending on science) allows you to make a predcition of the other (suicides). This prediction is better than a predcition based on the data of the US spendings alone.

Jochen Wilhelm said, "Both variables raise over time from 1999 to 2009, and the correlation of the variables is due to developments over time that affect both these variables. That should be quite obvious."

"knowing a value of one (say the US spending on science) allows you to make a prediction of the other "

Do you mean in the time interval from 1999 to 2009? That is not a prediction. That is calibration of events in a fixed time interval.

Do you mean, say, 1989 or 2019? It is not obvious that developments that affected spending will spuriously follow developments that affect suicides 10 years in either direction.

The year is the lurking variable behind the correlation between spendings and suicides. Let's please ignore "year" here. It's only about the observed correlation between spendings and suicides, ok? To understand my point you must pretend that "year" is completely unknown to us. We have only the value pairs of spending andsuicides.

If you would not know the spendings, what can you say about the suicides? -It's something between 5000 and 10000.

But if I tell you that the spendings are 20 bio, based on the correlation you can now say that the suicides must be somwhere close to 6000, plusminus 500 maybe. This is a much more precise prediction than the one you could make without knowing the spendings.

---

It's obvious here that it's actually the year that would be interesting. Giving the information that the spendings are 20 bio is just the same as giving the information that we are looking at the 2001 data (approximately), and there the number of suicides happen to be around 6000. The correlation allows us to use either of the two variables as a surrogate for the unknown "year" variable.

NOTE: I had not yet seen Jochen's most recent post when I posted this.

Joseph's post reminds me of discussions I've seen regarding the distinction between prediction and forecasting. E.g., see this discussion:

• https://stats.stackexchange.com/questions/65287/difference-between-forecast-and-prediction

I am also reminded of the way many textbooks and professors advise students to be very cautious about "extrapolating" beyond the range of the explanatory variables used to develop a model--e.g., see this page:

• https://online.stat.psu.edu/stat501/lesson/12/12.8

I think Joseph is arguing that extrapolating is even more risky than usual when the association is obviously spurious. That makes sense to me.

To Jochen's point about year, the examples on that website do not show ordinary scatter-plots, with X = number of suicides and Y = US spending, for example. Rather, they show trends over time for the two variables. Here is some Stata code to generate the more usual type of scatter-plot for that first example:

clear *

input year spending suicides

1999 18.079 5427

2000 18.594 5688

2001 19.753 6198

2002 20.734 6462

2003 20.831 6635

2004 23.029 7336

2005 23.597 7248

2006 23.584 7491

2007 25.525 8161

2008 27.731 8578

2009 29.449 9000

end

twoway scatter spending suicides || lfit spending suicides, ///

ytitle(Spending (billions of $)) ylabel(,angle(0)) xtitle(# of suicides)

* Uncomment the next line if you need to find and install -corrci-

* search corrci, sj

corrci spending suicides

The website reports r = 0.998, but I get r = 0.992 (95% CI, 0.969 to 0.998). I double-checked, and did not see any transcription errors. But notice that the upper limit of my 95% CI = the same value the website reports as r.

Jochen Wilhelm Again. I ask how long?

The correlation is good for 1999 to 2009, only. Do you agree?

This is a calibration. Spending = x suicides.

Year does not matter. You can determine year, suicides, or spending with the appropriate intercept. The determination is within the error of the calibration.

A scientific prediction in the given interval requires a scientific relationship. No scientific relationship is offered or possible from the given information. The statement, "the correlation of the variables is due to developments over time that affect both these variables," supposes that the developments can be discovered and causally associated. That is specious and unsupportable. There is no possible scientific prediction. There is a possible, but silly, calibration.

No offense intended, but this is an important distinction.

Joseph L Alvarez , I certainly agree that the estimated accuracy of the prediction is trust worthy only for the range of observed data. Extrapolations are always risky, particularily when there is no good theoretical model of the functional form of the relationship. You are certainly right to ring the alarm bell, as new data could well be from timesoutside the given range - and if we don't know about time (and its role here in this example), our prediction can be severely biased (plain wrong, so to say). This is a very general risk, whenever the correlation is spurious (at least to some relevant part). In this case we can not know if the data we used to find the correlation applies to new data we want to use for predictions. To be concrete: if the 20 bio figure for spending was from 1950 or from 2020, our prediction of the numberofsuicides based on that could be really wrong.

You introduce the word "calibration" here. I'm not sure if this word is correct in this context.

I also agree with your statement "a scienticic prediction needs a scientific relationship" (at least in the way I understand this statement). But I was not assuming that the predictionmust be scientific or scientifically justified (or whatever). I was just talking about a prediction, and the fact that knowledge about the value of another (correlated) variable is useful to improve the accuracy of the prediction.

Jochen Wilhelm Calibrate in the sense of establishing a scale. Calibrate spending to suicides or suicides to spending.

There is no prediction. There is estimation from the scale to the error of the calibration.

How is prediction accuracy improved? You have the spending data over time. This should be the best for predicting spending. You have calibrated spending to suicide. Suicide data is less good for predicting spending. You might feel more confident when the suicide estimate of spending agrees with the spending prediction, but how do you combine combine the values and errors to improve accuracy?

Again, how long? Yes, 1950 or 2020 data should be suspect. How far in time would you dare to extrapolate or interpolate?

This is not pedantic nor argumentative. These types of associations are made in health risk assessment and economics at an alarming rate. The associations are then extrapolated far beyond the data region.

Joseph L Alvarez , I think we are talking about different things.* I don't know how to solve this. Maybe my example was not well chosen, I will here try a different one: ice cream sold and drownings in swimming pools. They are supposedly correlated, and the lurking variable here is not time but the weather (or the temperature).

Consider now that we have data ONLY for ice cream consupmtion from a bunch of different locations, possibly from all over the world, and we have a hint on the average number of drownings in swimming pools and its variance.

Now if I ask you for the number of drownings in a particular location, what would you say? I think, the best guess you could give is the average, and you may note that this is a very vague prediction.

Now consider I additionally have the data of drownings in swimming pools (at the same locations), so that I can fit a linear regression model (i.e., I can estimate the intercept and the slope of that model from the observed data). I now tell you this model, it is [drowning] = 2 + 5*[ice cream] + error, given in suitable units. When you know this, you can do a better job to predict [drowning]: you can look up the amount of ice cream that is sold at that location and then use the regression equation to calculate a predcition of [drowning]. The prediction will have a smaller variance than the (global) average. This is what I meant when I was saying that a correlated variable helps to make more precise predictions, even if the correlation is spurious.

If the two variables drowning and ice cream are not correlatied (whether it's spurious or not), then the estimated slope of the model is expected to be close to 0 and the error is expected to be as large as the variance of drowning.

The formulation in this example (giving a location) will not allow any extrapolation. Instead of location, one could use the temperature. It is then possible to ask for the number of drowings at some temperature, which might be outside the range of observed temperature values (extrapolation, danger!)

---

* All clear that extrapolation is dangerous and that we may not share the same definition of the word "prediction", what are not the key issues here, anyway. But please let me try to get a common definition of "prediction", "estimate". This is excellently explained by Wolfgang Huber in:

https://stats.stackexchange.com/questions/17773/what-is-the-difference-between-estimation-and-prediction

TLDR: estimation is about parameters in a model, prediction is about response values. If the model is Y = aX+b, then the values of a and b are estimated (from observed data), and the value of Y can be predicted for any value of X (given a and b are known or estimated).

I'd like to add that a prediction within the span of observed X, but not precisely at any observed X, may be called an interpolation. A prediction for a value of X outside the span of observed values of X is called an extrapolation. If X is "time", an extrapolation to later times is called a forcast, an extrapolation to earlier times is a backcast (see https://en.wikipedia.org/wiki/Backcasting: " In statistics and data analysis, backcasting can be considered to be the opposite of forecasting").

I will therefore stay with "prediction" to mean giving/guessing/conjecturing a value of of some variable, based on some knowledge (what may be informal experience, a physical theory, or a statistical model).

To the original question:

A spurious correlation is not easily discovered, if the total information is limited. Data are sometimes given as, say, two categories in a table. If there is a correlation, there is no basis to question it. More information is needed to stratify or standardize. Shoe size and vocabulary are correlated. We can suspect age - shoe size and age - vocabulary will sort the difference, but absent the information on age, cannot be shown.

One method to question a correlation is estimate the size of a confounder necessary to explain the correlation. Examine confounders that might nullify the correlation.

The problem of smoking and lung cancer was mentioned in an answer. Smoking - lung cancer were correlated in observational studies. The RR was large, >10. There were many claims of confounding, because it was an observational study. Cornfield presented evidence of possible confounding size and found that proposed confounders were too small.

• Cornfield J, Haenszel W, Hammond EC, Lilienfeld AM, Shimkin MB, Wynder EL (1959). Smoking and lung cancer: Recent evidence and a discussion of some questions. Journal of the National Cancer Institute 22 (1):173-203. https://academic.oup.com/ije/article/38/5/1175/666926

Note: Beware of RR. Ratios are nonlinear and can appear large, while the odds ratio is small. Also, a large RR when the absolute risk is small, indicates a possibility of measuring only noise.

Thanks а lot again to everyone for the answers. Regards, Sergey.