You can detect the influence of an observation on correlation coefficient by deleting it and recalculating r. An easier way may be considering Dfbeta values from a regression model where your variables are entered as dependent and independent variables. You should have a look at those observations with extreme Dfbeta values (Dfbeta for nonconstant term in the model).

Or, you can try another correlation coefficient like Spearman's which is more resistant to outliers.

A quick search of google scholar failed to produce analytic or empirical (simulation) research addressing your question. If simulations designed to address this issue don’t exist, this would be an interesting study.

In an X-Y distribution for which r=0, the points appear as a circle in Cartesian X-Y space, the regression line parallels the X axis, and many points are outliers (note my use of this term here identifies points that lie far-off the regression estimate--or line, rather than simply the values having large absolute magnitude--i.e., deviation from the sample mean).  As r approaches the limit |1.0|, the magnitude of outliers diminishes, and the number of outliers becomes zero when absolute r=1.  The magnitude of the outliers assessed as raw numbers will increase if the data are squared, and will decrease if a radical (square-root) of the data is applied. However, what if standardized residuals are compared?

Unless they are perfect (no residuals), regression models have larger residuals--implying outliers--for raw scores of greater absolute magnitude. This well-known phenomenon is called regression toward the mean.

Less-well known, even for very simple metrics, the accuracy of the r model in predicting scores for individual observations is extremely poor, even for strong |r|--and most published regression models are only able to predict values at or near the mean with accuracy. These are values that represent “no difference” or “I can’t decide” in Likert-type ratings. Yet, researchers are generally most interested in predicting ratings near the poles of the rating (i.e., measurement) scale.

In studies of satisfaction, it is not unusual to find that most of the data lie at the extreme positive (or negative) pole. If it were possible for some transformation to change this distribution into another distribution that did *not* have outliers, what does this mean? Clearly, the result cannot be interpreted in terms of the original units, but rather in terms of the transformed units: it is no longer satisfaction ratings, but transformed satisfaction ratings. Is there some valid mapping between cognition/ perception/rating and the transformation(s) selected by the researcher? Is it the same mapping for all of the observations? Is this transformation/ mapping unique to the particular variables/phenomena addressed in the study, or is it generalizable to other phenomena? Why and how and when?

Improving (maximizing) the ability of r and multiple regression models to predict outlying (extreme) observations is easily accomplished using a non-parametric machine-learning algorithm, which has been reported to increase performance of the models in hold-out validity analysis (open-source articles cited when possible):

• https://odajournal.com/2013/09/20/maximizing-the-accuracy-of-multiple-regression-models-using-unioda-regression-away-from-the-mean/

The performance (p-value and effect strength--ESS) of this algorithm is invariant over any monotonic transformation of the raw data:

• Yarnold. P.R., & Soltysik, R.C. Maximizing Predictive Accuracy. Chicago, IL: ODA Books, 2016. DOI: 10.13140/RG.2.1.1368.3286

The current evolution of this algorithm identifies all of the statistically valid optimal (non)linear models in the sample. Models vary with respect to parsimony: they require no distributional assumptions, always produce a valid p value, and explicitly maximize the ability of the model to correctly predict point values over the domain of measurements:

• https://odajournal.com/2016/09/19/novometric-analysis-with-ordered-class-variables-the-optimal-alternative-to-linear-regression-analysis/
• https://odajournal.com/2016/09/19/novometrics-vs-multiple-regression-analysis-age-and-clinical-measures-of-pcp-survivors/

Linear models are susceptible to confounding by Simpson’s paradox:

• https://odajournal.com/2017/04/18/what-is-optimal-data-analysis/

Accordingly, normative standardization separately by group is appropriate for static designs,

• Yarnold, P.R. (1996).  Characterizing and circumventing Simpson’s paradox for ordered bivariate data. Educational and Psychological Measurement, 56, 430-442.
• https://odajournal.com/2013/09/20/assessing-hold-out-validity-of-cta-models-using-unioda/    

and ipsative standardization within subject is appropriate for dynamic multi-subject designs involving one group

• https://odajournal.com/2013/10/23/ipsative-standardization-is-essential-in-the-analysis-of-serial-data/

or more than one group

• https://odajournal.com/2016/05/31/using-machine-learning-to-model-dose-response-relationships-via-oda-eliminating-response-variable-baseline-variation-by-ipsative-standardization/

and is also appropriate for single-case longitudinal designs

• https://odajournal.com/2013/11/07/ascertaining-an-individual-patients-symptom-dominance-hierarchy-analysis-of-raw-longitudinal-data-induces-simpsons-paradox/
• https://odajournal.com/2013/09/19/the-use-of-unconfounded-climatic-data-improves-atmospheric-prediction/

Maximum-accuracy non-linear models can identify linear effects that linear models are unable to identify, and they obviate paradoxical confounding when data are appropriately normalized: please see previously cited review article, What is optimal data analysis?

I am still unclear on why you transformed the data and why the correlation should depend on the data being sensitive to outliers.

Have you tried Spearman's rank correlation on the raw data? Does it work well?

The original algorithm, discovered in 1991, versus Spearman r:

• https://odajournal.com/2015/12/02/unioda-vs-spearman-rank-%CF%81-between-raters-reliability-of-scores-on-the-adverse-drug-reaction-probability-scale/

The current (novometric) algorithm versus polychoric r and versus the initial (1991) algorithm:

• https://odajournal.com/2016/11/11/novometric-vs-oda-reliability-analysis-vs-polychoric-correlation-with-relaxed-distributional-assumptions-inter-rater-reliability-of-independent-ratings-of-plant-health/

Depending on what you mean by "outlier", r could vary a lot from the presence of an outlier, either making r small or large.

My best advice is to plot the bivariate data, transformed and not, and have a look.  It just may be that your data is curvilinear in a way that made the square transformation linearize them. Impossible to know without looking.

Here are examples of outliers changing Pearson r from small to large and from large to small.  It's in R code, but I'm sure you get the idea.  A1/B1 and X1/Y1 have no outliers.  Note also that Spearman correlation is mostly robust to these effects of outliers.

*        *        *

### Small r made large by outlier

A1 = c(1,2,3,4,5,6,7)

B1 = c(5,5,5,5,5,5,6)

cor.test(A1,B1)

plot(A1,B1)

A2 = c(1,2,3,4,5,6,7,100)

B2 = c(5,5,5,5,5,5,5,100)

cor.test(A2,B2)

plot(A2,B2)

### Large r made small by outlier

X1 = c(1,2,3,4, 5, 6, 7)

Y1 = c(2,4,6,8,10,12,13)

cor.test(X1,Y1)

plot(X1,Y1)

X2 = c(1,2,3,4, 5, 6, 7,100)

Y2 = c(2,4,6,8,10,12,13, 8)

cor.test(X2,Y2)

plot(X2,Y2)

### Spearman correlation is robust against effect of outliers

cor.test(A1, B1, method="spearman", exact=FALSE)

cor.test(A2, B2, method="spearman", exact=FALSE)

cor.test(X1, Y1, method="spearman", exact=FALSE)

cor.test(X2, Y2, method="spearman", exact=FALSE)

Novometric (globally-optimal) solutions to the four test problems

The first two examples each have one perfect (100% accurate) optimal (maximum accuracy) model, regardless if A or B is treated as the class variable.

Predicting A1 given B1:

If A16 (100% correct, ESS=100)

Predicting B1 given A1:

If B15 (100% correct, ESS=100)

Predicting A2 given B2:

If A25 (100% correct, ESS=100)

Predicting B2 given A2:

If B27 (100% correct, ESS=100)

**

Predicting X1 given Y1:

The third example has six perfect optimal models:

If X12 (100% correct, ESS=100)

If X14 (100% correct, ESS=100)

...

If X112 (100% correct, ESS=100)

In this case a priori selection heuristics would be used to identify the optimal model. Here are the options in UniODA and MegaODA software: one primary and one secondary selection heuristic may be specified—the secondary is used to select among models tied with respect to the first heuristic (if that happens):

•  The PRIMARY command specifies the primary criterion for choosing among multiple optimal solutions.  MAXSENS (or MAXPAC) is maximum sensitivity.  MEANSENS (MEANPAC) is the mean of the sensitivities of the separate classes.  SAMPLEREP (SREP) selects the pattern of predicted class membership most closely resembling the sample class membership.  BALANCED (BAL) selects the solution in which the sensitivity of the actual classes is most similar amongst each other.  SENS (PAC) selects the solution with maximum sensitivity of class value.  DISTANCE (DIST) selects the solution with smallest maximum (over all cutpoints) distance between the cutpoints and their boundaries.  RANDOM (RAND) selects a randomly chosen solution.  GENMEAN is used only when GEN is in effect: it selects the solution with maximum mean (weighted) sensitivity over all GEN groups.  GENSENS selects the solution with the maximum (weighted) sensitivity of group value.

The solution predicting Y1 given X1 is easily computed, as in the prior examples, and has six optimal models. At this junction, readers may wish to demonstrate this for themselves.

 **

 Predicting X2 given Y2:

Here there are three perfect optimal models:

If X22 (100% correct, ESS=100)

If X24 (100% correct, ESS=100)

If X26 (100% correct, ESS=100)

Predicting Y2 given X2:

Here there are three perfect optimal models:

If Y21 (100% correct, ESS=100)

If Y22 (100% correct, ESS=100)

If Y23 (100% correct, ESS=100)

**

What I'd like to know

• What is Spearman (and Pearson) r for these problems (thanks, in advance, to anyone who shares requested information)?
• In particular, when the Spearman (and Pearson) r is used to make point predictions (for each observation), what is the predictive accuracy?
• For each Spearman (and Pearson) model, what is the effect strength normed against chance (for ESS, 0 = predictive accuracy expected by chance, 100 = perfect accuracy)?

**

Real data may yield more complex solutions

In globally-optimal (novometric) maximum-accuracy models, association matrices are not necessarily symmetric:

• https://odajournal.com/2016/10/03/matrix-display-of-pairwise-novometric-associations-for-ordered-variables/
• https://odajournal.com/2016/10/05/matrix-display-of-pairwise-novometric-associations-for-mixed-metric-variables/

**

(Not) evaluating potential cross-generalizability or exact discrete 95% CIs for model and chance

I computed these novometric models by hand--they are training models (i.e., developed using all of the data in the sample). I didn't compute the jackknife (leave-one-out) validity, nor obtain the exact discrete 95% CIs for each model (and chance), due to my present time constraints...

• https://odajournal.com/2016/05/06/determining-jackknife-ess-for-a-cta-model-with-chaotic-instability/
• https://odajournal.com/2014/10/10/discrete-95-confidence-intervals-for-oda-model-and-chance-based-classifications/

Paul, the problem with the questions you are asking is the same problem with your injecting your software as a proposed solution to various questions on this site:  statistical tests address specific hypotheses, and your software doesn’t address the hypotheses being asked.

In this case, bivariate correlation tests to see if two variables vary together. It doesn’t make any predictions.  And it certainly doesn’t ask how to dichotomize the variables. Neither are the point at all.

So trying to compare how your method stacks up against correlation for a given data set on predictions, point predictions, predictive accuracy, effect strength, or what have you, is completely irrelevant to the discussion.

It is as if you have devised the perfect tool to cut a round pizza into eight perfect slices.  And I come to you and say, “I need a tool to cut this plywood to cover this broken window.”  And you say, “Here, I’ll cut it into eight perfect slices.”  And then I say, “I need a tool to cut this cloth into this pattern to make a shirt.”  And you say, “Here, I’ll cut it into eight perfect slices.”

Or to think about a two sample t-test, to which you have also proposed alternative solutions:  a t-test compares the means of two groups.  If I want to compare medians, I might use Mood’s median test.  If I want to compare the 95th percentile, I might use quantile regression.  If I want to compare if two groups vary stochastically, I might use Mann–Whitney test, and so on.

The point is that all of these tests test distinct hypotheses.  The tool you propose may be optimal for some task, but that doesn't mean it is a desirable approach for all problems.  If it doesn't address the hypothesis I am interested in investigating, it has no use for the question I'm trying to answer.

The problem lies not in my questions, but rather in your reactions to my questions.

We are both familiar with legacy statistics. In fact, I developed new legacy methods early in my career, and I published two best-selling books on legacy multivariable and multivariate statistical methods. And, I am intimately familiar with the ODA paradigm—this is where the problem starts.

The mischaracterization of ODA reflected in your “8 equal slices” analogy demonstrates that you don't really understand what ODA is or how it works. In reality, ODA is exactly the opposite of the way you depict it. ODA can in fact model any hypothesis.  Indeed, both of the books published about ODA present numerous research hypotheses that cannot be addressed using any existing legacy method, for which ODA models identified strong support.

Until you read the newest ODA book and gain a clear fact-based understanding of the ODA paradigm, we can't really have a productive informed discussion about ODA. Similarly, if we both fully understood ODA, but you had clear understanding of legacy statistics whereas I did not, then I would be unable to make informed comments and generalizations about the differences between ODA and legacy statistics.

Many researchers have received letters from editors rejecting a manuscript on the basis of comments made by uninformed, unqualified reviewers who had no basis for rational decision-making concerning the content of the manuscript in question. This practice is an unethical disservice to science and scientists. Similarly, it is a disservice to the forum and unethical to spread misinformed falsehoods by posting comments criticizing something with which the poster is insufficiently informed.

When confronted by a new paradigm, objective open-minded scientists are naturally expected to make the effort to read about and understand the method in order to make an informed evaluation of its merits and shortcomings. Until that happens, further discussion is pointless.

Thanks to everyone for their answers.  There is a lot for me to follow up on.  However, I found that what I am looking for is bootstrapping: "you should draw random samples of the same size as your data set. If you have outliers in your data and these outliers are very influential for your correlation estimate, the bootstrap estimate of the standard deviation should become large."

So I will be looking for more details, both so that I understand how it works, and so that I can apply the methods to my data set.

Robert, there are easy ways to see if an outlier strongly affects a correlation without using bootstrap.  I'm not sure what you are trying to accomplish here.

If I transform data, I want to know whether I should reject the transformation on the grounds that it depends more on outliers than the untransformed data.

Here's what I think:  Plot a histogram for each variable, and then plot the bivariate scatter plot for each correlation.  And also look at the plots for the transformed data.  I think it will be clear whether the transformations are justified, or if outliers are distorting things. 

Also, as suggested above, deleting suspected outliers and doing the correlations and transformations will also be indicative.

I don't think there's any thing more complicated you need to do in this case.

I see I forgot to add one more thing: this is in the context of a preregistration, so I need to be able to specify in advance what my criterion for rejection will be, and my decision must be reproducible. 

Then, just use bootstrapping without any transformations.

Maybe you could use non-parametric correlation. (Kendall, Spearman), and not worry about the transformation issue?

(The question / problem isn't particularly clear to me.)