The concept of robust estimates has been developed in statistics a long time ago. The median is more robust estimator of the central tendency than the mean; rank correlation is more robust than linear correlation. In general, robustness refers a statistical estimator being insensitive to small departures from the idealized assumptions for which an estimator is optimized. ‘Small’ has two meanings: (i) small departures for all data points, or (ii) large departures for a small number of data points. The latter case leads to outlier issue for statistical procedures.

Various sorts of robust estimators were developed in statistics. (i) M-estimates from maximum-likelihood (ML) concept that are the most relevant for model-fitting parameters. (ii) L-estimates are linear combinations of order statistics, which are most relevant to estimations of central value and central tendency, such as median or Tukey’s trimean as the weighted average of the 1-st, 2-nd and 3-rd quartiles in a distribution with weights ¼, ½, and ¼, respectively. (iii) R-estimates based on rank tests, such as Wilcoxon, Kolmogorov-Smirnov, or the Spearman rank-order correlation.

For normally distributed data, the more deviant the points (the bigger outliers) the greater weight is applied to them in parameter estimators. It is this property that makes normality based estimators such prone to the loss of robustness due to outliers. In contrast, when distribution data tails are larger than for normal, all deviant points get the same relative weight. This makes the estimator more robust, such as the ML estimator with the mean absolute deviation (rather than mean squared deviation for normal). When the data distribution tails are even larger, such as e.g. for Cauchy-Lorentz distributions, the relative data weight first increasing with deviation and then starts decreasing, so that very deviant points (the true outliers) are not included at all in the parameters’ estimation.

I would recommend this book. David Olive is a very good person in this area:

https://www.amazon.com/Robust-Multivariate-Analysis-David-Olive/dp/3319682512/ref=sr_1_1?keywords=david+olive+robust+multivariate&qid=1563659348&s=books&sr=1-1

Best wishes, David Booth