It will depend on what type of software you are using. If you are using the statistical software R, then you can download and use many functions from the R package outliers, see link below:

https://cran.r-project.org/web/packages/outliers/outliers.pdf

Both variables have non-negative domain, the graph reveals the presence of heteroscedasticity. In order to detect outliers here I recommend to transform your data, use using logs/Box-Cox to make the distribution normal. Then standard methods for detecting outliers will be applicable. To make your visualization easier to read, let me recommend this poster (you can find the section on data transformation useful as well):

Poster A Visual Framework for Longitudinal and Panel Studies (with ...

Keep all data points for transparency. And the so-called abnormal outer points are often the most interesting, after all :)

Thank you so much for your quick replies!

Andrew Paul McKenzie Pegman Precisely what I want is to make a specific class for those "abnormal" outer points, those are the ones of especial value. But I need some statistics to cut them off the main body of points. Make sense?

You must keep outliers. After all, they're real data points too and make the results more accurate and interesting. You cannot just arbitrarily delete points. It's better to be transparent :)

Andrey Davydenko Carl Alexander Frisk

Please notice that I'm not presenting here any kind of regression analysis. This is just a representation of two variables. There is no intention of getting rid of the ones that are outside the "imaginary line" of regression. What I want is to get the dots that are statistically OUTSIDE the "main body" of points that seems to be relatively confined within the (X>20,Y>20) "rectangle". Make sense? Sorry, I should have clarified this in advanced :)

The task "to get the dots that are statistically OUTSIDE the "main body" of points that seems to be relatively confined within the (X>20,Y>20)" sounds like you want to identify "unusual" observations.

In order to do that you need 1) to assume you have some distribution here 2) for each point you need to evaluate the probability that it does not belong to the distribution assumed. The easiest way to do it is using the transformations I described above. Try sqrt here, then "unusual" observations will become more evident even using the same scatterplot analysis.

Andrey Davydenko Thank you so much for your quick feedback!

I did the sqrt transformation of the variables x and y.

What are the following data treatments that I have to do?

Or can I operate manually from here?

Thank you

Guillermo Domínguez Huerta :

Looks great! I think now you can do the following:

You can apply the standard Three-Sigma Rule of Thumb for sqrt(x) and sqrt(y).

If abs(obs-mean(obs))>3*sd(obs) ===> you've got something unusual (as you named it "abnormal").

Good luck!

P.S. sqrt(x) and sqrt(y) are now close to being normally distributed, so you can apply the standard approaches described, e.g., here:

https://www.kdnuggets.com/2017/02/removing-outliers-standard-deviation-python.html

Of course, there can be some probability that the "outliers" you detect are, in fact, "normal" observations, but with high probability there's something wrong with these observations, they can occur due to some data collection problems or some unusual events.

Andrey Davydenko Thank you so much!!

Just to clarify, what I have to do is:

1) Get the sqrt of the values.

2) Run the python script on those transformed values.

3) The previous script will remove the "outliers".

?

Thank you,

Guillermo

Guillermo Domínguez Huerta

Glad it helped.

Yes, the script may help define the rule for "outliers" for your data set assuming that sqrt(x) and sqrt(y) are more or less normally distributed.

Instead of Python you can use Excel or any software available.

Btw, the rationale behind the above approach:

1) If you deal with count data (or something based on count data), it is likely they follow a Poisson distribution:

https://www.dur.ac.uk/physics/students/labs/skills/data/poisson/

2) the Poisson distribution is well approximated by a Gaussian distribution :

https://www.semanticscholar.org/paper/The-Effects-of-a-Square-Root-Transform-on-a-Poisson-Thacker-Bromiley/6c201e68c94fa8825249f6be91e7d11949316943?p2df

3) Methods for detecting outliers in Gaussian distribution are well-known, so the idea is to use them after applying the transformation.

P.S. You can try logs as well: seems you have no zero cases, logs might work as well.

P.P.S. You may want to use Jarque–Bera test to see if the distribution of transformed values has "good" properties. https://en.wikipedia.org/wiki/Normality_test

Andrey Davydenko Thank you so much for your generous explanations.

May I insist to fully understand this please?

My data look closer to a normal distribution after the transformation.

**The 3-sigma rule is applied to normal or near-normal distributions. When I followed this method, I got 24 points considered (by both variables X and Y) "outliers".

**But following the other method (I don't know the name, standard approach?):

--> values >abs(mean+2*SD) OR 97 points that are outliers in both variables X and Y.

--> values >abs(mean+3*SD) OR 48 points that are outliers in both variables X and Y.

Which one is the best approach here?

This is always assuming that we have really normalized the data, of course.

Thank you,

Guillermo

Guillermo Domínguez Huerta

This is a 2-sigma approach:

values >abs(mean+2*SD) OR abs(mean+3*SD) OR =abs(mean+3*SD) OR =

Use = instead of > and