Assumptions under rule of huge error?

Hasanthi anuradha Pathberiya @Hasanthi_Pathberiya

08 August 2015 1 564 Report

To use rule of huge error in detecting outliers in a data set is it necessary for the data set to be normally distributed?

http://education.mrsec.wisc.edu/research/topic_guides/outlier_handout.pdf

Peter Samuels

Dear Hasanthi,

I completely disagree with this guide. In my opinion, outliers should never be removed unless there is some evidence to doubt their validity.

If you wish to analyse a data set in a way that this not adversely affected by outliers then my advice would be to use a robust or a nonparametric technique.

I would also suggest that removing outliers in the way suggested in the paper will not guarantee normality in your data set.

I would further suggest that you seem to be approaching this issue the wrong way round: rather than changing your data set to fit the technique, the technique should be chosen to fit your (valid) data set. This means checking your data set for normality before using a parametric test.

Badges
Science topic

Similar topics
Moths
Data

More Hasanthi anuradha Pathberiya's questions See All

What does this reachability plot means?

10 November 2015 8,512 0 View

Can I fit a support vector regression model for financial data (exchange rate returns)?

Here are the variables Response variable: exchange rate return at five minute frequency Explanatory variables: exchange rate return at previous lags Other exchange rates Time around release of...

05 June 2015 5,142 5 View

How can I use a regime switch model (RSM) to identify different regimes in exchange rate behavior?

I am interested in studying the behavior of exchange rates. I would like to use R as a software. Can anyone please explain the algorithm that I should follow in order to fit a regime switch model?...

11 December 2013 8,460 3 View

Do we have to have a series which follows normal distribution in order to fit a regime switch model?

What is a regime switch model? Do we have to have a series which follows normal distribution in order to fit a regime switch model?

07 August 2013 6,706 2 View

Comparing correlation structures

I have a data set of stock returns. I am considering the correlation between mean return and volatility within different hours. I want to compare volatility structures across different hours. e.g:...

05 June 2013 3,245 3 View

How to identify clusters of volatility in different time intervals in financial time series data?

I am interested in analyzing disparities in volatility in different five minute intervals in different trading days in the Forex market. Now I want to know whether the volatility in such intervals...

05 June 2013 6,109 4 View

How to compare two groups with only two measurements?

Hiiiii everyone! I have an enquiry on statistical analysis. I was looking for many forum and it's still cannot solve my problem. I want to compare means of two groups of data but only with two...

03 March 2021 8,796 3 View

Can we classify HV and LV lines based on line length?

I have dataset which shows the length of power lines. I need to classify the lines based on the line length. Is there a rule to classify the High voltage (HV) and low voltage (LV) lines based on...

03 March 2021 4,116 4 View

What's the best way to measure growth rates in House sparrow chicks from day 2 to day 10?

What's the best way to measure growth rates in House sparrow chicks from day 2 to day 10? Since, the growth curve from day 2 to 10 won't be like the "Logistic curve" it might not follow logistic...

03 March 2021 1,401 3 View

How to increase a model accuracy ?

dear community, my model is based feature extraction from non stationary signals using discrete Wavelet Transform and then using statistical features then machine learning classifiers in order to...

03 March 2021 6,994 5 View

1. What is the impact of having different scales in a survey? and how can we solve this problem before and after data collection (Literature)?

1. What is the impact of having different scales in a survey? and how can we solve this problem before and after data collection (Literature-based reflection)? Thank you for your time and for...

03 March 2021 2,870 3 View

What degree of missing data in a particular case/participant results in Expectation-Maximisation no longer working?

I'm dealing with a mediation model and am using the PROCESS module in SPSS. Due to SPSS and PROCESS being limited in the imputation methods - being unable to handle multiple imputation - the other...

02 March 2021 4,362 1 View

Can NFL theorem be valid in infinite search space in ML?

NFL theorem is valid for algorithms training in fixed training set. However, the general characteristic of algorithms in expanded or open dataset has not been proved yet. Could you show your...

01 March 2021 1,189 3 View

Does anyone know how to plot my station to a section plot on Ocean Data View (ODV)?

I made a vertical section plot with WOA .nc file on ODV, and now I want to plot my stations (csv. file) in it. Does anyone know how to import my points to the section? They are two different...

01 March 2021 3,610 1 View

What test should I use to compare two or three means, if data is not normally distributed but sample size is > 30 in each group?

I personally feel this as a most confusing question. As per my understanding, if sample size is > 30, and sample is skewed, we can still use paramteric approach, as it will follow Z...

01 March 2021 1,587 6 View

How to address this issue in the analysis of Real-Time PCR results?

To dear Researchers, I was analyzing a series of concentration for estimation of Real-Time PCR efficiency. The concentration was 1:10. I used MS-excel to evaluate Slope. The result of slope was -8...

01 March 2021 8,683 4 View