If you want to build a prediction model from your experiment, you might find the attached useful for RSM work. Best, D. Booth

Thank you very much!

Dear @Nurlan : Large residuals are caused by poor execution of your experiments (there are latent variables that cause very significant effects on your response and you also have very poor control over your experimentation). You are badly advised. You can eliminate outliers only when the orthogonality of the experimental plan is not broken down (when there are few outlier). But you should NEVER replace an experiment that you don't like with another one that if you like it since you would be cheating, that is, you would be giving trend to your results. If you do not like your results, you must replace the entire experimental plan and in the execution, you must pay more attention to the control of factors that do not enter the study (latent variables).

NEVER: " As far as I know the unusual observations and large residuals points must be totally excluded from data and RSM must be performed again with new data."

A consequence of POOR CONTROL: "However, right after I remove these points and rub RSM, the results show new unusual observations and large residuals, and again I remove them. The same thing happens which leads to an undesirable reduction in data. "

Dear Raúl González-García, thank you for the answer, actually I am using the data that have been collected by other researchers, and as far as I know design of experiment was not considered for that experiment. In such case the data does not depend on my choice and it is what it is. If I leave unusual obs and large residuals as they are, the results show undesirable p-values and low R square values. To develop a significant model what else must be modified in data to get desirable results? Is it possible to develop a model based on such data or the feasibility of developing a model depends entirely on the pre-experimental choice of the reseracher who conducts the experiment?

Dear Nurlan Amirov :

"To develop a significant model what else must be modified in data to get desirable results?"

To develop a significant model:

1.- If the data comes from a DOE: variables must be chosen that greatly impact the response and / or have very good control in experimentation (control of experimental error), or

2.- If the data comes from historical data: There is no option. Maybe you should have a lot of data.

In either case, you should not change data that you do not like for data that you like since that is a trend and it would not be worth doing a DOE.

"Is it possible to develop a model based on such data or the feasibility of developing a model depends entirely on the pre-experimental choice of the reseracher who conducts the experiment?"

Yes, it is possible to develop a model based on these data. The results must be independent of the experimenter and the analyzer. Although the skill of the experimenter, without a doubt, is something that affects the analysis. But, the knowledge of the analyzer is also crucial. For example, in your case, you have not explored the transformation of the response variable, which may be necessary. The appropriate transformation of the response can be found with a Box-Cox graph. The transformation of the response also dramatically changes the coefficients of the model. Another situation that you may not have explored is the pruning of the model.

Nurlan -

You said "...after I remove these points and rub RSM, the results show new unusual observations and large residuals, and again I remove them."

I haven't worked with RSM, but that is something another ResearchGate member asked about recently with regard to "outliers," and I recall working with someone who wanted to do that: removing potential 'outliers' in consecutive groups. The logical conclusion would be very few remaining data of arbitrarily good 'fit.' That is not a good idea. If you had such extraordinarily 'dirty' data, you would be in a very bad position. Potential outliers should not be so easily removed. You would need good reason. Recall that 1% of the data points SHOULD be outside of the 99% prediction intervals.

Have you accounted for heteroscedasticity? I can see how taking out large points with large estimated variance of the prediction error could leave a set of data where you would then obtain the same result with the next iteration of what you were doing. Heteroscedasticity is a natural phenomenon that should be expected under ideal circumstances. Perhaps you should check on that.

Regardless, removing a lot of data as potential outliers without specific reasons in each case based on other knowledge is most likely not a good idea.

Cheers - Jim

Nurlan -

Above I asked "Have you accounted for heteroscedasticity?  I can see how taking out large points with large estimated variance of the prediction error could leave a set of data where you would then obtain the same result with the next iteration of what you were doing." 

What I meant was that for the largest y-values, did you tend to have the largest predictions, with heteroscedastic of the estimated residuals?  You would generally have larger estimated variance of the prediction error, as larger variance of estimated residuals contributes greatly to this, though the model also matters.  (See the description/abstract in the Overview of https://www.researchgate.net/publication/333642828_Estimating_the_Coefficient_of_Heteroscedasticity.) 

..........

So here is the point: 

Do the estimated residuals with the largest magnitudes generally belong to y-values with the largest predictions?  Are these the y-values you are removing?  If so, you may just be seeing heteroscedasticity, and this is being confused for the presence of outliers. 

Cheers - Jim

IMO outliers should not be removed unless they represent KNOWN BLUNDERS. Otherwise study them for the information they contain. Example attached. Best, D. Booth