Dear Loany,

maybe I don't quite understand your question, but why do you want to reduce your model if it contains only the most significant terms? You are right: In general, the model should be as simple as possible. That's why you select the most significant factors in the Screening DoE. With the most significant factors, you plan experiments for modeling by adding star points to your original plan (assuming you have non-linear effects). And after the subsequent analysis, you build your model by neglecting all non-significant effects. If you achieve the same lack of fit with your reduced model and the original model, I would always use the simpler model because it has less parameters despite having the same fit.

Best regards.

Thanks Alexander,

Let me clarify my question. The outcome of the analysis is that only factors A, B and ABC are significant, but the model still contain all non-significant factors also. My question is how to reduce (if at all, but you advocate for reducing) and keep the hierarchy (maintaining a non-significant factor when it is part of a significant interaction). Notice that option 3b attempts to keep non-significant interactions also, when part of a greater-level significant interaction (for example, keeping AB because it is part of ABC).

But your answer (very useful) points me to ask another question:

Yes, there is a screening phase where I select the most significant factors and move on to another design where I can study interactions and curvature better. Do you mean I dont need to worry about the model in the screening phase? Just worry about it at a subsequent step when I have done a design that allows me to have less confounding factors, or a clearer alias structure (easier to interpret)? That makes sense. But still in this step there are many ways to arrive to the reduced equation...

Hello, Loany,

I'm so glad I could help you. Sorry, I overlooked that not all effects are significant but only the main effects A, B and the interaction ABC.

Basically, models with many parameters are less accurate than models with less parameters. In addition, "large" models usually have the ability to make inaccurate predictions. I have been able to confirm this in my previous investigations. Therefore, I can only recommend reducing the model in dependence of the quality of your model fit. On the other hand, I always pay attention to the hierarchy of the model. As far as I know, if a model does not contain the main effects even if they are not significant, a change of the coding method (coded or real values) can cause instability. A model reduction logically provides a model. But if I'm not in the modeling phase yet, I wonder why I should be concerned with model reduction. After all, the experimental effort in the screening phase is minimal. Usually only as many experiments are carried out here that I can make a statement about which of the investigated factors are significant and whether a non-linearity exists. But I agree with you, it is an important and very interesting topic.

Thanks a lot. This is so helpful. But your answer led me to another question :D

I always assumed the factors in the equation from the Model (which contains the coefficients given by the ANOVA) need to be substituted by their coded values (from -1 to 1). Is this true? Should the model be "transformed" to be able to use real values on it?

Hey Loany,

you have very interesting and important questions I think. Well, basically, you're not forced to code the factors. With uncoded factor levels, you may not be able to interpret the effects. This is especially the case if the ranges for the factor levels are different: e.g. for the factor temperature: 20 °C to 160 °C and for the factor speed: 500 rpm to 1500 rpm. Comparing the effects will be difficult here. In this case also the coefficients of the model will differ clearly from that with coded factors. Therefore I can only recommend coding the factors.

Best regards

Thanks a Million! Yes, I am working with the coded equation. A bit uncomfortable (the need to transform the variable) but easier to understand. But I have stopped attempting to model the results of the initial screening studies (resolution III), and decided to keep the hierarchy of the model paying attention to the presence of main factors involved in significant interactions, but not to the non-significant intermediate interactions unless the residuals or fitting parameters (like r2(adj) recommeds so. I mean, as in the original example, if ABC is significant, just make sure A, B and C are in the model, although the insignificant AB might be absent (already covered by the presence of A and B as main factors).

Thanks a lot for your guidance

You are welcome!

keep only the significant factors

Hello,

I would look at

• the size of the effects
• rsquared(predicted) and
• whether I understand the ABC interaction.

If ABC is only just significant, I would drop it and only keep A and B. The reasoning behind that is that 3-factor-interactions are rarely important, particularly if so many lower order terms are not significant.

If not, I would compare the full model (i.e. including C, AB, AC, BC, ABC) with the model containing only A and B with respect to rsquared(predicted). If rsquared(predicted) is worse for the full model (as it often is, due to overfitting), I would also only include A and B.

If rsquared(predicted) is better, I would try to understand ABC from the subject matter. Is it really plausible that AB depends on the level of C, although individually they are not significant? Or did I possibly make a mistake in one of the experiments? Particularly if each setting is run only once, a big ABC is caused by a wrong result for one setting. If settings are realized several times: Are there outliers?

In other words: I would be very sceptical and make sure that I understand, what has happened, before I include ABC. However, if I do include it, I would also observe hierarchy.

I hope this was helpful.

Wilhelm Kleppmann

Very helpful Wilhelm Kleppmann ,

Very important points to consider. Thanks

What is the weighting of the significance of ABC compared to the significance of A or B?

@Norge

Do you mean we should also consider the coefficients? what do you mean by "weighting"?