Box Behnken design significant lack of fit in second order model - How to proceed?

When dealing with experimental design and modeling, encountering a significant lack of fit terms despite a significant second-order model can be challenging. Here are some suggestions on how to proceed in this situation:

Check model assumptions: Before proceeding, ensure that the assumptions of your modeling technique are met. This includes checking for normality, homoscedasticity, and independence of residuals. If any of these assumptions are violated, it could impact the model's performance.

Investigate the lack of fit: Explore the nature of the lack of fit term. It could be due to factors beyond the scope of your current model, such as interactions or non-linear effects that are not captured by the second-order model.

Consider alternative models: If the lack of fit term remains significant, you can consider alternative modeling approaches. These may include exploring higher-order models (e.g., third-order or higher) or non-linear models to capture additional complexity in the system. Keep in mind that as the model complexity increases, the risk of overfitting also increases, so be cautious and consider the trade-off between model complexity and the available data.

Collect more data: If the lack of fit term remains significant and the model complexity is appropriate, you may need to collect more data. Collecting additional data points, especially at specific levels or combinations of factors, can provide more information and help improve the model's fit.

Seek expert advice: Consulting with experts or statisticians experienced in experimental design and modeling can provide valuable insights and guidance specific to your situation. They can help evaluate the model, suggest alternative modeling techniques, or recommend adjustments to the experimental design.

Regarding additional documentation, here are some resources you may find helpful:

"Design and Analysis of Experiments" by Douglas C. Montgomery
"Response Surface Methodology: Process and Product Optimization Using Designed Experiments" by Raymond H. Myers, Douglas C. Montgomery, and Christine M. Anderson-Cook
"Applied Regression Analysis and Generalized Linear Models" by John Fox

These books cover various aspects of experimental design, modeling, and analysis, including higher-order models and strategies for dealing with lack of fit. Additionally, exploring academic journals or online resources specific to your field of study can provide further guidance on modeling techniques and addressing the lack of fit.

Remember, careful analysis and consideration of model assumptions, data collection, and expert advice are key to improving the model's fit and understanding the underlying system more accurately.

Antonio Trasierras

Hi again. Sorry for the late reply. Thanks, Jakub Jagielski and Ma'Mon Abu Hammad for your kind suggestions.

In the meantime, we have conducted three additional experiments at the center point. Furthermore, I have been studying the diagnostic plots of the regression, which are now attached.

In my opinion, several points could be problematic. For example, in the residuals vs leverage plot, points 5, 8, and 11 are out of Cook's distance dashed lines so they could be outliers. On the other hand, the scale-location plot shows that the homoscedasticity assumption could be not met. Lastly (the QQ plot seems ok) the residual vs fitted plot shows that again, samples 5, 8, and 11 may be outliers, and they do not follow a linear pattern. Should we take more samples at these points?

On the other hand, the summary of the obtained model is shown below. Remark that the analysis was done with the rsm package (R).

Call:

rsm(formula = Target ~ SO(x1, x2, x3), data = datos_exp)

Estimate Std. Error t value Pr(>|t|)

(Intercept) 84.21043 5.37188 15.6762 2.737e-07 ***

x1 31.79483 4.65218 6.8344 0.0001331 ***

x2 -0.46876 4.65218 -0.1008 0.9222198

x3 26.93490 4.65218 5.7897 0.0004099 ***

x1:x2 0.03971 6.57918 0.0060 0.9953320

x1:x3 -2.36949 6.57918 -0.3601 0.7280582

x2:x3 1.52293 6.57918 0.2315 0.8227555

x1^2 -20.56312 6.29909 -3.2645 0.0114507 *

x2^2 -12.93203 6.29909 -2.0530 0.0741618 .

x3^2 -16.89441 6.29909 -2.6820 0.0278378 *

---

Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Multiple R-squared: 0.9305, Adjusted R-squared: 0.8524

F-statistic: 11.91 on 9 and 8 DF, p-value: 0.0009589

Analysis of Variance Table

Response: Target

Df Sum Sq Mean Sq F value Pr(>F)

FO(x1, x2, x3) 3 13893.0 4631.0 26.7467 0.0001601

TWI(x1, x2, x3) 3 31.7 10.6 0.0611 0.9788688

PQ(x1, x2, x3) 3 4630.5 1543.5 8.9146 0.0062464

Residuals 8 1385.1 173.1

Lack of fit 3 1188.5 396.2 10.0762 0.0146535

Pure error 5 196.6 39.3

Stationary point of response surface:

x1 x2 x3

0.73008108 0.02699194 0.74717247

Stationary point in original units:

Temp Wgt Time

145.951216 0.656748 93.622761

Eigenanalysis:

eigen() decomposition

$values

[1] -12.78555 -16.68458 -20.91943

$vectors

[,1] [,2] [,3]

x1 -0.02636044 -0.2877568 0.95734067

x2 0.98152697 -0.1889919 -0.02978063

x3 0.18949919 0.9388707 0.28742296

Regarding Ma'Mon Abu Hammad's suggestions:

2. We do not expect interactions between x2 (Wgt) and any other variable to be significant. Moreover, the second-order model does not show any interaction term to be significant.

3 and 4. How can I be sure that the complexity of the second-order model is enough? Perhaps by the residuals vs fitted plot?

5. Thanks for the suggestion and the bibliography. I will read it and try to figure out how to proceed.

How can I measure MMP-1 protein activity or protein level from liver tissue?

Binary Logistic Generalized Linear Mixed model in SPSS. Is it doable?

Looking for Materials with high thermal expansion coefficient and high temperature resistance?

Guidelines for survey (question type) analysis?

What are the values needed for gabor filter's parameter for cifar10 data?

Can we block unwanted images during the x-ray prediction?

No title

What techniques can I use to determine the miscibility of polymer blends?

Sample dilution for quantitative analysis using ELISA?

The procedure compute?

How to manually calculate p value in ANOVA? what is the equation or formula for it?

Is it necessary to report effect size (ES) in statistics of biomedical research given that we already report p values routinely?

How to properly reach normality in residuals with lmer?

How could I get the P-value from the mean, SEM, and n of different groups in the t-test?

How to compare expected and counted values statistically with different p values for each genotype?

T-test and Mann-Whitney U test on groups with different sample sizes and variances?

I am trying to simulate phase change of refrigerant inside pipe extracting heat from panel in comsol. Can anyone help with how to simulate this?

Which statistical test to use for comparing percentages between multiple groups?

In multi-label classification, how do we find the P values and confidence interval for the performance evaluation metrics?

How to fixate four variables in the backwards step selection in SPSS?