Dear Hamed, I will give you an insight into your question may be it can guide you if not, just ignore, though it would have been important to tell us the type of data you are dealing with:

I would advise that you carry out a RMSE since it is widely used as a standard statistical metric to assess model performance, of say air quality, economics, climatic research.....etc. The MAE is another useful metric that is also employed in model evaluation. To my knowledge both of the said metrics have been used in several research to examine model performance for the last many decades but there is no agreed metric indicated as the most appropriate one. 

A gain if you have unbiased distribution of the error SE and RMSE will be the same.

I

Hi Hamed,

Since you have carried out 3 replication tests it is advisable to check lack of fit test. Moreover, try to check normality plot and predicted vs actual plot. A regression model exhibits significant lack-of-fit when it fails to adequately describe the functional relationship between the experimental factors and the response variable. Lack-of-fit can occur if important terms from the model such as interactions or quadratic terms are not included. It can also occur if several, unusually large residuals result from fitting the model. Do not forget to eliminate significant terms from your model in order to improve its ability for prediction. This can be done through backward elimination technique.

Another suggestion. Try to use different parameter combination which is not considered in your design matrix but within your experimental range. Herein, you need to replace the factors (A,B and C)  in your model with this parameter combination and calculate the predicted value of your response. Then do real experiment using this combination to compare between the predicted value and the actual value. This is the best way to validate your model.

Dear Cliff & Omar,

Thanks for your comments which there were interesting.

Please see attached file about RMSE and MAE. What is your idea how error can be calculated for each observation since I have carried out three replication tests. Can it be considered (Xi-XAvg)?

Also I have used point prediction tool of DX7 software and predicted the responses then I have carried out 3 replication tests for each 2 targets (there were two prediction points).

Therefore now I have values for predicted models (including SE mean, SE Pred., Confidence intervals (CI) and predicted intervals (PI) which is measured by software (The Excel sheet you already have it). My question is that what is difference between CI and PI? How it is calculated by software? And which of them is more useful for me CI or PI for interpretation of results?.  

Please check the below Link:

https://en.wikipedia.org/wiki/Confidence_interval

https://en.wikipedia.org/wiki/Prediction_interval

It should be noted that I used historical data design of DX7 software. I had prepared ANOVA tables for each response and R2 of model was in range of 0.97-0.999 for different responses.  

Now I have predicted responses by software for 2 optimum points (see former Excell file of Point Prediction; it is predictions for one of prediction point) and results of real tests (including 3 replications). Therefore for real data, average amount and standard deviation and confidence interval can be determined easily.

Also I have made regression analysis between predicted responses and average amounts of 3 experiments of real tests and the R2 was more than 0.99. Is it sufficient?

from the other side I think if the results of all 3 real experiments will be in lower and higher confidence interval, it means the prediction model was successful. What is your idea?

Thanks and best regards,

Hamed