Hamed,

since the outcome is continuous, you may assume the residuals are independent identical normal distributed, rather the Y1, Y1 is normal for each given level, you have no way to verify.

As your data coding, use linear regression or GLM, Say SAS reg, or SAS GLM, give similar results. See following coding and attachment.

Question? what is 0 0 0 dtabd for x1 x2 x3? If no meaning you may delete it.

With it, the model will assume the difference between 1 and 0 , 0 and -1 for each factor x1,x2,x3 are the same. If not, you may change coding.

hope help

data

try;

input

note $ X1 X2 X3 Y1;

cards

;

++- 1 1 -1 67.5

-++ -1 1 1 0.5

+-- 1 -1 -1 8.9

--+ -1 -1 1 0.4

+++ 1 1 1 56.9

--- -1 -1 -1 8.7

+-+ 1 -1 1 6.6

-+- -1 1 -1 69.4

000 0 0 0 37.1

;

data

try; set try;

y2=log(y1);

proc

reg;

model

y1=x1 x2 x3;

run

;

proc

GLM;

class

x1 x2 x3;

model

y1=x1 x2 x3/solution;

run

;

Why do they show completely different p-values while the model estimates are exactly the same?

Actually, both analyses should give identical results. Maybe the problem here is that the GLM tests are performed using the Chi² distribution. Maybe useing F-tests here would be better.

Generally, how can I find that which model (GLM or standard least squares) should be used for analysis when I have no idea about response distribution?

The response distribution (more specifically: the distribution of the *errors*) must be known or assumed. If it is not known, the assumption should be reasonable. There is not much more to do. One may look at the actual residual distribution to see if it is strongly contraditory to the assumption, one may also simulate data following the assumption and see if the "real" reasults are in strong contradiction to the simulated results.

does it mean that least squares model can not be suitable as it is affected by outliers?

If 0.144 is a physically impossible or really non-sensical value, you should anyway remove it. Otherwise you actually have too little data to judge from the data alone that this is an outlier. Typically the best option is then todo the analysis onec with and once without this value; if the results are grossly different, some careful reasoning is required (otherwise it doesn't matter anyway).

Outliers affect all analyses, not only least-squares. But only for least-squares it is relatively simple to detect outliers (given enough data). This does not mean that least-squares is more sensitive to the presence of outliers than other methods, except, for sure, robust methods (that usually have other drawbacks).

Paul, can you elaborate how ODA exactly applies to this problem? How does ODA decribe the relationship between the predictors and the response?

It was kind of hard for me to see the results of your outcome one my screen. Something that I have seen when using center points in DOE and not using the DOE analysis option in the software is that the software can think each level, -1, 0 and 1 are all categorical levels. I've had sofware give error messages if I use a design like yours and look for interactions. The software assumes that when you ask for X1*X2, you will have X1[-1]*X2[-1], X1[-1]*X2[0], X1[-1]*X2[1], X1[0]*X2[-1], X1[0]*X2[0], X1[0]*X2[-1], X1[1]*X2[-1], X1[1]*X2[0], and X1[1]*X2[1]. Since you don't have these 9 different combinations, the software might try to compensate some other way. 

What software did you use in this analysis? Is there a reason you didn't use the DOE section to analyze the data? If you remove the center point from the analysis, GLM and Least Squares should give the same results. When you add in the center point, I can see things being different for the reasons above. 

I did get a chance to analyze the Y1 data quickly. Since you only have a single center point and not a series of axial points, you can't really analyze for a quadratic term. I found that terms X1, X2, X3, X1X3 and X2X3 were statistically significant. The center point is not significant. As Jochen will tell you, there is a difference between statistical and practical significance. Which is for you to decide.

Because of the small number of samples in the analysis and the fact I pooled insignificant terms, the statistical significance of some of these terms might be (most likely are) bloated. 

Typically a glm function uses different estimation (such as ML) than regression, and may report z-scores instead of t-scores. So you would expect to see some difference there. But there is another issue with this dataset which is a problem. You have 8 data points and with a full factorial you are also fitting 8 parameters (X1, X2, X3, X1*X2,  X1*X3,  X2*X3, X1*X2*X3, const) so you're basically overfitting. E.g. (the m are the X coded 0/1 instead of -1/1)

. drop if descriptor=="000"

. anova y1 m1 m2 m3

Source | Partial SS df MS F Prob > F

-----------+----------------------------------------------------

Model | 5078.11389 3 1692.70463 3.89 0.1113

m1 | 463.601246 1 463.601246 1.06 0.3604

m2 | 3599.76141 1 3599.76141 8.27 0.0452

m3 | 1014.75124 1 1014.75124 2.33 0.2015

Residual | 1741.3251 4 435.331275

-----------+----------------------------------------------------

. regress y1 m1 m2 m3

------------------------------------------------------------------------------

y1 | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

m1 | 15.225 14.7535 1.03 0.360 -25.73727 56.18727

m2 | 42.425 14.7535 2.88 0.045 1.46273 83.38727

m3 | -22.525 14.7535 -1.53 0.202 -63.48727 18.43727

_cons | 9.8 14.7535 0.66 0.543 -31.16227 50.76227

------------------------------------------------------------------------------

glm with ML

. glm y1 m1 m2 m3, family(gaussian) link(identity)

------------------------------------------------------------------------------

| OIM

y1 | Coef. Std. Err. z P>|z| [95% Conf. Interval]

-------------+----------------------------------------------------------------

m1 | 15.225 14.7535 1.03 0.302 -13.69132 44.14132

m2 | 42.425 14.7535 2.88 0.004 13.50868 71.34132

m3 | -22.525 14.7535 -1.53 0.127 -51.44132 6.39132

_cons | 9.8 14.7535 0.66 0.507 -19.11632 38.71632

------------------------------------------------------------------------------

glm with iterated least squares

. glm y1 m1 m2 m3, family(gaussian) link(identity) irls vce(oim) noheader nolog

------------------------------------------------------------------------------

| OIM

y1 | Coef. Std. Err. z P>|z| [95% Conf. Interval]

-------------+----------------------------------------------------------------

m1 | 15.225 14.7535 1.03 0.302 -13.69132 44.14132

m2 | 42.425 14.7535 2.88 0.004 13.50868 71.34132

m3 | -22.525 14.7535 -1.53 0.127 -51.44132 6.39132

_cons | 9.8 14.7535 0.66 0.507 -19.11632 38.71632

------------------------------------------------------------------------------

Note that -anova- and -regress- give the same p-values and glm gives the same coefficients and quite similar p-values except for m2, but generally m2 is significant whichever way you look at it.

Now see what happens when fitting the full factorial model:

glm with ML

. glm y1 m1##m2##m3, family(gaussian) link(identity) noheader nolog

------------------------------------------------------------------------------

| OIM

y1 | Coef. Std. Err. z P>|z| [95% Conf. Interval]

-------------+----------------------------------------------------------------

1.m1 | .1999998 19.8 0.01 0.992 -38.60729 39.00729

1.m2 | 60.7 19.8 3.07 0.002 21.89271 99.50729

|

m1#m2 |

1 1 | -2.100001 28.00143 -0.07 0.940 -56.9818 52.7818

|

1.m3 | -8.3 19.8 -0.42 0.675 -47.10729 30.50729

|

m1#m3 |

1 1 | 6 28.00143 0.21 0.830 -48.8818 60.8818

|

m2#m3 |

1 1 | -60.6 28.00143 -2.16 0.030 -115.4818 -5.718202

|

m1#m2#m3 |

1 1 1 | 42.4 38.34254 1.11 0.269 -32.75 117.55

|

_cons | 8.7 14.00072 0.62 0.534 -18.7409 36.1409

------------------------------------------------------------------------------

glm with iterated least squares

. glm y1 m1##m2##m3, family(gaussian) link(identity) noheader nolog irls vce(oim)

------------------------------------------------------------------------------

| OIM

y1 | Coef. Std. Err. z P>|z| [95% Conf. Interval]

-------------+----------------------------------------------------------------

1.m1 | .1999998 19.8 0.01 0.992 -38.60729 39.00729

1.m2 | 60.7 19.8 3.07 0.002 21.89271 99.50729

|

m1#m2 |

1 1 | -2.100001 28.00143 -0.07 0.940 -56.9818 52.7818

|

1.m3 | -8.3 19.8 -0.42 0.675 -47.10729 30.50729

|

m1#m3 |

1 1 | 6 28.00143 0.21 0.830 -48.8818 60.8818

|

m2#m3 |

1 1 | -60.6 28.00143 -2.16 0.030 -115.4818 -5.718202

|

m1#m2#m3 |

1 1 1 | 42.4 38.34254 1.11 0.269 -32.75 117.55

|

_cons | 8.7 14.00072 0.62 0.534 -18.7409 36.1409

------------------------------------------------------------------------------

. regress y1 m1##m2##m3, noheader

------------------------------------------------------------------------------

y1 | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

1.m1 | .1999998 19.8 0.01 0.994 -251.3829 251.7829

1.m2 | 60.7 19.8 3.07 0.201 -190.8829 312.2829

|

m1#m2 |

1 1 | -2.100001 28.00143 -0.07 0.952 -357.8919 353.6919

|

1.m3 | -8.3 19.8 -0.42 0.747 -259.8829 243.2829

|

m1#m3 |

1 1 | 6 28.00143 0.21 0.866 -349.7919 361.7919

|

m2#m3 |

1 1 | -60.6 28.00143 -2.16 0.276 -416.3919 295.1919

|

m1#m2#m3 |

1 1 1 | 42.4 38.34254 1.11 0.468 -444.7882 529.5882

|

_cons | 8.7 14.00072 0.62 0.646 -169.196 186.596

------------------------------------------------------------------------------

"which modelshould be used for analysis when I have no idea about the response distribution"

What you really care for is the distribution of the residuals and that gets tougher in multiple-regression situations. You try to make reasonable assumptions. The other question is how to choose a model, for which there is no definitive answer. In principle you can compare R2 between different least-squares solvers and AIC between ML solvers, but you must judge carefully. 

From your experiment you have 9 linear equations with 9 unknown variables. No matter which technique or software you use to solve these equations, you get the same solution. In order to get an estimate of the uncertainties associated with the solution, you need more equations than unknown variables (unless you know the uncertainty of each observation a priori). You can get that by repeating your experiment a number of times, for example. Once you have the uncertainties associated with the solution, you can easily check if any of the solution parameters are significantly different from zero or not, e.g. using a Student t test.

I recommend that you use the least squares method first. It provides you with best estimates and standard uncertainties of model parameters quickly and easily, independently of the type of distributions underlying your data. If you have outliers in your data, you either have poor data or a poor model. You have to fix that by taking more data or refining your model, rather than using socalled robust statistics.