Noel is right .... bu look for confounding!!!!!!!!!!!!!!!!!!!!!

Wirat ..... You do have problems with your model .... Most of the time, some reserchers are happy with hight R**2 values.... But often times, that is because of multicollinearity problem .... Your are getteing a allmos perfect fit ..... That is unusual because of the test you have to right ..... The same is valid, for theSE .... You should review, your data.

What is the problem, with the three factors interacions? Which are the levels of each factor? .... Is the factorial experiment a complete factorial?....... Whith your information we can not say anymore.

If the three factor interacion gives you 0 degrees of freedon or in any other factor or intearction, that may be due to CONFOUNDING.

What is the problem with three factors interaction???????

Best regards.

Rafael Roman

It's hard to say for sure with the limited information you provide. But, if I had to guess, I might say that either:

A: your model is too complex given the quantity or quality of the available data

B: some or all of your predictors are correlated

C: A and B

Are you running a factorial design in Minitab? It looks like you are.

If you compare R^2(adj) vs R^2 (pred) they are very different. This tells me that your data are fairly random. I have had this type of issue come up before when I use random data as my response. It looks like you have a very significant intercept term. If all of your values are around 69.551, this can increase your R^2. However, when you ask, what is my predicted response given blah, blah and blah, the answer is always 69.551. This leads to a low Predicted value. Do you need a transformation of your data? What do your residuals look like?

It looks like the data comes from a Designed Experiment, in particular a 2^3 factorial design. With the coded data, that Minitab uses, the Std Err will be exactly the same for all of the terms of the same type, main(linear) effects, 2-way interactions, etc.

The software also uses coded variables for it's analysis. They are coded as (-1, 1). So, they are already centered. The designs are also orthogonal. So, VIF is 1.00 for each term.

No problem Alex.

I wanted to clear that up for anyone that looks at the data.

It's interesting to me, as someone with a fairly broad background in stats, to see how others attack a problem. I know a lot a biostatisticians and economists that never took a class in Design of Experiments from an Industrial Engineering or Stats department that make the suggestions you made.

Meanwhile, a lot of the industrial engineers I know think the data is already centered and has a low VIF for all the terms.

Most of the time I would make the same suggestions you did. If I am dealing with observational data, I go through the same process you suggested. I'm consistently amazed at how often your suggestions are not used. It's a real shame.

Dear Wirat (and all contributors),

I think the comments from Andrew and Alex are quite sound. However, I would suggest to examine the data before analysing the results on R^2 or factorial associated testing. I do not know what is your T variable (Temperature -not in percent-, Thallium?, Thorium?, Titanium?) but it is clear that you are analysing relations between Al (Aluminium), Ga (Gallium), Li (Lithium). The concentrations of chemical elements in percent, ppm, or other units constitute what are known as "compositional data". When applying the standard statistical methods to (raw) concentrations you can find anomalous results. Specially dramatic is the effect of "spurious correlation" (discussed by K. Pearson, 1896, more than a century ago). It makes any correlation between components of a composition useless. In fact, correlation is an artifact produced by the fact that the only information in a vector of concentrations (a composition) is the ratios between the components, and not their absolute values. I am afraid that your analysis is near to be meaningless.

I would recommend you to have a look at compositional data analysis, whose origins can be found in Geology and Geochemistry. Compositional data analysis is considered to be initiated by J. Aitchison (see his contributions in the eighties and his seminal book (1986)). Afterwards, the field has been developed for a more consistent analysis (see, for instance, the book in Wiley, edited by V. Pawlowsky-Glahn and A. Buccianti, or the review paper by R. Tolosana-Delgado, "Uses and misuses of compositional data in sedimentology", Sedimentary Geology 280 (2012) 60-79).

A consequence of these developments is that concentrations should be log-ratio transformed in order to facing most standard statistical analyses which involve expectation, covariance, distances, projections, ..., i.e. most statistical techniques.

Dear All

I really appreciate your contribution and expertise.

Anyway, my experiment was done by 2^3 factorial design (like Prof.Ekstrom mentioned) in MInitab software. I and my colleague are working on a new study of semiconductor. Al (Aluminium), Ga (Gallium),and Li (Lithium) are dopants ( by molar percent) in the solution of Zinc Oxide The response (%T) is the "light transmission value". We expect to see the effect of those three dopants on %transmission. (I do not think that there is any correlation between 3 factors)

Basically, I used the DOE in the field of traditional ceramics, it reinforces me that DOE is applicable for ceramics but by doing so with the Nanoceramics, it is giving me odd results.

However, I we did the main effect plots, it shows pretty good trends but when compared to P-Value, they are not that good.

I would appreciate and looking forward to further discussion. I can provide more detail, just in case.

PS. Thank to Prf. McMahon, Prof. Ekstrom , Prof. Russell, and Prof. Egozcue

It is possible that you need to use a logit transform on your data. to do this, you will take the percent transmittance (%T) and divide that by (100-%T), then take the Ln of that. So, you will have, Ln(%T/(100%-%T))=Y'.

How many times did you replicate your results?

Something I did in the past with a similar design, I replicated it 3 times. (24 samples) When I analyzed the data, I found that 1 factor was significant. When I took the average of my 3 replicates and used that as a response, (8 responses) I found 3 terms were significant.

Did you just use the 3-way interaction as your error term? What does your Pareto chart look like with the 3-way interaction included?

I used average data from 3 samples to analyse. However, I tried to use individual data to analyse, the result remains the same with lower R^2. Besides, the pareto shows no significant factor.

Do you mind if I send you the master file of my experiment? Please take a look this attached file.

I think I see the problem. You use the GLM function for analysis. It assumes that your 3 levels are discrete, not continuous. If you use your DOE analysis, it assumes that you have 2 levels and a center point.

Check out Sheet 1 and Sheet 2 on the Excel workbook. I copied the data I analyzed and the ANOVA table it produced. I used the factorial analysis option in the DOE submenu.

In the summarized data, %Ga is significant.

When I looked at all the data, 27 samples, everything was significant.

Dear Prof. Ekstrom,

Thank you so much for your assistance. The problem has solved. I will keep that in mind.

I think you should use transformation before analysis the data.

Which transformation method is suitable for these data?

I am a bit lost with respect to the original question.

However, I would like to suggest a transformation typical of compositional data. The isometric log-ratio transformation (ilr) is a generalisation of the logit transformation. It is equivalent to the selection of an orthonormal (Cartesian) system of coordinates. For a composition of three parts Al, Li, Ga (three part simplex) it gives two coordinates called balances, for instance:

b_1 = (sqrt(2/3)) log[ Al / (Li Ga)^(1/2) ]

b_2 = (sqrt(1/2)) log[ Li / Ga ]

Note that does not matter which are the units of the composition as b_1 and b_2 are scale invariant.

Dear All,

I have a question since, the replication (as Prof. Ekstrom mentioned) improves the significance of an experimental result by providing a large group of samples. But, the analyzed result of non-replicated data set compared to replicated data set is so huge different (considering by P-Value).

I my opinion, if factors are significant in replicated data set it should show some significant P-value in the non-replicated data. For example, non-replicated data might show P-value close to 0.05 but it was to far.

Another question is why do we need to transform the data prior the test?

If we didn't expect replicating the experiment to provide a significant improvement in the quality of our estimates, then when fitting a model with p parameters, we might as well stop collecting data after p+1 data points. If our data contained no measurement errors, this would in fact be sufficient. But this is never the case. Instead, consider the following:

Suppose we do a simple experiment where we drop a ball (so initial velocity = 0) from a height = h0 at t = t0, and record the position (h1, h2, ...hi, ..., hn) with respect to time at n subsequent time points, (t1,t2, ...ti,...tn). Suppose that our detector has some random, normal, i.i.d. measurement errors with s.d. sigma. From such a data set, we would be able to calculate g, the acceleration due to gravity from h(t) = h0 - 1/2 * g * (t-t0)^2. It should come as no surprise that the standard error of our estimate for g depends on the measurement error sigma, and if sigma is large the p-values will not be significant.

If we repeat the previous experiment m times, so that for the ith time point we have m measurements (ti.1,ti.2,...ti.j,...ti.m), then the uncertainty in the position hi is now sigma/(m^1/2). In general, the p-values are non-linear functions in the uncertainty of the data. One consequence of this non-linearity is that the effects of modest reductions in uncertainty through noise reduction or averaging may be larger than we might intuitively expect.

We often transform the data to improve the numerical stability of computations. I can't really provide a good explanation for why it works, but text books on data analysis and machine learning often provide convincing examples that it works in practice.

Prof. McMahon

This quot made me awake "It should come as no surprise that the standard error of our estimate for g depends on the measurement error sigma, and if sigma is large the p-values will not be significant".

For the transformation data, I am still not convincing. From my side, (industrial practice), when we do transformation the data, to normal distribution ,to determine Cpk, for example. It would pass the Cpk value we want but we know it is not the real data so, when we analyze the data we still are looking for the actual one.

But the thing, is that convincing me, is that we transform data to do S-hat. And it is good for industrial practice.

This is such a nice discussion.

Please tell me more.

Thank you so much.

Looking at your Minitab ouput, the fact that you have one of your factors with a t-value of 4.00 but with a ("large") p-value (0.156) suggests that you have very little degrees of freedom for your MS error and, consequently, you may be overfitting the experimental data. I would do a backward elimination procedure to reduce the number of terms in the model and - hopefully - decrease the MSError.

Noel is right .... bu look for confounding!!!!!!!!!!!!!!!!!!!!!

Wirat ..... You do have problems with your model .... Most of the time, some reserchers are happy with hight R**2 values.... But often times, that is because of multicollinearity problem .... Your are getteing a allmos perfect fit ..... That is unusual because of the test you have to right ..... The same is valid, for theSE .... You should review, your data.

What is the problem, with the three factors interacions? Which are the levels of each factor? .... Is the factorial experiment a complete factorial?....... Whith your information we can not say anymore.

If the three factor interacion gives you 0 degrees of freedon or in any other factor or intearction, that may be due to CONFOUNDING.

What is the problem with three factors interaction???????

Best regards.

Rafael Roman

In the design WIrat used, it confounds 2-way interactions with 3-way interactions. The design needs to be looked at as a 2-level design with center points not a 3 level design. That was the issue with the original data analysis. 

Hi Wirat lerdprom,

Have a look at this link, it can be of help: http://blog.minitab.com/blog/adventures-in-statistics/regression-analysis-how-do-i-interpret-r-squared-and-assess-the-goodness-of-fit . All the best.

http://blog.minitab.com/blog/adventures-in-statistics/regression-analysis-how-do-i-interpret-r-squared-and-assess-the-goodness-of-fit

The sample size might be too small, the number of parameters are relative too high. The model is overfitting. I think you only have 8 obervations, and the the three way interaction is used as regression error with 1 degree freedom.  For regression, The ratio between the sample size and unoknown parameters should be at least 5. At least 10 is recommended. If you only have 8 observations, even eliminate the two way interactions, you still have problem for doing regression.

Please check, is there a weight (or count variable) to represent the actual number of observations for  each given A1 Ga,Li.

Comment.

As far as I understand, the easiest way to detect confunding is by looking at the columns of the design matrix. When there is confounding, at least one of the columns of the matrix is identical to another, which means that is the same linear combination. Obviously this makes that the design matrix is not of full rank, the determinant is zero, and no inverse exist. This problem is solved by ignoring one of the columns and by understanding that something is confunded with something but we know that the effect is usually negligible (This is the principle of confunding).

The other ways to detect confunding is making a correlation table between the columns of the design matrix. In case of confunding you will find correlations of 1.0 between at least 2 columns.

With SAS GLM or any program that uses the constraint that the last constant is zero, the easiest way to detect confunding is by looking at the type I and type III sum squares, in the case of confunding, the sums of squares and degrees of freedom are ZERO for any of the effects, but still you can not detect which effects are confunded.