In reference to the last pdf you posted (Comparing outputs), I think it's first important to explain why the GLM estimates (Labeled 2 & 3 in the pdf) differ from the ANOVA/GLM estimates (Labeled 1 in the pdf).

First, you have primarily labeled *post-hoc* tests in the pdf ("Dunnett Test" section of #1, and everything included for #2 & 3). post-hoc tests are peformed following significant global tests to determine specifically where significant differences lie. The first two lines highlighted in #1 (rows for "intercept" & "treatment" in the ANOVA table) actually are global tests (intercept really isn't, but that's not important here).

GLM and ANOVA will produce identical *global* test results (they truly are two sides of the same coin), but, in this case the post-hoc methods in #1 will not produce the same results as #2 & 3 because they are using different methods to compare specific means: #1 states that it is using the Dunnett test, while #2 & 3 are using Wald tests. These tests are not equivalent, and therefore produce different results.

The Dunnett test (from what I've briefly skimmed, it's not one I'm familiar with) has a t-distribution and adjusts df based on the total number of comparisons being made. The Wald tests have Chi-squared distribution with 1 df each. With fewer degrees of freedom, the critical points and p-values are lower.

So to answer your questions above:

1- (assuming #2 & 3 are the "overparameterized" models) The different models are producing different results because they are using different post-hoc methods. I think which one makes more sense really depends on how conservative you want to be and what the research question is.

2- GLM with a log link and GLM with an identity link produce different results because the link function changes the estimated variance in relation to the means because the log transformation is not linear. The differences, however, are very small. I'm sure there are many reasons to use a log-link, but you have to decide if a log transformation of the outcome is appropriate to your research.

3- Remember, the ANOVA is a global test, and actually says nothing about the significance of *specific* inter-group comparisons. The post-hoc test (Dunnett) provides a different results than the default GLM output because they use different methods for comparisons.

Either GLM or ANOVA would be appropriate for unbalanced sample sizes since they are equivalent *global* tests. The "appropriate" post-hoc tests, however, are probably going to change between applications. Dunnett seems to be fine and able to handle different sample sizes.

Personally, I would select ANOVA post-hoc (Dunnett, Bonferronni, Scheffe's method, etc) if:

*I really wanted my estimates to be conservative (and very defensible)

*I had a lot of levels

*I needed to compare multiple levels to multiple other levels (not just one reference level).

I would use and GLM/Wald if:

*I was less concerned with being conservative

*I had continuous predictors (ANOVA falls apart on these unless you create categories)

*Only had 2-3 levels (multiple comparisons are less worrisome here)

Hope that helps.

Hi Elena, I am not sure I can answer your question in a general way that covers all statistical packages. I think some of the answer may be historical. A very long time ago ANOVA routines were very limited. In some cases they could only do a typical experimental ANOVA, by which I mean all groups were of equal size. The GLM procedure, which to the best of my knowledge started with SAS (PROC GLM) was needed for anything more general, such as unbalanced designs--I think ANOVA routines handle this now. Also, I think there is some difference as to whether all ANOVA routines will handle ANCOVA, a continuous covariate. GLM routines I have seen differ on how they incorporate interactions, particularly between continuous and categorical variables. As for how the routines test the contrasts, that's obviously very specific to the software package. The underlying math is the same, so in any case you should get the same result if both the ANOVA and GLM routines can compute the same model (I am assuming we are talking about fixed, not random, effects ANOVA). So, I don't really know that you need to justify using a GLM routine as you should get the same results. However, if you cannot obtain the contrasts you need from the GLM routine available to you, then an ANOVA routine would be the better choice. Also, GLM routines can be a more convenient choice than regression routines with some packages as it avoids the work of converting categorical predictors into dummy variables. Bob

Thank you, Bob.

No, the results are not the same (see the attached example), this is the problem... The difference between the groups analyzed by one-way ANOVA vs. ANOVA in Generalized Linear Model (called GLZ, STATISTICA) are different. Basically, ANOVA/GLZ detects more subtile differences as significant compared to plain ANOVA or ANOVA in General Linear Model module of the same package. The last two produce identical results (as expected).

The computation in GLM is apparently different.

/E

Seems that you are talking past each other. Elena is asking about "generalized" linear models and Robert is talking about "General" linear model. The former is for discrete response (binary, poisson etc.) , whereas ANOVA and General linear models are for continuous normally distributed data. The choice between General and ANova is about balance in the design, whereas the choice to use Generalized linear model is about the data type of the response...normal vs. discrete data types or other continuous data types like beta, gamma etc.

Then the example provided by Elena includes other free parameters like how to handle scale parameter and link funcitons. I suggest some reading on Generalized LInear models texts such as:

http://www.amazon.com/Generalized-Edition-Monographs-Statistics-Probability/dp/0412317605

Thank you, John. You are talking about generalized linear model in general and areas of their application, but this is not really what I am asking about.

To be specific, let's talk about generalized lm module in STATISTICA - since ANOVA as an option is offered in GLZ, I would like to know under which circumstances it can (and should) be used, how the between-group differences are calculated and why the results are different from those obtained using ANOVA module.

I know this book, will take a look once again...

Looks like you need a STATISTICA user. Short of that, I would put together some test data with various combinations of assumptions and run several of these test problems to reverse engineer what they are doing. Hopefully a generous STATISTICA user will show up soon!!

yes a seasoned Statistica user would be great. I posted the same question to StatSoft forum, but it is awfully quiet there...

I believe Bob is correct above. The results for GLM analysis of a model that contains a single categorical variable will produce identical results to an ANOVA of the same model. If the processes are calculated in different ways, then the differences should be incredibly small. However, the output provided by the two models are likely to be very different.

The only output that ANOVA hypothesis test will give you is whether there are *any* differences present in the data (this is why it is referred to as a "global" test). This appears in the first output in the pdf you provided (note that it refers to a Wald test). This would be equivalent to a Wald test in GLM testing all coefficients are simultaneously equal to zero.

In GLM, the default output will Wald test *individual* coefficients to determine if they are zero. This is not equivalent to an overall Wald test which tests them all individually. If you had not selected the identity link function, instead of log, you could calculate the mean of each treatment group by adding the coefficients for the intercept and the respective treatment. The p-values displayed are testing if each treatment effect is different than treatment 7 (the referent group - note there is no effect estimated for group 7).

In short, you're getting different outputs because the p-values in the two outputs (ANOVA and GLM) are making different comparisons, not because they are calculated using different methods. Also, you should use the identity link function to compare means, instead of the log function, unless you're interested in the log transform of the mean.

You should use ANOVA if you're interested in seeing if there is any difference between the means across treatment groups, and use GLM if you're interested in seeing if specific treatment groups were different than a referent treatment.

Basically, GLZ in Statistica is the same as GLM in any other statistical package. Since your data is balanced with continuous response, either using ANOVA or General regression model (not Generalized linear model), both give you the same answer.

Only when you have your response variables that are other types rather than continuous such as count, binary etc, in the cases, you should use GLM (generalized linear models) with certain link functions (for Poisson (count response), log and for binary or binomial, logit etc.). The link function for Normal is Identity not log.

Hope this helps.

Thank you William and Hwan-Jin, your answers are much appreciated. Yes, GLZ in Statistica is the same as GLM in other stat. packages. The dataset in my example is balanced and could be analyzed by simple ANOVA, of course, but this is just an example to test performance of different options, which turns out to be useful, because the test results are difficult to explain…

I was also sure that Generalized LM with NORMAL distribution and IDENTITY link will produce an output for the between-group comparison that is identical to General LM, however, it does not (see pdf attached); regardless of the post-hoc test. that is used in General LM. Therefore, my “WHY?” remains for this part.

The difference from zero as a main test and then for individual treatments is also quite clear. What is not clear –

1. Why there are different outcomes for “sigma-restricted” vs. “overparameterized” models? And why the latter makes more sense?

2. Why General LM and Generalized LM with IDENTITY link do not produce the same results?

3. Simple ANOVA provides much more conservative estimate for the deviation from the reference group compared to the Generalized LM. What strategy one should follow when analyzing data that are somewhat unbalanced (number of replicates per treatment varies 3 to 5)?

I am not new to neither statistics nor Statistica, but have to admit that this is confusing:)

I'm not familiar with Statistica but the GLM output you show is for a Dunnett test - this will be more conservative than uncorrected coefficients.

The overparameterized model allows the regression to mimic the standard ANOVA output and estimates means for each condition. The sigma-restricted parameterization estimates the parameters as deviations from the grand mean that sum to zero (the restriction).

Hi All, I zeroed in on the use of GLM as in PROC GLM, but I agree with all who remarked on generalized linear models which have a variety of link functions available. I do not know much of anything about Stasitica. I think that the discussion has pretty well covered the distinction of a generalized linear model with link function from the simpler general linear model. Using a linear link function it is possible to recover the parameter estimates of an ANOVA or other linear model, although it's not clear to me why one would do this when one does not need any of the additional features of a generalized linear model or related general linear mixed model. Bob

Thank you, Thom.

Bob: there are several sets of data, some of them are not normally distributed and unbalanced. For these data, using GLM with log-link produces a better p-plot compared to ANOVA. Would this justify for the application of the former for ALL data sets (just to use a uniform approach)?

In reference to the last pdf you posted (Comparing outputs), I think it's first important to explain why the GLM estimates (Labeled 2 & 3 in the pdf) differ from the ANOVA/GLM estimates (Labeled 1 in the pdf).

First, you have primarily labeled *post-hoc* tests in the pdf ("Dunnett Test" section of #1, and everything included for #2 & 3). post-hoc tests are peformed following significant global tests to determine specifically where significant differences lie. The first two lines highlighted in #1 (rows for "intercept" & "treatment" in the ANOVA table) actually are global tests (intercept really isn't, but that's not important here).

GLM and ANOVA will produce identical *global* test results (they truly are two sides of the same coin), but, in this case the post-hoc methods in #1 will not produce the same results as #2 & 3 because they are using different methods to compare specific means: #1 states that it is using the Dunnett test, while #2 & 3 are using Wald tests. These tests are not equivalent, and therefore produce different results.

The Dunnett test (from what I've briefly skimmed, it's not one I'm familiar with) has a t-distribution and adjusts df based on the total number of comparisons being made. The Wald tests have Chi-squared distribution with 1 df each. With fewer degrees of freedom, the critical points and p-values are lower.

So to answer your questions above:

1- (assuming #2 & 3 are the "overparameterized" models) The different models are producing different results because they are using different post-hoc methods. I think which one makes more sense really depends on how conservative you want to be and what the research question is.

2- GLM with a log link and GLM with an identity link produce different results because the link function changes the estimated variance in relation to the means because the log transformation is not linear. The differences, however, are very small. I'm sure there are many reasons to use a log-link, but you have to decide if a log transformation of the outcome is appropriate to your research.

3- Remember, the ANOVA is a global test, and actually says nothing about the significance of *specific* inter-group comparisons. The post-hoc test (Dunnett) provides a different results than the default GLM output because they use different methods for comparisons.

Either GLM or ANOVA would be appropriate for unbalanced sample sizes since they are equivalent *global* tests. The "appropriate" post-hoc tests, however, are probably going to change between applications. Dunnett seems to be fine and able to handle different sample sizes.

Personally, I would select ANOVA post-hoc (Dunnett, Bonferronni, Scheffe's method, etc) if:

*I really wanted my estimates to be conservative (and very defensible)

*I had a lot of levels

*I needed to compare multiple levels to multiple other levels (not just one reference level).

I would use and GLM/Wald if:

*I was less concerned with being conservative

*I had continuous predictors (ANOVA falls apart on these unless you create categories)

*Only had 2-3 levels (multiple comparisons are less worrisome here)

Hope that helps.

Yes, William, thank you.

Good morning Elena! Let´s go to talke about the General LM and Generalized LM.

a) Both General LM and the Generalized LM use the same canonical link function (Identity link fuction);

b) If the dataset follows the assumption of normality and homocedastic the results of both analysis will be the same;

c) Otherwise, if the dataset does not meet the same variance between the treatments the results will be different. This is because the typical estimation method between procedures are diferent. Generalized LM procedure uses Maximum Likelihood instead General LM that uses Ordinary Least Squares. So, General LM requires that Sigma2=identical matriz.

d) These differences gives the advantage for the Generalized LM to consider, during the proccess of calculation, the heterogeneity (Sigma2 different identical matriz). This is why we have some diferences in the results of both procedure.

Best regards Elena.