This is an important point especially with sparse data in LMs, GLMs and Survival models

Peng and MacKenzie (in two references below) show that V(B_hat) is reference subclass dependent so we may write V(B_hat)= V_r(B_hat). where r is the reference subclass and B_hat  is beta hat the estimated MLE.

In the Linear Model r should be chosen such that it is the most numerous subclass, say r_max - this minimizes  V_r(B_hat). In this way one does not miss significant differences which one might otherwise miss by a suboptimal choice.

lung cancer In observational studies there is usually some choice of reference subclass. Consider lung cancer survival and treatment. Choices include: Palliative care, Surgery and Radiotherapy and Chemotherapy jointly.

Also in GLMs the choice is more complicated but choosing r= r_max  is usually good. There are some exceptions and we give tools to check this.

In addition the choice of r is related to Multi-collinearity (MC) in LMs, GLMs and beyond. We examine the relationship between V_r(B_hat) and the Condition Number of V_r(B_hat). They are highly linearly and positively related. So that choosing r= r_max minimizes V_r(B_hat) and minimizes the MC.

We have obtained analytical forms in low dimensional cases and  have conducted exhaustive simulation studies.

References:

@inCollection{XuPeng:14,

author = {Peng, D. and MacKenzie, G.},

title = { Discrepancy and choice of reference subclass

in categorical regression models},

editor = {MacKenzie, G. and Peng, D.},

booktitle = {Statistical Modelling in Biostatistics and Bioinformatics: selected

papers},

publisher = {Springer},

pages = {159-184},

year = 2014,

address = {Munich, Germany},

isbn = {978-3-319-04578-8},

note = {Published},

____________________________________

Choice of Reference Subclass in Parametric Regression Models with

Categorical Variables

Defen Peng

The Centre for Biostatistics, Department of Mathematics and Statistics, University of Limerick, Ireland.

email: [email protected]

and

Gilbert MacKenzie

ENSAI, Rennes, France and The Centre for Biostatistics, Department of Mathematics and Statistics,

University of Limerick, Ireland.

email: [email protected]

Currently with Biometrics.

Condition number is a measure of

Arjumand,

Taken in isolation, each regression coefficient is with all other variables held to zero. You've changed the meaning of zero for your other intervention. This will be especially impacting if you have the factorial intervention X intervention term in your model, as the meaning of it being held to zero is very different.

Does that help?

Pat

Thanks for the response. Can you able to provide any reading material on this?

Arjumand,

I'd point you to a multiple regression text. Cohen, Cohen, Aiken, & West specifically has a good treatment of this kind of thing.

Pat

This is an important point especially with sparse data in LMs, GLMs and Survival models

Peng and MacKenzie (in two references below) show that V(B_hat) is reference subclass dependent so we may write V(B_hat)= V_r(B_hat). where r is the reference subclass and B_hat  is beta hat the estimated MLE.

In the Linear Model r should be chosen such that it is the most numerous subclass, say r_max - this minimizes  V_r(B_hat). In this way one does not miss significant differences which one might otherwise miss by a suboptimal choice.

lung cancer In observational studies there is usually some choice of reference subclass. Consider lung cancer survival and treatment. Choices include: Palliative care, Surgery and Radiotherapy and Chemotherapy jointly.

Also in GLMs the choice is more complicated but choosing r= r_max  is usually good. There are some exceptions and we give tools to check this.

In addition the choice of r is related to Multi-collinearity (MC) in LMs, GLMs and beyond. We examine the relationship between V_r(B_hat) and the Condition Number of V_r(B_hat). They are highly linearly and positively related. So that choosing r= r_max minimizes V_r(B_hat) and minimizes the MC.

We have obtained analytical forms in low dimensional cases and  have conducted exhaustive simulation studies.

References:

@inCollection{XuPeng:14,

author = {Peng, D. and MacKenzie, G.},

title = { Discrepancy and choice of reference subclass

in categorical regression models},

editor = {MacKenzie, G. and Peng, D.},

booktitle = {Statistical Modelling in Biostatistics and Bioinformatics: selected

papers},

publisher = {Springer},

pages = {159-184},

year = 2014,

address = {Munich, Germany},

isbn = {978-3-319-04578-8},

note = {Published},

____________________________________

Choice of Reference Subclass in Parametric Regression Models with

Categorical Variables

Defen Peng

The Centre for Biostatistics, Department of Mathematics and Statistics, University of Limerick, Ireland.

email: [email protected]

and

Gilbert MacKenzie

ENSAI, Rennes, France and The Centre for Biostatistics, Department of Mathematics and Statistics,

University of Limerick, Ireland.

email: [email protected]

Currently with Biometrics.

Condition number is a measure of

@Gilbert, I find your answer puzzling?  --- I learned early in my studies that the choice of reference class is arbitrary, so importance of categorical predictors should not be evaluated based on p-values associated with individual category coefficients (Wald  tests).  

Categorical predictors should be tested based on the change in deviance (or likelihood) with and without the entire class variable.  Testing full vs reduced models effectively. These tests are invariant to selection of the reference class and are not subject to missing important effects due to poor selection of the reference class.

The significance of (change in deviance, likelihood ratio)  tests represent the importance of the factor in total.  Subsequent analyses of significant effects then typically focuses on describing the nature of these effects by estimating means and confidence intervals per group or using multiple comparison procedures to estimate group differences.

What am I missing here?

@John,

I see what Gilbert is talking about. I learned the same thing as you in my studies. Something that I am getting into now as a data analyst makes Gilbert's argument more interesting. If we have say 100-300 samples in each category, then the choice of variable is arbitrary.

My current data consists of 30-40 samples in one group and 600+ samples in another group. Since I don't care what group is the reference, I will select the larger group to be the reference group. If I do care about which group is the reference group, I will select it.  

It also depends on how you make something a reference group. Some software has a default coding of reference groups versus experimental groups of (0, 1). This coding scheme will make life difficult if you change the reference group. Other software has a coding scheme of (-1, 0, 1). In this coding scheme, I don't think there is a "true" reference group.   

Paul,  How do your methods compare with classification trees and random forests?  They seem to have similarities, but from your references I can't see a description of the algorithm....

Dear John

Sorry I missed your response until now.

What you say is true -  the likelihood is invariant - so that some key properties are invariant - we make this point in the papers. We all learned this at College and taught our students this.

But its not the whole story,  we can still improve secondary aspects of the model. 

(i) If you are using Wald tests choosing the largest subclass is optimal in the sense that you cannot miss effects that might be missed by a sub-optimal choice.

(ii) If you are interested in multicollinearity - eg in sparse data applications - then choosing the largest subclass minimizes the multicollinearity as measured by the condition number.

So this means that you can move away from the traditional approach - provided of course you have the flexibility to choose your subclass. Some traditionalists argue that the reference subclass must be fixed by subject-specific considerations. But we view the choice as a multi-criteria decision at least in observational studies. For example any choice of reference can be evaluated in terms of Vr(B) and the  condition number of the corresponding VC matrix which depends on r.

There are also many interesting results on condition numbers and Vr(B) in terms of (multiple) subclass structures and how to generalize the basic LM model results to GLMs and beyond. All of this adds considerable  insight tnto the area.

Best

Gilbert