It helps to start with the meaning of the intercept – this is the estimated value of the response when all the predictor values are zero. This intercept is actually calculated in a multiple regression by including (hidden to you) an extra variable which is a constant – that is a set of 1’s for each and every observation in the data set. When you add in a categorical variable State, the software t creates a set of 1 and 0 values for each and every State but it has to drop one to avoid exact collinearity.

It cannot estimate this – that is dataset and model with a constant and 5 dummy variables

Constant Alab Calif Texas Utah Wyoming

Alabama 1 1 0 0 0 0

Calif 1 0 1 0 0 0

Texas 1 0 0 1 0 0

Utah 1 0 0 0 1 0

Wyoming 1 0 0 0 0 1

So it estimate this – that is dataset and model with a constant and 4 dummy variables, dropping one of the states, here Alabama ( arbitrarily - the first alphabetically)

Constant Calif Texas Utah Wyoming

Alabama 1 0 0 0 0

Calif 1 1 0 0 0

Texas 1 0 1 0 0

Utah 1 0 0 1 0

Wyoming 1 0 0 0 1

The key to interpretation is to realize that the intercept still gives he estimated value of the response when all the predictor values are zero. So the intercept is the estimated poverty when the environmental variables are all zero and in the State of Alabama – the dropped State. The regression coefficient for each of the other States gives you the difference in the poverty value in that State (with a value 1) as compared to the base category of Alabama. That is in this dummy variable coding you are modelling differentials. I often choose the coding such that the omitted category has the lowest value on the response. So I do an initial fit with whatever the software gives me and then choose a suitable coding. Other times I choose the modal category (the most commonly occurring in the data) as the referent category so that the intercept will be well estimated. The standard errors of the differentials are determined by how many observations fall into that category.

There are other forms of coding of categorical variables that are used.

This is pretty comprehensive

http://www.ats.ucla.edu/stat/sas/webbooks/reg/chapter5/sasreg5.htm

This explains the basics in SPSS

http://www.ats.ucla.edu/stat/spss/webbooks/reg/chapter3/spssreg3.htm

It helps to start with the meaning of the intercept – this is the estimated value of the response when all the predictor values are zero. This intercept is actually calculated in a multiple regression by including (hidden to you) an extra variable which is a constant – that is a set of 1’s for each and every observation in the data set. When you add in a categorical variable State, the software t creates a set of 1 and 0 values for each and every State but it has to drop one to avoid exact collinearity.

It cannot estimate this – that is dataset and model with a constant and 5 dummy variables

Constant Alab Calif Texas Utah Wyoming

Alabama 1 1 0 0 0 0

Calif 1 0 1 0 0 0

Texas 1 0 0 1 0 0

Utah 1 0 0 0 1 0

Wyoming 1 0 0 0 0 1

So it estimate this – that is dataset and model with a constant and 4 dummy variables, dropping one of the states, here Alabama ( arbitrarily - the first alphabetically)

Constant Calif Texas Utah Wyoming

Alabama 1 0 0 0 0

Calif 1 1 0 0 0

Texas 1 0 1 0 0

Utah 1 0 0 1 0

Wyoming 1 0 0 0 1

The key to interpretation is to realize that the intercept still gives he estimated value of the response when all the predictor values are zero. So the intercept is the estimated poverty when the environmental variables are all zero and in the State of Alabama – the dropped State. The regression coefficient for each of the other States gives you the difference in the poverty value in that State (with a value 1) as compared to the base category of Alabama. That is in this dummy variable coding you are modelling differentials. I often choose the coding such that the omitted category has the lowest value on the response. So I do an initial fit with whatever the software gives me and then choose a suitable coding. Other times I choose the modal category (the most commonly occurring in the data) as the referent category so that the intercept will be well estimated. The standard errors of the differentials are determined by how many observations fall into that category.

There are other forms of coding of categorical variables that are used.

This is pretty comprehensive

http://www.ats.ucla.edu/stat/sas/webbooks/reg/chapter5/sasreg5.htm

This explains the basics in SPSS

http://www.ats.ucla.edu/stat/spss/webbooks/reg/chapter3/spssreg3.htm

You set in up wrong .

This it

Ala 0 0 0 0

Cal1 0 0 0

TX 01 0 0

UT 0 0 1 0

WY 0 0 0 1

You set it up wrong The correct way is as follows:

Ala 0 0 0 0

Cal1 0 0 0

TX 01 0 0

UT 0 0 1 0

WY 0 0 0 1

No it is correct . You are forgetting the hidden constant which is vital for understanding what you are doing.

Sometimes it is just due to multicolinearity reasons: When one variable is linear dependent to another variable, then SPSS excludes the dependent variable. Try to manually investigate if the dropped variable is linear with 1 or more other variables of your model.

Suggestions:

The variables excluded were non-numeric (Thanks Dr. Kelvyn Jones for explaining hidden constant) or step-wise had been used instead of 'enter' , or there is a multicolinearity

When using dummy variables, you need a comparison group in order to be able to interpret the coefficients in the regression analysis. SPSS is automatically excluding one state to provide you with this comparison group.

Dear when you introduce all the states in your analysis you are basically facing a problem called "Dummy Variable Trap".

SPSS automatically exclude one category which is now your reference category. In you example let's suppose SPSS exclude 'Ala', now 'Ala' is your reference category and all other states will show your the relative differences from 'Ala' State.

Read "Introduction to Econometrics" by "Dougherty".

Ask me if you haven't a soft copy. I will send you.

Best of Luck for your research

To my mind Laura and Muhammad have identified the issue clearly – the 5 States are a categorical data set; and one way of dealing with this is to accept the SPSS default and effectively drop one of them so it becomes the constant – so that your are modelling differentials. But it would be good to hear from Abu Sufiyan if that is what is really going on.

It is also sometimes useful to get out the estimates associated with all 5 states; this is the way to do it.

1) For each of the environmental predictor variables – centre them around a convenient middle value such as the mean or median; this is not a necessary step but makes for easier interpretation – each predictor becomes (xi – xbari)

2) Create the full dummy variable set – so 5 States becomes 5 dummies (do not drop any)

State Alab Calif Texas Utah Wyoming

Alabama 1 0 0 0 0

Calif 0 1 0 0 0

Texas 0 0 1 0 0

Utah 0 0 0 1 0

Wyoming 0 0 0 0 1

3) Add all 5 dummies to the model, request that the Constant not be included and estimate the model – there will not be exact multicollinearity and there is not a dummy variable trap because the Constant ( the set of 1’s) is not there.

see this for SPSS syntax

http://pic.dhe.ibm.com/infocenter/spssstat/v20r0m0/index.jsp?topic=%2Fcom.ibm.spss.statistics.help%2Fsyn_regression_origin_and_noorigin_s.htm

There will be in effect an intercept for each State and this will be given directly and not as a differential. Thus Beta_ Alab will give the predicted mean poverty in Alabama for when the environmental predictors are at their typical value across the 5 States.

This formulation (which I call separate coding as opposed to the default contrast or reference base coding) is often useful particularly when the categories have no natural base.

If you have many more States I would consider random effects as compared to fixed effects analysis and the former naturally models differentials from the overall country effect – but that is another story!

SPSS drop one variable is to avoid perfect collinearity, when you have 5 you have to leave out one variable for as a baseline or reference category. you can choose baseline by yourself based on your interpretation

Hmmm, this is my first post to these forums, but I'm posting because nobody seems to know the answer to this rather straightforward question.  It's basically a degrees of freedom issue.  Let me explain degrees of freedom quickly.  One place it's found is the denominator of the variance: (N - 1).  You might ask, why N-1?  Why not N?  Well, let's think about the mean.  If we have an N of 5, and values of 5,6,7,8,9.  And I tell you the mean is 7.  If I five you this mean (7), and give you the values of 5,6,7,8 (but not 9).  Can you figure out the last value?  YES!  That's a loss of a degree of freedom from calculating the mean.

Okay, when you derive dummy coding variables you also lose a degree of freedom because you already know the categorical variable.  Let's take the above 5 vector coding:

Alabama 1 0 0 0 0

Calif........ 1 1 0 0 0

Texas...... 1 0 1 0 0

Utah........ 1 0 0 1 0

Wyoming 1 0 0 0 1

Now, let's take away the first column vector (the one that's all ones).  We have:

Alabama 0 0 0 0

Calif........ 1 0 0 0

Texas...... 0 1 0 0

Utah........ 0 0 1 0

Wyoming 0 0 0 1

Without the states shown, can you still differentiate the states from the four values provided in each vector?  YES!  However, let's delete another vector:

Alabama 0 0 0

Calif........ 0 0 0

Texas...... 1 0 0

Utah........ 0 1 0

Wyoming 0 0 1

Now can you perfectly differentiate the states?  NO!  Alabama and Calif will get confused.  You only lose ONE degree of freedom, not TWO.

Ok, let's outline another concept.  When doing linear regression (or parametric correlations, like Pearson's), you can use either continuous variables as the predictor or you can use dichotomous (two value) variables.  Either meets the assumptions, although each in a slightly different way.  You can't use categorical variables with more than two conditions though.  That's where the dummy coded variables come in.  We turn our categorical variable into a series of dichotomous variables.

Ok, one more concept.  When creating dichotomous variables, the actual values don't matter.  It could be 0 vs 1, or 1 vs 2, or 123 vs 987.  It just doesn't matter (other than for some contrasts among dummy coding variables.

So, here's the wrap-up.  When creating dummy coding variables for a categorical variable to do regression with it, you'll always need one LESS dummy coding variable than there are conditions in the original categorical variable.

Yeah sure, if you leave all five in, you'll create a perfect collinearity condition (aka, singularity), but that sort of misses the point.  It's all about degrees of freedom.

It's also severely annoying that SPSS doesn't just automatically do this for us, and there's no reason that it shouldn't.  It does precisely this (under the hood) when using categorical variables (aka, fixed factors) in GLM.  Also, when doing logistic regression, it DOES have an option to specify your categorical variables.  IBM has been severely and grossly lazy in not building this option into regression.

You might ask, "why not just use GLM?"  Well, I often like to find best-fit regression models, and that's not automatically available in GLM, especially the Backward, Forward, and Stepwise best-fit options.

Regards to All,

Elroy Sullivan, PhD

Elroy

intriguing post but - after thinking about it - I do not agree with relating the dropping of one category as akin  to reducing degrees of freedom; or at least I do not find it particularly helpful.

1) degrees of freedom - the calculation of  is n-m where m is the number of parameter that have been estimated or constrained - if you used just n then it would not make much difference if the n was of a reasonable size it does not generally halt estimation. 

2) the purpose of degreed of freedom as in the case of a variance is to get an unbiased estimate ; that is its conceptual raison d'etre.

3 ) there are many ways of motivating what you are doing in d-o- ff and Fisher originally  got it from a multidimensional geometry perspective that many did not understand - hence his debate with Pearson

4 You can use a variety of different codings and as I said in my second post I often use no intercept and retain all the dummies and drop none

5 The reason for dropping one if you do use an intercept is because you cannot invert the matrix XTX matrix it is singular - the determinant will be zero - that is you are dealing with exact collinearity and the software tries to help by automatically dropping one of the dummies (by default in SPPS  the one with a label at the top of the alphabet). What you have is intrinsic aliasing which if you use and intercept and do not drop a dummy means the model has insufficient information to estimate the parameter. The sum of the dummies would equal  the 1's of the intercept - that is one or the columns is exactly expressible as a linear combination of  its other columns, That is the nub of the probe