The following paper can be helpful for you.

https://odajournal.com/2013/11/11/univariate-and-multivariate-analysis-of-categorical-attributes-with-many-response-categories/

The following paper can be helpful for you.

https://odajournal.com/2013/11/11/univariate-and-multivariate-analysis-of-categorical-attributes-with-many-response-categories/

Hello Ammar,

How to handle a variable such as your "Harvard" example depends on your specific research aims. If what you want to know is, does _any_ degree from Harvard (vs. none) help differentiate cases? If so, code it as a dummy variate (e.g., 0, 1) instead of a count variable (for persons with multiple degrees).

If you want to know, does having the number of degrees from university X (whether Harvard or something else) help to differentiate cases (beyond the yes/no coding above), then use the count, but don't think of it as a dummy variate any longer. These counts will tend to be really skewed, however.

I'm not clear as to how inclusion of university and degree type automatically induces collinearity, except in extreme cases (a facetious example: "International Beauty School" as institution and "Cosmetology" as the degree).

Good luck with your work.

Thank you David Morse , my research aim is to find out if the educational background of the founders of a company has an effect on the success of the company.

So yes as you already mentioned I would like to know that having a degree vs not having one from Harvard has an effect on my target variable. However, I also believe/assume that having 2 degrees from Harvard is better than having 1 degree from Harvard which is then better than having no degree from Harvard. I also believe/assume that having a degree from Harvard and also having a degree from MIT differentiates the cases (Both would have a 1 here). In case I do go with that (0, 1, 2 or 3 for University X) I understand that it won't be counted as a dummy variable but does that still make sense? Since my dataset has 17,000 variables it is also possible that for some universities it might still be a 1 or 0 since people never attended it more than once. Is it fine to have some of my universities in dummy variable form (1 or 0) and others as ordinal (0, 1, 2 or 3).

I am using random forest and decision tree classifier in Python (Sklearn) so I am not sure if it makes a difference to have a dummy (1 or 0) or an ordinal (0, 1, 2 or 3). Since these are numerical values I believe that the algorithm should give the variable weight according to the values. I am completely fine with assuming that a 2 for University X might be 2 times as weighted as a 1 for University X. Additionally, do you think that a 0 value in University X (having only 1s & Os) compared to a 0 value in University Y (having 0, 1, 2 or 3) are treated the same as having no effect at all? Or is it possible that a 0 value in University X might relatively be different from having a 0 value in University Y?

Another alternative solution could be to just use dummies (1 or 0) for Universities and then also introduce another variable that counts the number of degrees taken by the person from all universities. What do you think?

Also could you please elaborate on how the data could be skewed when using count?

By multicollinearity, I was just referring to the practice where one of the dummy variables is always supposed to be dropped since we take k-1 dummies.

If possible could you please provide a reference from a book or a research paper that will justify all this since I need it for academic writing?

Thank you once again.

Mohammed Ashikur Rahman The solution in you mentioned paper does not seem so obvious.

Could you please be more specific and explain?

Thank you.

Hello again Ammar,

I think that you'll find trying to pinpoint specific universities as explanatory might be less helpful than you might imagine, especially with so many possibilities. You could collapse these into plausible subsets (e.g., "Ivy League" institutions).

Second, if you are really sure that number/count of degrees from the same institution is useful, then, by all means, include that as a variable. Note that a count variable and a status (yes/no) variable for a given institution will exhibit collinearity. But your approach allows for such collinearity.

To your question, why skewed: Because, of your set of thousands of records, the majority will have a zero value for institution X, some will have one degree from institution X, and fewer yet will have more than one degree. Hence, skewed.

Finally, forest methods, tree methods, and the like are remarkably opportunistic, so overfitting is an ever-present concern. Be sure your approach allows for sufficient data to be used for validation purposes.

Good luck with your work.

Dear David Morse,

Thank you once again for your help. So far I have used the approach of taking the 50 most frequently occurring universities and while keeping the count of individual universities as a variable. So it's a lot similar to dummy variables, however since I summed up all the dummy variables per person some universities and subjects have more than binary values.

I thought about collapsing them to Ivy Leagues and others however, the dataset has extremely large number of unique and I would have to do this manually for each university. I am happy to hear if you have a more automated solution. Also the same approach is taken for subject.

Below is what I finally implemented:

If University X was attended once it would have a 1 as a value.

If University X was attended twice then it would have 2 as a value.

If University X was attended thrice then it would have 3 as a value.

If University X was not attended then it would have 0 as a value.

Exact same process is used for subjects.

My results actually have been great on a 70 - 30 split of train and test. However, I just need a book or paper to justify if this makes sense. I would appreciate if you could recommend something.