I attended a deep learning course yesterday (I'm new to ML and am relatively new to programming in general) and the instructor mentioned that we need to normalise data for input into the deep learning network.
Unfortunately, he didn't go into any real detail about how to do this.
I have Googled and figured out how to normalise numeric data (e.g. sensor readings), but I can't find any solutions for dealing with categorical data. The instructor gave an example of a NN trying to predict house prices.
For example, one value in the dataset might be outer wall colour, which obviously isn't a number (to begin with), i.e. it might be brown/grey/cream/red brick/etc.
How do I normalise this sort of data?
Is it simply a case of assigning each category a number, and then normalising that number dataset?
e.g. brown = 1, grey = 2, cream = 3, red brick = 4, and so on?
...and then normalising that 1-4 dataset?
Is it as simple as that or is there more to it?
Thanks in advance for thoughts.
(I don't specifically need this knowledge for a project right now, it's more a matter of interest)
Hi,
Complementing Sobhan answer, you do not normalize categorical data. You use normalization (0-1) and standardization (mean = 0 and variance =1) to avoid tha some variables have more influence in learning than others due to different scales. For example, if you have a sensor measurement ranging from 1-6 and other ranging from 8000-9000, the measurement of the first sensor will have very low influence in the decision, but it doesn't necessarily mean that it is less important.
In case of NNs, the best way to deal with categorical variables is doing something called one-of-K or one-hot encoding. In you example, instead of one variable called color, you create one variable for each color. Then, for each house example, you assign 1 to the corresponding color and 0 to the remaining. If you are using Decision Trees, you can use the encoding you mentioned, one number for each color, although some softwares accept "names" as inputs (they do the encoding for you).
Regards,
Ricardo
This is a classic example of regression versus classification (the later for your example since you want to predict a discrete color). To add to the discussion, in the case of SVMs (see libSVM @ https://www.csie.ntu.edu.tw/~cjlin/libsvm/) you would assign each category/class a number/label and every feature in your training dataset would map to one of these classes. Having trained a model you can input a test feature vector and the output will be one of the labels (the correct one if enough training data and the trained model is discriminative enough).
If you are using python sklearn library, you can directly encode categorical variables by applying the following two encoders:
http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelEncoder.html
http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html
Sobhan Sarkar what if the unique values in columns are very large (larger than 20). for example, there is a dataset on Kaggle about house prices prediction and one column in it is the name of the neighborhood. so, should I use one-hot encoder on it?. also another question (the same dataset), if I have a column with discrete value such as the number of rooms, how can I deal with it?
thank you.
- Badges
- Science topic