Can anyone suggest me how to normalize one single dataset having 2 different sets of kernel features (preferably in Matlab) to be able to feed to a machine learning engine.
1- I have done the normalization separately and then combine them but the results are not good enough.
2- I used Matlab normc() and normr() to normalize integrally and also the results were even worse.
Consider true normalization. Change each variable to a normal quantile. See Van der Waerden Scores and BLOM. You will obtain perfect Normal(0,1) variables this way.Each ranked raw value (x) is changed into a standard normal variable (z) corresponding to the percentile of x. You obtain a robust test procedure when you use such quantiles.
as I explained in my questions, I have two different set of features (with two different scaling and max), so this technique won't work
I suggest for each data set separately, subtract data from its mean value and divide the obtained value to the standard deviation of the dataset
thanks for the reply @Elham. this standard deviation you mentioned is the std of each column or each sets of features or the whole features at once?
Hi, you should calculate mean and std for each set of features separately, and normalize each data set according to its mean and std.
Consider true normalization. Change each variable to a normal quantile. See Van der Waerden Scores and BLOM. You will obtain perfect Normal(0,1) variables this way.Each ranked raw value (x) is changed into a standard normal variable (z) corresponding to the percentile of x. You obtain a robust test procedure when you use such quantiles.
@Raid: That is like a weighted normalizing process, I like it. Thanks...
Hi Elham,
It is a well known procedure that goes back many years. Conover's text on nonparametrics statistics devotes a chapter on "Tests with ARE Greater than 1". I was mesmerized by this chapter when I was a student taking a course on nonparametrics.
http://www.wiley.com/WileyCDA/WileyTitle/productCd-0471160687.html
- Badges
- Science topic