I am trying to transform my data which is having the right skewness to obtain normally distributed data. I have been using different transformation techniques like logarithmic, Box-cox and then I came across quantile transformation. The quantile transformation gives better-transformed data than other techniques. But I have seen the choice of the number of quantiles does change the final model error estimates as I am performing a regression problem.
Swarnendu Sekhar Ghosh -
Let me ask you if you are transforming data because you do not think you can use skewed data, say, in a linear regression (which includes polynomial linear regression)? If so, that is not a problem. Examples can be found through the following:
Invited presentation for mathematical statisticians at the US Energy Information Administration:
https://www.researchgate.net/publication/319914742_Quasi-Cutoff_Sampling_and_the_Classical_Ratio_Estimator_-_Application_to_Establishment_Surveys_for_Official_Statistics_at_the_US_Energy_Information_Administration_-_Historical_Development, 2017, using prediction.
It is nice if the estimated residuals are approximately 'normally' (Gaussian) distributed, but not such a big problem, but the unconditionally distributed data themselves can be distributed any way.
If you are looking at count data, that's different. You could look at poisson regression.
Anyway, I suggest that you be certain that you need quantile regression before you proceed, if you are not certain now.
Cheers - Jim
Sir James R Knaub,
I intend to utilize Gaussian process regression(GPR) which is a Bayesian nonparametric regression technique that assumes a gaussian zero mean prior and a Gaussian likelihood. So this critical assumption of the technique that the sample data comes from a population that follows a multivariate normal distribution makes me transform the observable variable to better pose the problem. A clearer picture of the problem, I am using 3 observables and they have a skewness of (0.5,0.17,1.62). When I checked for the normality of the variables from the histogram they seem to have right skewness, and they even fail to pass the Anderson darling test and Kolmogorov Smirnov test. After I transformed those variables using the quantile transformation to agree with the theory of GPR, they seem to pass the aforementioned normality tests and even their quantile-quantile plot seems to agree. But, I am a little confused with how to choose the number of quantiles for the transformation. Thank you for your response I will surely look into your suggested work.