I want to transform my data to achieve normality, but all the regular transformations have been unsuccessful. Transforming using the power of 1.5 (3/2) gives me normal distributed data. Is this an acceptable transformation to use?
Transformed value = value^(3/2)
Thanks!
Thank you for your reply. Regarding your second point - some of my variables are not expected to be normally distributed (for example, rating how pleasant a pleasant or ugly picture is - I expect them to be clustered at the positive or negative end respectively). However I would like to use them in a regression model. Is it better in this case to transform them prior to putting them into the regression?
I would also like to comment that for most modelling applications we will not expect to find, nor do we require, approximately normally distributed data. In particular, for an ordinary least squares fitting procedure to produce the maximum likelihood estimate, we require that the measurement errors or residuals be normally distributed.
Also, for point number 2 from Jochen's first post, it is also possible that the model is correctly specified but the measurement errors are not normally distributed. One can often make a theoretical argument that the errors should be normally distributed based on the measurement procedure. If the residuals should be normally distributed, but they are not, then a model misspecification error seems relatively more likely. But if the measurement errors should theoretically be poisson distributed, then obviously the fact that the residuals are not approximately normally distributed does not suggest the model is incorrect.
A couple points, some made by others.
1) Even for OLS regression, the data do not have to be normal, the residuals do.
2) One problem with transformations, especially odd ones like ^3/2, is that the results of your model are less interpretable. In the old days, when computers were nonexistent or scarce and (a little later) slow, this was sometimes a good compromise. OLS regression has nice properties that make it fast.
But times have changed!
For example, quantile regression is available (in R and SAS and probably other packages) and it makes no assumptions about the distribution of the residual. There is also robust regression. Or, if you want to go the transformation route, there is MARS.
I wrote a paper on these for the recent SAS global forum: Alternative methods of regression when OLS is not right (linked below). It concentrates on SAS but might be useful even if you use different software.
http://support.sas.com/resources/papers/proceedings15/3412-2015.pdf
Short answer: Yes. People often use Box-Cox transformation or Tukey Ladder of Powers transformation, both of which an give fractional powers.
Use this command in STATA drawnorm to achieve normality, it saves time and very effective as well. You can also use Tukey ladder of powers in STATA to get normal distribution
In R, the boxcox function in the MASS library will perform the the Box–Cox Transformation. It can transform a single variable. Or can transform the dependent variable of a linear model.
The problem with BoxCox, as I noted, is that it frequently gives transformations that are substantively nonsensical.
Instead of forcing the substance to follow the statistics, we should let the statistics follow the substance. There are better ways than transformations.
- Badges
- Science topic