I have a dataset that is heavily skewed to the right. I have tried to use lg(n) +1, but the data still remains skewed. What options do I have from this point? In my analysis plan I have, besides regular descriptive data, also planned to do logistic and linear regression, Cox and/or poisson regression, depending on the different research questions at hand. I've noticed that several regressions are a bit picky when it comes to this...
your dependent variable - Is it discrete, binary, continuous , categorical?
Ulf -
A few thoughts:
Poisson regression is one thing. Here is a good resource, I think:
https://onlinecourses.science.psu.edu/stat504/node/168
But if you have linear regression for continuous data, where both the size (from regressors, x), and variable of interest, y, are skewed, then transformation is not needed. If this is heteroscedastic, then weighted least squares (WLS) regression is appropriate, preserving the error structure. Some transform to do hypothesis tests, which are often misleading in the best of circumstances. Estimating variances of prediction errors, and prediction intervals (analogous to confidence intervals) can be much more useful. Even model bias is indirectly included there, but testing models using separate test data can be very helpful.
Transformations may often make your results difficult to interpret.
Cheers - Jim
Try this
Article TO DETERMINE SKEWNESS, MEAN AND DEVIATION WITH A NEW APPROAC...
There are stronger transformations, for example a reciprocal transformation.
But you might want to think about the distributional assumptions for each of the models you want to build. Ordinary least squares (OLS) linear regression, logistic regression, and Poisson regression do not have the same assumptions of data distribution.
If your data are not meeting the assumptions of OLS regression, you might consider a generalized linear model. For right-skewed data, you might transform the dependent variable so that the residuals of the analysis are normally distributed for OLS regression. But another approach is to consider that a gamma distribution might be more appropriate for the data. (See plots here: https://en.wikipedia.org/wiki/Gamma_distribution). Note that the gamma distribution assumes continuous data.
Likewise, the negative binomial distribution can have a variety of shapes (https://en.wikipedia.org/wiki/Negative_binomial_distribution), but it is used for discrete data, making it appropriate for count data, but more flexible than Poisson.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Regression