Dear all,
I measured a variable that takes values between 0 and 0.1 (with a minimum of 0.00053). This variable will be used in a regression analysis, but it has values of skewness and kurtosis of 3.8 and 14.3, respectively, hence requiring a transformation in order to reduce those values.
I first thought about a log transformation. However, in this way, the resulting values of the variable will be negative, and I would avoid this. Another option is multiplying all values for 1,000 and then use a log transformation. But, how can I justify this choice to referees?
Have you ever experienced this problem? How have you solved it?
Thank you for your attention to this matter.
Best
Lorenzo,
Thanks. First, negative values are not a problem. That only affects the regression intercept, which will not be of interest after transformation. It does not affect the regression coefficients associated with predictors.
Second, you have multimodal data, not remotely normal. It's highly unlikely your residuals will be normal. It looks like possibly something about your data collection might be responsible for the spikes. Regardless, a transformation will generally not get rid of this problem. I suggest you look into non-parametric techniques, such as using bootstrapped standard errors for your regression coefficients.
Pat
In psychometrics, log-transform is the most common way to increase the normality of your distribution. This method is routinely use to analyze response times, which are never normally distributed. An alternative approach would be to fit your distribution with a specific function, and then conduct your analysis on the individual parameter estimates of the fitted function. Again, this approach is sometimes used in latency analysis. Here is a paper that illustrates this approach and its rationale in the context of experimental psychology:
Base in Econometric , The Skeweness near to 0, the Kurtosis must be near to 3, the mean it is a Symmetric series. The Skewenesss and Kurtosis depend Standard Deviation (Its the gab between mean and items ) so according the output maybe need to review the items, number of items(n - more items much better), the weekly,monthly or for year. Its much better to work in year items. From my point of view review your items,n and work with year series.
Lorenzo,
What does the distribution look like? It sounds like the large majority of responses are near zero, but looking at the graphed distribution can still help with identifying appropriate solutions.
Also, what's your concern about negative values?
Pat
Thank you all for your responses.
Dear Andrea,
could you send me the full paper. I cannot download it from the link.
Dear Claudio,
I have cross-sectional data. What do you mean by working with year series?
Dar Patrick,
attached you can see the distribution of the data.
I prefer not having negative values to avoid problems of interpretations of results after regression analysis. I will use a Tobit model. Do you think having negative values is not an issue?
All the Best
Lorenzo,
Thanks. First, negative values are not a problem. That only affects the regression intercept, which will not be of interest after transformation. It does not affect the regression coefficients associated with predictors.
Second, you have multimodal data, not remotely normal. It's highly unlikely your residuals will be normal. It looks like possibly something about your data collection might be responsible for the spikes. Regardless, a transformation will generally not get rid of this problem. I suggest you look into non-parametric techniques, such as using bootstrapped standard errors for your regression coefficients.
Pat
I second the opinion of Patrick M. How were data points generated?
Are data censored:
https://en.wikipedia.org/wiki/Censored_regression_model
You might want to apply a Tobit regression model:
https://en.wikipedia.org/wiki/Tobit_model
Bootstrapping looks attractive, but I'd probably still start by running conventional regression models and looking at the residuals as multimodality is to be expected if your predictors are multimodal. You can also fit this kind of data with mixture models or by taking account of censoring if it is present.
We think that, skewness and/or kurtosis are not something reduce or manuplate. They are only the descriptive outcomes to describe your continuous data, and you should act by taking them into account. We just published a new article, which can describe any skewed data, with new mean and deviation approaches. j.ponte.2017.2.34
Article TO DETERMINE SKEWNESS, MEAN AND DEVIATION WITH A NEW APPROAC...
Can i delete some of variables which has more than ±2 skewness and kurtosis to get better scores?
- Badges
- Science topic