I am trying to find the best fit for my data. When I plot the Cullen & Frey graph, it shows that my data is closer to a gamma fitting. However when it is fitted, with several distributions, for comparison, it shows that lognormal distribution is the best fit. This is shown both graphically, & using standard goodness-of-fit tests such as Kolmogorov-Smirnov & Anderson-Darling?
Shouldn't the Cullen and Frey graph results be consistent with the actual fitting results?
.
not really : C&F just compare distributions in the (skewness², kurtosis) space ; this is a good summary but still only a summary of the properties of a distribution
it is better used to choose a reduced set of candidate distributions (in other words, use C&F to reject the unlikely candidates) and then go for goodness of fit a select the best result
(for C&F, do not forget to bootstrap !)
.
.
not really : C&F just compare distributions in the (skewness², kurtosis) space ; this is a good summary but still only a summary of the properties of a distribution
it is better used to choose a reduced set of candidate distributions (in other words, use C&F to reject the unlikely candidates) and then go for goodness of fit a select the best result
(for C&F, do not forget to bootstrap !)
.
.
skewness and kurtosis are high order moments and their sampling distribution can be quite wide specially for small samples
bootstrapping the (skewness², kurtosis) couple gives you a better feeling of the sampling distribution and may help you not to reject some candidates which might seem a little away from your (one and only) empirical couple
see Fig.2 in the document below
https://cran.r-project.org/web/packages/fitdistrplus/vignettes/paper2JSS.pdf
.
Thank you Fabrice. I will clarify my research further to benefit more from your experience:) I have to add that I am not a statistician, I am an Electrical Engineer, so most of these concepts are new to me.
My data is quite large, 50,000 plus samples. Moreover, it is real time data packets, and I wanted to fit its byte size to a suitable distribution, to predict network bandwidth requirement. So, I am thinking that I should retain its original sequencing. Am I right on posting this restriction?
My understanding of bootstrapping is that it re-samples by shuffling the data to create new sample sets. If I am correct in my initial understanding of how to find a suitable distribution model for my data, then shuffling will not serve my purpose!
With this added information, do you still recommend using bootstrap?
.
if you just want to have an idea of the distribution of packet sizes, you do not bother about the order !
so you can safely resample
now if you were (for instance) interested in the distribution of sizes of two consecutive packets, then you would have to take order into account and resample among consecutive couples of packets ...
(oh ... and bootstrapping is not reshuffling : if you have a size N sample, bootstrapping ("vanilla" version) is just sampling N times with replacement from your sample ; this gives you one bootstrap sample from which you calculate your statistic ; then you sample another bootstrap sample again and again until ... you're tired of the process !
.
50000 samples may sound large but for log normal distributions (which can lead to very large rare events) or even weibull it may not be so humongous !
.
- Badges
- Science topic