I am trying to fit some probability distribution functions to a set of data and estimate the parameters of the equation. Which software to use in such a case and what is the generic procedure? The curves should look something like in the figures.
Figure citation:
(Tiwari, B., Ajmera, B., & Dhital, S. (2017). Characteristics of moderate-to large-scale landslides triggered by the Mw 7.8 2015 Gorkha earthquake and its aftershocks. Landslides, 14(4), 1297-1318. )
Hello Kaushal,
The standard approach to fitting a probability distribution to data is the goodness of fit test. This compares the observed frequency (f) to the expected frequency from the model (fhat) for any number of classes, such as you show in your graphs. The 20th century approach is the chisquare goodness of fit test, using the chisquare statistic. The 21st century approach is the non-Pearsonian chisquare statistic, called G, which is based on a likelihood ratio. The formula for the G statistic, along with the derivation, is shown at:
https://en.wikipedia.org/wiki/G-test
Statistical packages are listed at this site.
G can be calculated from the numbers shown in your graphs for f (observed frequency in each class) and for fhat (expected frequency in each class).
Note that G = -2 ln LR
where LR is the likelihood ratio, lnLR is the evidential support, and the likelihood ratio test of goodness of fit is calculated from the chisquare distribution, with degrees of freedom = number of classes over which you compare f to fhat. As a standard point of reference people use p 0.05 as adequate fit. For one of your graphs you have 18 classes, hence 17 degrees of freedom. For a Gstatistic of G=20 on 17 degrees of freedom the p-value is 0.0511.
That was a bit terse, but I hope it was sufficient.
~David S
You would usually determine these values of the parameters (of the distribution model) under which the observed data is most likely (-> maximum likelihood estimates). One can literally search these values (usually done by an optimazation algorithm), but sometimes there are analytic solutions available. For instance, the maximum likelihood estimates of the parameters of the normal distribution are µ: m = sum(x)/n and σ²: s² = sum(x-m)²/n. For the log-normal distribution, the solution is the same, only using log(x) instead of (x).
Distributions can be fit to data with the function fitdistr() (package MASS) in R (www.r-project.org). There is also an add-on package "fitditrsplus". You can find many examples in the web, e.g. here: http://www.di.fc.ul.pt/~jpn/r/distributions/fitting.html
Dear Kaushal
Distribution fitting toolbox of MATLAB software is good. Also EZYFIT tool box for MATLAB or EZYFIT software is suitable to obtain unknown parameters of user defined probability density function.
Best regards
Keep also an eye on possible outliers in your data base. Estimation of parameters, outlier detection and goodness of fit tests should be done in a consecutive way. - When choosing your statistical tests, be careful for the recommended sample size. Some tests work with very small sample sizes, others don't.
I usually use JMP. First, you obtain a histogram then find and click the option for fitting parametric distributions. Then you will find a number of distributions fitted to your histogram. You can obtain p-value too.
You can use MathWave EasyFit software, which allows you to automatically or manually fit a large number of distributions to your data and select the best model.
Also, there are tests that show how well the distribution you selected fits to your data.
I have used it often and it gave great results.
Best regards!
Dear Kaushal,
if I were you, I would use R (www.r-project.org). It is an opensource software with a huge number of libraries; further there are several IDEs such as RStudio, which can help you in writing, debugging and plotting your code.
Best regards.
You can fit the kind of curves with the sm package in R.
Useful libraries in R include: logspline, kernSmooth, MASS
Available functions include: density, hist, ...
Venables and Ripley, "Modern Applied Statistics with S" is an excellent reference (cf. Ch 5.6).
Thank you everyone for your valuable suggestions. My problem is solved with your feedback.
Dear Kaushal,
You can try to fit with Matlab function, for example:
histfit(data,40,'exponential')
good luck!
Long
I think the fitdist function in R is very good package. Not only it does the fitting, it also calculates AIC (Akaike Information Criteria), BIC (Bayesian Information Criteria) which will allow you to pick the best model which minimizes KL distance in information theoretic sense.
- Badges
- Science topic