I have results of porosity analysis from CT data which is a very skewed distribution (looks like Pareto distribution or similar). I have data from a few samples which I want to compare about the pore size. It is clear from the histogram, that I cant use arithmetic average because it is not a normal distribution. May I use median or are there some more sophisticated approaches to deal with data like this?
There is nothing wrong with the average to estimate the expected value of the distribution.
It is more a problem that you cannot easily give a typical value of the distribution. I would describe your data as some kind of distribution and give the parameters. For instance, your data fits well to y = 10000 / (3x)³.
Dear Pavel,
Pareto distributions are a very interesting topic and its applications cover a large range of applications (like statistical analysis of extreme wind speeds (1), statistical analysis of low grazing angle and high resolution X-band sea clutter (2), precipitation events (3) , ocean engineering (4), extreme earthquake events (5) and many others).
An interesting extension of basic analysis is provided by (6).
If you have to perform some basic analysis, you'll find fairly accurate information on the Wikipedia page in English language: I am also attaching an article.
Hope this helps!
Best regards
Marco
1. J.D. Holmes, W.W. Moriarty, Application of the generalized Pareto distribution to extreme value analysis in wind engineering, Journal of Wind Engineering and Industrial Aerodynamics, Volume 83, Issues 1–3, 1999, Pages 1-10
2. M. Farshchian and F. L. Posner, "The Pareto distribution for low grazing angle and high resolution X-band sea clutter," 2010 IEEE Radar Conference, Washington, DC, 2010, pp. 789-793, doi: 10.1109/RADAR.2010.5494513.
3. Victor Korolev, Andrey Gorshenin. (2020) Probability Models and Statistical Tests for Extreme Precipitation Based on Generalized Negative Binomial Distributions. Mathematics 8:4, 604. Online publication date: 16-Apr-2020.
4. Chen, B., Zhang, K., Wang, L. et al. Generalized Extreme Value-Pareto Distribution Function and Its Applications in Ocean Engineering. China Ocean Eng 33, 127–136 (2019). https://doi.org/10.1007/s13344-019-0013-9
5. Pisarenko, V., Sornette, D. Characterization of the Frequency of Extreme Earthquake Events by the Generalized Pareto Distribution. Pure appl. geophys. 160, 2343–2364 (2003). https://doi.org/10.1007/s00024-003-2397-x
6. Alexandru V. Asimit, Edward Furman & Raluca Vernic (2016) Statistical Inference for a New Class of Multivariate Pareto Distributions, Communications in Statistics - Simulation and Computation, 45:2, 456-471, DOI: 10.1080/03610918.2013.861627
Maybe your data is normal with upper outliers. For example, how do you know that you aren't seeing multiple Processes? D. Booth
Irregular or heavily skewed distribution can also be analysed with finite mixture model and gamma shape mixture (GSM).
Important R packages that can be of help include: mixR and ForestFit.
You can also read:
Podlaski R. (2017). Forest modelling: the gamma shape mixture model and simulation of tree diameter distributions. Annals of Forest Science, 74(2):29
The analysis by Jochen Wilhelm (above) suggests a simple transformation, assuming your pore size is a diameter rather than a volume, Thus z=x^-3 gives a "number of pores of a given size if a unit volume were filled with pores of that size" ... this may then lead to a a sensible interpretation of the data, depending on the context, comparing an observed number (of a given size) against a "maximum number" .. or volume occupied by pores of a given size compared with volume filled by pores of smaller or larger sizes.
You can try to use skew distributions to analyse the values of the distribution. More realistic approach is provided by Azzalini, 1985 on his remarkable paper regarding skew normal distribution. There are many skew distributions are available in the litetature; like as, skew t distribution, skew catchy, skew chi and skew logistics. These distributions may be useful to model the data both left and right heavy tailed skewed data sets.
In power-law distributions, the mean and variance say nothing, you can check this by a log-log scale (maybe is a log-normal distribution or straight line). If you have a straight line in log-log scale the slope gives information. The best option would be using a nonparametric approximation. I would recommend the Kolmogorov Smirnoff Test (which is based on the Cumulative (CPD), which is better due to is not necessary to use a number of bins) or the U-Mann test. An approximation with surrogate data also is possible dependent on the properties that you want to verify. Also, I recommend you to use boxplots for the data in order o verify which approximation is better for the differentiation problem (centrality or dispersion).
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics