Ehsan,

I believe there is a minor mistake in your statement that makes all diference...

The assumption behind parametric tests is that the mean (which is what will be tested) is normally distributed, not raw data. Is that important? Yes!

When raw data is normally distributed, mean distribution is always Normal. But if raw data is not normally distributed, mean still can presents a Normal distribution, depending on the simetry of the raw data and the sample size. For instance, it is easy to demonstrate, by simulations, that samples from uniform raw data quickly shows a Normal mean distribution. It is not surprise, since the sum of two uniforms results in a triangular distribution.

Exponential distribution needs greater sample sizes due to its skewness, but is not so difficult. Addicionally, experiments with simulated data have show that even with exponential raw data, parametric tests as t-test and ANOVA are robust enough to present good results.

The formula exp[-A*x^B] looks somehow strange. Are you sure that you want use right this formula? Exponential distribution can be described with the only parameter A - the density function is PDF(x)=A*exp[-Ax] for all x>=0. Exponential trend is usually described by two parameters A*B^x. How do your data look like? Are they discrete or continuous (in the case you are working with counts, Poisson model might be apropriate)?

Semi log graph should do

Hi Igor,

counts are typically analysed using Poisson regressions, where you could compare the mean (lambda) between groups. You may experience some difficulties like an excess of zeros or overdispersion (the poisson distribution has no extra variance parameter) , this could be dealt with by using zero-inflated versions and for example negative binomial models.

If your observed x are well away from zero, a normal approximation may be feasible.

Could you provide some insight of the distribution of x and of the process of generating x?

Just to clarify: Do you know A and B or do you want to estimate A and B from the data of each group? Exponentially distributed data of the form f(x)~\lambda Exp[-\lambda x] \foreach x>0 can be checked for \lambda just as for any argument of any other pdf.

The most useful and simplest way is to increase your sample size. Following Central Limit Theorem, as the sample size increases, the mean distribution tends to Normal and you can use regular parametric tests, like ANOVA. Usually, n=30 is considered a good number, but it will depends on your data distribution.

How big are your samples now? It is possible to increase them?

Dear Igor

One of the main assumptions in inferential statistics specially comparisons is the normality of the data. Otherwise, you should apply kruskal wallis if you aim to compare more than two means. But, be aware, if the distribution of all the treatments are same you cal transform them to make normal distribution.

In such distribution, the mean is not reliable enough, if your data with exponential distribution is large, you can take samples of them and compare the mean of samples. Although the distribution of your data is exp. but the sample distribution is always normal if it's proportion to total data is low.

Ehsan,

I believe there is a minor mistake in your statement that makes all diference...

The assumption behind parametric tests is that the mean (which is what will be tested) is normally distributed, not raw data. Is that important? Yes!

When raw data is normally distributed, mean distribution is always Normal. But if raw data is not normally distributed, mean still can presents a Normal distribution, depending on the simetry of the raw data and the sample size. For instance, it is easy to demonstrate, by simulations, that samples from uniform raw data quickly shows a Normal mean distribution. It is not surprise, since the sum of two uniforms results in a triangular distribution.

Exponential distribution needs greater sample sizes due to its skewness, but is not so difficult. Addicionally, experiments with simulated data have show that even with exponential raw data, parametric tests as t-test and ANOVA are robust enough to present good results.

Eik Vettorazzi had the most appropriate answer thus far. This is Poisson distributed data and Poisson regression is the standard technique. While he mentioned several cautions regarding using it that are important to keep in mind, the typical case involves simply entering your group predictors and dependent variable into a Poisson regression model.

The several recommended non parametric tests are not necessarily wrong but it's always best to use a parametric model that fits the data when one is available.

Log transformation could be best way to deal with such a data. Based on the number of cases, application of parametric test could be best. However, checking the distribution of the residuals could be still important to not ignore the non-parametric options.

I would suggest a log based transformation followed by a goodness of fit test (Shapiro-Wilk, Kolmogorov-Smirnov, Anderson-Darling or Cramér-von Mises test can be used)

on each treatment group (Gauss aproximation for the null hypothesis that the values of the analysis variable are a random sample from the specified theoretical distribution). If the result is not significant for ALL the different groups, you can compare them using a t-test (2 groups) or ANOVA (3 or more groups); otherwise (when the test is significant for at least one group), you should compare them using a non-parametric test, such as the Mann-Whitney (for two groups) or the Kruskal-Wallis test (for three or more groups).

Thank you very much, everyone, for your quick and informative responses! I used the form exp[-A*x^B] because it looks like B can be

Hi Igor,

from looking at your data, I think you run into different problms. And I really wonder if your 'experimental' approach is correct.

Anyway, as both the bin classes and the sample size of each of the groups are different, the only thing your lists have in common is the number of bins.Hence, you should probably go with either simple correlations or ranks.

Both groups have in common that the counts decrease the bigger the bin classes are.

So these groups have at least a similar trend.

Maybe, ANOVA or PCA might help you - but this depends on what you really want to conclude from it.

Thanks a lot, Dierk, for looking at the data! Perhaps my histogram bins were confusing. In the attached file (sample2) the data are the same, but a constant bin size of 2 is used for both groups. It is clear that in group 2 there is a much longer "tail" of the distribution towards larger values of x. That is why I wonder if there is a useful technique not just to test for differences in means or ranks, but to produce estimates of parameters A and B for each group and compare these.

Hi Igor,

o.k., I see your point - I checked your second sample.

I really doubt that you can do much more than simple correlation.

There is evidently nothing for classical statistics as the data is not normally distributed.

However, how many groups are you analyzing in total?

Do you think the two groups 'behave the same' (are members of the same population)? Or do you think they are different?

If you maximize your sample size (number of groups), Sandro's answer will most likely be correct and you might successfully find something more by using ANOVA.

In addition, principal component analysis - as I suggested - might take a while for computation with a large dataset, but you can possibly identify logic connections de novo - which might require additional experiments for verification.

Initially, you will need to specify (many) control populations, to rule out the possibility of a high variability even in your controls.

hmm, Poisson regression would be fine here, I think.

check this out, if you have R

library(xlsx) #version should be >0.5.2, there is a bug in earlier version with rowIndex

dta

Eik is correct, IF the two groups belong to TWO different populations.

But, what if they are considered two samples of ONE population with generally high variance?

@Dierk: isn't this like asking in a simple ANOVA model, what will you get if you forget that you have groups? You would observe a population with high variance if you ignore the group structure.

@Eik: Right. ANOVA would not 'care'.

Simple pearson gives r=0.97, which is a nice correlation as such, without any transformation or else.

ANOVA will not provide meaningful information with only a few sample groups, especially with varying bin number or group sizes.

Thus, I am convinced that Igor will need plenty groups as input for ANOVA, as Sandro pointed out.

Dear Eik and Dierk,

Thanks a lot for very interesting comments and an R example! The negative binomial modeling approach looks promising!

I wonder what you (and anyone else interested) would think about the following approach: Fit the frequency distribution for each group to a smooth function, e.g. exp[-A*x^B] and obtain group-specific parameter values A and B. Then fit the data from both groups using say different values of A, but a common value of B. Then fit the data from both groups using say different values of B, but a common value of A. Finally rank these 3 models (i.e. 1 = different A and B, 2 = common A and different B, 3 = common B and different A) say by AICc. This would allow one to see how much (or how little) improvement in model fit would result from using group-specific parameter values instead of using common values. Does this make sense? I would be grateful for your comments!

To really answer your original question (or the title of your question), a generalized linear model (GLM) with inverse link-function and exponential or gamma error distribution, would be sufficient for analysing exponentially distributed data.

But when reading forward, I realize that you actually have count data, and that "exponential" rather describes the functional form of your model. As some others have pointed out, Poisson regression (i.e., a GLM wil log-link and Poisson error), or negative binomial regression are good alterantives for such cases.

Andreas, thank you, this is informative! What you and others have been saying here, particularly about negative binomial regression, sounds very useful! However, I wonder about the following: suppose there is a biologically-motivated model for explaining these data. As an example, suppose it has the shape P=exp[-A*x^B], as mentioned earlier, where P is the probability density function. How can the parameter values A and B for model fits to data from different groups be compared statistically? For example, would the method I mentioned above using AICc ranking work? Or would Monte Carlo techniques (e.g. fitting the model to bootstrapped data to generate distributions of parameter values for data from different groups, and then compare these distributions) be helpful?

Dear Igor,

you should explain what is under your control in your experiment, what is the outcome, and how it is measured, where comes randomness in play and so on. Then you can set up a statistical model, specifying appropriate effects and interactions (eg different effects between groups) and error terms. In P=exp(-A*X^B), where are the counts you mentioned?

I supposed, that you can control x and observe some counts of something yet unknown to us. isn't that the case?

Dear Eik,

Perhaps I misunderstood your point. The counts refer to cells which express a signal of certain intensity (x). Suppose there is some biologically motivated function P (x) which describes the probability for a cell to express signal with intensity x. Does this make sense?

It makes very much sense!!!

So, by just seeing the graph Eik provided, it looks that Factor1 and Factor2 are from two different populations. Moreover, Igor provided two different data sets pertaining the same information except with changing bin sizes. It implies that for every measurement of intensity x, Igor knows its exact value and thus determine which bin it belongs. Would it not be handy that Igor provide the actual readings instead of categorizing it by binning with outcome either Factor 1 or 2; that information will be very handy to analyze either using negative binomial distribution or by log transformation ( all is relevant as said earlier). Comments?

just one comment on log-transformation of counts - do not!

http://ckwri.tamuk.edu/fileadmin/user_upload/A-Litt/OHara_and_Kotzke_2010_-_do_not_log_transform_counts.pdf

In my analysis I used x as explanatory variable (eg under some control by Igor) but now I think its the other way round - right? So the observed zero counts in the data aren't actually "observed" in the meaning of "under condition x in (x,y) we observed no cells expressing" - but "no cells expressed x in (x,y)". Correct?

Dear Sartaj and Eik,

Attached is a file containing the same data, but without binning - instead listing actual values of x in the two groups. Hopefully this will be useful. Also, thank you for the reference from O'Hara - this is quite interesting! Certainly it makes sense that using something like a negative binomial distribution on count data is better than "artificial" log transformation with addition of a constant to handle zeros.

Eik, you are absolutely right that the variable x is not "controlled". Rather the data set comes from counting cells which happened to express particular intensities of signal x.

What I am still wondering about is how can such data be analyzed with a "custom" probability distribution P(x), which is motivated biologically rather than statistically. Is there a useful technique for comparing parameter values generated by fitting such a function to data from different groups? Thanks a lot for your interest!

so each cell is observed for its own, you can measure x for each cell separately? And apart from being treated either with A or B these cells are otherwise equal?

Yes, you are absolutely right that x can be measured separately for each cell. All the cells are the same (from the same population). The treatment can be considered binary - for example group 1 is untreated, and group 2 treated. A and B are parameters for a hypothetical probability distribution. They presumably differ between groups. Hopefully this helps and thanks again for your interest!

Igor, you talk about the data being counts - I'm assuming it's in millions of cells or something like that? Although it is discrete, it seems to be like it could be safely treated as continuous, although the zeros would still be a problem for log transformation.

There are also some patterns in your data that should possibly be looked at - assuming that the order you have the data in on your spreadsheet is the order they were collected, it doesn't look to be randomly distributed. For example, there are a lot of zeros between the 20th and 40th and between the 80th and 100th data points in group 1, and between the 40th and 60th data points in group 2. If these are data from say different experimental runs, or were analysed by different people or on different instruments or on different days, or came from different cell stocks, then there are some worrying inconsistencies. Not enough to think that there isn't a difference in means between the groups, but maybe enough to not have too much faith in the parameters you estimate in your model.

Hi Igor, you may find some information in the the book entitled "Applied Life Data Analysis" by Wayne B. Nelson, Pages 459-467 helpful. Copyright 1982, Published by John Wiley & Sons, Inc

Thanks a lot, Aedan, for your observation and comments! I will talk to the people who generated these data about the potential non-random effects. But I am inclined to stay away from log transformation, e.g. because of the abundance of zeros and for other reasons outlined in the paper provided by Eik.

Thank you, Axel, for informative comments! I agree that Eik's approach is very useful for this situation and I will try it! However, it is purely statistical, and I wonder what can be done with models which are motivated biologically rather than statistically. For example, consider the case that some equations which describe biological processes can result in a distribution P(x) for the probability that a cell expresses the signal at intensity x. For example, suppose this P(x) has 2 parameters A and B. The null hypothesis then would be that both A and B are the same for treated and untreated groups, and the alternative hypothesis would be that at least one of them differs between groups. My preliminary thoughts on how to approach this are the following:

1) Fit the function P(x) to the data from both groups. For example, the "observed" Pobs(x) could be the frequencies of counts in bins of x. The errors of these estimates of Pobs(x) in each bin could be approximated say by the Poisson distribution.

2) Perform the fitting of P(x) to Pobs(x) 4 times: 1. keep both parameters A and B constant (in common) for both groups, 2. keep parameter A in common, but allow parameter B to be different for each group, 3. keep parameter B in common, but allow parameter A to be different for each group, 4. allow both parameters A and B o be different for each group.

3) Calculate AICc for the fits of these 4 models and compare them. For example, if model 4 has much lower AICc than the rest, this suggests that the data are better explained by having different parameters A and B for different groups.

4) Perform bootstrapping on the data from both groups and fit model version 4 (with different parameters). Then compare parameter distributions corresponding to fits to the different groups. If these distributions don't overlap, this also suggests that that the data are better explained by having different parameters A and B for different groups.

Does any of this make sense? As always, I would appreciate comments from anyone interested.

I think Yes. In theory of numerical analysis there is idea that existed (any, both theoretical, and experimental) data can be checked on subject of possible errors. As I could understand the idea by simple words, laws of nature should be described by continuous relations. By similar basis, in the theory of numerical analysis there are approaches how to find points in data which do not put under such conditions (via analysis of derivatives of higher orders, which should tend to zero). One can see in more details in [K.S.Kunz, Numerical Analysis, McGraw-Hill Book Comp., Inc., 1957, Chapter 3]

I think Kolmogorov-Smirnov test and a QQ-plot can do the job for you.

Hi Igor,

Are your sure the data is exponentially distributed? Could potentially be a power law. I would rank the frequency of events and display the data on a log-log plot. If you have a straight -line you may have a scale -free mechanism underlying the data.

Good luck!

Tom

Thank you, Sergei and Axel!

Amit: I am missing your point. Please explain more if you are interested.

Thomas: thank you! This is an interesting idea.

Based on the new data sample3, here is the graph and both distributions are distinctively different. I have fit the polynomial for both the funcitons and goodness of fit is extreemly high with R2~0.99 to 0.993. Igor, you can generate data based on these equations with confidence and also fit an exponential function if you so desire. Any comments?

BTW, fitting an exponential function to the data, yet avoiding log transformation is incongruent in terms of logic as log transformation is indeed expressing nothing but the same relationship. For example Y = b0 * exp^(b1 * X) is in fact the same thing as ln(Y) = ln(b0) + (b1 * X).

Estimation for xGrp2 -- xGrp2 id, f(6a) i(b0 0) --

Power: Y = b0 * (X^b1) OR ln(Y) = ln(b0) + (b1 * ln(X))

Exponential: Y = b0 * (e^(b1 * X)) OR ln(Y) = ln(b0) + (b1 * X)

--------------------------------------------------------------------------------

Variable Power Exponential

b0 9.974e-07 1.0460933

1.74 10.36

0.0847 0.0000

b1 3.851162 .03814904

31.74 45.90

0.0000 0.0000

Statistics

N 128 128

r2_a .97571316 .98512162

Estimation for xGrp1 -- xGrp2 id, f(6a) i(b0 0) --

Power: Y = b0 * (X^b1) OR ln(Y) = ln(b0) + (b1 * ln(X))

Exponential: Y = b0 * (e^(b1 * X)) OR ln(Y) = ln(b0) + (b1 * X)

--------------------------------------------------------------------------------

Variable Power Exponential

b0 9.152e-10 .02870534

2.10 14.91

0.0382 0.0000

b1 4.8013421 .04807048

47.29 79.25

0.0000 0.0000

Statistics

N 120 120

r2_a .98912057 .99525586

graph for the x values for Group2 was not attached previously

From the maximum likelihood formulation it is immediately obvious that the appropriate test is a test for homogeneity of means with the expected values proportional to the group sizes. I believe a chi-square test for homogeneity would be adequate.

Dear Sartaj,

Thanks a lot for your interest in these data and useful analyses! Indeed functions of the type you used are quite reasonable, and a difference between groups is very apparent. What I am thinking to do next is to come up with a biologically-motivated functional form, and then use that for analysis.

Vic: your comment is very interesting. Can you explain more?

Igor, its my pleasure! BTW did you notice that I have provided you three sets of models: polynomial, power and exponential. Parsimonious and best fit model being exponential with 99% variability explained here are the equations for your data:

For Group 1: X = 0.02870534 * EXP(0.04807048 * n )

For Group 2: X = 1.0460933* EXP(0..03814904 * n )

*where n is from 1 to N (sample size)

Back to your original question, "what is the appropriate method to compare these two distributions" is bootstrap analysis of comparing coefficients or other nonparametric tests.

Best -

Igor, my point was that you can compare visually the model exponential distribution versus the sample distribution you have on a qq-plot(R can be handy here). Follow this link on how to achieve this - https://www.inkling.com/read/r-cookbook-paul-teetor-1st/chapter-10/recipe-10-22

If the two distributions are alike they will deviate little from the diagonal line. You can use log axes for clarity and focus on tail towards the zero values.

Another way is K-S test which is a non-parametric method for comparing distributions. It tells you how good the two distributions match irrespective of the model assumption. It calculates a distance metric D for the comparison. Smaller the D, more similar the distributions. Generally, we hope to values of D smaller than 0.01 but this is just arbitrary and may depend on context.

You can read about KS test here and this page also has a link for doing the test online http://www.physics.csbsju.edu/stats/KS-test.html

Dear Igor!

There are two common approaches to the comparison of two samples of exotic (non-Gaussian) distributed random variables.

First. Assuming that there are samples from the same distribution, the amount can be constructed, for example of squared difference, normalized by the product of the standard deviations to be distributed (also presumably) a chi-square or the ratio of the standard deviations (it will be Fischer distribution with corresponding degrees of freedom). Your real-world data will show you whether this hypothesis must be REJECTED or not.

Another way is to use the criteria for comparison of samples that do not depend on the distribution such as Kolmogorov-Smirnov test.

All recipes are well described in the ancient but ever living book J.H.Pollard.A handbook of numerical and statistical techniques. CUP, 1977. (Поллард, Справочник по вычислительным методам статистики, М.Финансы и статистика, 1982).

From there you can take a word for the selection of a suitable procedure in commercial packages.

Best regards.

V.S.

@ Amit: the q-qplot is a descriptive tool for comparing a given dataset against a known (model) distribution . Its not the inferential tool to quantify the differences between two given set of data. Please look at www.csub.edu/ssricrem/spss/SPSS11-4/11-4.htm‎. In our case both sample data (grouped by either 1 or 2) arise from exponential distribution and thus there is no differences in terms of distribution but there are differences in parameters. If you see my analysis, I have already quantity those parameters and they are significantly different. Igor has already shred his data and if you practically show us graphically using R that will be pretty awesome.

@Sartaj - Generally, QQplots have been used with one dataset compared against a known distribution. If I make it a non-parametric comparison, I can use both distributions from samples. This is less followed because of complexity of interpretation as you have to drop the model parameters for non-parametric statistics. But this might be the easiest way possible even if two distributions are same, without worrying about parameters. Its not even comparing the means and std deviation. It actually turns out ot be independent of parameters.If you want to compare two exponential distributions and say OK they are both exponential but have different parameters, this wont help you. I am not aware of other tests for that. You can simple estimate parameters and compare the numbers then.

@Sartaj your statement may be resulting from SPSS probably. No tool I know can do it directly for you. But this doesn't imply it is not doable or wrong. I ll see if I can do it for you.

This is not a problem about what is usually called "exponentially distributed data". Your data are counts and you hypothesise that the logarithms of the mean values equal A x^B for some covariate x and unknown coefficients A and B. Possibly it is reasonable to assume that the counts in each cell are Poisson distributed. In that case you can now estimate A and B by maximum likelihood. The statistical package "R" contains several functions for solving the resulting optimization problem. The log likelihood (if the counts are independent and Poisson distributed) is sum_i N_i log(lambda_i) - lambda_i where lambda_i = A x_i^B. You want to maximize this quantity through choice of A and B.

Hi Igor. I worked your sample3 for two data series. They have different value o media, different number of items N, but almost the same statistical adimensional structures. I anex a one page with graphs.

If we present the same sample using two different measurement units it is obvious that the numerical expected value of each one results different -for example, if you use british units in one case and international units in the other one- but the structure remains even if you employ two different fitting models. That is my conclusion.

Thanks for the samples because it permitted me to explain the method I recommend once again. Best wishes for 2014, emilio

As usual I took two mesages to send the file. thanks, emilio

You can try to make a logarithmic data transformation to reduce the variance. Then, you can test the goodness of fit to the Gaussian distribution of each group (Kolmogorov-Smirnov, Anderson-Darling, etc.). If all groups are normally distributed, you can use the standard parametric comparative tests. Other option is the using of non-parametric comparative tests, which do not require any distribution adjustment.

If you want to pairwise compare the groups I would use a Mann–Whitney U test (http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U), that will tell you if the means two populations are the same or different. But because you have more than one comparison (n choose 2, where n is the number of groups) you have to correct for that. The easier method is the Bonferroni correction (http://en.wikipedia.org/wiki/Bonferroni_correction), but it's very conservative, so if you have many groups you won't see differences between them.

Here is how it works: Each comparison has a 5% chance of error (WIth alpha = 0.05), so the probabilty of all being correct is 0.95^n, with 10 comparison = 0.6. You have a 40% chance of being wrong if your p-values are around 0.05! So Bonferroni corrects for that so (1-newAlpha)^n = 0.95 (Keep an effective alpha of 0.05)

Approximating: (1 - n*newAlpha) = (1 - desiredAlpha) = (1 - 0.05) = 0.95

0.05/n = newAlpha

So if you have 10 comparisons, the alpha required is 0.005.

So basically if you have 3 groups (A, B and C) and comparing them with the MWW test you get these p-values:

A--B: 0.03

A-C: 0.4

B-C:0.001

Then the p-value is significant if it is lower than 0.05/3 = 0.017. Therefore the only significant comparison would be B-C and those distributions are significantly different.