Eli

this may present slightly more issues than you think:

answer to Q1 is sort of yes - t-test is an option

It looks like you need An independent t-test (a two sample t-test), or student's t-test, which is an inferential statistical test that determines whether there is a statistically significant difference between the means in two unrelated groups which it appears you have here.

To run this you need One independent, categorical variable that has two levels (ie 1 for one variable A and 2 for the other variable B) and One dependent variable (containing A and B observations)

An independent t-test requires that the dependent variable is approximately normally distributed within each group. You can test for this using a multitude of tests, but the Shapiro-Wilks Test or a Q-Q Plot, are very common. You can run these tests in a software package such as SPSS

If you find that either one or both of your group's data is not normally distributed you have two options: (1) transform your data so that the data becomes normally distributed or (2) run the Mann-Whitney U Test which is a non-parametric test that does not require the assumption of normality.

Your Independent T-Test for Two Samples will test samples from different populations or test to see if there is a violation of the assumption of the homogeneity of variance (that is equal variances assumed). An independent t-test assumes the variances of the two groups you are measuring to be equal. If your variances are unequal, this can affect the Type I error rate. The assumption of homogeneity of variance can be tested using Levene's Test of Equality of Variances, which can also be determined in SPSS.

This test for homogeneity of variance provides an F statistic and a significance value (p-value). If it is greater than 0.05, the group variances can be treated as equal. However, if p < 0.05, you have unequal variances and you have violated the assumption of homogeneity of variance.

If the Levene's Test for Equality of Variances is statistically significant, and therefore indicates unequal variances, you can correct for this violation by not using the pooled estimate for the error term for the t-statistic, and also making adjustments to the degrees of freedom using the Welch-Satterthwaite method. In all reality, you will probably never have heard of these adjustments as software packages such as SPSS hide this information and simply labels the two options as "Equal variances assumed" and "Equal variances not assumed" without explicitly stating the underlying tests used.

Why not just use the confidence intervals that the software provides?  Values whose 95% confidence intervals do not overlap are statistically different.

T test is not appropriate for your situation since EC50 values are not means.

I guess your aim when comparing these EC50 values is to evaluate relative potency. If this is the case, then you need take another direction for statistical analysis.

Hi thank you all so very much for taking the time to answer my questions. I'll respond to each comment separately:

1. While, the process of checking for normality and equal variance definitely applies to fitting the dose-response curves to the assay data (and great description, by the way), I can not check these assumptions directly for the EC50s. They are predictions, and not part of a replicated set of values around a mean. Fortunately, I found an excellent book (cowritten by the CEO of GraphPad Prism): "Fitting Models to Biological Data using Linear and Nonlinear Regression" by Harvey Motulsky and Arthur Christopoulos. In this guide, they describe using a t-test in essentially the manner in which I described above. There are some issues with this methodology, that Motulsky and Christopoulos point out: not only does the data need to be normally distributed, but the SE values of the EC50s need to be accurate, which we can not be sure of. It appears that comparing nested models is my best bet (I will describe how they say to accomplish this below).

2. I could be wrong, but I think that comparing confidence intervals is not a foolproof method of finding significant differences. It is true that non-overlapping 95% confidence intervals occur around means that are significantly different, yet there are cases when overlapping confidence intervals still have significantly different means: http://blog.minitab.com/blog/real-world-quality-improvement/common-statistical-mistakes-you-should-avoid

3. I agree that a t-test is not appropriate. Here is how Motulsky and Christopoulos described comparing nested models to look for differences in EC50 values between two curves:

i. Fit curves separately to each compound, weighting the values if necessary to satisfy assumptions.

ii. Check for dependency between parameters and systematic runs (autocorrelation)

ii. Check confidence intervals of parameters (Hillslope, bottom and top asymptote, and LogEC50). Consider constraining certain parameters if this improves the fit and makes biological sense.

So, now we would have two separate curves (one for each compound) with entirely different parameter estimates.

iii. Run a global fit model in which the two compounds share the LogEC50 parameter.

iv. Use an F test or AICc values to compare the simpler model (global fit) to the more complex model (separate curve fits for each compound). To calculate F ratios and AICcs, you combine the sum of squares for each curve within the simple and complex models.

If having a model with different LogEC50s produces a better fit (according to the F test or AICc) for each compound, then a model with the same LogEC50s for each compound we can conclude that the EC50s are significantly different.

The F test will test the null hypothesis that the global fit model (simpler model) is better than the more complex model (different LogEC50s). A p-value less than 0.05 suggests that we would expect to see the simpler model fit the data this much better than the more complex model less than 5% of the time. And we can reject the null. 

So, what I plan on doing is comparing all compounds to each other in this manner. Since I have four compounds, there are six combinations of compounds and six associated F tests. I will Bonferonni correct: 0.05/6 = 0.0083

Thanks again and please let me know if you see any issues with this method of comparing models to test differences between compound EC50s.

Best Wishes,

Eli

I'll address point 2.

I guess first off, the example you linked to didn't give enough information to conclude anything from it.  (It doesn't say how they calculated the confidence intervals, or what test they used to determine that the means are significantly different, or where the data comes from, or what the original data was.)

But that's besides the point.   Of course, two different statistical tests are not the same, and won't always have the same results.  Using t-tests is not the same as using confidence intervals for means. (And there are several different ways of calculating confidence intervals.)

Bottom line:  There is no "foolproof" statistical method.

I guess you mean by logEC50 logDose since logEC50 is just a unique value. You can model dose-response relationship by logDose or Dose itself and select the model that best fits.

I had a glance to those t-tests or F-tests and they seem artificial to me. They seem to be forced to adapt your case.

I am not sure if two EC50 values can be compared statistically. Confidence intervals might give you an idea. Comparison of them means comparing two different curves. We have Deviance to evaluate logistic curve fits.

If you wish to find out the best compound (which has a greater EC50) statistically, you had better calculate relative potency and its confidence interval if compounds have parallel logit scale dose-response relationships. That way would be statistically valid.

In other words, those t and F tests are not foolproof at all.

Hi All, 

Thanks for your feedback! It's been immensely challenging figuring out how to analyze this dataset. While the model comparison method I described above works well when looking for significant differences in parameter estimates, if the curves, whose parameters we are comparing, are asymmetrical around the inflection point we can not compare EC50s (the inflection point is no longer the EC50). Unfortunately, asymmetrical curves fit my data better (e.g. a Gompertz curve) for the most part.

Sooooooo....I am resorting to the confidence interval evaluation. I will just mention in the discussion that we are fairly confident in, for instance, compound A being more fungicidal, herbicidal, or whatever than compound B, because their 95% confidence intervals do not overlap.

JMP Pro tests for parallelism and gives potency values. The only problem with that, I think, is that the curves that fit my compounds are typically not in the least bit parallel. In almost every case, they have a low p-value for the Chi-squared parallelism test.

Confidence interval comparison it is! Unless anyone has more suggestions, which I am afraid to even ask about because this data analysis has been taking me months to complete :(

Best wishes in life and in science,

Eli