That depends pretty much on your specific experiment, your questions, and your hypotheses. For instance (just to name 1 out of a million): are all treatments completely different of are there "related treatments"? Such knowledge could theoretically be used for the analysis. However, the effort to optimally using such information is often not worth the gain of additional insight.

Generally, I would not recommend using LSD, because It is too liberal. Better use Tukey's HSD.

Paul, Jenkins wrote that he did ANOVA. Doesn't this imply that there is one continuous dependent and one categorical independent (presumably with more than 2 categories) variable? Doesn't this further imply that the errors are normal or N is large enough to rely on the central limit theorem (why else should N matter)? If it was a within-subjects design, wouldn't he have mentioned repeated-measures and paired tests instead?

This is not sarcastic. I am honestly asking you to find out if I miss something or understod something wrongly.

Hey guys,

slow down. First, we used the randomized complete block design (RCBD). We tested treatment effects on yield response of water spinach. We had 5 treatments and 8 replications. The analysis of variances is based on yield (kg) per treatment. Analysis of Variance was based on agricultural statistical analysis by Gomez and Gomez (1986) using the RCBD format.

Sources of variations include:

Replications (r-1)

Treatment (t-1)

Experimental Error (t-1) (r-1)

Total Error (tr)-1

Theoretically, how would you know which treatment mean yield was significantly different among the experimental treatments on water spinach?

ANOVA certainly does tell you that there is a high significant difference. Thus, one practical way of knowing as proposed by Williams (2010) "Research design and statistical analysis" is to compute the Fisher's LSD test. LSD can only be computed when ANOVA F omnibus is significant. The central idea of the LSD is to compute the smallest significant difference (i.e. LSD) between two means as if these means had been the only means to be compared (i.e. with a t test) and the declare significant any difference larger than the computed LSD across and within the experimental treatment groups (Williams 2010). I hope this further elaborate my interest.

Jochen Wilhelm, I did compute the Tukey's HSD and result was not as ambitious as others (LSD, Neuman-Keuls Test for Contrasts on Ordered Means, etc) using StatPlus. I will however weight my options and see which ones best described the data and outcomes relative to other analyses.

So, I hope things doesn't get out hand between you two. I would recommend that the discussion remain as professional, respectful, and cordial as much as possible even if we share divergent views. This is what critiques is about. We learn through these kinds of exchanges.

Best fellows.

Bye the way, I sincerely appreciate the discussions, because it gets me thinking. However, hope things don't get too spice up. Thanks for the insights!

Paul, it does not make the impression to me that you are interested in a discussion about anything. Your question you referred to is only sarcasm to me, and therefor I am not going to participate in this game. As it looks to me I think you only try to sell your books about with bootstrap methods, presented like the "emperor in new clothes".

Hi Paul,

Thanks so much for the references you listed. However, I am following a straight set of form of analysis and I really appreciate the engagements.

Best

I recommend Tukey's HSD in this case.

I too recommend the tukey HSD. It will be very easy to see what treatments are significantly different using this method. Each treatment is given a letter and if the treatment is assigned a different letter then they are significantly different.

I have just re read your comments on the previous page and see that you have carried out the tukey test- did it not give you the results you were hoping for?

Hi Hannah,

I did Post Hoc Test with all the possible tests included using StatPlus. I think selecting the optimal test and results would have some tradeoffs that i need to consider. The LSD test results show 7 significant differences, while the HSD shows 4 significant outcomes. Certainly each tests has its own advantages and disadvantages. I think it is a matter of choice to select, which result best describe the specific research objective.

Yes definitely. Good luck with all that data.

At the Natural Resources Institute (University of Greenwich) we have a stats expert that can help us with stats dilemmas so I think we are very fortunate. Sounds like you are becoming an expert yourself.

Thanks Hannah. I think the discussions here in RG (including yours) give me more eyeballs to look into the results. Thanks.

Jenkins,

I'd like to add to the discussion by pointing out there is literature to support your approach.   If you use the ANOVA to determine that there is some significant difference amongst the means at 5%, then applying simulation studies have suggested that testing all pairwise comparisons using t-tests with homogeneous variance and df from the ANOVA (so called LSD method), also using 5%, is appropriate (see, e.g., Rencher, Multivariate Statistical Inference and Applications, Wiley1998).   That is, simulations have suggested that the 5% error rate (for false positives) is 'usually maintained.'  However, as others in this thread have suggested, many statisticians would not recommend using LSD, even though it has Fisher's name attached to it, because the error rate could be larger than 5%.  Tukeys test is theoretically proven, and simulation tested, to maintain a 5% error rate.   So while you think that getting 7 'positive results' via LSD via 5 'positive results' via Tukeys is a good thing, a statistician is suspicious whether the additional 2 sig. results are really just 'false positives.'

Also, I think that Paul's point re: being careful with parametric tests, like ANOVA, LSD, and Tukeys, are well taken.   It is crucial  for small data sets to check the normality and homogenous variance assumptions.   I recommend using individual value, residual and normal probability plots to check thee normality and variance assumptions. If you do not have homogeneity of variance, then an alternate approach to using ANOVA then t-tests (LSD) is to simply apply Welch 2-sample t-tests (i.e., using heterogenous variance) to all comparisons, but now you would need to apply a multiple comparison correction (like Bonferronis (conservative) or FDR (more liberal) to keep the error rate at 5%.  While this second approach deals with non-homogenoeus variance, it will likely be even mopre conservative than Tukeys (i.e., you may find even less than 5 'positive results.'   But the point is, you'd be using the right statistical model.

I hope these comments are useful.

Al

Thanks for the comments, but I assure you that whatever the path I take. It is in the best interest of the farmers I worked with. So, enjoy Thanksgiving if you celebrate one. If not, have a great week!

Testing the homoscedasticity assumptions is a natural reaction, but there are at least five papers indicating that this approach is unsatisfactory. References can be found in my 2012 book Modern Statistical Methods for the Social Sciences, CRC. Tukey's method has been known go be unsatisfactory, even under normality, since 1980. Fisher's method is even worse. (Again, details are in my book.) There are numerous papers indicating that non-normality is much more serious than once thought. Also, there are now many robust methods that can make a substantial difference in terms of both power and Type I errors. One of the simpler methods is based on a generalization of  the T3 procedure. Details are in my book. The R function lincon can be used to apply it. There are other options, but a brief explanation is impossible. If you can, take a look at my book.  Hope this helps.

LSD, Tukey's, and Schiefe's tests compare pairs of means. There is a provision for comparing groups of means. Kindly refer to Gomez and Gomez for further details which will of course further improve the outcomes.

Hi Mohammad, thanks for the inputs and I did use Gomez and Gomez, in fact and as you know, their book is one of the very few agronomic research design and statistical analysis text that each and every agronomists and agricultural researchers should have.

In case it helps, Fisher's LSD and related homoscedastic methods have been obsolete since 1980 due to a paper by Dunnett. Even when standard assumptions are true, Fisher does not control FWE except when dealing with three groups. Under general conditions standard methods use the wrong standard error which can result in poor  control over the Type I error probability and poor power. Skewness and heavy-tailed distributions also create serious concerns under general conditions. Also, using Fisher or Tukey-Kramer only after a significant F alters their properties in a negative manner as   explained by Bernhardson (1975). Many new and improved methods are now available.

For details, see for example 

Wilcox (2012).Modern Statistics for the Social and  Behavioral Sciences: A Practical Introduction  New York: Chapman & Hall/CRC press

or

Wilcox, R. R. (2012).  Introduction to Robust Estimation and

Hypothesis Testing  3rd Edition. New York: Elsevier

Unfortunately, standard training typically ignores the many advances and insights that have occurred during the last 50 years.