Also, one of the existed post hoc tests, Dunnett's test, has been designed to compare each of the means, from a small or large number of experimental groups, against a control group mean.
Post hoc tests are run to confirm where the differences that occurred between groups; they should only be run when you have shown an overall statistically significant difference in group means (i.e., a statistically significant one-way ANOVA result).
Post hoc tests are designed for situations in which the researcher has already obtained a significant omnibus F-test with a factor that consists of three or more means and additional exploration of the differences among means is needed to provide specific information on which means are significantly different from each other.
Question should be more precised. If results by employing ANOVA are statistical significant you will use post hoc test to investigate where is the significant difference among your groups. You may use Sheffe test, Bonferroni, Tukey Test etc.
Also, one of the existed post hoc tests, Dunnett's test, has been designed to compare each of the means, from a small or large number of experimental groups, against a control group mean.
There are a lot of post-hoc tests. The most common post-hoc tests are: Bonferroni Procedure,Duncan’s new multiple range test (MRT),Dunn’s Multiple Comparison Test,Fisher’s Least Significant Difference (LSD),Holm-Bonferroni Procedure, Newman-Keuls, Rodger’s Method, Scheffé’s Method, Tukey’s Test, Dunnett’s correction, Benjamin-Hochberg (BH) procedure. You can use SPSS software ( https://spss-64bits.en.softonic.com) to determine those that you want from these tests. For example Fisher’s Least Significant Difference (LSD) identify which pairs of means are statistically different and it used t-values.
I am skeptical about the validity of the F test part of the ANOVA. It is hyper sensitive to non normality and significance might be false, or lack of significance might be false. However, the Mean Square Error could be considered an improved estimate of the variance and used for subsequent multiple comparison tests. (Maybe I am a bit of a heretic, but I see the partitioning method of the ANOVA as its only value, unless the sample size is almost impractically large.)
Unfortunately, if your procedure is post hoc and not preplanned you have to test every treatment against every other treatment. A very good test is to use the Bonferroni correction with say a series of t tests after getting a variance estimate from the partitioning procedure of the ANOVA. If you plan the tests before any analysis or examination of the data (best before the data are obtained), then you can restrict your analysis to comparing against a control (that must be designated before data collection). This makes the test more practical (reduces costs in terms of required sample size to obtain a useful level of power) and also more likely to detect what you seek without enormous sample sizes.
Using the R language via RStudio which is free from the Internet is as good as SPSS or SAS and is becoming more popular.
In addition, the comparison of means with the aid of the selected post hoc test should make sense, in order that researchers may draw a comprehensible conclusion.
Using the Dunne, Tukey, Scheffe and the Bonferroni post hoc tests used with a one-way ANOVA are used to compare means to determine if there is a significant difference among the means. The F test aids in additional comparison of significance among the means.. the LSD is least significant difference.
Post-hoc (Latin, meaning “after this”) means to analyze the results of your experimental data. They are often based on a familywise error rate; the probability of at least one Type I error in a set (family) of comparisons. The most common post-hoc tests are:
Bonferroni Procedure
Duncan’s new multiple range test (MRT)
Dunn’s Multiple Comparison Test
Fisher’s Least Significant Difference (LSD)
Holm-Bonferroni Procedure
Newman-Keuls
Rodger’s Method
Scheffé’s Method
Tukey’s Test (see also: Studentized Range Distribution)
Dunnett’s correction
Benjamin-Hochberg (BH) procedure
When you run Analysis of Variance (ANOVA), the results will tell you if there is a difference in means. However, it won’t pinpoint the pairs of means that are different. Duncan’s Multiple Range Test will identify the pairs of means (from at least three) that differ. The MRT is similar to the LSD, but instead of a t-value, a Q Value is used.
ANOVA test tells you whether you have an overall difference between your groups, but it does not tell you which specific groups differed – post hoc tests do. Because post hoc tests are run to confirm where the differences occurred between groups, they should only be run when you have a shown an overall statistically significant difference in group means (i.e., a statistically significant one-way ANOVA result). Post hoc tests attempt to control the experimentwise error rate (usually alpha = 0.05) in the same manner that the one-way ANOVA is used instead of multiple t-tests. Post hoc tests are termed a posteriori tests; that is, performed after the event (the event in this case being a study).
Types of test:
There are a great number of different post hoc tests that you can use. However, you should only run one post hoc test – do not run multiple post hoc tests. For a one-way ANOVA, you will probably find that just two tests need to be considered. If your data met the assumption of homogeneity of variances, use Tukey's honestly significant difference (HSD) post hoc test. Note that if you use SPSS Statistics, Tukey's HSD test is simply referred to as "Tukey" in the post hoc multiple comparisons dialogue box). If your data did not meet the homogeneity of variances assumption, you should consider running the Games Howell post hoc test.