There are many different tests and several versions of the t-test. The variant described in your second link is for two independent samples. If the data are in correlated pairs, then the paired difference t-test is more appropriate. This is simply a test on whether the average of the differences between pairs of values differs from zero. Both tests rely on the data being normally distributed (the differences in the paired case). In the independent case there is no exact test without an assumption, e.g. the two populations have the same variance, but the version given is an approximate test. A slightly better version in this case is the Welch test. You should be able to find these tests on Wikipedia.

All these tests are reasonably robust for non-normal data, but if the data are very skewed or tend to produce many extreme values, alternative tests could be considered, especially if the sample sizes are very different. For large samples you may use z (normal) tables instead of t, but it is preferable to use statistical software such as R and not worry about z or t.

One alternative, the Mann-Whitney, may be used for independent samples; the Wilcoxon signed rank test may be used for paired data. Again, look these up for necessary conditions and what exactly they are testing.

The t-test is a hypothesis test. Its applicability (practical significance, usability)  is NOT related to the research subject (like Diary Science) but to the statistical question and the data.

We typically face variable (i.e. non-constant) data. Replications of an experiment under really identical conditions are not possible, so every replication gives more or less different results. We can not perfectly predict what the result of an experiment will be. But we can quantitatively model our uncertainty. This is based on probability theory.

Based on this theory we can assign probabilities for observations/events/data (I will further just call it "data"). These assignments require a probability model that may be theoretically derived or simply justified by "experience". Having such a model one can calculate the probability of data. The model has to be specific (it has to be fully specified). This specification includes that a particular, specific "effect" (like for instance a difference in means) is precisely stated. But we do not know the effect. We only have noisy data.

Now there are two strategies to continue: We may estimate a "reasonable effect" by assuming all possible different values for the effect and see under which the observed data is most likely (or probable). This is the "maximum likelihood estimate". Or we may want to "test" if the data "too unlikely" given a particular assumed or "hypothesized" value of the effect. The hypothesized effect is called the "null hypothesis", because the test aims to "nullify" (reject) this particular hypothesis about the effect. Quite often, this particular hypothesis is that the effect is just zero. The test will then give a probability to get data that is at least as "extreme" as the actually observed data, under the statistical model that the effect is zero. When this probability is very low, we interpret it as a discredit of the (null-)hypothesis under which the probability was calculated; the data (sufficiently) convinces us to disbelieve/reject the null hypothesis (and thus believe that the effect is non-zero).

As already said, what probability model and what test to use depends on the kind of data and the kind of question. If the data is continuous, (essentially) unbounded, and when it is resonable to assume that the values scatter symmetrically around a common center (=the mean), then the normal probability distribution is the appropriate probability model, and t-tests are the appropriate tests about hypotheses regarding the mean values (or about differences between mean values).* Statistically, t-tests are tests about expected values (what is a statistical term). T-tests are performed to get an idea if we have reasons to "(statistically) expect a difference" (either to a given value [=one-sample test] or between two groups [=two-sample test]).

Sometimes not the expectations are of primary interest but the question is simply about a "stochastic difference". The null hypothesis is then: "if I have one observation from each group, the probability that the observation from the one group is larger than the observation from the other group is 0.5" (it is equally likely that the larger observation is from either group). This hypothesis is tested with the Mann-Withney test that Terry mentioned.

Data like counts (like the number of cattles in a cot) are not compatible with the assumptions to use the normal probability model. Often a Poisson probability model is more appropriate, if the story becomes more complex a negative binomial probability model may be have to used. Binomial data (can take only two different values liek yes/no, present/absent, died/survived...) calls for a binomial probability model. According to the models, different tests will be used.

If the question is not about means or expectations (or parameters of some other models like the Piosson rate-parameter or the binomial proportion-parameter), but about entire distribution, again different tests are employed. A frequent question is if frequency distributions change under different conditions. When the data is discrete (like counts of categories), a chi-squared test would be used. When the data is continuous, a Kolmogorov-Smirnov-Test would be used. Just to give an impression how the kind of data and the kind of the statistical question determin the "appropriate" test to use.

* When the data doe not scatter symmetriocally around a common center, often a transformation (like taking the logarithms) can make the transformed data behave "well" enough" so that the t-test can be performed using the transformed values. Additionally, t-tests also work "well-enough" when there are lots of data available (large sample sizes). The explanation is a bit difficult and involves the "central limit theorem", and the answer to the question "when is a sample large enough" depends on the kind and severity of the actual violation of the assumptions. There is not golden rule. However, a rough rule of thumb could be: n>20 should work well enough in most cases, n>50 should work work enough even in most "rather pathological" cases. On the other hand, the most likely "mistake" resulting from the violations is that the test will overestimate the likelihood of the data under the nuöll-hypothesis. This means that it will be too unlikely to reject the null hypothesis. This is bad for a personal research aim, but when a test sais that the data is really unlikely under the null hypothesis even when the assumptions are violated, than one is typically on the save side to reject the null hypothesis.

When there are three or more levels for the nominal variable, a simple approach is to run a series of t tests between all the pairs of levels. For example, we might be interested in the heights of athletes in three sports, so we could run t test for each pair of sports. (Note that this approach is not the same as a paired t test. That comes later.) A more powerful approach is to analyze all the data in one go. The model is the same, but it is now called a one-way analysis of variance (ANOVA), and the test statistic is the F ratio. So t tests are just a special case of ANOVA: if you analyze the means of two groups by ANOVA, you get the same results as doing it with a t test.

The term analysis of variance is a source of confusion for newbies. In spite of its name, ANOVA is concerned with differences between means of groups, not differences between variances. The name analysis of variance comes from the way the procedure uses variances to decide whether the means are different. A better acronym for this model would be ANOVASMAD (analysis of variance to see if means are different)! The way it works is simple: the program looks to see what the variation (variance) is within the groups, then works out how that variation would translate into variation (i.e. differences) between the groups, taking into account how many subjects there are in the groups. If the observed differences are a lot bigger than what you'd expect by chance, you have statistical significance. In our example, there are only two groups, so variation between groups is just the difference between the means.

I won't bother with trying to represent this model as an equation like Y = mX + c. Suffice to say that it can be done, simply by making an X variable representing sex that has the value 0 for females and 1 for males, say (or vice versa). So it is also a "linear" model, even though we don't normally think about it as a straight line. The parameters in the model are simply the mean for the females and the mean for the males.

http://www.sportsci.org/resource/stats/ttest.html

Vesna,

"The term analysis of variance is a source of confusion for newbies. In spite of its name, ANOVA is concerned with differences between means of groups, not differences between variances."

I don't subscribe to this point of view. I fact I think that the confusion is caused  by the misconception that the ANOVA was about "differences in means". Sure, the two-sample t-test is a special case that is mathematically identical to the ANOVA, but the generalization that ANOVA is a tool to analyze "differences between means" is not correct. The ANOVA really *is* an analysis of variances, not of means, and it compares the residual variance of two nested models (nested means that the response is the same and the smaller model has only predictors that are also in the larger model).

The ANOVA is not about comparing means. Means can be compared ONLY by pair-wise t-tests. There is no other way. You can do these t-tests using a pooled variance estimate to get more robust estimates of the standard errors. Such tests are called "post hoc tests" because they can be performed only after the entire data is collected (also from the other groups that are not actually compared in a particular t-test). Such a pooled variance is calculated during the ANOVA, so programmers used to combine ANOVA and post-hoc tests, and the unfortunate misconeption rose that "hoct-hoc" would mean "after ANOVA", what is logically wrong, too. Some argue that the ANOVA is required to keep a FWER of the "family of t-tests". This is also generally wrong (some few exceptions are the so-called ANOVA-protected tests, but these are, well, exceptions). Appropriate post-hoc tests control the FWER absolutely independent of any ANOVA.

So it is actually simple: t-test compare means. This can onyl be done by pair-wise comparisons, and a pooled standard error could be used to make it more robust. ANOVAs compare nested models to how much of the variance in the data is "explained" by a predictor (or a whole set of predictors).

Dear Sir, Terry Moore Jochen Wilhelm ·

Thank you very much for your nice opinion.

Dear All Colleagues 

Thank you very much for sharing your valuable ideas / opinions on RG network for my question.

I am trying to learn a lot of! 

T-tests and analysis of variance (ANOVA) are widely used statistical methods to compare group means. For example, the independent sample t-test enables you to compare annual personal income between rural and urban areas and examine the difference in the grade point average (GPA) between male and female students. Using the paired t-test, you can also compare the change in outcomes before and after a treatment is applied.

For a t-test, the mean of a variable to be compared should be substantively interpretable. Technically, the left-hand side (LHS) variable to be tested should be interval or ratio scaled (continuous), whereas the right-hand side (RHS) variable should be binary (categorical). The ttest can also compare the proportions of binary variables. The mean of a binary variable is the proportion or percentage of success of the variable. When sample size is large, t-tests and z-test for comparing proportions produce almost the same answer.

http://rt.uits.iu.edu/visualization/analytics/docs/ttest.php

Dear Vesna,

I know that ANOVA is frequently reported in the context of a comparison of means. Almost always, at least in the biological/biomedical literature that I am aware of, ANOVA is reproted unreflectedly because the software used provides the actually interesting post-hoc tests as a part of the analysis conducted when the menu item "ANOVA" is clicked.

I know that many introductory courses for biologists and physicians state that ANOVA is a tool to compare group means, often claiming that they test a so-called omnibus-hypothesis that all group means are equal (versus the alternative that this is not the case). Although the ANOVA is particularily sensitive to this alternative is this a kind of abuse. I think that RA Fisher would agree to say that ANOVA is a nice tool to quickly and roughly check if is would be worth to take the pain and calculate all the post-hoc tests. Given you have only paper and pencil that Fisher had, and not our possibilities to use fast computers and comfortable software. So there is a confusion of the purpose: the purpose of ANOVA was to save work by smartly abusing a tool, and not to actually test differences between means.

I would be glad if you could show me some real example where authors really used the ANOVA to test differences between means (except for the two-sample case, for sure!). This must be an example where *no* post-hoc tetst are performed (because this is what I say: the actual comparisons are done by these post-hoc tests; the ANOVA is either completely uninteresting or used only to control the FWER [in these rather exceptional cases of "ANOVA-protected post-hoc tests" like Fisher's LSD tests]).

Dear All Colleagues

Thank You very much! Really, All you are  excellent and active members of RG network. Please continue it!

Good Luck! 

Thank you very Dear Ayaz!