Dominique, you would just report the results as non-significant.  Lets suppose that you have 30 males and 30 females each providing one reaction time to some stimulus in your experiment.  You could report the entire ANOVA summary table which might look something like the following:

ANOVA Summary Table for Reaction Time

____________________________________________

Source            SS           df          MS           F         p

Gender           6.9            1           6.9         1.5      0.23

Error              266.8       58          4.6

Total              270.5       59

____________________________________________

You could report this in the text as “It was concluded that there is no significant difference in mean reaction times due to gender, (F(1,58) = 1.5, ns).”  Since you have only two independent groups, why not opt for a simpler test, the independent groups t-test.  In the case of just two groups, the square root of F will give you the t value.  You could just report this as (t(58) = 1.22, ns.) or ((t(58) = 1.22, p = .23).  Just for your own education, you might want to demonstrate to yourself that anytime to do an F test with 1 and n degrees of freedom it is equivalent to a t(squared) with n degrees of freedom. 

A post hoc test would be used to determine which of several groups might be different in means.  The ANOVA is generally a preparatory test that protects Type 1 error rates and is amenable to fairly complex data structures.  In your case, you don’t need any of the complex ways of partitioning variance components that can be provided by ANOVA.  Any post hoc test when you have only two groups to start with would degenerate into a plain old t-test and the conclusions would not change.  If you had a more complex structure and the entire ANOVA showed non-significant differences, then you would make an omnibus conclusion that you did not detect any differences.  You would use a post hoc (after the fact) test only if one or more sources of variance was significant.  This is different situation than creating a set of a priori (before the fact) contrasts, even though the computations might be the same as a post hoc test.  It is not the computations alone that provide for being able to make a statistical inference, it is about the design and what you believe and need to falsify before doing the analysis.     

Reaction time data is generally positively skewed.  You might want to transform your data before doing any analyses beyond a simple plot of the data.  Two transformations’ come to mind when I am thinking of reaction time data.  The first is a reciprocal transformation.  Where X’ – 1/X.  This will yield speed rather than time and it generally tends to reduce the skewness in the data.  The other transformation is a magnitude (log) transformation where X’ = log(X).  This will give you relative magnitudes of reaction time that may be more linear and will arrange themselves more like a normal distribution.  There are others, but these two seem reasonable with a logical basis to use either of them.  In any event it would probably be wise to transform your data so that it is at least symmetrical, if not mound shaped with well defined tails.

It is also likely that there is a problem of outliers with this kind of data.  An outlier is a datapoint that is sampled from some distribution other than the one you think you are sampling from.  Consider measuring reaction time as a pure neural reception and motor reaction time.  If one is momentarily distracted, the resulting point will not be just neural processing time, it will be neural processing plus distraction time.  The contaminated datapoints are outliers, whether they are extreme datapoints or not.  Just because a datapoint may be extreme does not necessarily make it an outlier, either. So you will have to decide on how you wish to handle outliers, particularly when reaction time is prone to contamination.  Since you may not be able to make high quality decisions, you may wish to find some technique with reasonable power that is robust and not overly influenced by outlying points.  You might trim the top and bottom 10% of  your data or use a process known as Winsorizing where you replace (impute) extreme datapoint values with the next most extreme datapoint values.  You might find a test for differences  other than means, or use a modified t-test where you calculate a robust standard error.

There are other ways that distributions differ, besides location (mean, median, mode, mid mean, trimean, trimmed mean, Winsorized mean, etc).  Their variability may be different (variance, standard deviation, range, midspread, average deviation from some measure of location, etc).  They may differ on shape (skewness, kurtosis, j-shape, etc) or clumping such as mixture distributions that look bi-modal or multimodal.  There may be unusual observations that are either legitimate extreme data points or outliers.  Each of these may be interesting in its own right and may tell you a lot about the samples (and populations from which they are drawn).    

Your second hypothesis regarding males being more competitive than females might be addressed by a test for difference in variances, but I’m not sure I understand what you are trying to assess or how you are going to measure it..  What specifically is your substantive hypothesis and how would you express this as a statistical hypothesis?  How do you operationally define competitiveness?  Are you manipulating competitiveness in some way?  How is competitiveness related to reaction time?  It is often better to figure out what you want to do before you attempt to assess how well you’ve done it. 

Best wishes,

David Drehmer

If one way ANOVA was not significant you would report that there was no significant difference in comparisons between A and B, however if post hoc analysis showed significant comparisons with respect to sex then you would report that post hoc a nalysis revealed significant differences with respect to sex showing the t value for example if Tukey HSD was used.

Dear Dominique,

I'm not quite understanding the "gender as a subscale" part of your question. Was gender your main Independent Variable? If so, you should be running a t-test or a Mann-Whitney U test, not an ANOVA. If not, then perhaps you are doing a two way ANOVA.

Generally speaking, if a one way ANOVA is not significant you should not run any post hoc test. This means you should spend more time talking about the descriptive statistics and the assumption checking rather than the result itself. A good place to start is an error bar chart, the Shapiro-Wilk normality test for each group and the Levene's test for homoscedasticity.

Your DV is "reaction times" - a variable for which I would not assume normal distribution. There should be some sensible probabalistic models for reaction times. Without knowing the topic well I would gusee that an exponential distribution (or, more generally, a gamma disribution) should more appropriate. So instead of using an ANOVA I would think that a generalized linear model of a some more sensible famliy should be more approproate.

Dominique, you would just report the results as non-significant.  Lets suppose that you have 30 males and 30 females each providing one reaction time to some stimulus in your experiment.  You could report the entire ANOVA summary table which might look something like the following:

ANOVA Summary Table for Reaction Time

____________________________________________

Source            SS           df          MS           F         p

Gender           6.9            1           6.9         1.5      0.23

Error              266.8       58          4.6

Total              270.5       59

____________________________________________

You could report this in the text as “It was concluded that there is no significant difference in mean reaction times due to gender, (F(1,58) = 1.5, ns).”  Since you have only two independent groups, why not opt for a simpler test, the independent groups t-test.  In the case of just two groups, the square root of F will give you the t value.  You could just report this as (t(58) = 1.22, ns.) or ((t(58) = 1.22, p = .23).  Just for your own education, you might want to demonstrate to yourself that anytime to do an F test with 1 and n degrees of freedom it is equivalent to a t(squared) with n degrees of freedom. 

A post hoc test would be used to determine which of several groups might be different in means.  The ANOVA is generally a preparatory test that protects Type 1 error rates and is amenable to fairly complex data structures.  In your case, you don’t need any of the complex ways of partitioning variance components that can be provided by ANOVA.  Any post hoc test when you have only two groups to start with would degenerate into a plain old t-test and the conclusions would not change.  If you had a more complex structure and the entire ANOVA showed non-significant differences, then you would make an omnibus conclusion that you did not detect any differences.  You would use a post hoc (after the fact) test only if one or more sources of variance was significant.  This is different situation than creating a set of a priori (before the fact) contrasts, even though the computations might be the same as a post hoc test.  It is not the computations alone that provide for being able to make a statistical inference, it is about the design and what you believe and need to falsify before doing the analysis.     

Reaction time data is generally positively skewed.  You might want to transform your data before doing any analyses beyond a simple plot of the data.  Two transformations’ come to mind when I am thinking of reaction time data.  The first is a reciprocal transformation.  Where X’ – 1/X.  This will yield speed rather than time and it generally tends to reduce the skewness in the data.  The other transformation is a magnitude (log) transformation where X’ = log(X).  This will give you relative magnitudes of reaction time that may be more linear and will arrange themselves more like a normal distribution.  There are others, but these two seem reasonable with a logical basis to use either of them.  In any event it would probably be wise to transform your data so that it is at least symmetrical, if not mound shaped with well defined tails.

It is also likely that there is a problem of outliers with this kind of data.  An outlier is a datapoint that is sampled from some distribution other than the one you think you are sampling from.  Consider measuring reaction time as a pure neural reception and motor reaction time.  If one is momentarily distracted, the resulting point will not be just neural processing time, it will be neural processing plus distraction time.  The contaminated datapoints are outliers, whether they are extreme datapoints or not.  Just because a datapoint may be extreme does not necessarily make it an outlier, either. So you will have to decide on how you wish to handle outliers, particularly when reaction time is prone to contamination.  Since you may not be able to make high quality decisions, you may wish to find some technique with reasonable power that is robust and not overly influenced by outlying points.  You might trim the top and bottom 10% of  your data or use a process known as Winsorizing where you replace (impute) extreme datapoint values with the next most extreme datapoint values.  You might find a test for differences  other than means, or use a modified t-test where you calculate a robust standard error.

There are other ways that distributions differ, besides location (mean, median, mode, mid mean, trimean, trimmed mean, Winsorized mean, etc).  Their variability may be different (variance, standard deviation, range, midspread, average deviation from some measure of location, etc).  They may differ on shape (skewness, kurtosis, j-shape, etc) or clumping such as mixture distributions that look bi-modal or multimodal.  There may be unusual observations that are either legitimate extreme data points or outliers.  Each of these may be interesting in its own right and may tell you a lot about the samples (and populations from which they are drawn).    

Your second hypothesis regarding males being more competitive than females might be addressed by a test for difference in variances, but I’m not sure I understand what you are trying to assess or how you are going to measure it..  What specifically is your substantive hypothesis and how would you express this as a statistical hypothesis?  How do you operationally define competitiveness?  Are you manipulating competitiveness in some way?  How is competitiveness related to reaction time?  It is often better to figure out what you want to do before you attempt to assess how well you’ve done it. 

Best wishes,

David Drehmer