Hey again, Sami!

Stefano makes important points. I only attended to your italicized question regarding how to conduct the Bonferroni in your application--my bad, thanks Stefano.

Regardless of the study N, it is appropriate (common practice in journals which I read) to report which p-values meet the experimentwise (Bonferroni) criterion for statistical significance, as well as which p-values failed the Bonferroni criterion but met the per-comparison criterion (p

Hello Sami,

The choice to use a non-parametric test is typically conditioned by the data set not conforming to the usual assumptions for parametric tests (interval/ratio strength scale for the measure, with model-data residuals being normally and independently distributed, and equal variance for groups/levels) rather than a result of sample size. (The lone exception to this might be for questions about normality: the Central limit theorem indicates that as sample size approaches infinity, the distribution of sample means approaches normality, no matter what the shape of the parent population.

With that many DVs, the power of any statistical test (parametric, non-parametric, exact, bootstrap) will be low unless the effects are very strong.

However, to answer your question, it sounds as if you have three sets of DVs rather than one set of nine DVs.

If you'd like to explore non-parametric multivariate tests, here are two good places to start:

Hettmansperger, T.P. (1984). Statistical inference

based on ranks. New York: Wiley. (See chapter 6, pp. 279-296.)

Article outlining these methods: sign test, wilcoxon test,

mann-whitney test, in S code is Sheu, C-F. & O'Curry, S. (1996).

Implementation of nonparametric multivariate statistics with S.

Behavior Research Methods, Instruments & Computers, 28(2), 315-318.

Good luck with your work!

Dear Morse,

Thank you for your answer. I mean, I have to use non-parametric tests because of the fact that my sample size is so small and accordingly almost all assumptions of parametric tests (normality, etc.) are violated.

I start to examine the resources that you suggest.

Thank you again.

Sincerely,

Sami.

Sami, your question is not clear and you seem to be confused. When you compare two groups in terms of a characteristic, you can use Mann Whitney U. Although Kruskal Wallis would give the same result, this is prefered by many.

Manova is something very different and not comparable, since it is about more than one dependent variables.

I don't see why you need a correction.

What is your hypothesis and variables? You should explain more.

Dear İyisoy,

I apologize for my complicated style. In short, my problem is:

I have two groups and nine dependent variables: Intelligence scores (5), report notes (2), teacher assessments (2).

I have to use the Mann-Whitney U test to look for a difference in terms of dependent variables between the two groups because the assumptions of parametric tests have been violated. I have to run a total of nine analyzes. Should I use any correction when determining significance value (p)? Or should I use the standard value (.05)?

If I had 3 groups and 1 dependent variable, I would run Kruskal-Wallis. Then I would do a paired comparison to find the difference. I was going to use a correction for p here. For example, with the Bonferroni correction, I would divide my p-value by 3 and I would question the H0 hypothesis accordingly.

Do I need to make a correction for my current work? That's my question.

I hope it is more clearly.

Thank you again for your interest.

Hi, Sami!

You have to adjust the p value which is used to establish statistical significance. Adjustment is based on the number of tests of statistical hypotheses being evaluated--in your case, nine tests. Here is a link: http://www.cogsci.ucsd.edu/~dgroppe/STATZ/Abdi-Bonferroni2007-pretty.pdf

For more information, google on "Sidak Bonferroni-type multiple comparisons procedure"

There is a more vigorous non-parametric test than U, if you are interested:

https://odajournal.com/2014/12/27/unioda-vs-mann-whitney-u-test-comparative-effectiveness-of-laxatives-3

https://odajournal.com/2014/12/09/unioda-vs-mann-whitney-u-test-sunlight-and-petal-width

I would personally use the raw uncorrected p-values, avoiding to claim results as “significant” or “not significant.” Article Statistical Rituals: The Replication Delusion and How We Got There

Rather focus on descriptive statistics, your research hypothesis and effect sizes.

If you really really need to compare them to an arbitrary threshold, i.e. 0.05, then yes, you should correct for the overall number of tests performed, i.e. nine.

Also, with that sample size the p-values under the null might not be uniform as you expect them to be if you interpret them as an "observed error rate" so using an arbitrary threshold like 0.05 is even more problematic.

************************************

The WMW test was designed to test the null hypothesis that P(X

Hey again, Sami!

Stefano makes important points. I only attended to your italicized question regarding how to conduct the Bonferroni in your application--my bad, thanks Stefano.

Regardless of the study N, it is appropriate (common practice in journals which I read) to report which p-values meet the experimentwise (Bonferroni) criterion for statistical significance, as well as which p-values failed the Bonferroni criterion but met the per-comparison criterion (p

You first need to run test for testing normality. If that test confirms normality then you can proceed with usual ANOVA and multiple comparison. If not then non-parametric pairwise comparison can be done. This is available in several softwares.

Assumptions required by ANOVA:

https://en.m.wikipedia.org/wiki/One-way_analysis_of_variance

https://stats.stackexchange.com/questions/6350/anova-assumption-normality-normal-distribution-of-residuals

Dear Yarnold, Nembrini and Shahbaz;

Thank you for your answers.

Firstly, in fact, I thought that the U test will be a suggestion of the reviewers. That's why I did not use more vigorous or robust tests. However, the studies you sent will guide me in my future work. I'm grateful for that.

Secondly, afterward, I thought that it would be reasonable to report the value of p I used and leave the assessment to the readers. I have also seen articles on the evaluation of the p-value in various Facebook groups (for example, in Prof. Uli Schimmack's psychological methods group or PsychMap). I looked at them again. My idea became definite with these studies.

I would like to repeat that I am wrongly talk about N. Our need for non-parametric testing was not derived from N, but because of the fact that assumptions of parametric tests did not meet.

I am aware of the importance of starting research with Power analysis; however, our sample was a very rare patient group (6-17 aged children with absence epilepsy who do not have any other psychological or neurological diagnosis, and also they have to two medication maximum etc.). Even if we cannot meet the power analysis parameters since it is difficult to reach such a sampling, we were able to get funding from our university. By the way, we had to travel more than 1000 kilometer to reach 19 children with this feature. When we talk about them with our professors, our university stated that they can provide funding in the future.

I read the answers here and various handbooks. I also contacted other professors. Then I tested significance by dividing p-value by 5 for intelligence test and 2 for other structures. As a result, our article was accepted yesterday.

Thank you again for all the information.

Best regards,

Sami.