Hello Mahdi,

The example posted at https://www.statisticshowto.datasciencecentral.com/benjamini-hochberg-procedure/ is pretty clear, but let's break it down a bit further.

1. Run all of the tests desired, and note the "ordinary" p-value reported (directional or non-directional, as consistent with your research question) for each.

2. Put them in ascending order, from smallest observed p-value to largest observed p-value.

3. Rank the results. Smallest p-value = rank value of 1; largest p-value = rank value of c (where c is total number of comparison tests run).

4. Choose a target familywise Type I risk (the "false discovery" rate).

5. Starting with the result that was rank 1, do this:

a. Compute a per-comparison probability threshold = (rank / c) * FW_risk

b. If the ordinary p-value was less than the per-comparison threshold, declare that result to be statistically significant.

6. Repeat step 5 until you encounter a non-significant result or exhaust the list.

As an example, pretend that the fourth-ranked result (out of 10 total) had an ordinary p-value reported of .0257, and your FW_risk was chosen to be .10 (10%):

The per-comparison alpha level would be = (4/10) * .10 = .04.

Since the observed ("ordinary") p-value was .0257, it is less than the per-comparison threshold for this case, so result #4 would be declared statistically significant. You would then proceed to evaluate result #5 (against the per-comparison alpha level of 5/10 * .10 = .05).

Good luck with your work.

I just cross-checked that in R, comparint it to p.adjust(p, method="BH").

Good news: tha's all correct :)

p

Nowadays, there are many ways to apply the Benjamini-Hochberg FDR correction.

If you know nothing about the BH and its other q-values versions, better to start with the classical Benjamini-Hochberg (BH) 1995 procedure. This procedure works mainly on sorted p-values.

[ref. Benjamini, Y. Hochberg (1995). Controlling the False Discovery Rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society Series B, 57, 1, 289-300.]

This algorithm is well explained with an example in the 1995 paper:

1. Convert all your m tests statistics(correlations, or T tests) into m p-values.

2. Choose significance level alpha. People like to use 0.1 or 0.05 (but other levels