Hello Husam,

If the principal purpose of your study was to use your sample of 1,000 in order to make inferences about the level of awareness in your target population of 30,000 then your concern is with the precision of any estimate from the sample, and not so much with statistical power (to reject some null hypothesis, if it happens to be false).

If your principal purpose was in comparing subgroups with respect to level of awareness (e.g., males vs. females; students in science majors vs. students in non science majors), or something similar, then perhaps statistical power is pertinent. However, it sounds as if the data are already collected and analysis already completed. At this point, so-called post hoc power offers very little insight to you, as your sample results are already known, and your statistical decision already made.

At best, post hoc power offers the answer to a question that almost nobody would have asked: If a similar study were undertaken, to test the same (null) hypothesis, using the same alpha level (Type I error risk), the same sample size, the same measure(s), the same statistical method, _and_ the target effect size for such a study was set at a value equal to what you obtained in your study, what would be the probability of detecting that effect (or larger), if it existed in the population? Not only is it a mouthful just to declare, I doubt that any other researcher would have chosen your obtained ES as the target ES of interest.

If you could elaborate on what your goal is for desiring some sort of power estimate, after having run the study, then maybe you might get a different answer from someone. (Please note, doing this just to satisfy some reviewer's suggestion that this exercise is in anyway helpful is largely a waste of your time and any reader's time spent reading it.)

Good luck with your work!

Let us consider the estimation of sample size for a cross-sectional study.

In order to estimate the required sample size, we need to know the following:

p: The prevalence of the condition/ health state. If the prevalence is 32%, it may be either used as such (32%), or in its decimal form (0.32).

q: i. When p is in percentage terms: (100-p)

ii. When p is in decimal terms: (1-p)

d (or l): The precision of the estimate. This could either be the relative precision, or the absolute precision. This will be discussed later in this post.

Za [Z alpha]: The value of z from the probability tables. If the values are normally distributed, then 95% of the values will fall within 2 standard errors of the mean. The value of z corresponding to this is 1.96 (from the standard normal variate tables).

The formula for estimating sample size is given as:

(Za)^2[p*q]      where the symbol ^ means ‘to the power of’; * means ‘multiplied by’

N=      d^2                                      that is, “Z-alpha squared into pq; upon d-square”

substituting the values of Za, we get:

N= (1.96)^2[p*q]

d^2

We can round off the value of Za (1.96) to 2, to obtain:

N= (2)^2[p*q]

d^2

or, N= 4pq/ d^2      that is, “4 pq by d-square”

Example:

I wish to conduct a cross-sectional study on awareness of Hepatitis B among school children. A literature search reveals that other investigators have reported knowledge to range from 5% to 20% among students of grades 6 through 8. What should the size of my sample be?

The formula requires us to input the value of d (precision). If the absolute precision is known, there is no problem. However, often we can only input a relative precision. Where do we get the value of relative precision from?

Typically, relative precision is taken as a proportion of ‘p’. The maximum permissible limit is 20% of ‘p’.

In the above example, if ‘p’ is 20%, then ‘d’ will be (20/100)*20= 0.2*20= 4 {Taking a relative precision of 20%}.

This means that we will be able to detect a ‘p’ (prevalence) of 18% or more {half the value of relative precision on either side of ‘p’–> +/- 2%: 18% to 22%}.

That is, by taking a relative precision of 20% of ‘p’, the study will be able to detect the true awareness level if the actual prevalence is 18% or more. If the actual prevalence is less than 18%, however, the study will be unable to detect it accurately.

Therefore, the larger the value of ‘p’ (prevalence), the larger the possible value of ‘d’ (relative precision), keeping ‘d’ fixed (say, at 20% of ‘p’). If the prevalence is 50%, ‘d’ (20% of ‘p’) would then be 0.2*50= 10 (as compared to ‘d’ = 4 when ‘p’ = 20%).

The reverse is also true: the smaller the value of ‘p’, the smaller the value of ‘d’. A smaller ‘d’ implies a larger sample size. Therefore, the choice of ‘p’ is crucial.

We can now input the values in the formula to obtain the sample size:

For the calculation, we will take ‘d’ as 4. This yields:

N= (4*20*80)/ (4*4)

= 400 this sample size will enable us to detect the truth if the prevalence is between 18-22% (or more).

If we took ‘p’= 5, then the sample size would be:

N= (4*5*95)/(1*1)                                           [‘d’= 0.2*5= 1]

= 1900 this sample size will enable us to detect the truth if the prevalence is between 4-6% (or more).

So should I take ‘p’= 20% or ‘p’=5%?

That depends upon:

1. The location of the original study- if you are planning to conduct the study in an urban area, use the prevalence reported by studies conducted in urban areas, and vice versa.

2. The available resources (time, manpower, money, etc.). Aim for the largest feasible sample size. The size should be adequate to yield 80% power. Do not unnecessarily increase the sample size unless the intention is to obtain greater power. If so, please mention the same in the methodology section.

3. The results of your pilot study. If you have conducted a pilot study, the prevalence obtained from that study should be taken as ‘p’. This will be much more accurate than any other external value.

Note 1: If you have multiple objectives, you must calculate the required sample size for each objective, then choose the largest sample size thus obtained. This will ensure adequate power for all objectives, else the study will lack power for one or more objectives. That is, you may not be able to detect a significant result where it actually exists because you failed to include enough subjects to detect it.

Note 2: It is advisable to mention a range rather than a single value for sample size. This is standard practice in the west, but not in India. A range may be obtained by calculating the sample size for different values of ‘p’.

I recommend https://www.danielsoper.com/statcalc/default.aspx

Hello David Morse , thank you for your answer,

Indeed, I already collected my sample, and the primary outcome of the study was to measure the level of awareness in the target population. However, I did compare subgroups in my population to find what predicts a high awareness score (e.g., do participants with higher SES score higher on awareness level? and does having the disease predict a higher score as well? etc).

The interpretation of post hoc power analysis that you provided was very helpful, thank you for explaining it.

So for a future study, I need to find a similar study in a similar population and take their ES to measure the sample size I need, is that right? Also, what do I do if I could not find a similar study? do I just skip power analysis?

Hello again Husam,

In general, declaration of a target ES may be the most difficult part of a priori thinking about what would comprise a suitable sample size for testing your specific hypothesis(es) in your study. Ideally, the ES chosen should represent the smallest, non-ignorable degree of difference/relationship that you would like your study to be capable of detecting, if it existed in the population.

How to set the target ES?

1. Use expert consensus.

2. Use existing definitions (e.g., if well-accepted sources declare an ES of x.xx as being "clinically important").

3. Determine for yourself what ES would cause you or decision-makers to behave differently about the phenomenon. (For example, how much of a price difference would be required for you to change vendors for your purchase of, say, a fill-up of fuel for your vehicle?)

If none of these help, then consider:

4. Review existing literature (as you propose) to see what sorts of ES values have been reported previously.

5. Running a pilot study, to see what sort of ES is observed.

Good luck with your work.

Getasew Amogne Aynalem , Thank you for your detailed answer on the steps and their meanings! It is very clear and helpful.

Can you please recommend a textbook or a source that dives deeper in these types of equations?

Getasew Amogne Aynalem , So i want know that what power is assumed if we calculated sample size through this equation. I am ph,d, student my guide ask this question. My sample size is coming 384 using z=1.96, p=0.5 and d= 0.05. If your are able to answer then please show the calculation or reference so I can present to my research progress committee.

David Morse Cristian Ramos-Vera Husam Aldean Mahdi Abuhayyeh , dear all veteran, please feel free to give input based on your expertise.

Thanks and waiting for help.

Hello Sharad,

Please note that the formula that was provided by Getasew has to do with estimating needed sample size for purposes of estimating a population parameter (here, proportion of cases) with some desired degree of precision and at some target risk level of being entirely wrong (by not capturing the true value within the confidence interval estimate); it's not a power function determination.

If your goal is to estimate some population characteristic that is suitably expressed as a proportion (e.g., a dichotomous variable), then the approach works well for simple random sampling.

If, however, you want to test a specific hypothesis and want that test to be run with a specific a priori power level to detect an effect at least as large as some specific a priori value, then you'll need to try another approach. Here's a link that you may find helpful for this latter possibility: Article G*Power 3: A flexible statistical power analysis program for...

and the site for downloading the g*power software: https://www.psychologie.hhu.de/arbeitsgruppen/allgemeine-psychologie-und-arbeitspsychologie/gpower

Good luck with your work.

Dear Husam,

G*Power is a free statistical power analysis and sample size calculation software.I think it would be helpful for you.

https://www.psychologie.hhu.de/arbeitsgruppen/allgemeine-psychologie-und-arbeitspsychologie/gpower

https://www.psychologie.hhu.de/fileadmin/redaktion/Fakultaeten/Mathematisch-Naturwissenschaftliche_Fakultaet/Psychologie/AAP/gpower/GPowerWin_3.1.9.7.zip

And you can use the following online calculator also-

https://sample-size.net/