Assume you take one reading of 100 observations, i.e. n = 100. These observations are the readings of subjects given a stimulus (treatment), but the reading is one time. There is no control group. The condition may reflect real life situation---in many cases in real life, this is what we face.

Is the treatment effective?

EXCEEDING THRESHOLD: Assume further that we are using 0.95 confidence interval where the critical Z(0.95) = 1.65. See attached table.

(1)   Z = (Xi - X^) / S

... where Xi = individual observation; X^ = observed mean; and S = sample standard deviation. H0: No effective treatment Z(obs) < 1.65 and HA: Effective treatment Z(obs) > 1.65. 

TYPE 2 ERROR: Run a second test to double if we made a type II error. Find the value for ZB. Since we have no group comparison and no control group, we may select the estimated mean (μ) as the reference as the threshold to derived Cohen's effect size d (This is by analogy because we do not have two groups). From this logic, then solve for ZB. Recall that:

(2)   (Za - ZB) = d (sqrt(n))

Solve for ZB, we have:

(3)   ZB = d sqrt(n) - Za

... where Za = 0.05 because using 0.95 CI. With ZB value, look in the Z-table for % probability of Type II error. If within acceptable range, then (1) above may be usable.

Everything depends on the exact type of data and the shape of the distribution.

The Propensity score matching methods are the most commonly used: match treated and untreated observations on the estimated probability of being treated (propensity score).

Hi, I think you need to look at the attached book. You will find such methodologies clearly presented.

Cheers

Audil,

The book that Zerihun posted has some nice information on impact evaluation and methods. However, there is no credible way in which to measure causal impacts in the scenario that you have given. Without baseline information on poverty (and the process through which your units are selected for the poverty alleviation program...note that this can be relaxed under randomized program rollout) and a comparison group that was not exposed to the program, there is no way for you to infer whether the levels of poverty that you observe are do to the program or something unrelated. All of the methods in the book that Zerihun posted require at least a comparison group (and most are only defensible with measurements both pre- and post-treatment). Fundamentally, in order to measure an impact, one must calculate a difference, whether that be a within unit comparison (before and after) or between group comparison (or even better, both). In your scenario, there is no difference to be taken.

Good luck,

Merlin

Thanks Merlin, Paul, and everyone else.

Paul, can you please suggest a research paper based on similar methodology.

Khaki Audil