I am currently seeking to develop common criteria to evaluate the success of different policy initiatives to tackle the informal economy. To do this, one needs to develop a common measuring rod (e.g., the cost per job moved from the informal economy into the formal economy; the tax revenue-to-cost ratio). The first thing that struck me is whether it is valid to use such criteria to evaluate a policy when it was not perhaps its original objective. The second thing is to select common criteria that can be used. What common measure/s would you use? And does anybody know any literature that discusses the issues involved in developing common measures to compare different policy initiatives?
It is very difficult to identify a common set of indicators that measures the impact of a policy initiative. This is because a policy initiative does not have just a short term effect, and normally the long run effects are more important for local development. Also, over time, a policy initiative may be followed and influenced by other cumulative factors and it is difficult to extract the weight of its influence. Moreover, a policy initiative may have different effects depending on the “fertile soil” (legislative framework, local/regional endowment, the openness of the economy, etcetera).
In my opinion it is important to find those indicators that measure the welfare of citizens and the social progress inside the region/nation.
Most indicators to measure the impacts of policy initiatives might be somewhat generic but to be really effect they need to be specific. This can be accomplished if SPECIFIC GOALS are identified and specific metrics are developed that apply to the goals/policy. These should be measured/monitored frequently and the results communicated to those in power and those that are to benefit.
Conversely those metrics would also help to understand why the goals/policy outcome have not been reached.
Therefore, the short answer from me would be develop specific goals, objects, etc. for any policy initiative and indicators should follow.
Michael, thank you for these references. They are most useful. This is a little outside of my usual field of enquiry so any references on these matters is greatly appreciated.
Just to broaden this a little - the other element to consider here is 'who' is it successful for? I have found Bacchi's work useful in going back a step, understanding how the policy intervention is framed 'what is the problem?', then exploring who it impacts. So while an intervention may be designed to achieve a particular outcome, when we measure only for that outcome we can miss some of the unintended consequences (which can lead to policy failure).
Lisa, we are finding that 'who' a policy is successful for is very important. A policy that is successful for one set of stakeholders may not be successful for other stakeholders. And the policy measures selected tend to reflect which stakeholders vested interests are given prominence, which usually reflects the power dynamics in particular situations.
The policy space reflects a clear mix of complex interwoven processes, motivations and contexts. These have a huge analytic purchase when devising a measuring criteria or tool. For instance, why are policy formulation steps often divorced from the practical implementation phases in resource constrained or political unstable contexts even if citizens are involved in the formulation process?. Generate tools in the form of a one size fit all may be misleading. Social Policies may operate a different set of measuring tools from Public policies or even policies from the Corporate world. They have clearly different triggers and motivations. ODI did some work on Evidence based policy-making- it could be useful in thinking through your work. goodluck!
Arising out of the discussion of the above question, we have produced the attached Working Paper. I hope that readers find it useful.
Technical Report Assessing the cross-national transferability of policy measu...
- Badges
- Science topic
- Similar topics
- Linguistics
- Composition Studies
- Publication