Do we have any research that scientifically, rigorously or with some level of objectivity measure the structural logic of policies? Or is there anyway measure we use to say that this is a coherent, consistent, congruent policy with integrity (the content)? Or in another way, are there any existing measures one can use to measure the evolution of a policy's causal logic over time?
The existing measure is it also used as an indicator for the potential success of a policy?
Thank you so much
It may be easier to answer your question if the policy environment is contextualized. However, let me put two cents' worth.
Social policies, and more particularly educational policies, evolve due to many contestations of policy actors who, for example, may be parents, special interest groups, contractors, politicians, teachers, staff and so on. Many policy analysts describe the policy context and contestations as "messy". Given this scenario, there can be hardly any structural logic to policies.
Policy analysis uses a number of frameworks for analysis including the policy trajectory approach which considers the context of influences, text production, outcomes and results at three levels: micro, meso and macro. The utilization of policy networks is another approach to analysis.
With regard to your particular interest, that of a measure of integrity, coherence and so on, I have not come across any such objective measure. Policies invariably relate to human activities and measures of their success and failure are measured by the degree the policy outcomes solve the reasons for making them. Often this involves interviews and some objective measures of outcomes which are usually analyzed using qualitative analysis.
@Hassan
Thank you so m much for your insight. The motivation for the question stems from a body of literature I.e Schneider and Ingram 1990, who suggest that one must study the structural logic of the policy as it will or could lead to more successful implementation. Subsequent to that Dr Howlett et al. Suggested that the policy designs should be evaluated on their coherence, structure, congruence, goodness of fit and integrity. However, there is no objective measure to study the integrity of the policy objectively except for falling back on expert advise, intuition, reason etc. Which are all subjective
Current book on policy analysis/design makes the call that policy causal logic/structural logic must be mapped and should be adopted as part of the evaluation/analysis process. However, even with mapping (and the body of literature behind it) mapping on it's own will not be suffice because you can add conceptual components to the policy design but its relationship to the rest of the policy isn't obvious. It would seem that it's exactly those TypeIII errors in design that has a significant impact on the potential policy implementation outcomes. E.g the impact on implementation Fidelity
So, my question is if there are measures/models to more objectively evaluate the causal logic/structural logic of the policy text.
I agree with our colleague Hassan - he quotes a classic example. Strategic level policy rarely has any rigorous underlying theory of change, but is pragmatic, ideological or populist. Sometimes that may well be a good thing. "Politics is the art of the possible" (Harold Wilson). Small scale policy often does have a theory of change underlying it. In terms of large scale policy I would use some sort of discourse or content analysis to determine the underlying thinking or strands of thinking, probably not coherent. Then the outcomes of the policy would have to be compared against these. Whilst you can use quant data for part of this, in the end the synthesis would have to be done by the researchers. Bias can be avoided to some extent by taking a team approach and by seeking peer involvement and critique of the model or by adapting group techniques to seek expert opinion in fora or through simplified Delphi type exercises. The starting point though would be analysis of the relevant statements regarding the policy direction and also inductive thinking about the policy itself (what tradition does it appear to come from, is there a relevant evidence base etc). It is notoriously difficult to assess strategic policy outcomes but it is worth trying.
@Andrew McCulloch
Yes, I agree with the arguments of Hassan as they are fundamentally the bases for policy sciences.
However, I think my interest is in the advancement of the field and the recognition that the structural logic of a policy may have a big impact on the implementation outcomes. Some even suggest that in evaluation one could blame failure more to policy design (noun) than on implementation.
For me, the collaborative approach as suggested by you in concert with mapping as suggested by current research could help objectively or more rigorously analyse the content.
Additionally, for me I think another problem is the "tacit information" or policy elements that we have in policy. They are difficult to trace or rigorously evaluate, and makes policy feedback very difficult. Policy feedback and learning is important to the advancement of policy to include new information to contextualize the policy more to its environment. "Approximating it well against its environment" as suggested by John Peter.
On your starting point suggestion. Yes, that would be a good start. Having identified the data underlying the policy, and relevance (meaning)of the policy. As policy pinching often in 3rd world states tilt toward failure because of the possible mismatch between the policy and its environment. The missing link however, will be the "structural /causal logic"of the policy in Popper's tradition.
That is where my interest to the question comes from, as current scholarship hints at this but doesn't clearly provide methods to evaluate the structural logic.
Thanks,
- Badges
- Science topic
- Similar topics
- Philosophy
- Esthetics
- Interpretation