Syed Muhammad Imran Haider Zaidi sb 

Thanks for the contribution and valuable comments. Your suggestions are quite helpful.

Dear Munir, these are really relevant questions. Comparisons of two similar or different policies could be done using policy evaluation tools. And Policy evaluation tools use a range of research methods to systematically investigate the effectiveness of policy interventions, implementation and processes, and to determine their merit, worth, or value in terms of improving the social and economic conditions of different stakeholders. There are a range of policy evaluation tools that help to make a comparison of policy development regulatory analysis, comprehensive impact evaluation, and performance evaluation.

Dear Mebratu Kelecha

Thanks for the reply and opening new dimensions of policy evaluation tools to compare policies.

Plz suggest good policy evaluation tools available in the market to explore the strengths of such tools.

In my experience, comparative studies require consideration of legal, cultural, and in the case of IT technical differences.  If you are considering a quantitative approach, appropriate control and/or interaction variables need to identified.  If your comparison is qualitative, comparators (see my article on the Oregon Shines process) are needed. If coding is important, then intercoder reliability testing is crucial.  The most robust comparisons, such as a project I worked on 1997-1999, require a fuzzy logic model that allows the inclusion of both quantitative and qualitative elements in the same model.

Comparisons of two different policies within the same country is more problematic, in my view.  Policies may be related, as you suggest in your example  (i.e IT policy and Right to Information policy).  However, the social goals, intentions of the policy-makers, and numerous other factors intrude, not infrequently resulting in computational intractability. For instance, Right to Information policy perspective, (e.g. US Freedom of Information Act) may have significantly different definitions of key terms and divergent views on central concerns (e.g. privacy and IT security perspectives).  

If you refine your research question(s), I might be able to comment more specifically.  Good luck.

Dear Munir Ahmed,

You can choose one of the three broad types of policy evaluation designs, randomized experiments, quasi-experiments and non-experiments. The choice of one particular design is basically based on the objective of evaluation (in your case comparison): what do you really want to compare? This is to mean it is the goals of evaluation that regulate the parameters of comparison/evaluation.  For example, in the i) Pre-Test/Post-Test designs, you can compare the intervention before and after the intervention, ii) Time-series designs, you can look at changes over time to determine trends, iii) The longitudinal study is another type of time-series design, and iv) In a post-test only design, evaporators observe the intervention at one point in time after the intervention, etc. I hope this help!

Dear William P Kittredge

Thanks for excellent explanation for quantitative and qualitative approaches. Comments on the comparisons of two policies are very helping  and interesting.

Actually my intentions of comparing similar or different policies are :

Which rules are contradicting in the policies?

Which rules are supporting in the policies?

Dear Mebratu Kelecha

Thanks for contribution and very valuable comments about types of policy evaluation tools and design.

Agree to all contributors.

@William P Kittredge, For example if we want to compare IT policy of two developing countries say Pakistan & India. Then what kind of analytical tools/parameters need to be adopted for the comparison?

@Asmat, First, 

First, let me point out that IT policy is not an area in which I can claim expertise. Therefore, I will be making general comments based on my broader policy analysis experience.

I suspect that IT policy is analogous to many financial policy areas, in that quantitative and qualitative factors interact. For instance, a balanced budget policy may exist but it may be regularly circumvented by 'off budget' expenditures.

In such environments, mixed mode estimations from structural equation models may be informative, assuming that appropriate data exists and is comparable.    

My OECD experience drove home the importance of comparability. My work in the Territorial Development Policy Committee (TDPC) highlighted the difficulties one encounters, even within countries. I remember all too well one extremely arduous project during which we were confounded by the differences between northern and southern Italy. In the north, the data we needed was readily available and had high validity. In the south, if it was available at all, it was incomplete and its provenance uncertain. This challenge alone made it difficult to come up with a suitable set of indicators for Italy as a whole and that was before we even tried to find comparability within the EU.  Our colleague @Munir has a challenging research agenda ahead of him.

Appropriate control variables that account for IT environmental differences between the two countries are critical. Environmental variables might account for the impact of different legal systems on IT policy or technical differences between the IT systems, to use two examples.

I also suspect that some variable interactions will have to be considered. My lack of expertise in IT policy limits me to this general comment. 

@William P Kittredge,

Thanks for addressing my question exclusively. I find good example cited by you. I remember my research work in Roma (2007), I encountered the bit similar situation as you mentioned.

I am of the opinion that Dear Munir has to develop analytical framework to do certain comparison. He can argue the reasons for adopting that specific framework. OECD as referred by you can also be a useful source. OECD evaluate various kinds of policy such as agriculture, education and health etc in its member countries including India. This can be a good starting point for him, I guess?

@Asmat Ali, Always a pleasure to work with a colleague, makes us all better.  OECD can be a good starting point, World Bank and others also do interesting work.  I served at OECD for 3 years and have some experience with the UN, USAID, AUSAID, WB and a few others.  The primary critique I offer it the western normative bias, which I believe influences analytical methods through, among other things, assumptions about data availability, and normative perspectives, the lens through which, for better or worse, much social science is conducted.  Bottom line: useful and in need of adaptation.

@William P Kittredge ,

Indeed it is a matter of pleasure for me as well to connect to you. Hope we can have joint venture in some area. Regards

Dear Munir,

Pls see this from @William P. Kittredge. It can be interesting.

http://ec.europa.eu/regional_policy/archive/conferences/evaluation2009/pres/ws7_kittredge.ppt

@William P Kittredge,

Thanks for sharing your  interesting experience at OECD and guiding me towards the research agenda.

It is my pleasure that I have received your experienced attentions.

@Asmat Ali

First of all thanks for joining hot debate of policies comparison, contributing the discussion to align the research agenda as well as research methodology and guiding me to achieve the  research intentions.

Thanks once again for sharing @William P. Kittredge document.

I will contribute my findings after reading the document.

We recommend Integrative Propositional Analysis (IPA) developed by Steven E. Wallis, Ph.D. It has an objective logical construct which inputs verifiable data and provides numerical output. If applied to two or more policies, the one that will achieve its objective will stand out from those whose results will be unanticipated, undesirable, and chaotic.

Dr. Wallis has created a game-like table-top version to that can be played by individuals or teams, inputting variables and parameters to generate the optimal policy. It is called ASK MATT and is based upon Strategic Knowledge Mapping.

Contact Dr. Wallis through www.meaningfulevidence.com or me at www.scipolicy.org.

Respected Franklin David Nash, thanks for quite informative reply and links. Certainly, will contact you, soon.

Sir Franklin David Nash thanks for prompted reply and surely your guidelines will help us in the analysis of policies..

Through the programs that they plan to implement, and by the method that they walk on.