With regard to developing countries. Lack of accurate data remains the biggest challenge even if present incorporating it in the policy is unusual

As well as the applicability of the policy remains an issue

Dear Mohamed A Yassin, thanks for initiating the debate.  I agree, accuracy of data is important. I endorse your words, the problem is, incorporating data in the policy is unusual. Much appreciated.

I recently co-authored a case study that you may find helpful. We analyzed the problem definition part of the policy process that led to the 2012 cuts to health coverage for refugees and refugee claimants in Canada. While centred on how that specific policy decision was made, the article provides a more broadly-applicable framework as well as some recommendations that aim to encourage data-driven public policy making. I hope you will find some of this information useful.

http://dx.doi.org/10.1017/S0008423916000020

Dear Paul Sheridan, thanks for sharing a quite useful and relevant document. But its full version is not available. Can you share that, please?

Dear Asmat Ali,

Your question I would re-phrased in the following way: "Do policy makers use LIE during policy making process? If you agree, then have a look at the discussion "Why do politicians lie?" on the link https://www.researchgate.net/post/Why_do_politicians_lie .

Dear Gennady Fedulov,

Thanks for letting me know the link.

I like the comment of Asmat Ali.

Yes they do

The parties use the data that can strenght their policy preferences and can support their preferred solutions. For the developing countries I suggest to read the book "Confessions of an economic hitman". It's an interesting book written by a practicioner that shows how wrong predictions based on biased data are used by the main international organization in order to create deficit and public debt in the developing countries (of course all of this can go on and on  because most of the time if not always there is a lot of corruption in it).

Dear Asmat, Actually our experience from Pakistan in health sector is that they dont. And its not for the lack of availability of data. We explored the issue using Public Choice Theory. Our findings from around 150 interviews of policy makers are: 

1. They dont use data. And its not because they dont have data or access to it. Even when data are available, they dont use it or trust it unless it agrees with their personal belief systems.

2. Its because they dont have to since the purpose of policy is seldom to enhance public good.

3. The key aim is enhance personal benefits and since there is no public or beneficiary feedback, the issue of public good takes a back seat.

4. We also found that there is seldom an awareness of an institutional mandate, of incentives others in the department at organisational level or competetion or cooperation between policy makers (game theory)

Hope this helps

Dear Arup Barman & Oluwafemi Samson Balogun, thanks for the compliments.

Dear Giorgio Giraudi, That is true i.e. only specific part of data that supports and strengthens certain policy objectives is utilized but it is misuse of data.  Thanks for suggesting a good book. Much appreciated.

Dear Adnan, thanks for superb contribution.  You benefited a very good framework, indeed. Would it be possible for you to share some more details of these interviews, please?

Being Pakistani ,my conclusion about the use of data in policy making is:-

Policy makers use data only if there are certain personal benefits embedded  for them regardless of data availability or accessibility.

Hi Munir,

This is not any exception in South Asian or in Asian Nations as I observed. In African also same. Data gap or the cut throat research are rarely use in developmental planning. People believe on profit for self. Let data remain somewhere and development go somewhere.

Well Said  Dear Arup Barman

"data remain somewhere and development go somewhere"

Dear Asmat. Please find a brief based on the study. We are writing up the results for publication.

https://www.researchgate.net/profile/Adnan_Khan21/publication/265883682_MAKING_DECISIONS_USING_INFORMATION_ABOUT_POPULATION_AND_HEALTH_IN_PAKISTAN/links/56e0ed8308aec4b3333d1a25.pdf

Technical Report MAKING DECISIONS, USING INFORMATION ABOUT POPULATION AND HEA...

The question is not DO they use but HOW they use.  Path breaking research some years ago by Judith Innes (Berkeley) about how different decision makers interpret ostensibly objective information (GIS map layers) in urban planning decisions. 

The Philippines recently passed a law creating the Philippine National Health Research System (PNHRS). One of the major objectives of the law was to ensure that policies are research based.  PNHRS has a committee that creates a National Unified Health Research Agenda - collating and prioritizing issues and concerns that affect health from different sources.  However there is nothing to compel policy makers to propose their contemplated policies nor for them to use studies that will help their policies.  The help of media in bringing out the issue may be needed.

Dear Adnan,

Thanks for the kind reply and the publication link. Certainly detailed results will be more appreciated when you finish with write up.

Dear Rachelle Alterman,

Thanks for contributing, mentioning a very valuable and greatly desired publication as well as raising a practical question. I agree to your point, How they use data is quite important but first it needs to be established that data is really used or not by the policy/decision makers of Pakistan for example. Adnan Khan has addressed it but my focus, as you have rightly triggered is geospatial/GIS. Therefore, your highlighted question is of course the next and complementing question to the question asked by me.

 Dear Juan M P Nañagas,

Thanks for sharing information from Philippines. You are right, the role of media is quite important.  

From the UK perspective, the use of data in the policy making process is perhaps better than certain nations. This is partly because of the conducive political model that is receptive towards new ideas and not using research data selectively as ammunition (see Weiss, 1979). At the same time, we are going through fiscal austerity where the public funding is shrinking, and such a situation demands better use of resources and transparency in spending public money. One of the ways to achieve this standard is through the use of research data to inform policy making process.

This was seen through my experience where some time ago, I produced a paper on the prevalence of drug use among Black and Minority Ethnic communities in Bristol, United Kingdom: https://www.researchgate.net/publication/295259903_Prevalence_of_Drug_Use_Among_Black_and_Minority_Ethnic_BME_Communities_in_Bristol. The paper highlights the factors influencing drug use among these communities, and the barriers preventing them from seeking treatment. I am pleased to report that the local government has been using the report to engage with these communities, and the non-governmental organisations in England and Ireland have been using this paper to persuade the government to improve the access to treatment for these communities.

Technical Report Prevalence of Drug Use Among Black and Minority Ethnic (BME)...

Good polies are based on the existing experiences which are envidenced by data so countries like Germany(FRG) as at 1950-todate among few others have over fifty years of policy decisions based on data, Their socioeconomic and macroeconomic performance is just a reflection.

Yes, policy decision makers do use data to inform their policy decisions. In the Australian context, the use of data to inform policy decision making has increased over the last ten 10 years. Contingent on the jurisdictional regulation and department, policy makers in Australia can obtain free data from the Australian Bureau of Statistics. These are usually complex mega-datasets which can be used for community service delivery policy making, such as aged care etc. However, internal teams may have their own data collection strategies and own their own datasets. Globally, data can be downloaded from publically available datasets.

 Dear Nasrul Ismail,

Thanks for sharing UK perspective and your very good publication. Glade to know that the publication was benefited for policy/decision making. Was your paper cited/referred by local government in some policy document or in written form that can be cited as an example from UK?

Dear Dastan Bamwesigye,

Good to learn from you that Germany is also benefiting data for decision/policy making. Is there any document that can cited to prove it? If so would you please that.

 Dear Lindy Lou West,

Thanks for contributing and describing experience from Australia.  It is also to note that for decision making data is obtained from Bureau of Statistics that is free of charge. Can you please share any document or official report, policy document and/or research article so that can be cited? It would be great favor if you can do.

Not all data from the ABS is free. However, data in this link below, is public and therefore freely available. If you go to the link, you could download the data from spreadsheets. Should you require specific data, the ABS has a great helpline and you may need to pay for the specific data you require.

Other countries, Sweden, Denmark, US too has free data on some websites.

http://www.abs.gov.au/AUSSTATS/[email protected]/DetailsPage/6401.0Dec%202015?OpenDocument

Dear Lindy Lou West ,

Thanks for actively participating. Much appreciated.

Asmat, I notice that your topic is on public organisations.

You may be interested to know that we are in the process of working on an international collaborative project and have sent out calls for chapters. The response so far is very good.  The topic is on emerging business models and managerial strategies in not for profit organisations. The new closing date is 30 March.

Dear Lindy Lou,

Please send link of call for chapter.

Related to the agricultural sector in the Americas my experience is that those countries involved in a process of results-oriented policies and budgeting are more data-intense users, not only in the formulation process but only for monitoring ande evaluation. Some examples are Jamaica (multisectorial), Canada, Chile and Peru. 

Dear Joaquin Arias,

Thanks for contributing with some good examples.

Let me tell in other interpretation. Data are used by bureaucrats, secretaries used by them without much thinking and parroted by a politician in decision making  with a false promise by politician in decision making. Thus, data remain as unused, if some used in appropriate sense of the term they are considered as academic theoretical use, not practical.

I fully agree with Arum Barman. I have been involved in many studies on the industrialisation of my country (World Bank, UNDP, ADB etc funded) but the recommendations were seldom used in decision making. 

In case of Pakistan research and data are barely used in policy formulation process. The area of political preference is selected first,  feasibility, benefit cost analysis follow  to fit the political preference.

The problem of social science research utilization in public policies arises basically from the fact that policy makers in Sub-Saharan Africa, in particular, have not endorsed the use of research output in policy making. Hence they rarely sponsor appropriate research inputs. Besides the lack of support to research, policy makers rarely buy into appropriate research outputs sponsored by other agencies because they don't make efforts to access them. Finally, they are usually accountable to political decisions which eventually determine the direction of public policies.

Dear Arup Barman,

Excellent comments. Agree with you fully. Perhaps another reason for not using data can be lack of training in benefiting data for policy making?

Dear Alfred Inkiriwang,

Thanks for joining the discussion. I appreciate that you shared your experience. That’s really important.

Mam Seemi Waheed,

Well said. Thanks for speaking of ground realities. I am glad to see that  we share a  common sphere of interest i.e. Governance. Hope I will learn more from you.

Dear Theophilus Fadayomi,

Thanks for describing experience from your region. You said well, politicians have not endorsed the use of research output in policy making. They don't make efforts to access them. Unfortunately, it is almost the same case in many developing countries. Don’t you think the reason may be cultural background, as well?

There are numbers of causes for not using data in policy planning

In Indonesia, in a formal planning process, such as regular midterm (5 year) development planning process, collecting data is one of formal technocratic procedure.But there there are numbers of causes for not using reliable data in planning

- data is not available

- Planners are unaware about the availability of data

- Data is not accessible due to lack of communication between institutions or due to distrust

Another issue: many planners have no capacity to use data in an appropriate ways. 

Any policy maker that fails to use relevant data to plan is destined to fail. Every  govt. and non-govt. bodies require data

In response to the comments on my contribution, I don't think that the non-use of relevant data in policy making is cultural, it is still a reflection of the misconception of policy making; in the sense that it (i.e. policy making)is not considered to be a tool of development planning but that of political agenda.

Dear Munir Ahmad,

Thanks for contributing and adding valuable comments.

Dear Ernan Rustiadi,

Welcome to the discussion. Good to learn about Indonesia where data is collected but rarely used. I agree that many planners may not have capacity to use data in an appropriate ways.

Dear Okechukwu Eme,

Thanks for joining. Well said, every  govt. and non-govt. bodies require data.

Dear Theophilus Fadayomi,

I appreciate your generous contribution. Policy (making) is not considered to be a tool of development planning but that of political agenda, agree with you.

Dear Asmat Ali,

I think absence of evidence based policy has remained the dominant challenge in many countries. Particularly, in developing countries, you can find dozens of policy makers interested in counting with a portfolio of policy proposals or initiatives when making decisions. But, their own knowledge about a certain topic is not only always enough but poor. For worst, they can hardly turn to organizations and individuals specialized in different public policy issues, nor have a team that can help them with that. On the contrary, they often begin and end with a personal or political interest. What an incumbent political party can promote is the sole criteria for policy making, not really what is public!

Dear Mebratu Kelecha,

Agree with you. But what can be the possible reasons for not using data as an evidence in policy making?

In my opinion the researchers and the report writers are also to blame for this phenomena.  In many cases the policy makers are political appointees. The research data presented are usually not in a readers friendly format. Graphs, bar charts, statistics and elaborations need "translations" to know what they present or mean. Reports are usually 100 pages booklets and seldom have a good 2 page summary. Politicians have not the knowledge, academic staff, or the time to digest the research data. So policy makers will take the recognisable words related to  issues that are current from the research data and package it with a  political consensus rhetoric and will give that to his/her subordinates to come up with a policy. By the time this policy  become part of a government regulation or a presidential decree or a ministerial regulation, the research data evidence is lost in the "translation." That is why in the US, lobbyist, and special interest groups in Washington make a good living. These groups use research data to their full extent for advancing a particular favourable and profitable cause. Maybe data producing researchers have to keep the above process in mind.

Dear Alfred Inkiriwang,

Excellent contribution. You have highlighted so many good dimensions of the root causes. Perfect. Much appreciated.  

Presumably in the decision-making should have a basis for accountability and is intended to limit the possibility of deviation or minimized the risks in order to achieve maximum results.

Basis in decision making can be data and/or based on facts that can be substantiated, but they all must meet the relevant criteria in deciding step and the right purpose.

Dear Teguh Yuniarto ,

Thanks for contributing with quite good arguments that make sense. 

Dear Alfred Inkiriwang

Interesting perspective. Here is an article that is around 60 y old but still relevant, particularly for the point you raise about access to info and lobbyists

https://msuweb.montclair.edu/~lebelp/DownsEcThDemocJPE1957.pdf

Dear Adnan Khan, I read the article written by Anthony Downs and fully agree it is still very, very relevant in politics. You are right, his arguments on information and lobbyists is enlightening. Indonesia is a country with a multiple party democracy  and what is described in Downs article is exactly the current political situation in Indonesia. I am presently teaching Political Media Literacy as part of a course on Politics and Media in the USA  for graduate students and certainly will make this article  a reading assignment. Thank you very much for the link you provided.  

Dear Adnan Khan,

Excellent. Thanks for sharing a valuable paper on this forum.

Dear Alfred Inkiriwang.

Thanks for continuously contributing.

In my opinion------

Policy makers must use data for making good policies-

Because- Policy originates in the following way--

Origin of a thought leads to philosophical concept (a systematic school of thought)

This concept makes a tool called 'policy' which is the axis for complete system based on philosophy including anticipated results.

So without collected 'data' philosophical concept can not be formed and substantiated.

A substantiated philosophical concept only can give strong axis for the system(Policy)

Implementation of policy and exercising rendered data will run the system more affectively

Dear Siva Prasad Kodukula,

Your contribution is much appreciated especially the notion of axis. 

 Policy process  referred  to  as  operationalization  which  takes  a  non-empirical  concept and gives it some empirical representation.  The  concept  of    policy  has  an  intuitive, common sense  logic  which   partly  explains  how  it  has  become  naturalized  in  a diverse  range  of  policy settings.  Knowledge  of  the  policy  process  is  itself  an  important  resource,  which  raises  the  issue  of  knowledge management  and  the capacity  of  policy-makers  to  analyze  research  results  and  to  distinguish between  good  and  dubious  research  findings.  policy makers working in both the community and government sectors are increasingly using the language of policy but  not analyzing data itself.

Dear Kiran Grover,

I appreciate your generous feedback. Certainly,  Knowledge of the policy making process is itself important. Policy makers working in both the community and government sectors are increasingly using the language of policy but not analyzing data itself, agreed.

Asmat,

You might also want to consider the functional definitions of the term 'data'.  I believe that political decision makers are using 'data' in their decision making.  However, their functional definition may include considerations and factors not included in other definitions.  

Our respected colleague Adnan Khan's research points out that 'data' that does not accord with belief systems is not considered.  

However, aren't belief systems central to decision making?  

Einstein is reported to have rejected quantum theory because 'God doesn't play dice', making the exclusion of belief systems untenable when considering the decision making process and its relationship to 'data'.  

Respected William P Kittredge,

Thanks for adding and contributing more to this forum.  Well said, and I agree too the semantics may be different. Belief system although is not science but plays central role in making decisions in certain cultures like Pakistan. Can you explain more functional definition of Data.

Regards

asmat

Dear Respected Colleague Asmat,

As it happens, I have several Pakistani colleagues and former graduate students with whom I've discussed the situation there.  Two years ago, I made a 10 day trip to Karachi and had many meetings with professional associations, including the Institute of Chartered Accountants of Pakistan, foundations, including the Aman Foundation, and several universities.  Since my visit, I've done several webinars, including a recent discussion of tax incidence with ICAP members.

Dear Respected Colleague Asmat,

First, I accept that belief systems play a central decision-making role in all cultures. Belief systems differ but their central role is, in my experience, universal. I believe that this is simply a human characteristic. 

The initial question involved the degree to which 'data' is used in policy-making. To ensure that we have a common understanding, and at the risk of boring you, let me define how I am using the term 'data' and some of the related terms with respect to policy making.

Data are raw inputs. Data may be quantitative or qualitative. Analysis and interpretation of data produces information.

Information is employed to formulate, implement, and evaluate policy.

Normative considerations determine what data are relevant. Only relevant data are included for analytical purposes. Normative perspectives also determine the relative importance of a given datum.

Consider our learned colleague's interpretation of his findings. His assertion that 'data' is not being used and, as a result, the public good is not being optimized, depends entirely on his definition of 'data', assuming we can agree on what constitutes 'the public good'. 

Consider this hypothetical, highly simplified, example:

An individual, call him VA, with policy-making authority within a given governmental system, wishes to initiate a vaccination program for children under conditions of limited resources and competing budget priorities common to all public policy decision-making. 

Advocates for many different policy choices, using various definitions of the public good, all assert that their policy should receive priority. Some of those advocates are VA's peers, that is they participate in, and their support is required for, policy formulation and implementation, and their perspectives will influence policy evaluation.

VA's policy-making data includes some easily quantifiable data, such as the budget constraint and recent child mortality statistics. Even apparently unambiguously 'factual' data such as these may be subject to interpretation. For the sake of simplicity, I'm going to assume universal agreement.

However, other data must be considered. The relative importance of this initiative within VA's broader agenda, VA's relative importance and influence among his peers, and the internal and external incentives VA and his peers face are all important qualitative data, although I do not pretend this is an exhaustive list. These qualitative factors, generally excluded by social science scholars, are what I have called practical data.

The exclusion of practical data from the analysis compromises the validity of the findings, in my opinion. I believe that this conclusion can be extended to the vast majority, if not almost all, of the social science research related to policy, public administration and the allied fields. 

NOTE: Government form makes little difference. The distinction between democratically elected policymakers, those who have risen through an oligarchy (e.g. China), and a dictator, such as the apparently hereditary ruler of North Korea, lies primarily in the policy-maker selection process, which precedes the policy-making, implementation and evaluation process discussed above.

Yes, I think that policy makers control laws, including serves and by what benefits them.

Asmat -

Interesting thread. 

In my experience, mostly at the US Energy Information Administration, but elsewhere, and informed by the legendary exploits of W. Edwards Deming, and the offhand behavior of many politicians, and other information over many years, I think it accurate to say that very often decision-makers, at all levels, tend to disregard data, because they do not trust what they do not understand.  This is in agreement with my theory that the skills and motivation needed to obtain a position of leadership are often not relevant or even incompatible with the skills and motivation needed to perform the duties of such a position. The desire to even become knowledgeable in an area before making a basically arbitrary decision, or a self-serving one, is often simply not present.

The use of data is particularly problematic, because so many people - perhaps especially such 'leaders' - are easily confused by data and therefore find it of no value ... when they aren't misusing it. 

Sad but very often true. 

And as I think I saw it mentioned in some postings above, statisticians often do not communicate practical concepts well, and often do not even understand what is practical.  We too can fail to do what is necessary so that even an interested decision-maker can do what is necessary to perform well. 

Best wishes - Jim

How decision makers use information ('data") has been extensively conceptualized and researched, beginning with John Forester's Planning in the Face of Power. Journal of the American Planning Association, Volume 48, 1982 - Issue 1 

Pages 67-80 | Published online: 26 Nov 2007.  He also published a book with the same title . He and several subsequent scholars have demonstrated, time after time, how decision maker use (often ignore or interpret) information in ways that serve their objectives.  the use of data is often far from what the creators of the data set had expected under "rational" thinking.  See also Judith Innes' work on related topics.   

It is not surprising that a "...decision maker [will] use (often ignore or interpret) information in ways that serve their objectives," but how often are they even competent to understand data and data analyses, even if they wanted to do so?  I expect that differs somewhat by level of organization, type of organization, political climate (history, geography), application ... and for those at some levels, yes, intent ... but often interesting to my experience was lack of practical competence to understand data and statistical inference, even within the agency generating the data.  I also have substantial experience providing reason to believe that this is a problem for many statistical agencies providing official statistics, as well as a problem for the user communities.  Further, when communications 'experts' tell statisticians to 'dumb down' statistics to the level of a twelve-year-old child, this, to me, means such users should not be making any conclusions from it. 

Has there been a good study on statistical competence of decision-makers?  

In 2012 the Philippine National HealthRresearch System was made into law and one of its major objectives was to influence policy through research.  Most policy makers (as many have said in this forum) do not usually look at the research and that it may be up to researchers to make it easy for them to do so including simplifying the language.  Generally researchers should have several version - one for media, one for co-researchers, one for policy makers, maybe even one for regulatory agencies.  It is also important to show policy makers (a.k.a. politicians) what the benefits will be in terms of increasing votes or approval from the community.

Juan -

I generally agree that you have a good approach except that I have to assume that your last sentence meant the benefit of using accurate information, not in only presenting something in support of a previously stated position.  For example, the government statistical agency where I worked on official energy statistics was independent of the rest of the Department of Energy, and produced information without political bias for others, in and out of the government to use.  Other organizations used or misused various parts of what we produced, but the idea was to provide it all without prejudice.  - Jim

Yes James - the version of a study for researchers may not be as useful to the media but the information is not changed and of course everyone can (and they sometimes do) use it for their own purposes.

Dear Dr. Asmat,

Despite the huge quantity of readily available data, policy makers in the developing countries are intentionally ignoring it unless it fits their personal interests. It is really a very disappointing behavior which represents one of the aspects of corruption. There are many studies addressed this issue and you can check some at the following links.

Best regards

SM Najim

http://www.bard.edu/civicengagement/usfp/docs/gookins.pdf

http://www.scielo.br/pdf/csp/v22s0/02.pdf

http://www.tandfonline.com/doi/full/10.1080/10919392.2015.1125187

Dear @SM Najim, Thanks for contributing and sharing quite useful documents. Unfortunately, it is going to be proven that policy makers in the developing countries are intentionally ignoring it unless it fits their personal interests. I agree with you.

Best Regards

asmat

Policy actors often find it hard to understand statistical terms unless it is translated to simple lay man's language. The situation is even dire in low and middle income countries where some elected official do not have more than primary r secondary school education. Furthermore, corruption and self-interest dwarf any chance of public good in policy formulation and the public do not trust the politicians hence no feedback to the community who the policy is designed to affect.

Dear@ Segun Emmanuel Ibitoye, thanks for contributing.

Sometimes researchers do not make it clear what policy makers ought to do with the findings. Findings often do not speak directly to the interventions that are required or evaluate the best interventions. We also need to take policy makers and other users of research through the journey of our research and in a way educate what we are doing.

@ Melinda Potgieter, thanks for sharing your views.

Dear Ali ji,

You have raised a very pertinent question. As far as my experience of contributing in Policy formulation and implementation goes, I would like to share following-

1) Policy makers do get access to plenty of data. However, they prefer to go by their own opinions, direction of political bosses or experts’ opinions/ reports/ recommendations.

2) They seldom do statistical analysis other than primitive graphical interpretations, max - min value and average, etc.

3) Relevant research (from the perspective of policy practitioners) by academics can play an important role in better policy formulation and implementation.

4) Relevant academic works, particularly those based on Qualitative approach and/or mixed methods can become a game changer and help in improving efficacy of policies.

Deleted research item The research item mentioned here has been deleted