Policy Concepts in 1000 Words: ‘Evidence Based Policymaking’

(extra long podcast download, plus lecture/Q&A from UC Denver)

See also: ‘Evidence-based Policymaking’ and the Study of Public Policy

The term ‘Evidence Based Policymaking’ is in common currency in media and social media. It often represents an ideal which governments fail to reach. A common allegation is that policymakers ignore and/ or do not understand or act on the correct evidence. However,  if you look at policy studies, you tend to find highly critical discussions of the concept, and the suggestion that people are naïve if they think that EBPM is even a possibility. Some of this is simply to do with a lack of clarity about what EBPM means. Some of it is about the claim in policy studies that people don’t understand the policy process when they make EBPM claims. We can break this down into 2 common arguments in policy studies:

1. EBPM is an ideal-type, only useful to describe what does not and cannot happen

EBPM should be treated in the same way as the ideal-type ‘comprehensively rational policymaker’.  By identifying the limits to comprehensive rationality, we explore the implications of ‘bounded rationality’. For example, by stating that policymakers do not have the ability to gather and analyse all information, we identify the heuristics and short cuts they use to gather what they can. This may reveal their biases towards certain sources of information – which may be more important than the nature of the evidence itself. By stating that they can only pay attention to a tiny fraction of the issues for which they are responsible, we identify which issues they put to the top of the agenda and which they ignore. Again, there is a lot more to this process than the nature of the evidence – it is about how problems are ‘framed’ by their advocates and how they are understood by the policymakers held responsible for solving them.

2. Scientists use evidence to highlight policy problems, but not to promote policy change

The policy literature contains theories and studies which use the science of policymaking to explain how policymaking works. For example, ‘punctuated equilibrium’ studies use bounded rationality to identify long periods of policymaking stability and policy
continuity punctuated by profoundly important bursts of instability and change. In some cases, policymakers ignore some evidence for years, then, very quickly, pay disproportionate attention to the same evidence. This may follow the replacement of some policymakers by others (for example, after elections) or a ‘focusing event’ which prompts them to shift their attention from elsewhere. Further, studies of policy diffusion use bounded rationality to identify emulation in the absence of learning; the importation of a policy by a government which may not know much about why it was successful somewhere else. In
such cases, a policy may be introduced as much because of its reputation as the evidence of its transferable success. In other studies, such as the ‘advocacy coalition framework’, we identify a battle of ideas, in which different groups seek to gather and interpret evidence in very different ways. EBPM is about the dominant interpretation of the world, its major events and the consequences of
policy so far.

In each case, the first overall point is that policymakers have to make important decisions in the face of uncertainty (a lack of information), ambiguity (uncertainty about how to understand a problem and its solution) and conflict (regarding how to interpret information and draw conclusions). They do so by drawing on policymaking short cuts, such as by using information from sources they trust, and by adapting that information to the beliefs they already hold. The second point is that, even in ‘Westminster’ systems, there are many policymakers involved. We may begin with the simple identification of a single, comprehensively rational policymaker at the heart of the process, but end by identifying a complicated picture in which many actors – in many levels or types of government – influence how evidence is portrayed and policy is made.

In this context, a simple appeal for the government to do something with ‘the evidence’ may seem naïve. Such an appeal to the evidence-base relating to a particular policy problem is incomplete without a prior appeal to the evidence-base on the policy process. Instead of bemoaning the lack of EBPM, we need a better understanding of bounded-EBPM to inform the way we conceptualize the relationship between information and policy. This is just as important to the scientist seeking to influence policymaking as it is to the scientist of policymaking. The former should identify how the policy process works and seek to influence it on that basis – not according to how we would like it to be. To understand only one aspect of EBPM is to reject EBPM.

See also:

This post is one of many on EBPM. The full list is here: https://paulcairney.wordpress.com/ebpm/

A ‘decisive shift to prevention’: how do we turn an idea into evidence based policy?

Weible et al on how to use policy theory to guide groups seeking to influence policymaking