The Politics of Evidence

This is a draft of my review of Justin Parkhurst (2017) The Politics of Evidence (Routledge, Open Access)

Justin Parkhurst’s aim is to identify key principles to take forward the ‘good governance of evidence’. The good governance of scientific evidence in policy and policymaking requires us to address two fundamentally important ‘biases’:

1. Technical bias. Some organisations produce bad evidence, some parts of government cherry-pick, manipulate, or ignore evidence, and some politicians misinterpret the implications of evidence when calculating risk. Sometimes, these things are done deliberately for political gain. Sometimes they are caused by cognitive biases which cause us to interpret evidence in problematic ways. For example, you can seek evidence that confirms your position, and/ or only believe the evidence that confirms it.
2. Issue bias. Some evidence advocates use the mantra of ‘evidence based policy’ to depoliticise issues or downplay the need to resolve conflicts over values. They also focus on the problems most conducive to study via their most respected methods such as randomised control trials (RCTs). Methodological rigour trumps policy relevance and simple experiments trump the exploration of complex solutions. So, we lose sight of the unintended consequences of producing the ‘best’ evidence to address a small number of problems, and making choices about the allocation of research resources and attention. Again, this can be deliberate or caused by cognitive biases, such as to seek simpler and more answerable questions than complex questions with no obvious answer.

To address both problems, Parkhurst seeks pragmatic ways to identify principles to decide what counts as ‘good evidence to inform policy’ and ‘what constitutes the good use of evidence within a policy process’:

‘it is necessary to consider how to establish evidence advisory systems that promote the good governance of evidence – working to ensure that rigorous, sys­tematic and technically valid pieces of evidence are used within decision-making processes that are inclusive of, representative of and accountable to the multiple social interests of the population served’ (p8).

Parkhurst identifies some ways in which to bring evidence and policy closer together. First, to produce evidence more appropriate for, or relevant to, policymaking (‘good evidence for policy’):

1. Relate evidence more closely to policy goals.
2. Modify research approaches and methods to answer policy relevant questions.
3. Ensure that the evidence relates to the local or relevant context.

Second, to produce the ‘good use of evidence’, combine three forms of ‘legitimacy’:

1. Input, to ensure democratic representative bodies have the final say.
2. Throughput, to ensure widespread deliberation.
3. Output, to ensure proper consideration the use of the most systematic, unbiased and rigorously produced scientific evidence relevant to the problem.

In the final chapter, Parkhurst suggests that these aims can be pursued in many ways depending on how governments want to design evidence advisory systems, but that it’s worth drawing on the examples of good practice he identifies. Parkhurst also explores the role for Academies of science, or initiatives such as the Cochrane Collaboration, to provide independent advice. He then outlines the good governance of evidence built on key principles: appropriate evidence, accountability in evidence use, transparency, and contestability (to ensure sufficient debate).

The overall result is a book full of interesting discussion and very sensible, general advice for people new to the topic of evidence and policy. This is no mean feat: most readers will seek a clearly explained and articulate account of the subject, and they get it here.

For me, the most interesting thing about Parkhurst’s book is the untold story, or often-implicit reasoning behind the way in which it is framed. We can infer that it is not a study aimed primarily at a political science or social science audience, because most of that audience would take its starting point for granted: the use of evidence is political, and politics involves values. Yet, Parkhurst feels the need to remind the reader of this point, in specific (“it is worth noting that the US presidency is a decidedly political role”, p43) and general circumstances (‘the nature of policymaking is inherently political’, p65). Throughout, the audience appears to be academics who begin with a desire for ‘evidence based policy’ without fully thinking through the implications, either about the lack of a magic bullet of evidence to solve a policy problem, how we might maintain a political system conducive to democratic principles and good evidence use, how we might design a system to reduce key ‘barriers’ between the supply of evidence by scientists and its demand by policymakers, and why few such designs have taken off.

In other words, the book appeals primarily to scientists trained outside social science, some of whom think about politics in their spare time, or encounter it in dispiriting encounters with policymakers. It appeals to that audience with a statement on the crucial role of high quality evidence in policymaking, highlights barriers to its use, tells scientists that they might be part of the problem, but then provides them with the comforting assurance that we can design better systems to overcome at least some of those barriers. For people trained in policy studies, this concluding discussion seems like a tall order, and I think most would read it with great scepticism.

Policy scientists might also be sceptical about the extent to which scientists from other fields think this way about hierarchies of scientific evidence and the desire to depoliticise politics with a primary focus on ‘what works’. Yet, I too hear this language regularly in interdisciplinary workshops (often while standing next to Justin!), and it is usually accompanied by descriptions of the pathology of policymaking, the rise of post-truth politics and rejection of experts, and the need to focus on the role of objective facts in deciding what policy solutions work best. Indeed, I was impressed recently by the skilled way in which another colleague prepared this audience for some provocative remarks when he suggested that the production and use of evidence is about power, not objectivity. OMG: who knew that policymaking was political and about power?!

So, the insights from this book are useful to a large audience of scientists while, for a smaller audience of policy scientists, they remind us that there is an audience out there for many of the statements that many of us would take for granted. Some evidence advocates use the language of ‘evidence based policymaking’ strategically, to get what they want. Others appear to use it because they believe it can exist. Keep this in mind when you read the book.

Three ways to communicate more effectively with policymakers

By Paul Cairney and Richard Kwiatkowski

Use psychological insights to inform communication strategies

Policymakers cannot pay attention to all of the things for which they are responsible, or understand all of the information they use to make decisions. Like all people, there are limits on what information they can process (Baddeley, 2003; Cowan, 2001, 2010; Miller, 1956; Rock, 2008).

They must use short cuts to gather enough information to make decisions quickly: the ‘rational’, by pursuing clear goals and prioritizing certain kinds of information, and the ‘irrational’, by drawing on emotions, gut feelings, values, beliefs, habits, schemata, scripts, and what is familiar, to make decisions quickly. Unlike most people, they face unusually strong pressures on their cognition and emotion.

Policymakers need to gather information quickly and effectively, often in highly charged political atmospheres, so they develop heuristics to allow them to make what they believe to be good choices. Perhaps their solutions seem to be driven more by their values and emotions than a ‘rational’ analysis of the evidence, often because we hold them to a standard that no human can reach.

If so, and if they have high confidence in their heuristics, they will dismiss criticism from researchers as biased and naïve. Under those circumstances, we suggest that restating the need for ‘rational’ and ‘evidence-based policymaking’ is futile, naively ‘speaking truth to power’ counterproductive, and declaring ‘policy based evidence’ defeatist.

We use psychological insights to recommend a shift in strategy for advocates of the greater use of evidence in policy. The simple recommendation, to adapt to policymakers’ ‘fast thinking’ (Kahneman, 2011) rather than bombard them with evidence in the hope that they will get round to ‘slow thinking’, is already becoming established in evidence-policy studies. However, we provide a more sophisticated understanding of policymaker psychology, to help understand how people think and make decisions as individuals and as part of collective processes. It allows us to (a) combine many relevant psychological principles with policy studies to (b) provide several recommendations for actors seeking to maximise the impact of their evidence.

To ‘show our work’, we first summarise insights from policy studies already drawing on psychology to explain policy process dynamics, and identify key aspects of the psychology literature which show promising areas for future development.

Then, we emphasise the benefit of pragmatic strategies, to develop ways to respond positively to ‘irrational’ policymaking while recognising that the biases we ascribe to policymakers are present in ourselves and our own groups. Instead of bemoaning the irrationality of policymakers, let’s marvel at the heuristics they develop to make quick decisions despite uncertainty. Then, let’s think about how to respond effectively. Instead of identifying only the biases in our competitors, and masking academic examples of group-think, let’s reject our own imagined standards of high-information-led action. This more self-aware and humble approach will help us work more successfully with other actors.

On that basis, we provide three recommendations for actors trying to engage skilfully in the policy process:

1. Tailor framing strategies to policymaker bias. If people are cognitive misers, minimise the cognitive burden of your presentation. If policymakers combine cognitive and emotive processes, combine facts with emotional appeals. If policymakers make quick choices based on their values and simple moral judgements, tell simple stories with a hero and moral. If policymakers reflect a ‘group emotion’, based on their membership of a coalition with firmly-held beliefs, frame new evidence to be consistent with those beliefs.
2. Identify ‘windows of opportunity’ to influence individuals and processes. ‘Timing’ can refer to the right time to influence an individual, depending on their current way of thinking, or to act while political conditions are aligned.
3. Adapt to real-world ‘dysfunctional’ organisations rather than waiting for an orderly process to appear. Form relationships in networks, coalitions, or organisations first, then supply challenging information second. To challenge without establishing trust may be counterproductive.

These tips are designed to produce effective, not manipulative, communicators. They help foster the clearer communication of important policy-relevant evidence, rather than imply that we should bend evidence to manipulate or trick politicians. We argue that it is pragmatic to work on the assumption that people’s beliefs are honestly held, and policymakers believe that their role is to serve a cause greater than themselves. To persuade them to change course requires showing simple respect and seeking ways to secure their trust, rather than simply ‘speaking truth to power’. Effective engagement requires skilful communication and good judgement as much as good evidence.

---

This is the introduction to our revised and resubmitted paper to the special issue of Palgrave Communications The politics of evidence-based policymaking: how can we maximise the use of evidence in policy? Please get in touch if you are interested in submitting a paper to the series.

Full paper: Cairney Kwiatkowski Palgrave Comms resubmission CLEAN 14.7.17

The impact of multi-level policymaking on the UK energy system

In September, we will begin a one-year UKERC project examining current and future energy policy and multi-level policymaking and its impact on ‘energy systems’. This is no mean feat, since the meaning of policy, policymaking (or the ‘policy process’), and ‘system’ are not clear, and our description of the components parts of an energy system and a complex policymaking system may differ markedly. So, one initial aim is to provide some way to turn a complex field of study into something simple enough to understand and engage with.

We do so by focusing on ‘multi-level policymaking’ – which can encompass concepts such as multi-level governance and intergovernmental relations – to reflect the fact that the responsibility for policies relevant to energy are often Europeanised, devolved, and shared between several levels of government. Brexit will produce a major effect on energy and non-energy policies, and prompt the UK and devolved governments to produce relationships, but we all need more clarity on the dynamics of current arrangements before we can talk sensibly about the future. To that end, we pursue three main work packages:

1. What is the ‘energy policymaking system’ and how does it affect the energy system?

Chaudry et al (2009: iv) define the UK energy system as ‘the set of technologies, physical infrastructure, institutions, policies and practices located in and associated with the UK which enable energy services to be delivered to UK consumers’. UK policymaking can have a profound impact, and constitutional changes might produce policy change, but their impacts require careful attention. So, we ‘map’ the policy process and the effect of policy change on energy supply and demand. Mapping sounds fairly straightforward but contains a series of tasks whose level of difficulty rises each time:

1. Identify which level or type of government is responsible – ‘on paper’ and in practice – for the use of each relevant policy instrument.
2. Identify how these actors interact to produce what we call ‘policy’, which can range from statements of intent to final outcomes.
3. Identify an energy policy process containing many actors at many levels, the rules they follow, the networks they form, the ‘ideas’ that dominate discussion, and the conditions and events (often outside policymaker control) which constrain and facilitate action. By this stage, we need to draw on particular policy theories to identify key venues, such as subsystems, and specific collections of actors, such as advocacy coalitions, to produce a useful model of activity.

2. Who is responsible for action to reduce energy demand?

Energy demand is more challenging to policymakers than energy supply because the demand side involves millions of actors who, in the context of household energy use, also constitute the electorate. There are political tensions in making policies to reduce energy demand and carbon where this involves cost and inconvenience for private actors who do not necessarily value the societal returns achieved, and the political dynamics often differ from policy to regulate industrial demand. There are tensions around public perceptions of whose responsibility it is to take action – including local, devolved, national, or international government agencies – and governments look like they are trying to shift responsibility to each other or individuals and firms.

So, there is no end of ways in which energy demand could be regulated or influenced – including energy labelling and product/building standards, emissions reduction measures, promotion of efficient generation, and buildings performance measures – but it is an area of policy which is notoriously diffuse and lacking in co-ordination. So, for the large part, we consider if Brexit provides a ‘window of opportunity’ to change policy and policymaking by, for example, clarifying responsibilities and simplifying relationships.

3: Does Brexit affect UK and devolved policy on energy supply?

It is difficult for single governments to coordinate an overall energy mix to secure supply from many sources, and multi-level policymaking adds a further dimension to planning and cooperation. Yet, the effect of constitutional changes is highly uneven. For example, devolution has allowed Scotland to go its own way on renewable energy, nuclear power and fracking, but Brexit’s impact ranges from high to low. It presents new and sometimes salient challenges for cooperation to supply renewable energy but, while fracking and nuclear are often the most politically salient issues, Brexit may have relatively little impact on policymaking within the UK.

We explore the possibility that renewables policy may be most impacted by Brexit, while nuclear and fracking are examples in which Brexit may have a minimal direct impact on policy. Overall, the big debates are about the future energy mix, and how local, devolved, and UK governments balance the local environmental impacts of, and likely political opposition to, energy development against the economic and energy supply benefits.

For more details, see our 4-page summary

Powerpoint for 13.7.17

Cairney et al UKERC presentation 23.10.17

Policy Concepts in 1000 Words: The Policy Process

We talk a lot about ‘the policy process’ without really saying what it is. If you are new to policy studies, maybe you think that you’ll learn what it is eventually if you read enough material. This would be a mistake! Instead, when you seek a definition of the policy process, you’ll find two common responses:

1. Many will seek to define policy or public policy instead of ‘the policy process’.
2. Some will describe the policy process as a policy cycle with stages.

Both responses seem inadequate: one avoids giving an answer, and another gives the wrong answer!

However, we can combine elements of each approach to give you just enough of a sense of ‘the policy process’ to continue reading the full ‘1000 words’ series:

1. The beauty of the ‘what is policy?’ question …

… is that we don’t give you an answer. It may seem frustrating at first to fail to find a definitive answer, but eventually you’ll accept this problem! The more important outcome is to use the ‘what is policy?’ question to develop analytical skills, to allow you to define policy in more specific circumstances (such as, what are the key elements of policy in this case study?), and ask more useful and specific questions about policy and policymaking. So, look at the questions we need to ask if we begin with the definition, ‘the sum total of government action, from signals of intent to the final outcomes’: does action include statements of intent? Do we include unintended policy outcomes? Are all policymakers in government? What about the things policymakers choose not to do? And so on.

2. The beauty of the policy cycle approach …

… is that it provides a simple way to imagine policy ‘dynamics’, or events and choices producing a never-ending sequence of other events and choices. Look at the stages model to identify many different tasks within one ‘process’, and to get the sense that policymaking is continuous and often ‘its own cause’. It’s not a good description of what actually happens, but it describes what some might like to happen, and used by many governments to describe what they do. Consequently, we can’t simply ignore it, at least without providing a better description, a better plan, and a better way for governments to justify what they do.

There are more complicated but better ways of describing policymaking dynamics

This picture is the ‘policy process’ equivalent of my definition of public policy. It captures the main elements of the policy process described – albeit in different ways – by most policy theories in this series. I present it here to give you enough of an answer – to ‘what is the policy process?’ – to help you ask more questions.

In the middle is ‘policy choice’

At the heart of most policy theory is ‘bounded rationality’, which describes (a) the cognitive limits of all people, and (b) how policymakers overcome such limits to make decisions (in the absence of NZT). In short, they use ‘rational’ and ‘irrational’ shortcuts to action, but these are provocative terms to prompt further reading (on, for example, ‘evidence-based policymaking’).

‘Rational’ describes goal-oriented activity: people may have limits to their attention and ‘information processing’, but they find systematic ways to respond, by setting goals and producing criteria to find the best information. ‘Irrational’ describes aspects of psychology: people draw on habit, emotions, their ‘gut’ or intuition, well-established beliefs, and their familiarity with information to make often-almost-instant decisions.

Surrounding choice is what we’ll call the ‘policy environment’

Environment is a metaphor we’ll use to describe the combination of key elements of the policy process which (a) I describe separately in further 1000 words posts, and (b) policy theories bring together to produce an overall picture of policy dynamics.

There are 5 or 6 key elements. In the picture are 6, reflecting the way Tanya Heikkila and I describe it (and the fact that I had 7 boxes to fill). In real life, I describe 5 because I have 5 digits on each hand. If you are Count Tyrone Rugen you have more choice.

Policy environments are made up of:

1. A wide range of actors (which can be individuals and organisations with the ability to deliberate and act) making or influencing policy at many levels and types of government.
2. Institutions, defined as the rules followed by actors. Some are formal, written down, and easy to identify. Others are informal, reproduced via processes like socialisation, and difficult to spot and describe.
3. Networks, or the relationships between policymakers and influencers. Some are wide open, competitive, and contain many actors. Others are relatively closed, insulated from external attention, and contain few actors.
4. Ideas, or the beliefs held and shared by actors. There is often a tendency for certain beliefs or ‘paradigms’ to dominate discussion, constraining or facilitating the progress of new ‘ideas’ as policy solutions.
5. Context and events. Context describes the policy conditions – including economic, social, demographic, and technological factors – that provide the context for policy choice, and are often outside of the control of policymakers. Events can be routine and predictable, or unpredictable ‘focusing’ events that prompt policymaker attention to lurch at short notice.

This picture is only the beginning of analysis, raising further questions that will make more sense when you read further, including: should policymaker choice be at the centre of this picture? Why are there arrows in the cycle but not in my picture? Should we describe complex policymaking ‘systems’ rather than ‘environments’? How exactly does each element in the ‘policy environment’ or ‘system’ relate to the other?

The answer to the final question can only be found in each theory of the policy process, and each theory describes this relationship in a different way. Let’s not worry about that just now! We’ll return to this issue at the end, when thinking about how to combine the insights of many theories.

 

 

 

Policy in 500 Words: The Policy Process

We talk a lot about ‘the policy process’ without really saying what it is. If you are new to policy studies, maybe you think that you’ll learn what it is eventually if you read enough material. This would be a mistake! Instead, when you seek a definition of the policy process, you’ll find two common responses.

1. Many will seek to define policy or public policy instead of ‘the policy process’.
2. Some will describe the policy process as a policy cycle with stages.

Both responses seem inadequate: one avoids giving an answer, and another gives the wrong answer!

However, we can combine elements of each approach to give you just enough of a sense of ‘the policy process’ to continue reading:

1. The beauty of the ‘what is policy?’ question is that we don’t give you an answer. I give you a working definition to help raise further questions. Look at the questions we need to ask if we begin with the definition, ‘the sum total of government action, from signals of intent to the final outcomes’.
2. The beauty of the policy cycle approach is that it provides a simple way to imagine policy ‘dynamics’, or events and choices producing a sequence of other events and choices. Look at the stages to identify many different tasks within one ‘process’, and to get the sense that policymaking is continuous and often ‘its own cause’.

There are more complicated but better ways of describing policymaking dynamics

This picture is the ‘policy process’ equivalent of my definition of public policy. It captures the main elements of the policy process described (in different ways) by most policy theories. It is there to give you enough of an answer to help you ask the right questions.

In the middle is ‘policy choice’. At the heart of most policy theory is ‘bounded rationality’, which describes (a) the cognitive limits of people, and (b) how they overcome those limits to make decisions. They use ‘rational’ and ‘irrational’ shortcuts to action.

Surrounding choice is what we’ll call the ‘policy environment’, containing: policymakers in many levels and types of government, the ideas or beliefs they share, the rules they follow, the networks they form with influencers, and the ‘structural’ or socioeconomic context in which they operate.

This picture is only the beginning of analysis, raising further questions that will make more sense when you read further, including: should policymaker choice be at the centre of this picture? Why are there arrows (describing the order of choice) in the cycle but not in my picture?

Take home message for students: don’t describe ‘the policy process’ without giving the reader some sense of its meaning. Its definition overlaps with ‘policy’ considerably, but the ‘process’ emphasises modes and dynamics of policymaking, while ‘policy’ emphasises outputs. Then, think about how each policy model or theory tries, in different ways, to capture the key elements of the process. A cycle focuses on ‘stages’ but most theories in this series focus on ‘environments’.

 

 

 

 

 

Policy concepts in 1000 or 500 words

Imagine that your audience is a group of scientists who have read everything and are only interested in something new. You need a new theory, method, study, or set of results to get their attention.

Let’s say that audience is a few hundred people, or half a dozen in each subfield. It would be nice to impress them, perhaps with some lovely jargon and in-jokes, but almost no-one else will know or care what you are talking about.

Imagine that your audience is a group of budding scientists, researchers, students, practitioners, or knowledge-aware citizens who are new to the field and only interested in what they can pick up and use (without devoting their life to each subfield). Novelty is no longer your friend. Instead, your best friends are communication, clarity, synthesis, and a constant reminder not to take your knowledge and frame of reference for granted.

Let’s say that audience is a few gazillion people. If you want to impress them, imagine that you are giving them one of the first – if not the first – ways of understanding your topic. Reduce the jargon. Explain your problem and why people should care about how you try to solve it. Clear and descriptive titles. No more in-jokes (just stick with the equivalent of ‘I went to the doctor because a strawberry was growing in my arse, and she gave me some cream for it’).

At least, that’s what I’ve been telling myself lately. As things stand, my most-read post of all time is destined to be on the policy cycle, and most people read it because it’s the first entry on a google search. Most readers of that post may never read anything else I’ve written (over a million words, if I cheat a bit with the calculation). They won’t care that there are a dozen better ways to understand the policy process. I have one shot to make it interesting, to encourage people to read more. The same goes for the half-dozen other concepts (including multiple streams, punctuated equilibrium theory, the Advocacy Coalition Framework) which I explain to students first because I now do well in google search (go on, give it a try!).

I also say this because I didn’t anticipate this outcome when I wrote those posts. Now, a few years on, I’m worried that they are not very good. They were summaries of chapters from Understanding Public Policy, rather than first principles discussions, and lots of people have told me that UPP is a little bit complicated for the casual reader. So, when revising it, I hope to make it better, and by better I mean to appeal to a wider audience without dumping the insights. I have begun by trying to write 500-words posts as, I hope, improvements on the 1000-word versions. However, I am also open to advice on the originals. Which ones work, and which ones don’t? Where are the gaps in exposition? Where are the gaps in content?

This post is 500 words.

https://paulcairney.wordpress.com/1000-words/

https://paulcairney.wordpress.com/500-words/

5 images of the policy process

A picture tells a thousand words but, in policy studies, those words are often misleading or unclear. The most useful images can present the least useful advice, or capture a misleading metaphor. Images from the most useful theories are useful when you already know the theory, but far more difficult to grasp initially.

So, I present two examples from each, then describe what a compromise image might look like, to combine something that is easy to pick up and use but also not misleading or merely metaphorical.

Why do we need it? It is common practice at workshops and conferences for some to present policy process images on powerpoint and for others to tweet photos of them, generally with little critical discussion of what they say and how useful they are. I’d like to see as-simple but more-useful images spread this way.

1. The policy cycle

The policy cycle is perhaps the most used and known image. It divides the policy process into a series of stages (described in 1000 words and 500 words). It oversimplifies, and does not explain, a complex policymaking system. We are better to imagine, for example, thousands of policy cycles interacting with each other to produce less orderly behaviour and less predictable outputs.

For students, we have dozens of concepts and theories which serve as better ways to understand policymaking.

Policymakers have more use for the cycle, to tell a story of what they’d like to do: identify aims, identify policies to achieve those aims, select a policy measure, ensure that the selection is legitimised by the population or its legislature, identify the necessary resources, implement, then evaluate the policy.

As such, this story still pops up from time to time:

Really helpful slide from SG Chief Social Researcher on the Gov. 'Policy Cycle'

Useful for anyone working in the Policy Sphere. @SPRExchange pic.twitter.com/jtLEGhD0oD

— TILLY ROBINSON-MILES 🌻 (@TRobinsonMiles) December 9, 2020

Yet, most presentations from policymakers, advisers, and practitioners modify the cycle image to show how messy life really is:

The messy policy process and the role of evidence #evidence4gov @effectiveserv pic.twitter.com/g8kLOhxet5

— Majella McCloskey (@MajellaMcCl) June 27, 2017

Discussing the challenges of evidence based policy including complexity of the landscape @OECD #evidence4gov pic.twitter.com/1yu3UIKKjO

— Rob Davies (@r0bdavies) June 26, 2017

Update 18.2.20: I knocked this up one in my garage for the book The Politics of Policy Analysis. See in particular what you need as an analyst versus policymaking reality, which argues that the cycle (or 5-step policy analysis) describes what policy analysts would like to do (not what happens).

2. The multiple streams metaphor

The ‘multiple streams’ approach uses metaphor to describe this messier world (described in 1000 words and 500 words). Instead of a linear cycle – in which policymakers define problems, then ask for potential solutions, then select one – we describe these ‘stages’ as independent ‘streams’. Each stream – heightened attention to a problem (problem stream), an available and feasible solution (policy stream), and the motive to select it (politics stream) – must come together during a ‘window of opportunity’ or the opportunity is lost.

Many people like MSA because it contains a flexible metaphor which is simple to pick up and use. However, it’s so flexible that I’ve seen many different ways to visualise – and make sense of – the metaphor, including literal watery streams, which suggest that when they come together they are hard to separate.  There is the Ghostbusters metaphor which shows that key actors (‘entrepreneurs’) help couple the streams. There is also Howlett et al’s attempt to combine the streams and cycles metaphors (reproduced here, and criticised here).

However, I’d encourage Kingdon’s space launch metaphor in which policymakers will abort the mission unless every factor is just right.

3. The punctuated equilibrium graph

Punctuated equilibrium theory is one of the most important approaches to policy dynamics, now backed up with a wealth of data from the Comparative Agendas Project. The image (in True et al, 2007) describes the result of the policy process rather than the process itself. It describes government budgets in the US, although we can find very similar images from studies of budgets in many other countries and in many measures of policy change.

It sums up a profoundly important message about policy change: we find a huge number of very small changes, and a very small number of huge changes. Compare the distribution of values in this image with the ‘normal distribution’ (the dotted line). It shows a ‘leptokurtic’ distribution, with most values deviating minimally from the mean (and the mean change in each budget item is small), but with a high number of ‘outliers’.

The image helps sum up a key aim of PET, to measure and try to explain long periods of policymaking stability, and policy continuity, disrupted by short but intense periods of instability and change. One explanation relates to ‘bounded rationality’: policymakers have to ignore almost all issues while paying attention to some. The lack of ‘macropolitical’ attention to most issues helps explain stability and continuity, while lurches of attention can help explain instability (although attention can fade before ‘institutions’ feel the need to respond).

Here I am, pointing at this graph:

@CairneyPaul providing much food for thought on multi-level decision making in energy policy at @UKERCHQ assembly '17 pic.twitter.com/zKoImOhGNZ

— Christian Brand (@_chris_brand_) July 13, 2017

4. The advocacy coalition framework ‘flow diagram’

The ACF presents an ambitious image of the policy process, in which we zoom out to consider how key elements fit together in a process containing many actors and levels of government. Like many policy theories, it situates most of the ‘action’ in policy networks or subsystems, showing that some issues involve intensely politicized disputes containing many actors while others are treated as technical and processed routinely, largely by policy specialists, out of the public spotlight.

The ACF suggests that people get into politics to turn their beliefs into policy, form coalitions with people who share their beliefs, and compete with coalitions of actors who share different beliefs. This competition takes place in a policy subsystem, in which coalitions understand new evidence through the lens of their beliefs, and exercise power to make sure that their interpretation is accepted. The other boxes describe the factors – the ‘parameters’ likely to be stable during the 10-year period of study, the partial sources of potential ‘shocks’ to the subsystem, and the need and ability of key actors to form consensus for policy change (particularly in political systems with PR elections) – which constrain and facilitate coalition action.

5. What do we need from a new image?

I recommend an image that consolidates or synthesises existing knowledge and insights. It is tempting to produce something that purports to be ‘new’ but, as with ‘new’ concepts or ‘new’ policy theories, how could we accumulate insights if everyone simply declared novelty and rejected the science of the past?

For me, the novelty should be in the presentation of the image, to help people pick up and use a wealth of policy studies which try to capture two key dynamics:

1. Policy choice despite uncertainty and ambiguity.

Policymakers can only pay attention to a tiny proportion of issues. They use ‘rational’ and ‘irrational’ cognitive shortcuts to make decisions quickly, despite their limited knowledge of the world, and the possibility to understand policy problems from many perspectives.

2. A policy environment which constrains and facilitates choice.

Such environments are made up of:

1. Actors (individuals and organisations) influencing policy at many levels and types of government
2. Institutions: a proliferation of rules and norms followed by different levels or types of government
3. Networks: relationships between policymakers and influencers
4. Ideas: a tendency for certain beliefs or ‘paradigms’ to dominate discussion
5. Context and events: economic, social, demographic, and technological conditions provide the context for policy choice, and routine/ unpredictable events can prompt policymaker attention to lurch at short notice.

The implications of both dynamics are fairly easy to describe in tables (for example, while describing MSA) and to cobble together quickly in a SmartArt picture:

However, note at least three issues with such a visual presentation:

1. Do we put policymakers and choice at the centre? If so, it could suggest (a bit like the policy cycle) that a small number of key actors are at the ‘centre’ of the process, when we might prefer to show that their environment, or the interaction between many actors, is more important.
2. Do we show only the policy process or relate it to the ‘outside world’?
3. There are many overlaps between concepts. For example, we seek to describe the use and reproduction of rules in ‘institutions’ and ‘networks’, while those rules relate strongly to ‘ideas’. Further, ‘networks’ could sum up ‘actors interacting in many levels/ types of government’. So, ideally, we’d have overlapping shapes to denote overlapping relationships and understandings, but it would really mess up the simplicity of the image.

Of course, the bigger issue is that the image I provide is really just a vehicle to put text on a screen (in the hope that it will be shared). At best it says ‘note these concepts’. It does not show causal relationships. It does not describe any substantial interaction between the concepts to show cause and effect (such as, event A prompted policy choice B).

However, if we tried to bring in that level of detail, I think we would quickly end up with the messy process already described in relation to the policy cycle. Or, we would need to provide a more specific and less generally applicable model of policymaking.

So, right now, this image is a statement of intent. I want to produce something better, but don’t yet know what ‘better’ looks like. There is no ‘general theory’ of policymaking, so can we have a general image? Or, like ‘what is policy?’ discussions, do we produce an answer largely to raise more questions?

Update: please compare with the turtle diagram, below, and explored in more depth here.

___

Here I am, looking remarkably pleased with my SmartArt skills

😀😀😀Diagram on policy environment by @CairneyPaul #UKERCAA17 pic.twitter.com/hADP9gS5ds

— Sam Hampton (@samhampton) July 13, 2017

How do we get governments to make better decisions?

This is a guest post by Chris Koski (left) and Sam Workman (right), discussing how to use insights from punctuated equilibrium theory to reform government policy making. The full paper has been submitted to the series for Policy and Politics called Practical Lessons from Policy Theories.

Many people assume that the main problem faced by governments is an information deficit. However, the opposite is true. A surfeit of information exists and institutions have a hard time managing it.  At the same time, all the information that exists in defining problems may be insufficient. Institutions need to develop a capacity to seek out better quality information too.

Institutions, from the national government, to state legislatures, to city councils – try to solve the information processing dilemma by delegating authority to smaller subgroups. Delegation increases the information processing capacity of governments by involving more actors to attend to narrower issues.

The delegation of authority is ultimately a delegation of attention. It solves the ‘flow’ problem, but also introduces new ‘filters’.  The preferences, interests, and modes of information search all influence the process. Even narrowly focused smaller organizations face limitations in their capacity to search and are subject to similar forces as the governments which created them – filters for the deluge of information and capacity limitations for information seeking.

Organizational design predisposes institutions to filter information for ideas that support status quo problem definitions – that is, definitions that existed at the time of delegation – and to seek out information based on these status quo understandings.  As a result, despite a desire to expand attention and information processing to adapt to changes in problem characteristics, most institutions look for information that supports their identity.  Institutional problem definitions stay the same even as the problems change.

Governments eventually face trade-offs between the gains made from delegating decision-making to smaller subgroups and the losses associated with coordinating the information generated by those subgroups.

Governments get stuck in the same ruts as when the delegation process started: status quo bias that doesn’t adjust with change problem conditions.  There is a sense among citizens and academics that governments make bad decisions in part because they respond to problems of today with the policies of 10 years ago.  Government solutions look like hammers in search of nails when they ought to look more like contractors or even urban planners.

Governments should not respond simply by centralizing

When institutions become stultified in their problem definitions, policymakers and citizens often misdiagnose the problem as entirely a coordination problem.  The logic here is that a small group of actors have captured policymaking and are using such capture for their own gain.  This understanding may be true, or may not, but it leads to the “centralization as savior” fallacy.  The idea here is that organizations with broader latitude will be better able to receive a wider variety of information from a broader range of sources.

There are two problems with this strategy.  First, centralization might guarantee an outcome, but at the expense of an honest problems search and, likely, at the expense of what we might call policy stability.  Second, centralization may offer the opportunity for a broader array of information to bear on policy decisions, but, in practice will rely on even narrower information filters given the number of issues to which the newly centralized policymaking forum must attend.

More delegation produces fragmentation

The alternative, more delegation, has significant coordination challenges as we find bottlenecks of attention when multiple subsystems bear on decision-points.  Also, simply delegating authority can predispose subsystems to a particular solution, which we want to avoid.

We’d propose: Adaptive governance

• Design institutions not just to attend to problems, but to be specifically information seeking. For example, NEPA requires that all US federal decision-making regarding the environment undergo some kind of environmental assessment – this can be as simply as saying “the environmental will not be harmed” or as complex as an environmental impact statement.  At the same time, we’d suggest greater coordination of institutional actions – enhance communication across delegated units but also better feedback mechanisms to overarching institutions.
• Institutions need to listen to the signals that their delegated units give them. When delegated institutions come to similar conclusions regarding similar problems, these are key signals to broader policymaking bodies.  Listening to signals from multiple delegated units allows for expertise to shine.  At the same time, disharmony across delegated units on the same problems is a good indicator of disharmony in information search.  Sometimes institutions respond to this disharmony by attempting to reduce participation in the policy process or cast outliers as simply outliers.  We think this is a bad idea as it exaggerates the acceptability of the status quo.
• We propose ‘issue bundling’ which allows for issues to be less tied up by monolithic problem definitions. Policymaking institutions ought to formally direct delegated institutions to look at the same problem relying upon different expertise.  Examples here are climate change or critical infrastructure protection.  To create institutions to deal with these issues is a challenge given the wide range of information necessary to address each.  Institutions can solve the attention problems that emerge from the multiple sources by creating specific channels of information.  This allows for multiple subsystems  – e.g. Agriculture, Transportation, or Environmental Protection – to assist institutional decision-making by sorting issue specific – e.g. Climate Change – information.

Our solutions do solve fundamental problems of information processing in terms of sorting and seeking information – such problems are fundamental to humans and human-created organizations.  However, while governments may be predisposed to prioritize decisions over information, we are optimistic that our recommendations can facilitate better informed policy in the future.