Tag Archives: punctuated equilibrium theory

Evidence-based policymaking and the ‘new policy sciences’

image policy process round 2 25.10.18

[I wasn’t happy with the first version, so this is version 2]

In the ‘new policy sciences’, Chris Weible and I advocate:

  • a return to Lasswell’s vision of combining policy analysis (to recommend policy change) and policy theory (to explain policy change), but
  • focusing on a far larger collection of actors (beyond a small group at the centre),
  • recognising new developments in studies of the psychology of policymaker choice, and
  • building into policy analysis the recognition that any policy solution is introduced in a complex policymaking environment over which no-one has control.

However, there is a lot of policy theory out there, and we can’t put policy theory together like Lego to produce consistent insights to inform policy analysis.

Rather, each concept in my image of the policy process represents its own literature: see these short explainers on the psychology of policymaking, actors spread across multi-level governance, institutions, networks, ideas, and socioeconomic factors/ events.

What the explainers don’t really project is the sense of debate within the literature about how best to conceptualise each concept. You can pick up their meaning in a few minutes but would need a few years to appreciate the detail and often-fundamental debate.

Ideally, we would put all of the concepts together to help explain policymaker choice within a complex policymaking environment (how else could I put up the image and present is as one source of accumulated wisdom from policy studies?). Peter John describes such accounts as ‘synthetic’. I have also co-authored work with Tanya Heikkila – in 2014 and 2017 to compare the different ways in which ‘synthetic’ theories conceptualise the policy process.

However, note the difficulty of putting together a large collection of separate and diverse literatures into one simple model (e.g. while doing a PhD).

On that basis, I’d encourage you to think of these attempts to synthesise as stories. I tell these stories a lot, but someone else could describe theory very differently (perhaps by relying on fewer male authors or US-derived theories in which there is a very specific reference points and positivism is represented well).

The example of EBPM

I have given a series of talks to explain why we should think of ‘evidence-based policymaking’ as a myth or political slogan, not an ideal scenario or something to expect from policymaking in the real world. They usually involve encouraging framing and storytelling rather than expecting evidence to speak for itself, and rejecting the value of simple models like the policy cycle. I then put up an image of my own and encourage people to think about the implications of each concept:

SLIDE simple advice from hexagon image policy process 24.10.18

I describe the advice as simple-sounding and feasible at first glance, but actually a series of Herculean* tasks:

  • There are many policymakers and influencers spread across government, so find out where the action is, or the key venues in which people are making authoritative decisions.
  • Each venue has its own ‘institutions’ – the formal and written, or informal and unwritten rules of policymaking – so learn the rules of each venue in which you engage.
  • Each venue is guided by a fundamental set of ideas – as paradigms, core beliefs, monopolies of understanding – so learn that language.
  • Each venue has its own networks – the relationships between policy makers and influencers – so build trust and form alliances within networks.
  • Policymaking attention is often driven by changes in socioeconomic factors, or routine/ non-routine events, so be prepared to exploit the ‘windows of opportunity’ to present your solution during heightened attention to a policy problem.

Further, policy theories/ studies help us understand the context in which people make such choices. For example, consider the story that Kathryn Oliver and I tell about the role of evidence in policymaking environments:

If there are so many potential authoritative venues, devote considerable energy to finding where the ‘action’ is (and someone specific to talk to). Even if you find the right venue, you will not know the unwritten rules unless you study them intensely. Some networks are close-knit and difficult to access because bureaucracies have operating procedures that favour some sources of evidence. Research advocates can be privileged insiders in some venues and excluded completely in others. If your evidence challenges an existing paradigm, you need a persuasion strategy good enough to prompt a shift of attention to a policy problem and a willingness to understand that problem in a new way. You can try to find the right time to use evidence to exploit a crisis leading to major policy change, but the opportunities are few and chances of success low.  In that context, policy studies recommend investing your time over the long term – to build up alliances, trust in the messenger, knowledge of the system, and to seek ‘windows of opportunity’ for policy change – but offer no assurances that any of this investment will ever pay off

As described, this focus on the new policy sciences and synthesising insights helps explain why ‘the politics of evidence-based policymaking’ is equally important to civil servants (my occasional audience) as researchers (my usual audience).

To engage in skilled policy analysis, and give good advice, is to recognise the ways in which policymakers combine cognition/emotion to engage with evidence, and must navigate a complex policymaking environment when designing or selecting technically and politically feasible solutions. To give good advice is to recognise what you want policymakers to do, but also that they are not in control of the consequences.

From one story to many?

However, I tell these stories without my audience having the time to look further into each theory and its individual insights. If they do have a little more time, I go into the possible contribution of individual insights to debate.

For example, they adapt insights from psychology in different ways …

  • PET shows the overall effect of policymaker psychology on policy change: they combine cognition and emotion to pay disproportionate attention to a small number of issues (contributing to major change) and ignore the rest (contributing to ‘hyperincremental’ change).
  • The IAD focuses partly on the rules and practices that actors develop to build up trust in each other.
  • The ACF describes actors going into politics to turn their beliefs into policy, forming coalitions with people who share their beliefs, then often romanticising their own cause and demonising their opponents.
  • The NPF describes the relative impact of stories on audiences who use cognitive shortcuts to (for example) identify with a hero and draw a simple moral.
  • SCPD describes policymakers drawing on gut feeling to identify good and bad target populations.
  • Policy learning involves using cognition and emotion to acquire new knowledge and skills.

… even though the pace of change in psychological research often seems faster than the ways in which policy studies can incorporate new and reliable insights.

They also present different conceptions of the policymaking environment in which actors make choices. See this post for more on this discussion in relation to EBPM.

My not-brilliant conclusion is that:

  1. Policy theory/ policy studies has a lot to offer other disciplines and professions, particularly in field like EBPM in which we need to account for politics and, more importantly, policymaking systems, but
  2. Beware any policy theory story that presents the source literature as coherent and consistent.
  3. Rather, any story of the field involves a series of choices about what counts as a good theory and good insight.
  4. In other words, the exhortation to think more about what counts as ‘good evidence’ applies just as much to political science as any other.

Postscript: well, that is the last of the posts for my ANZOG talks. If I’ve done this properly, there should now be a loop of talks. It should be possible to go back to the first one and see it as a sequel to this one!

Or, for more on theory-informed policy analysis – in other words, where the ‘new policy sciences’ article is taking us – here is how I describe it to students doing a policy analysis paper (often for the first time).

Or, have a look at the earlier discussion of images of the policy process. You may have noticed that there is a different image in this post (knocked up in my shed at the weekend). It’s because I am experimenting with shapes. Does the image with circles look more relaxing? Does the hexagonal structure look complicated even though it is designed to simplify? Does it matter? I think so. People engage emotionally with images. They share them. They remember them. So, I need an image more memorable than the policy cycle.

 

Paul Cairney Brisbane EBPM New Policy Sciences 25.10.18

 

 

 

*I welcome suggestions on another word to describe almost-impossibly-hard

2 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), Policy learning and transfer, Psychology Based Policy Studies, public policy

Evidence-based policymaking and the ‘new policy sciences’

Circle image policy process 24.10.18

I have given a series of talks to explain why we should think of ‘evidence-based policymaking’ as a myth or political slogan, not an ideal scenario or something to expect from policymaking in the real world. They usually involve encouraging framing and storytelling rather than expecting evidence to speak for itself, and rejecting the value of simple models like the policy cycle. I then put up an image of my own and encourage people to think about the implications of each concept:

SLIDE simple advice from hexagon image policy process 24.10.18

I describe the advice as simple-sounding and feasible at first glance, but actually a series of Herculean* tasks:

  • There are many policymakers and influencers spread across government, so find out where the action is, or the key venues in which people are making authoritative decisions.
  • Each venue has its own ‘institutions’ – the formal and written, or informal and unwritten rules of policymaking – so learn the rules of each venue in which you engage.
  • Each venue is guided by a fundamental set of ideas – as paradigms, core beliefs, monopolies of understanding – so learn that language.
  • Each venue has its own networks – the relationships between policy makers and influencers – so build trust and form alliances within networks.
  • Policymaking attention is often driven by changes in socioeconomic factors, or routine/ non-routine events, so be prepared to exploit the ‘windows of opportunity’ to present your solution during heightened attention to a policy problem.

In most cases, we don’t have time to discuss a more fundamental issue (at least for researchers using policy theory and political science concepts):

From where did these concepts come, and how well do we know them?

To cut a long story short, each concept represents its own literature: see these short explainers on the psychology of policymaking, actors spread across multi-level governance, institutions, networks, ideas, and socioeconomic factors/ events. What the explainers don’t really project is the sense of debate within the literature about how best to conceptualise each concept. You can pick up their meaning in a few minutes but would need a few years to appreciate the detail and often-fundamental debate.

Ideally, we would put all of the concepts together to help explain policymaker choice within a complex policymaking environment (how else could I put up the image and present is as one source of accumulated wisdom from policy studies?). Peter John describes such accounts as ‘synthetic’. I have also co-authored work with Tanya Heikkila – in 2014 and 2017 to compare the different ways in which ‘synthetic’ theories conceptualise the policy process. However, note the difficulty of putting together a large collection of separate and diverse literatures into one simple model (e.g. while doing a PhD).

The new policy sciences

More recently, in the ‘new policy sciences’, Chris Weible and I present a more provocative story of these efforts, in which we advocate:

  • a return to Lasswell’s vision of combining policy analysis (to recommend policy change) and policy theory (to explain policy change), but
  • focusing on a far larger collection of actors (beyond a small group at the centre),
  • recognising new developments in studies of the psychology of policymaker choice, and
  • building into policy analysis the recognition that any policy solution is introduced in a complex policymaking environment over which no-one has control.

This focus on psychology is not new …

  • PET shows the overall effect of policymaker psychology on policy change: they combine cognition and emotion to pay disproportionate attention to a small number of issues (contributing to major change) and ignore the rest (contributing to ‘hyperincremental’ change).
  • The IAD focuses partly on the rules and practices that actors develop to build up trust in each other.
  • The ACF describes actors going into politics to turn their beliefs into policy, forming coalitions with people who share their beliefs, then often romanticising their own cause and demonising their opponents.
  • The NPF describes the relative impact of stories on audiences who use cognitive shortcuts to (for example) identify with a hero and draw a simple moral.
  • SCPD describes policymakers drawing on gut feeling to identify good and bad target populations.
  • Policy learning involves using cognition and emotion to acquire new knowledge and skills.

… but the pace of change in psychological research often seems faster than the ways in which policy studies can incorporate new and reliable insights.

Perhaps more importantly, policy studies help us understand the context in which people make such choices. For example, consider the story that Kathryn Oliver and I tell about the role of evidence in policymaking environments:

If there are so many potential authoritative venues, devote considerable energy to finding where the ‘action’ is (and someone specific to talk to). Even if you find the right venue, you will not know the unwritten rules unless you study them intensely. Some networks are close-knit and difficult to access because bureaucracies have operating procedures that favour some sources of evidence. Research advocates can be privileged insiders in some venues and excluded completely in others. If your evidence challenges an existing paradigm, you need a persuasion strategy good enough to prompt a shift of attention to a policy problem and a willingness to understand that problem in a new way. You can try to find the right time to use evidence to exploit a crisis leading to major policy change, but the opportunities are few and chances of success low.  In that context, policy studies recommend investing your time over the long term – to build up alliances, trust in the messenger, knowledge of the system, and to seek ‘windows of opportunity’ for policy change – but offer no assurances that any of this investment will ever pay off

Then, have a look at this discussion of ‘synthetic’ policy theories, designed to prompt people to consider how far they would go to get their evidence into policy.

Theory-driven policy analysis

As described, this focus on the new policy sciences helps explain why ‘the politics of evidence-based policymaking’ is equally important to civil servants (my occasional audience) as researchers (my usual audience).

To engage in skilled policy analysis, and give good advice, is to recognise the ways in which policymakers combine cognition/emotion to engage with evidence, and must navigate a complex policymaking environment when designing or selecting technically and politically feasible solutions. To give good advice is to recognise what you want policymakers to do, but also that they are not in control of the consequences.

Epilogue

Well, that is the last of the posts for my ANZOG talks. If I’ve done this properly, there should now be a loop of talks. It should be possible to go back to the first one in Auckland and see it as a sequel to this one in Brisbane!

Or, for more on theory-informed policy analysis – in other words, where the ‘new policy sciences’ article is taking us – here is how I describe it to students doing a policy analysis paper (often for the first time).

Or, have a look at the earlier discussion of images of the policy process. You may have noticed that there is a different image in this post (knocked up in my shed at the weekend). It’s because I am experimenting with shapes. Does the image with circles look more relaxing? Does the hexagonal structure look complicated even though it is designed to simplify? Does it matter? I think so. People engage emotionally with images. They share them. They remember them. So, I need an image more memorable than the policy cycle.

 

Paul Cairney Brisbane EBPM New Policy Sciences 25.10.18

 

 

*I welcome suggestions on another word to describe almost-impossibly-hard

2 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), Policy learning and transfer, Psychology Based Policy Studies, public policy, Storytelling

Taking lessons from policy theory into practice: 3 examples

Notes for ANZSOG/ ANU Crawford School/ UNSW Canberra workshop. Powerpoint here. The recording of the lecture (skip to 2m30) and Q&A is here (right click to download mp3 or dropbox link):

The context for this workshop is the idea that policy theories could be more helpful to policymakers/ practitioners if we could all communicate more effectively with each other. Academics draw general and relatively abstract conclusions from multiple cases. Practitioners draw very similar conclusions from rich descriptions of direct experience in a smaller number of cases. How can we bring together their insights and use a language that we all understand? Or, more ambitiously, how can we use policy theory-based insights to inform the early career development training that civil servants and researchers receive?

The first step is to translate policy theories into a non-technical language by trying to speak with an audience beyond our immediate peers (see for example Practical Lessons from Policy Theories).

However, translation is not enough. A second crucial step is to consider how policymakers and practitioners are likely to make sense of theoretical insights when they apply them to particular aims or responsibilities. For example:

  1. Central government policymakers may accept the descriptive accuracy of policy theories emphasising limited central control, but not the recommendation that they should let go, share power, and describe their limits to the public.
  2. Scientists may accept key limitations to ‘evidence based policymaking’ but reject the idea that they should respond by becoming better storytellers or more manipulative operators.
  3. Researchers and practitioners struggle to resolve hard choices when combining evidence and ‘coproduction’ while ‘scaling up’ policy interventions. Evidence choice is political choice. Can we do more than merely encourage people to accept this point?

I discuss these examples below because they are closest to my heart (especially example 1). Note throughout that I am presenting one interpretation about: (1) the most promising insights, and (2) their implications for practice. Other interpretations of the literature and its implications are available. They are just a bit harder to find.

Example 1: the policy cycle endures despite its descriptive inaccuracy

cycle

The policy cycle does not describe and explain the policy process well:

  • If we insist on keeping the cycle metaphor, it is more accurate to see the process as a huge set of policy cycles that connect with each other in messy and unpredictable ways.
  • The cycle approach also links strongly to the idea of ‘comprehensive rationality’ in which a small group of policymakers and analysts are in full possession of the facts and full control of the policy process. They carry out their aims through a series of stages.

Policy theories provide more descriptive and explanatory usefulness. Their insights include:

  • Limited choice. Policymakers inherit organisations, rules, and choices. Most ‘new’ choice is a revision of the old.
  • Limited attention. Policymakers must ignore almost all of the policy problems for which they are formally responsible. They pay attention to some, and delegate most responsibility to civil servants. Bureaucrats rely on other actors for information and advice, and they build relationships on trust and information exchange.
  • Limited central control. Policy may appear to be made at the ‘top’ or in the ‘centre’, but in practice policymaking responsibility is spread across many levels and types of government (many ‘centres’). ‘Street level’ actors make policy as they deliver. Policy outcomes appear to ‘emerge’ locally despite central government attempts to control their fate.
  • Limited policy change. Most policy change is minor, made and influenced by actors who interpret new evidence through the lens of their beliefs. Well-established beliefs limit the opportunities of new solutions. Governments tend to rely on trial-and-error, based on previous agreements, rather than radical policy change based on a new agenda. New solutions succeed only during brief and infrequent windows of opportunity.

However, the cycle metaphor endures because:

  • It provides a simple model of policymaking with stages that map onto important policymaking functions.
  • It provides a way to project policymaking to the public. You know how we make policy, and that we are in charge, so you know who to hold to account.

In that context, we may want to be pragmatic about our advice:

  1. One option is via complexity theory, in which scholars generally encourage policymakers to accept and describe their limits:
  • Accept routine error, reduce short-term performance management, engage more in trial and error, and ‘let go’ to allow local actors the flexibility to adapt and respond to their context.
  • However, would a government in the Westminster tradition really embrace this advice? No. They need to balance (a) pragmatic policymaking, and (b) an image of governing competence.
  1. Another option is to try to help improve an existing approach.

Further reading (blog posts):

The language of complexity does not mix well with the language of Westminster-style accountability

Making Sense of Policymaking: why it’s always someone else’s fault and nothing ever changes

Two stories of British politics: the Westminster model versus Complex Government

Example 2: how to deal with a lack of ‘evidence based policymaking’

I used to read many papers on tobacco policy, with the same basic message: we have the evidence of tobacco harm, and evidence of which solutions work, but there is an evidence-policy gap caused by too-powerful tobacco companies, low political will, and pathological policymaking. These accounts are not informed by theories of policymaking.

I then read Oliver et al’s paper on the lack of policy theory in health/ environmental scholarship on the ‘barriers’ to the use of evidence in policy. Very few articles rely on policy concepts, and most of the few rely on the policy cycle. This lack of policy theory is clear in their description of possible solutions – better communication, networking, timing, and more science literacy in government – which does not describe well the need to respond to policymaker psychology and a complex policymaking environment.

So, I wrote The Politics of Evidence-Based Policymaking and one zillion blog posts to help identify the ways in which policy theories could help explain the relationship between evidence and policy.

Since then, the highest demand to speak about the book has come from government/ public servant, NGO, and scientific audiences outside my discipline. The feedback is generally that: (a) the book’s description sums up their experience of engagement with the policy process, and (b) maybe it opens up discussion about how to engage more effectively.

But how exactly do we turn empirical descriptions of policymaking into practical advice?

For example, scientist/ researcher audiences want to know the answer to a question like: Why don’t policymakers listen to your evidence? and so I focus on three conversation starters:

  1. they have a broader view on what counts as good evidence (see ANZSOG description)
  2. they have to ignore almost all information (a nice way into bounded rationality and policymaker psychology)
  3. they do not understand or control the process in which they seek to use evidence (a way into ‘the policy process’)

Cairney 2017 image of the policy process

We can then consider many possible responses in the sequel What can you do when policymakers ignore your evidence?

Examples include:

  • ‘How to do it’ advice. I compare tips for individuals (from experienced practitioners) with tips based on policy concepts. They are quite similar-looking tips – e.g. find out where the action is, learn the rules, tell good stories, engage allies, seek windows of opportunity – but I describe mine as 5 impossible tasks!
  • Organisational reform. I describe work with the European Commission Joint Research Centre to identify 8 skills or functions of an organisation bringing together the supply/demand of knowledge.
  • Ethical dilemmas. I use key policy theories to ask people how far they want to go to privilege evidence in policy. It’s fun to talk about these things with the type of scientist who sees any form of storytelling as manipulation.

Further reading:

Is Evidence-Based Policymaking the same as good policymaking?

A 5-step strategy to make evidence count

Political science improves our understanding of evidence-based policymaking, but does it produce better advice?

Principles of science advice to government: key problems and feasible solutions

Example 3: how to encourage realistic evidence-informed policy transfer

This focus on EBPM is useful context for discussions of ‘policy learning’ and ‘policy transfer’, and it was the focus of my ANZOG talk entitled (rather ambitiously) ‘teaching evidence-based policy to fly’.

I’ve taken a personal interest in this one because I’m part of a project – called IMAJINE – in which we have to combine academic theory and practical responses. We are trying to share policy solutions across Europe rather than explain why few people share them!

For me, the context is potentially overwhelming:

So, when we start to focus on sharing lessons, we will have three things to discover:

  1. What is the evidence for success, and from where does it come? Governments often project success without backing it up.
  2. What story do policymakers tell about the problem they are trying to solve, the solutions they produced, and why? Two different governments may be framing and trying to solve the same problem in very different ways.
  3. Was the policy introduced in a comparable policymaking system? People tend to focus on political system comparability (e.g. is it unitary or federal?), but I think the key is in policymaking system comparability (e.g. what are the rules and dominant ideas?).

To be honest, when one of our external assessors asked me how well I thought I would do, we both smiled because the answer may be ‘not very’. In other words, the most practical lesson may be the hardest to take, although I find it comforting: the literature suggests that policymakers might ignore you for 20 years then suddenly become very (but briefly) interested in your work.

 

The slides are a bit wonky because I combined my old ppt to the Scottish Government with a new one for UNSW Paul Cairney ANU Policy practical 22 October 2018

I wanted to compare how I describe things to (1) civil servants (2) practitioners/ researcher (3) me, but who has the time/ desire to listen to 3 powerpoints in one go? If the answer is you, let me know and we’ll set up a Zoom call.

2 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), IMAJINE, Policy learning and transfer

5 images of the policy process

Cairney 2017 image of the policy process

A picture tells a thousand words but, in policy studies, those words are often misleading or unclear. The most useful images can present the least useful advice, or capture a misleading metaphor. Images from the most useful theories are useful when you already know the theory, but far more difficult to grasp initially.

So, I present two examples from each, then describe what a compromise image might look like, to combine something that is easy to pick up and use but also not misleading or merely metaphorical.

Why do we need it? It is common practice at workshops and conferences for some to present policy process images on powerpoint and for others to tweet photos of them, generally with little discussion of what they say and how useful they are. I’d like to see as-simple but more-useful images spread this way.

1. The policy cycle

cycle

The policy cycle is perhaps the most used and known image. It divides the policy process into a series of stages (described in 1000 words and 500 words). It oversimplifies, and does not explain, a complex policymaking system. We are better to imagine, for example, thousands of policy cycles interacting with each other to produce less orderly behaviour and less predictable outputs.

For students, we have dozens of concepts and theories which serve as better ways to understand policymaking.

Policymakers have more use for the cycle, to tell a story of what they’d like to do: identify aims, identify policies to achieve those aims, select a policy measure, ensure that the selection is legitimised by the population or its legislature, identify the necessary resources, implement, then evaluate the policy.

Yet, most presentations from policymakers, advisers, and practitioners modify the cycle image to show how messy life really is:

2. The multiple streams metaphor

NASA launch

The ‘multiple streams’ approach uses metaphor to describe this messier world (described in 1000 words and 500 words). Instead of a linear cycle – in which policymakers define problems, then ask for potential solutions, then select one – we describe these ‘stages’ as independent ‘streams’. Each stream – heightened attention to a problem (problem stream), an available and feasible solution (policy stream), and the motive to select it (politics stream) – must come together during a ‘window of opportunity’ or the opportunity is lost.

Many people like MSA because it contains a flexible metaphor which is simple to pick up and use. However, it’s so flexible that I’ve seen many different ways to visualise – and make sense of – the metaphor, including literal watery streams, which suggest that when they come together they are hard to separate.  There is the Ghostbusters metaphor which shows that key actors (‘entrepreneurs’) help couple the streams. There is also Howlett et al’s attempt to combine the streams and cycles metaphors (reproduced here, and criticised here).

However, I’d encourage Kingdon’s space launch metaphor in which policymakers will abort the mission unless every factor is just right.

3. The punctuated equilibrium graph

True et al figure 6.2

Punctuated equilibrium theory is one of the most important approaches to policy dynamics, now backed up with a wealth of data from the Comparative Agendas Project. The image (in True et al, 2007) describes the result of the policy process rather than the process itself. It describes government budgets in the US, although we can find very similar images from studies of budgets in many other countries and in many measures of policy change.

It sums up a profoundly important message about policy change: we find a huge number of very small changes, and a very small number of huge changes. Compare the distribution of values in this image with the ‘normal distribution’ (the dotted line). It shows a ‘leptokurtic’ distribution, with most values deviating minimally from the mean (and the mean change in each budget item is small), but with a high number of ‘outliers’.

The image helps sum up a key aim of PET, to measure and try to explain long periods of policymaking stability, and policy continuity, disrupted by short but intense periods of instability and change. One explanation relates to ‘bounded rationality’: policymakers have to ignore almost all issues while paying attention to some. The lack of ‘macropolitical’ attention to most issues helps explain stability and continuity, while lurches of attention can help explain instability (although attention can fade before ‘institutions’ feel the need to respond).

Here I am, pointing at this graph:

4. The advocacy coalition framework ‘flow diagram’

ACF diagram

The ACF presents an ambitious image of the policy process, in which we zoom out to consider how key elements fit together in a process containing many actors and levels of government. Like many policy theories, it situates most of the ‘action’ in policy networks or subsystems, showing that some issues involve intensely politicized disputes containing many actors while others are treated as technical and processed routinely, largely by policy specialists, out of the public spotlight.

The ACF suggests that people get into politics to turn their beliefs into policy, form coalitions with people who share their beliefs, and compete with coalitions of actors who share different beliefs. This competition takes place in a policy subsystem, in which coalitions understand new evidence through the lens of their beliefs, and exercise power to make sure that their interpretation is accepted. The other boxes describe the factors – the ‘parameters’ likely to be stable during the 10-year period of study, the partial sources of potential ‘shocks’ to the subsystem, and the need and ability of key actors to form consensus for policy change (particularly in political systems with PR elections) – which constrain and facilitate coalition action.

5. What do we need from a new image?

I recommend an image that consolidates or synthesises existing knowledge and insights. It is tempting to produce something that purports to be ‘new’ but, as with ‘new’ concepts or ‘new’ policy theories, how could we accumulate insights if everyone simply declared novelty and rejected the science of the past?

For me, the novelty should be in the presentation of the image, to help people pick up and use a wealth of policy studies which try to capture two key dynamics:

  1. Policy choice despite uncertainty and ambiguity.

Policymakers can only pay attention to a tiny proportion of issues. They use ‘rational’ and ‘irrational’ cognitive shortcuts to make decisions quickly, despite their limited knowledge of the world, and the possibility to understand policy problems from many perspectives.

  1. A policy environment which constrains and facilitates choice.

Such environments are made up of:

  1. Actors (individuals and organisations) influencing policy at many levels and types of government
  2. Institutions: a proliferation of rules and norms followed by different levels or types of government
  3. Networks: relationships between policymakers and influencers
  4. Ideas: a tendency for certain beliefs or ‘paradigms’ to dominate discussion
  5. Context and events: economic, social, demographic, and technological conditions provide the context for policy choice, and routine/ unpredictable events can prompt policymaker attention to lurch at short notice.

The implications of both dynamics are fairly easy to describe in tables (for example, while describing MSA) and to cobble together quickly in a SmartArt picture:

Cairney 2017 image of the policy process

However, note at least three issues with such a visual presentation:

  1. Do we put policymakers and choice at the centre? If so, it could suggest (a bit like the policy cycle) that a small number of key actors are at the ‘centre’ of the process, when we might prefer to show that their environment, or the interaction between many actors, is more important.
  2. Do we show only the policy process or relate it to the ‘outside world’?
  3. There are many overlaps between concepts. For example, we seek to describe the use and reproduction of rules in ‘institutions’ and ‘networks’, while those rules relate strongly to ‘ideas’. Further, ‘networks’ could sum up ‘actors interacting in many levels/ types of government’. So, ideally, we’d have overlapping shapes to denote overlapping relationships and understandings, but it would really mess up the simplicity of the image.

Of course, the bigger issue is that the image I provide is really just a vehicle to put text on a screen (in the hope that it will be shared). At best it says ‘note these concepts’. It does not show causal relationships. It does not describe any substantial interaction between the concepts to show cause and effect (such as, event A prompted policy choice B).

However, if we tried to bring in that level of detail, I think we would quickly end up with the messy process already described in relation to the policy cycle. Or, we would need to provide a more specific and less generally applicable model of policymaking.

So, right now, this image is a statement of intent. I want to produce something better, but don’t yet know what ‘better’ looks like. There is no ‘general theory’ of policymaking, so can we have a general image? Or, like ‘what is policy?’ discussions, do we produce an answer largely to raise more questions?

___

Here I am, looking remarkably pleased with my SmartArt skills

 

6 Comments

Filed under public policy

How do we get governments to make better decisions?

This is a guest post by Chris Koski (left) and Sam Workman (right), discussing how to use insights from punctuated equilibrium theory to reform government policy making. The full paper has been submitted to the series for Policy and Politics called Practical Lessons from Policy Theories.

Koski Workman

Many people assume that the main problem faced by governments is an information deficit. However, the opposite is true. A surfeit of information exists and institutions have a hard time managing it.  At the same time, all the information that exists in defining problems may be insufficient. Institutions need to develop a capacity to seek out better quality information too.

Institutions, from the national government, to state legislatures, to city councils – try to solve the information processing dilemma by delegating authority to smaller subgroups. Delegation increases the information processing capacity of governments by involving more actors to attend to narrower issues.

The delegation of authority is ultimately a delegation of attention. It solves the ‘flow’ problem, but also introduces new ‘filters’.  The preferences, interests, and modes of information search all influence the process. Even narrowly focused smaller organizations face limitations in their capacity to search and are subject to similar forces as the governments which created them – filters for the deluge of information and capacity limitations for information seeking.

Organizational design predisposes institutions to filter information for ideas that support status quo problem definitions – that is, definitions that existed at the time of delegation – and to seek out information based on these status quo understandings.  As a result, despite a desire to expand attention and information processing to adapt to changes in problem characteristics, most institutions look for information that supports their identity.  Institutional problem definitions stay the same even as the problems change.

Governments eventually face trade-offs between the gains made from delegating decision-making to smaller subgroups and the losses associated with coordinating the information generated by those subgroups.

Governments get stuck in the same ruts as when the delegation process started: status quo bias that doesn’t adjust with change problem conditions.  There is a sense among citizens and academics that governments make bad decisions in part because they respond to problems of today with the policies of 10 years ago.  Government solutions look like hammers in search of nails when they ought to look more like contractors or even urban planners.

Governments should not respond simply by centralizing

When institutions become stultified in their problem definitions, policymakers and citizens often misdiagnose the problem as entirely a coordination problem.  The logic here is that a small group of actors have captured policymaking and are using such capture for their own gain.  This understanding may be true, or may not, but it leads to the “centralization as savior” fallacy.  The idea here is that organizations with broader latitude will be better able to receive a wider variety of information from a broader range of sources.

There are two problems with this strategy.  First, centralization might guarantee an outcome, but at the expense of an honest problems search and, likely, at the expense of what we might call policy stability.  Second, centralization may offer the opportunity for a broader array of information to bear on policy decisions, but, in practice will rely on even narrower information filters given the number of issues to which the newly centralized policymaking forum must attend.

More delegation produces fragmentation

The alternative, more delegation, has significant coordination challenges as we find bottlenecks of attention when multiple subsystems bear on decision-points.  Also, simply delegating authority can predispose subsystems to a particular solution, which we want to avoid.

We’d propose: Adaptive governance

  • Design institutions not just to attend to problems, but to be specifically information seeking. For example, NEPA requires that all US federal decision-making regarding the environment undergo some kind of environmental assessment – this can be as simply as saying “the environmental will not be harmed” or as complex as an environmental impact statement.  At the same time, we’d suggest greater coordination of institutional actions – enhance communication across delegated units but also better feedback mechanisms to overarching institutions.
  • Institutions need to listen to the signals that their delegated units give them. When delegated institutions come to similar conclusions regarding similar problems, these are key signals to broader policymaking bodies.  Listening to signals from multiple delegated units allows for expertise to shine.  At the same time, disharmony across delegated units on the same problems is a good indicator of disharmony in information search.  Sometimes institutions respond to this disharmony by attempting to reduce participation in the policy process or cast outliers as simply outliers.  We think this is a bad idea as it exaggerates the acceptability of the status quo.
  • We propose ‘issue bundling’ which allows for issues to be less tied up by monolithic problem definitions. Policymaking institutions ought to formally direct delegated institutions to look at the same problem relying upon different expertise.  Examples here are climate change or critical infrastructure protection.  To create institutions to deal with these issues is a challenge given the wide range of information necessary to address each.  Institutions can solve the attention problems that emerge from the multiple sources by creating specific channels of information.  This allows for multiple subsystems  – e.g. Agriculture, Transportation, or Environmental Protection – to assist institutional decision-making by sorting issue specific – e.g. Climate Change – information.

Our solutions do solve fundamental problems of information processing in terms of sorting and seeking information – such problems are fundamental to humans and human-created organizations.  However, while governments may be predisposed to prioritize decisions over information, we are optimistic that our recommendations can facilitate better informed policy in the future.

2 Comments

Filed under agenda setting, public policy

Policy bubbles and emotional policymaking

I am at a workshop today on policy ‘bubbles’, or (real and perceived) disproportionate policy responses. For Moshe Maor, a bubble describes an over-reaction to a problem, and a negative policy bubble describes under-reaction.

For Maor, this focus on bubbles is one way into our increasing focus on the role of emotion in policymaking: we pay disproportionate attention to problems, and try to solve some but not others, based on the ways in which we engage emotionally with information.

This focus on psychology is, I think, gaining a lot of traction in political science now, and I think it is crucial to explaining, for example, processes associated with ‘evidence-based policymaking’.

In taking this agenda forward, there remain some outstanding issues:

How much of the psychology literature is already reflected in policy studies? For example, see the social construction of target populations (emotion-driven treatment of social groups), ACF (on loss aversion and the devil shift), and the NPF (telling stories to exploit cognitive biases).

What insights remain untapped from key fields such as organisational psychology? I’ll say more about this in a forthcoming post.

How can we study the psychology of policymaking? Most policy theory begins with some reference to bounded rationality, including PET and the identification of disproportionate information processing (policymakers pay disproportionate attention to some issues and ignore the rest). It is largely deductive then empirical: we make some logical steps about the implications of bounded rationality, then study the process in that light.

Similarly, I think most studies of emotion/ policymaking take insights from psychology (e.g. people value losses more than gains, or they make moral judgements then seek evidence to justify them) and then apply them indirectly to policymaking (asking, for example, what is the effect of prospect theory on the behaviour of coalitions).

Can we do more, by studying more directly the actions of policymakers rather than merely interpreting their actions? The problem, of course, is that few policymakers may be keen on engaging in the types of study (e.g. experiments with control groups) that psychologists have used to establish things like fluency effects.

How does policymaker psychology fit into broader explanations of policymaking? The psychology of policymakers is one part of the story. The other is the system or environment in which they operate. So, we have some choices to make about future studies. Some might ‘zoom in’ to focus on emotionally-driven policymaking in key actors, perhaps at the centre of government.

Others may ‘zoom out’. The latter may involve ascribing the same basic thought processes to a large number of actors, examining that process at a relatively abstract level. This is the necessary consequence of trying to account for the effects of a very large number of actors, and to take into account the role of a policymaking environment, only some of which is in the control of policymakers.

Can we really demonstrate disproportionate policy action? The idea of a proportionate policy response interests me, because I think it is always in the eye of the beholder. We make moral and other personal evaluative statements when we describe a proportionate solution in relation to the size of the problem.

For example, in tobacco policy, a well-established argument in public health is that a proportionate policy response to the health effects of smoking and passive smoking (a) has been 20-30 years behind the evidence in ‘leading countries’, and (b) has yet to happen in ‘laggard’ countries. The counterargument is that the identification of a problem does not necessitate the favoured public health solution (comprehensive tobacco control, towards the ‘endgame’ of zero smoking) because it involves major limits to personal liberties and choice.

Is emotion-driven policymaking necessarily a bad thing?

[excerpt from my 2014 PSA paper ] This is partly the focus of Alter and Oppenheimer (2008) when they argue that policymakers spend disproportionate amounts of money on risks with which they are familiar, at the expense of spending money on things with more negative effects, producing a ‘dramatic misallocation of funds’. They draw on Sunstein (2002), who suggests that emotional bases for attention to environmental problems from the 1970s prompted many regulations to be disproportionate to the risk involved. Further, Slovic’s work suggest that people’s feelings towards risk may even be influenced by the way in which it is described, for example as a percentage versus a 1 in X probability (Slovic, P. 2010: xxii).

Haidt (2001: 815) argues that a focus on psychology can be used to improve policymaking: the identification of the ‘intuitive basis of moral judgment’ can be used to help policymakers ‘avoid mistakes’ or allow people to develop ‘programs’ or an ‘environment’ to ‘improve the quality of moral judgment and behavior’. Similarly, Alter and Oppenheimer (2009: 232) worry about medical and legal judgements swayed by fluid diagnoses and stories.

These studies compare with arguments focusing on the positive role of emotions of decision-making, either individually (see Constantinescu, 2012, drawing on Frank, 1988 and Elster, 2000 on the decisions of judges) or as part of social groups, with emotional responses providing useful information in the form of social cues (Van Kleef et al, 2010).

Policy theory does not shy away from these issues. For example, Schneider and Ingram (2014) argue that the outcomes of social construction are often dysfunctional and not based on a well-reasoned, goal-oriented strategy: ‘Studies have shown that rules, tools, rationales and implementation structures inspired by social constructions send dysfunctional messages and poor choices may hamper the effectiveness of policy’. However, part of the value of policy theory is to show that policy results from the interaction of large numbers of people and institutions. So, the poor actions of one policymaker would not be the issue; we need to know more about the cumulative effect of individual emotional decision making in collective decision-making – not only in discrete organisations, but also networks and systems.

And finally: if it is a bad thing, should we do something about it?

Our choice is to find it interesting then go home (this might appeal to the academics) or try to limit the damage/ maximise the benefits of policymaker psychology to policy and society (this might appeal to practitioners). There is no obvious way to do something, though, is there?

2 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), public policy

The Psychology of Evidence Based Policymaking: Who Will Speak For the Evidence if it Doesn’t Speak for Itself?

Let’s begin with a simple – and deliberately naïve – prescription for evidence based policymaking (EBPM): there should be a much closer link between (a) the process in which scientists and knowledge brokers identify major policy problems, and (b) the process in which politicians make policy decisions. We should seek to close the ‘evidence-policy gap’. The evidence should come first and we should bemoan the inability of policymakers to act accordingly. I discuss why that argument is naïve here and here, but in terms of the complexity of policy processes and the competing claims for knowledge-based policy. This post is about the link between EBPM and psychology.

Let’s consider the role of two types of thought process common to all people, policymakers included: (a) the intuitive, gut, emotional or other heuristics we use to process and act on information quickly; and (b) goal-oriented and reasoned, thoughtful behaviour. As Daniel Kahneman’s Thinking, Fast and Slow (p 20) puts it: ‘System 1 operates automatically and quickly, with little or no effort and no sense of voluntary control. System 2 allocates attention to the effortful mental activities that demand it, including complex computations … often associated with the subjective experience of agency, choice and concentration’

The naïve description of EBPM requires System 2 (‘slow’) thinking, but what happens if most policymaking is characterised by System 1 (‘fast’)? The answer is ‘a whole bunch of cognitive shortcuts’, including:

  • the ‘availability heuristic’, when people relate the size, frequency or probability of a problem to how easy it is to remember or imagine
  • the ‘representativeness heuristic’, when people overestimate the probability of vivid events
  • ‘prospect theory’, when people value losses more than equivalent gains
  • ‘framing effects’, based on emotional and moral judgements
  • confirmation bias
  • optimism bias, or unrealistic expectations about our aims working out well when we commit to them
  • status quo bias
  • a tendency to use exemplars of social groups to represent general experience; and
  • a ‘need for coherence’ and to establish patterns and causal relationships when they may not exist (see Paul Lewis, p 7).

The ‘availability heuristic’ may also be linked to more recent studies of ‘processing fluency’ – which suggests that people’s decisions are influenced by their familiarity with things; with the ease in which they process information (see Alter and Oppenheimer, 2009). Fluency can take several forms, including conceptual, perceptual, and linguistic. For example, people may pay more attention to an issue or statement if they already possess some knowledge of it and find it easy to understand or recall. They may pay attention to people when their faces seem familiar and find fewer faults with systems they comprehend. They may place more value on things they find familiar, such as their domestic currency, items that they own compared to items they would have to buy, or the stocks of companies with more pronounceable names – even if they are otherwise identical. Or, their ability to imagine things in an abstract or concrete form may relate to their psychological ‘distance’ from it.

Is fast thinking bad thinking? Views from psychology

Alter and Oppenheimer use these insights to warn policymakers against taking the wrong attitude to regulation or spending based on flawed assessments of risk – for example, they might spend disproportionate amounts of money on projects designed to address risks with which they are most familiar (Slovic suggests that feelings towards risk may even be influenced by the way in which it is described, for example as a percentage versus a 1 in X probability). Alter and Oppenheimer also worry about medical and legal judgements swayed by fluid diagnoses and stories. Haidt argues that the identification of the ‘intuitive basis of moral judgment’ can be used to help policymakers ‘avoid mistakes’ or allow people to develop ‘programs’ or an ‘environment’ to ‘improve the quality of moral judgment and behavior’. These studies compare with arguments focusing on the positive role of emotions of decision-making, either individually (Frank) or as part of social groups, with emotional responses providing useful information in the form of social cues (Van Kleef et al).

Is fast thinking bad thinking? Views from the political and policy sciences

Social Construction Theory suggests that policymakers make quick, biased, emotional judgements, then back up their actions with selective facts to ‘institutionalize’ their understanding of a policy problem and its solution. They ‘socially construct’ their target populations to argue that they are deserving either of governmental benefits or punishments. Schneider and Ingram (forthcoming) argue that the outcomes of social construction are often dysfunctional and not based on a well-reasoned, goal-oriented strategy: ‘Studies have shown that rules, tools, rationales and implementation structures inspired by social constructions send dysfunctional messages and poor choices may hamper the effectiveness of policy’.

However, not all policy scholars make such normative pronouncements. Indeed, the value of policy theory is often to show that policy results from the interaction between large numbers of people and institutions. So, the actions of a small number of policymakers would not be the issue; we need to know more about the cumulative effect of individual emotional decision making in a collective decision-making environment – in organisations, networks and systems. For example:

  • The Advocacy Coalition Framework suggests that people engage in coordinated activity to cooperate with each other and compete with other coalitions, based on their shared beliefs and a tendency to demonise their opponents. In some cases, there are commonly accepted ways to interpret the evidence. In others, it is a battle of ideas.
  • Multiple Streams Analysis and Punctuated Equilibrium Theory focus on uncertainty and ambiguity, exploring the potential for policymaker attention to lurch dramatically from one problem or ‘image’ (the way the problem is viewed or understood). They identify the framing strategies – of actors such as ‘entrepreneurs’, ‘venue shoppers’ and ‘monopolists’ – based on a mixture of empirical facts and ‘emotional appeals’.
  • The Narrative Policy Framework combines a discussion of emotion with the identification of narrative strategies. Each narrative has a setting, characters, plot and moral. They can be compared to marketing, as persuasion based more on appealing to an audience’s beliefs (or exploiting their thought processes) than the evidence. People will pay attention to certain narratives because they are boundedly rational, seeking shortcuts to gather sufficient information – and prone to accept simple stories that seem plausible, confirm their biases, exploit their emotions, and/ or come from a source they trust.

In each case, we might see our aim as going beyond the simple phrase: ‘the evidence doesn’t speak for itself’. If ‘fast thinking’ is pervasive in policymaking, then ‘the evidence’ may only be influential if it can be provided in ways that are consistent with the thought processes of certain policymakers – such as by provoking a strong emotional reaction (to confirm or challenge biases), or framing messages in terms that are familiar to (and can be easily processed by) policymakers.

These issues are discussed further in these posts:

Is Evidence-Based Policymaking the same as good policymaking?

Policy Concepts in 1000 Words: The Psychology of Policymaking

And at more length in these papers:

PSA 2014 Cairney Psychology Policymaking 7.4.14

Cairney PSA 2014 EBPM 5.3.14

See also: Joseph Rowntree Foundation, Evidence alone won’t bring about social change

Discover Society (Delaney and Henderson) Risk and Choice in the Scottish Independence debate

4 Comments

Filed under Evidence Based Policymaking (EBPM), public policy