Category Archives: agenda setting

Taking lessons from policy theory into practice: 3 examples

Notes for ANZSOG/ ANU Crawford School/ UNSW Canberra workshop. Powerpoint here. The recording of the lecture (skip to 2m30) and Q&A is here (right click to download mp3 or dropbox link):

The context for this workshop is the idea that policy theories could be more helpful to policymakers/ practitioners if we could all communicate more effectively with each other. Academics draw general and relatively abstract conclusions from multiple cases. Practitioners draw very similar conclusions from rich descriptions of direct experience in a smaller number of cases. How can we bring together their insights and use a language that we all understand? Or, more ambitiously, how can we use policy theory-based insights to inform the early career development training that civil servants and researchers receive?

The first step is to translate policy theories into a non-technical language by trying to speak with an audience beyond our immediate peers (see for example Practical Lessons from Policy Theories).

However, translation is not enough. A second crucial step is to consider how policymakers and practitioners are likely to make sense of theoretical insights when they apply them to particular aims or responsibilities. For example:

  1. Central government policymakers may accept the descriptive accuracy of policy theories emphasising limited central control, but not the recommendation that they should let go, share power, and describe their limits to the public.
  2. Scientists may accept key limitations to ‘evidence based policymaking’ but reject the idea that they should respond by becoming better storytellers or more manipulative operators.
  3. Researchers and practitioners struggle to resolve hard choices when combining evidence and ‘coproduction’ while ‘scaling up’ policy interventions. Evidence choice is political choice. Can we do more than merely encourage people to accept this point?

I discuss these examples below because they are closest to my heart (especially example 1). Note throughout that I am presenting one interpretation about: (1) the most promising insights, and (2) their implications for practice. Other interpretations of the literature and its implications are available. They are just a bit harder to find.

Example 1: the policy cycle endures despite its descriptive inaccuracy

cycle

The policy cycle does not describe and explain the policy process well:

  • If we insist on keeping the cycle metaphor, it is more accurate to see the process as a huge set of policy cycles that connect with each other in messy and unpredictable ways.
  • The cycle approach also links strongly to the idea of ‘comprehensive rationality’ in which a small group of policymakers and analysts are in full possession of the facts and full control of the policy process. They carry out their aims through a series of stages.

Policy theories provide more descriptive and explanatory usefulness. Their insights include:

  • Limited choice. Policymakers inherit organisations, rules, and choices. Most ‘new’ choice is a revision of the old.
  • Limited attention. Policymakers must ignore almost all of the policy problems for which they are formally responsible. They pay attention to some, and delegate most responsibility to civil servants. Bureaucrats rely on other actors for information and advice, and they build relationships on trust and information exchange.
  • Limited central control. Policy may appear to be made at the ‘top’ or in the ‘centre’, but in practice policymaking responsibility is spread across many levels and types of government (many ‘centres’). ‘Street level’ actors make policy as they deliver. Policy outcomes appear to ‘emerge’ locally despite central government attempts to control their fate.
  • Limited policy change. Most policy change is minor, made and influenced by actors who interpret new evidence through the lens of their beliefs. Well-established beliefs limit the opportunities of new solutions. Governments tend to rely on trial-and-error, based on previous agreements, rather than radical policy change based on a new agenda. New solutions succeed only during brief and infrequent windows of opportunity.

However, the cycle metaphor endures because:

  • It provides a simple model of policymaking with stages that map onto important policymaking functions.
  • It provides a way to project policymaking to the public. You know how we make policy, and that we are in charge, so you know who to hold to account.

In that context, we may want to be pragmatic about our advice:

  1. One option is via complexity theory, in which scholars generally encourage policymakers to accept and describe their limits:
  • Accept routine error, reduce short-term performance management, engage more in trial and error, and ‘let go’ to allow local actors the flexibility to adapt and respond to their context.
  • However, would a government in the Westminster tradition really embrace this advice? No. They need to balance (a) pragmatic policymaking, and (b) an image of governing competence.
  1. Another option is to try to help improve an existing approach.

Further reading (blog posts):

The language of complexity does not mix well with the language of Westminster-style accountability

Making Sense of Policymaking: why it’s always someone else’s fault and nothing ever changes

Two stories of British politics: the Westminster model versus Complex Government

Example 2: how to deal with a lack of ‘evidence based policymaking’

I used to read many papers on tobacco policy, with the same basic message: we have the evidence of tobacco harm, and evidence of which solutions work, but there is an evidence-policy gap caused by too-powerful tobacco companies, low political will, and pathological policymaking. These accounts are not informed by theories of policymaking.

I then read Oliver et al’s paper on the lack of policy theory in health/ environmental scholarship on the ‘barriers’ to the use of evidence in policy. Very few articles rely on policy concepts, and most of the few rely on the policy cycle. This lack of policy theory is clear in their description of possible solutions – better communication, networking, timing, and more science literacy in government – which does not describe well the need to respond to policymaker psychology and a complex policymaking environment.

So, I wrote The Politics of Evidence-Based Policymaking and one zillion blog posts to help identify the ways in which policy theories could help explain the relationship between evidence and policy.

Since then, the highest demand to speak about the book has come from government/ public servant, NGO, and scientific audiences outside my discipline. The feedback is generally that: (a) the book’s description sums up their experience of engagement with the policy process, and (b) maybe it opens up discussion about how to engage more effectively.

But how exactly do we turn empirical descriptions of policymaking into practical advice?

For example, scientist/ researcher audiences want to know the answer to a question like: Why don’t policymakers listen to your evidence? and so I focus on three conversation starters:

  1. they have a broader view on what counts as good evidence (see ANZSOG description)
  2. they have to ignore almost all information (a nice way into bounded rationality and policymaker psychology)
  3. they do not understand or control the process in which they seek to use evidence (a way into ‘the policy process’)

Cairney 2017 image of the policy process

We can then consider many possible responses in the sequel What can you do when policymakers ignore your evidence?

Examples include:

  • ‘How to do it’ advice. I compare tips for individuals (from experienced practitioners) with tips based on policy concepts. They are quite similar-looking tips – e.g. find out where the action is, learn the rules, tell good stories, engage allies, seek windows of opportunity – but I describe mine as 5 impossible tasks!
  • Organisational reform. I describe work with the European Commission Joint Research Centre to identify 8 skills or functions of an organisation bringing together the supply/demand of knowledge.
  • Ethical dilemmas. I use key policy theories to ask people how far they want to go to privilege evidence in policy. It’s fun to talk about these things with the type of scientist who sees any form of storytelling as manipulation.

Further reading:

Is Evidence-Based Policymaking the same as good policymaking?

A 5-step strategy to make evidence count

Political science improves our understanding of evidence-based policymaking, but does it produce better advice?

Principles of science advice to government: key problems and feasible solutions

Example 3: how to encourage realistic evidence-informed policy transfer

This focus on EBPM is useful context for discussions of ‘policy learning’ and ‘policy transfer’, and it was the focus of my ANZOG talk entitled (rather ambitiously) ‘teaching evidence-based policy to fly’.

I’ve taken a personal interest in this one because I’m part of a project – called IMAJINE – in which we have to combine academic theory and practical responses. We are trying to share policy solutions across Europe rather than explain why few people share them!

For me, the context is potentially overwhelming:

So, when we start to focus on sharing lessons, we will have three things to discover:

  1. What is the evidence for success, and from where does it come? Governments often project success without backing it up.
  2. What story do policymakers tell about the problem they are trying to solve, the solutions they produced, and why? Two different governments may be framing and trying to solve the same problem in very different ways.
  3. Was the policy introduced in a comparable policymaking system? People tend to focus on political system comparability (e.g. is it unitary or federal?), but I think the key is in policymaking system comparability (e.g. what are the rules and dominant ideas?).

To be honest, when one of our external assessors asked me how well I thought I would do, we both smiled because the answer may be ‘not very’. In other words, the most practical lesson may be the hardest to take, although I find it comforting: the literature suggests that policymakers might ignore you for 20 years then suddenly become very (but briefly) interested in your work.

 

The slides are a bit wonky because I combined my old ppt to the Scottish Government with a new one for UNSW Paul Cairney ANU Policy practical 22 October 2018

I wanted to compare how I describe things to (1) civil servants (2) practitioners/ researcher (3) me, but who has the time/ desire to listen to 3 powerpoints in one go? If the answer is you, let me know and we’ll set up a Zoom call.

2 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), IMAJINE, Policy learning and transfer

Managing expectations about the use of evidence in policy

Notes for the #transformURE event hosted by Nuffield, 25th September 2018

I like to think that I can talk with authority on two topics that, much like a bottle of Pepsi and a pack of Mentos, you should generally keep separate:

  1. When talking at events on the use of evidence in policy, I say that you need to understand the nature of policy and policymaking to understand the role of evidence in it.
  2. When talking with students, we begin with the classic questions ‘what is policy?’ and ‘what is the policy process’, and I declare that we don’t know the answer. We define policy to show the problems with all definitions of policy, and we discuss many models and theories that only capture one part of the process. There is no ‘general theory’ of policymaking.

The problem, when you put together those statements, is that you need to understand the role of evidence within a policy process that we don’t really understand.

It’s an OK conclusion if you just want to declare that the world is complicated, but not if you seek ways to change it or operate more effectively within it.

Put less gloomily:

  • We have ways to understand key parts of the policy process. They are not ready-made to help us understand evidence use, but we can use them intelligently.
  • Most policy theories exist to explain policy dynamics, not to help us adapt effectively to them, but we can derive general lessons with often-profound implications.

Put even less gloomily, it is not too difficult to extract/ synthesise key insights from policy theories, explain their relevance, and use them to inform discussions about how to promote your preferred form of evidence use.

The only remaining problem is that, although the resultant advice looks quite straightforward, it is far easier said than done. The proposed actions are more akin to the Labours of Hercules than [PAC: insert reference to something easier].

They include:

  1. Find out where the ‘action’ is, so that you can find the right audience for your evidence. Why? There are many policymakers and influencers spread across many levels and types of government.
  2. Learn and follow the ‘rules of the game’. Why? Each policymaking venue has its own rules of engagement and evidence gathering, and the rules are often informal and unwritten.
  3. Gain access to ‘policy networks’. Why? Most policy is processed at a low level of government, beyond the public spotlight, between relatively small groups of policymakers and influencers. They build up trust as they work together, learning who is reliable and authoritative, and converging on how to use evidence to understand the nature and solution to policy problems.
  4. Learn the language. Why? Each venue has its own language to reflect dominant ideas, beliefs, or ways to understand a policy problem. In some arenas, there is a strong respect for a ‘hierarchy’ of evidence. In others, they key reference point may be value for money. In some cases, the language reflects the closing-off of some policy solutions (such as redistributing resources from one activity to another).
  5. Exploit windows of opportunity. Why? Events, and changes in socioeconomic conditions, often prompt shifts of attention to policy issues. ‘Policy entrepreneurs’ lie in wait for the right time to exploit a shift in the motive and opportunity of a policymaker to pay attention to and try to solve a problem.

So far so good, until you consider the effort it would take to achieve any of these things: you may need to devote the best part of your career to these tasks with no guarantee of success.

Put more positively, it is better to be equipped with these insights, and to appreciate the limits to our actions, than to think we can use top tips to achieve ‘research impact’ in a more straightforward way.

Kathryn Oliver and I describe these ‘how to’ tips in this post and, in this article in Political Studies Review, use a wider focus on policymaking environments to produce a more realistic sense of what individual researchers – and research-producing organisations – could achieve.

There is some sensible-enough advice out there for individuals – produce good evidence, communicate it well, form relationships with policymakers, be available, and so on – but I would exercise caution when it begins to recommend being ‘entrepreneurial’. The opportunities to be entrepreneurial are not shared equally, most entrepreneurs fail, and we can likely better explain their success with reference to their environment than their skill.

hang-in-there-baby

3 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), public policy, UK politics and policy

What can you do when policymakers ignore your evidence?

The first post in this series asks: Why don’t policymakers listen to your evidence? It is based on talks that I have been giving since 2016, mostly to tap into a common story told by people in my audience (and the ‘science community’ more generally) about a new era in politics: policymakers do not pay sufficient respect to expertise or attention to good quality evidence.

It’s not my story, but I think it’s important to respect my audience members enough to (a) try to engage with their question, before (b) inviting them to think differently about how to ask it, and (c) provide different types of solutions according to the changing nature of the question.

Instead of a really long post for (b) and (c), I’ve made it a bit like Ceefax in which you can choose which question to ask or answer:

  • Why don’t policymakers listen to your evidence? (go to page 154)
  • What can you do when policymakers ignore your evidence? Tips from the ‘how to’ literature from the science community (go to page 650)
  • What can you do when policymakers ignore your evidence? Encourage ‘knowledge management for policy’ (go to page 568)
  • How else can we describe and seek to fill the evidence-policy gap? (go to page 400)
  • How far should you go to privilege evidence? 1. Evidence and governance principles (go to page 101)
  • How far should you go to privilege evidence? 2. Policy theories, scenarios, and ethical dilemmas (go to page 526)
  • How far should you go to privilege evidence? 3. Use psychological insights to manipulate policymakers (go to page 300 then scroll down to point 3)

Some of this material will appear in work with Dr Kathryn Oliver and with my co-authors on a forthcoming report for Enlightenment 2.0 

See also:

Evidence-based policymaking: political strategies for scientists living in the real world

The Science of Evidence-based Policymaking: How to Be Heard

Evidence based policymaking: 7 key themes

I also do slides, such as:

Paul Cairney FUSE May 2018 

Paul Cairney Victoria May 2018

The Politics of Evidence-Based Policymaking: ANZSOG talks 

Talks and blogs: ANZSOG and beyond

This is me presenting those slides in Cambridge while being very Scottish, enjoying a too-heavy cold, and sucking a lozenge. Please note that I tend to smile a lot and make many sarcastic jokes while presenting, partly to apologise indirectly for all the self-publicity.

12 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), Psychology Based Policy Studies, public policy

Policy in 500 words: uncertainty versus ambiguity

In policy studies, there is a profound difference between uncertainty and ambiguity:

  • Uncertainty describes a lack of knowledge or a worrying lack of confidence in one’s knowledge.
  • Ambiguity describes the ability to entertain more than one interpretation of a policy problem.

Both concepts relate to ‘bounded rationality’: policymakers do not have the ability to process all information relevant to policy problems. Instead, they employ two kinds of shortcut:

  • ‘Rational’. Pursuing clear goals and prioritizing certain sources of information.
  • ‘Irrational’. Drawing on emotions, gut feelings, deeply held beliefs, and habits.

I make an artificially binary distinction, uncertain versus ambiguous, and relate it to another binary, rational versus irrational, to point out the pitfalls of focusing too much on one aspect of the policy process:

  1. Policy actors seek to resolve uncertainty by generating more information or drawing greater attention to the available information.

Actors can try to solve uncertainty by: (a) improving the quality of evidence, and (b) making sure that there are no major gaps between the supply of and demand for evidence. Relevant debates include: what counts as good evidence?, focusing on the criteria to define scientific evidence and their relationship with other forms of knowledge (such as practitioner experience and service user feedback), and what are the barriers between supply and demand?, focusing on the need for better ways to communicate.

  1. Policy actors seek to resolve ambiguity by focusing on one interpretation of a policy problem at the expense of another.

Actors try to solve ambiguity by exercising power to increase attention to, and support for, their favoured interpretation of a policy problem. You will find many examples of such activity spread across the 500 and 1000 words series:

A focus on reducing uncertainty gives the impression that policymaking is a technical process in which people need to produce the best evidence and deliver it to the right people at the right time.

In contrast, a focus on reducing ambiguity gives the impression of a more complicated and political process in which actors are exercising power to compete for attention and dominance of the policy agenda. Uncertainty matters, but primarily to describe the role of a complex policymaking system in which no actor truly understands where they are or how they should exercise power to maximise their success.

Further reading:

For a longer discussion, see Fostering Evidence-informed Policy Making: Uncertainty Versus Ambiguity (PDF)

Or, if you fancy it in French: Favoriser l’élaboration de politiques publiques fondées sur des données probantes : incertitude versus ambiguïté (PDF)

Framing

The politics of evidence-based policymaking

To Bridge the Divide between Evidence and Policy: Reduce Ambiguity as Much as Uncertainty

How to communicate effectively with policymakers: combine insights from psychology and policy studies

Here is the relevant opening section in UPP:

p234 UPP ambiguity

28 Comments

Filed under 500 words, agenda setting, Evidence Based Policymaking (EBPM), public policy, Storytelling

Evidence based policymaking: 7 key themes

7 themes of EBPM

I looked back at my blog posts on the politics of ‘evidence based policymaking’ and found that I wrote quite a lot (particularly from 2016). Here is a list based on 7 key themes.

1. Use psychological insights to influence the use of evidence

My most-current concern. The same basic theme is that (a) people (including policymakers) are ‘cognitive misers’ seeking ‘rational’ and ‘irrational’ shortcuts to gather information for action, so you won’t get far if you (b) bombard them with information, or (c) call them idiots.

Three ways to communicate more effectively with policymakers (shows how to use psychological insights to promote evidence in policymaking)

Using psychological insights in politics: can we do it without calling our opponents mental, hysterical, or stupid? (yes)

The Psychology of Evidence Based Policymaking: Who Will Speak For the Evidence if it Doesn’t Speak for Itself? (older paper, linking studies of psychology with studies of EBPM)

Older posts on the same theme:

Is there any hope for evidence in emotional debates and chaotic government? (yes)

We are in danger of repeating the same mistakes if we bemoan low attention to ‘facts’

These complaints about ignoring science seem biased and naïve – and too easy to dismiss

How can we close the ‘cultural’ gap between the policymakers and scientists who ‘just don’t get it’?

2. How to use policy process insights to influence the use of evidence

I try to simplify key insights about the policy process to show to use evidence in it. One key message is to give up on the idea of an orderly policy process described by the policy cycle model. What should you do if a far more complicated process exists?

Why don’t policymakers listen to your evidence?

The Politics of Evidence Based Policymaking: 3 messages (3 ways to say that you should engage with the policy process that exists, not a mythical process that will never exist)

Three habits of successful policy entrepreneurs (shows how entrepreneurs are influential in politics)

Why doesn’t evidence win the day in policy and policymaking? and What does it take to turn scientific evidence into policy? Lessons for illegal drugs from tobacco and There is no blueprint for evidence-based policy, so what do you do? (3 posts describing the conditions that must be met for evidence to ‘win the day’)

Writing for Impact: what you need to know, and 5 ways to know it (explains how our knowledge of the policy process helps communicate to policymakers)

How can political actors take into account the limitations of evidence-based policy-making? 5 key points (presentation to European Parliament-European University Institute ‘Policy Roundtable’ 2016)

Evidence Based Policy Making: 5 things you need to know and do (presentation to Open Society Foundations New York 2016)

What 10 questions should we put to evidence for policy experts? (part of a series of videos produced by the European Commission)

3. How to combine principles on ‘good evidence’, ‘good governance’, and ‘good practice’

My argument here is that EBPM is about deciding at the same time what is: (1) good evidence, and (2) a good way to make and deliver policy. If you just focus on one at a time – or consider one while ignoring the other – you cannot produce a defendable way to promote evidence-informed policy delivery.

Kathryn Oliver and I have just published an article on the relationship between evidence and policy (summary of and link to our article on this very topic)

We all want ‘evidence based policy making’ but how do we do it? (presentation to the Scottish Government on 2016)

The ‘Scottish Approach to Policy Making’: Implications for Public Service Delivery

The politics of evidence-based best practice: 4 messages

The politics of implementing evidence-based policies

Policy Concepts in 1000 Words: the intersection between evidence and policy transfer

Key issues in evidence-based policymaking: comparability, control, and centralisation

The politics of evidence and randomised control trials: the symbolic importance of family nurse partnerships

What Works (in a complex policymaking system)?

How Far Should You Go to Make Sure a Policy is Delivered?

4. Face up to your need to make profound choices to pursue EBPM

These posts have arisen largely from my attendance at academic-practitioner conferences on evidence and policy. Many participants tell the same story about the primacy of scientific evidence challenged by post-truth politics and emotional policymakers. I don’t find this argument convincing or useful. So, in many posts, I challenge these participants to think about more pragmatic ways to sum up and do something effective about their predicament.

Political science improves our understanding of evidence-based policymaking, but does it produce better advice? (shows how our knowledge of policymaking clarifies dilemmas about engagement)

The role of ‘standards for evidence’ in ‘evidence informed policymaking’ (argues that a strict adherence to scientific principles may help you become a good researcher but not an effective policy influencer)

How far should you go to secure academic ‘impact’ in policymaking? From ‘honest brokers’ to ‘research purists’ and Machiavellian manipulators (you have to make profound ethical and strategic choices when seeking to maximise the use of evidence in policy)

Principles of science advice to government: key problems and feasible solutions (calling yourself an ‘honest broker’ while complaining about ‘post-truth politics’ is a cop out)

What sciences count in government science advice? (political science, obvs)

I know my audience, but does my other audience know I know my audience? (compares the often profoundly different ways in which scientists and political scientists understand and evaluate EBPM – this matters because, for example, we rarely discuss power in scientist-led debates)

Is Evidence-Based Policymaking the same as good policymaking? (no)

Idealism versus pragmatism in politics and policymaking: … evidence-based policymaking (how to decide between idealism and pragmatism when engaging in politics)

Realistic ‘realist’ reviews: why do you need them and what might they look like? (if you privilege impact you need to build policy relevance into systematic reviews)

‘Co-producing’ comparative policy research: how far should we go to secure policy impact? (describes ways to build evidence advocacy into research design)

The Politics of Evidence (review of – and link to – Justin Parkhurt’s book on the ‘good governance’ of evidence production and use)

20170512_095446

5. For students and researchers wanting to read/ hear more

These posts are relatively theory-heavy, linking quite clearly to the academic study of public policy. Hopefully they provide a simple way into the policy literature which can, at times, be dense and jargony.

‘Evidence-based Policymaking’ and the Study of Public Policy

Policy Concepts in 1000 Words: ‘Evidence Based Policymaking’

Practical Lessons from Policy Theories (series of posts on the policy process, offering potential lessons for advocates of evidence use in policy)

Writing a policy paper and blog post 

12 things to know about studying public policy

Can you want evidence based policymaking if you don’t really know what it is? (defines each word in EBPM)

Can you separate the facts from your beliefs when making policy? (no, very no)

Policy Concepts in 1000 Words: Success and Failure (Evaluation) (using evidence to evaluate policy is inevitably political)

Policy Concepts in 1000 Words: Policy Transfer and Learning (so is learning from the experience of others)

Four obstacles to evidence based policymaking (EBPM)

What is ‘Complex Government’ and what can we do about it? (read about it)

How Can Policy Theory Have an Impact on Policy Making? (on translating policy theories into useful advice)

The role of evidence in UK policymaking after Brexit (argues that many challenges/ opportunities for evidence advocates will not change after Brexit)

Why is there more tobacco control policy than alcohol control policy in the UK? (it’s not just because there is more evidence of harm)

Evidence Based Policy Making: If You Want to Inject More Science into Policymaking You Need to Know the Science of Policymaking and The politics of evidence-based policymaking: focus on ambiguity as much as uncertainty and Revisiting the main ‘barriers’ between evidence and policy: focus on ambiguity, not uncertainty and The barriers to evidence based policymaking in environmental policy (early versions of what became the chapters of the book)

6. Using storytelling to promote evidence use

This is increasingly a big interest for me. Storytelling is key to the effective conduct and communication of scientific research. Let’s not pretend we’re objective people just stating the facts (which is the least convincing story of all). So far, so good, except to say that the evidence on the impact of stories (for policy change advocacy) is limited. The major complication is that (a) the story you want to tell and have people hear interacts with (b) the story that your audience members tell themselves.

Combine Good Evidence and Emotional Stories to Change the World

Storytelling for Policy Change: promise and problems

Is politics and policymaking about sharing evidence and facts or telling good stories? Two very silly examples from #SP16

7. The major difficulties in using evidence for policy to reduce inequalities

These posts show how policymakers think about how to combine (a) often-patchy evidence with (b) their beliefs and (c) an electoral imperative to produce policies on inequalities, prevention, and early intervention. I suggest that it’s better to understand and engage with this process than complain about policy-based-evidence from the side-lines. If you do the latter, policymakers will ignore you.

The UK government’s imaginative use of evidence to make policy 

What do you do when 20% of the population causes 80% of its problems? Possibly nothing.

The theory and practice of evidence-based policy transfer: can we learn how to reduce territorial inequalities?

We need better descriptions than ‘evidence-based policy’ and ‘policy-based evidence’: the case of UK government ‘troubled families’ policy

How can you tell the difference between policy-based-evidence and evidence-based-policymaking?

Early intervention policy, from ‘troubled families’ to ‘named persons’: problems with evidence and framing ‘valence’ issues

Key issues in evidence-based policymaking: comparability, control, and centralisation

The politics of evidence and randomised control trials: the symbolic importance of family nurse partnerships

Two myths about the politics of inequality in Scotland

Social investment, prevention and early intervention: a ‘window of opportunity’ for new ideas?

A ‘decisive shift to prevention’: how do we turn an idea into evidence based policy?

Can the Scottish Government pursue ‘prevention policy’ without independence?

Note: these issues are discussed in similar ways in many countries. One example that caught my eye today:

 

All of this discussion can be found under the EBPM category: https://paulcairney.wordpress.com/category/evidence-based-policymaking-ebpm/T

See also the special issue on maximizing the use of evidence in policy

Palgrave C special

3 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), Prevention policy, public policy, Storytelling, UK politics and policy

Three ways to communicate more effectively with policymakers

By Paul Cairney and Richard Kwiatkowski

Use psychological insights to inform communication strategies

Policymakers cannot pay attention to all of the things for which they are responsible, or understand all of the information they use to make decisions. Like all people, there are limits on what information they can process (Baddeley, 2003; Cowan, 2001, 2010; Miller, 1956; Rock, 2008).

They must use short cuts to gather enough information to make decisions quickly: the ‘rational’, by pursuing clear goals and prioritizing certain kinds of information, and the ‘irrational’, by drawing on emotions, gut feelings, values, beliefs, habits, schemata, scripts, and what is familiar, to make decisions quickly. Unlike most people, they face unusually strong pressures on their cognition and emotion.

Policymakers need to gather information quickly and effectively, often in highly charged political atmospheres, so they develop heuristics to allow them to make what they believe to be good choices. Perhaps their solutions seem to be driven more by their values and emotions than a ‘rational’ analysis of the evidence, often because we hold them to a standard that no human can reach.

If so, and if they have high confidence in their heuristics, they will dismiss criticism from researchers as biased and naïve. Under those circumstances, we suggest that restating the need for ‘rational’ and ‘evidence-based policymaking’ is futile, naively ‘speaking truth to power’ counterproductive, and declaring ‘policy based evidence’ defeatist.

We use psychological insights to recommend a shift in strategy for advocates of the greater use of evidence in policy. The simple recommendation, to adapt to policymakers’ ‘fast thinking’ (Kahneman, 2011) rather than bombard them with evidence in the hope that they will get round to ‘slow thinking’, is already becoming established in evidence-policy studies. However, we provide a more sophisticated understanding of policymaker psychology, to help understand how people think and make decisions as individuals and as part of collective processes. It allows us to (a) combine many relevant psychological principles with policy studies to (b) provide several recommendations for actors seeking to maximise the impact of their evidence.

To ‘show our work’, we first summarise insights from policy studies already drawing on psychology to explain policy process dynamics, and identify key aspects of the psychology literature which show promising areas for future development.

Then, we emphasise the benefit of pragmatic strategies, to develop ways to respond positively to ‘irrational’ policymaking while recognising that the biases we ascribe to policymakers are present in ourselves and our own groups. Instead of bemoaning the irrationality of policymakers, let’s marvel at the heuristics they develop to make quick decisions despite uncertainty. Then, let’s think about how to respond effectively. Instead of identifying only the biases in our competitors, and masking academic examples of group-think, let’s reject our own imagined standards of high-information-led action. This more self-aware and humble approach will help us work more successfully with other actors.

On that basis, we provide three recommendations for actors trying to engage skilfully in the policy process:

  1. Tailor framing strategies to policymaker bias. If people are cognitive misers, minimise the cognitive burden of your presentation. If policymakers combine cognitive and emotive processes, combine facts with emotional appeals. If policymakers make quick choices based on their values and simple moral judgements, tell simple stories with a hero and moral. If policymakers reflect a ‘group emotion’, based on their membership of a coalition with firmly-held beliefs, frame new evidence to be consistent with those beliefs.
  2. Identify ‘windows of opportunity’ to influence individuals and processes. ‘Timing’ can refer to the right time to influence an individual, depending on their current way of thinking, or to act while political conditions are aligned.
  3. Adapt to real-world ‘dysfunctional’ organisations rather than waiting for an orderly process to appear. Form relationships in networks, coalitions, or organisations first, then supply challenging information second. To challenge without establishing trust may be counterproductive.

These tips are designed to produce effective, not manipulative, communicators. They help foster the clearer communication of important policy-relevant evidence, rather than imply that we should bend evidence to manipulate or trick politicians. We argue that it is pragmatic to work on the assumption that people’s beliefs are honestly held, and policymakers believe that their role is to serve a cause greater than themselves. To persuade them to change course requires showing simple respect and seeking ways to secure their trust, rather than simply ‘speaking truth to power’. Effective engagement requires skilful communication and good judgement as much as good evidence.


This is the introduction to our revised and resubmitted paper to the special issue of Palgrave Communications The politics of evidence-based policymaking: how can we maximise the use of evidence in policy? Please get in touch if you are interested in submitting a paper to the series.

Full paper: Cairney Kwiatkowski Palgrave Comms resubmission CLEAN 14.7.17

2 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), Psychology Based Policy Studies, public policy

How do we get governments to make better decisions?

This is a guest post by Chris Koski (left) and Sam Workman (right), discussing how to use insights from punctuated equilibrium theory to reform government policy making. The full paper has been submitted to the series for Policy and Politics called Practical Lessons from Policy Theories.

Koski Workman

Many people assume that the main problem faced by governments is an information deficit. However, the opposite is true. A surfeit of information exists and institutions have a hard time managing it.  At the same time, all the information that exists in defining problems may be insufficient. Institutions need to develop a capacity to seek out better quality information too.

Institutions, from the national government, to state legislatures, to city councils – try to solve the information processing dilemma by delegating authority to smaller subgroups. Delegation increases the information processing capacity of governments by involving more actors to attend to narrower issues.

The delegation of authority is ultimately a delegation of attention. It solves the ‘flow’ problem, but also introduces new ‘filters’.  The preferences, interests, and modes of information search all influence the process. Even narrowly focused smaller organizations face limitations in their capacity to search and are subject to similar forces as the governments which created them – filters for the deluge of information and capacity limitations for information seeking.

Organizational design predisposes institutions to filter information for ideas that support status quo problem definitions – that is, definitions that existed at the time of delegation – and to seek out information based on these status quo understandings.  As a result, despite a desire to expand attention and information processing to adapt to changes in problem characteristics, most institutions look for information that supports their identity.  Institutional problem definitions stay the same even as the problems change.

Governments eventually face trade-offs between the gains made from delegating decision-making to smaller subgroups and the losses associated with coordinating the information generated by those subgroups.

Governments get stuck in the same ruts as when the delegation process started: status quo bias that doesn’t adjust with change problem conditions.  There is a sense among citizens and academics that governments make bad decisions in part because they respond to problems of today with the policies of 10 years ago.  Government solutions look like hammers in search of nails when they ought to look more like contractors or even urban planners.

Governments should not respond simply by centralizing

When institutions become stultified in their problem definitions, policymakers and citizens often misdiagnose the problem as entirely a coordination problem.  The logic here is that a small group of actors have captured policymaking and are using such capture for their own gain.  This understanding may be true, or may not, but it leads to the “centralization as savior” fallacy.  The idea here is that organizations with broader latitude will be better able to receive a wider variety of information from a broader range of sources.

There are two problems with this strategy.  First, centralization might guarantee an outcome, but at the expense of an honest problems search and, likely, at the expense of what we might call policy stability.  Second, centralization may offer the opportunity for a broader array of information to bear on policy decisions, but, in practice will rely on even narrower information filters given the number of issues to which the newly centralized policymaking forum must attend.

More delegation produces fragmentation

The alternative, more delegation, has significant coordination challenges as we find bottlenecks of attention when multiple subsystems bear on decision-points.  Also, simply delegating authority can predispose subsystems to a particular solution, which we want to avoid.

We’d propose: Adaptive governance

  • Design institutions not just to attend to problems, but to be specifically information seeking. For example, NEPA requires that all US federal decision-making regarding the environment undergo some kind of environmental assessment – this can be as simply as saying “the environmental will not be harmed” or as complex as an environmental impact statement.  At the same time, we’d suggest greater coordination of institutional actions – enhance communication across delegated units but also better feedback mechanisms to overarching institutions.
  • Institutions need to listen to the signals that their delegated units give them. When delegated institutions come to similar conclusions regarding similar problems, these are key signals to broader policymaking bodies.  Listening to signals from multiple delegated units allows for expertise to shine.  At the same time, disharmony across delegated units on the same problems is a good indicator of disharmony in information search.  Sometimes institutions respond to this disharmony by attempting to reduce participation in the policy process or cast outliers as simply outliers.  We think this is a bad idea as it exaggerates the acceptability of the status quo.
  • We propose ‘issue bundling’ which allows for issues to be less tied up by monolithic problem definitions. Policymaking institutions ought to formally direct delegated institutions to look at the same problem relying upon different expertise.  Examples here are climate change or critical infrastructure protection.  To create institutions to deal with these issues is a challenge given the wide range of information necessary to address each.  Institutions can solve the attention problems that emerge from the multiple sources by creating specific channels of information.  This allows for multiple subsystems  – e.g. Agriculture, Transportation, or Environmental Protection – to assist institutional decision-making by sorting issue specific – e.g. Climate Change – information.

Our solutions do solve fundamental problems of information processing in terms of sorting and seeking information – such problems are fundamental to humans and human-created organizations.  However, while governments may be predisposed to prioritize decisions over information, we are optimistic that our recommendations can facilitate better informed policy in the future.

2 Comments

Filed under agenda setting, public policy

Policy in 500 Words: Multiple Streams Analysis and Policy Entrepreneurs

NASA launch

In a previous post, I ask: if the policy cycle does not exist, what do we do? In this artificial policy cycle world, ‘comprehensively rational’ policymakers combine their values with evidence to define policy problems and their aims, ‘neutral’ bureaucracies produce many possible solutions consistent with those aims, and policymakers select the ‘best’ or most ‘evidence based’ solution, setting in motion a cycle of stages including legitimation, implementation, evaluation, and the choice to maintain or change policy.

In the real world, policymaking is not so simple, and three ‘stages’ seem messed up:

  • Defining problems. There is too much going on in the world, and too much information about problems. So, policymakers have to ignore most problems and most ways to understand them. They use ‘rational’ and ‘irrational’ short-cuts to help them pay attention to a manageable number of issues and address problems without fully understanding them. Problems get attention based on how they are ‘framed’: actors use evidence to reduce uncertainty, and persuasion to reduce ambiguity (they focus our minds on one way to understand a problem).
  • Producing solutions. When policymaker attention lurches to a problem, it’s too late to produce a new solution that is technically feasible (will it work as intended?) and politically feasible (is it acceptable to enough people in the ‘community’?). While attention lurches quickly, feasible solutions take time to develop.
  • Making choices. The willingness and ability of policymakers to select a solution is fleeting, based on their beliefs, perception of the ‘national mood’, and the feedback they receive from interest groups and political parties.

Don’t think of these things as linear ‘stages’. Instead, they are independent ‘streams’ which have to come together during a brief ‘window of opportunity’. All key factors – heightened attention to a problem (problem stream), an available and feasible solution (policy stream), and the motive to select it (politics stream) – must come together at the same time, or the opportunity is lost. If you think of the streams as water, the metaphor suggests that when they come together they are hard to separate.  Instead, a ‘window of opportunity’ is like a space launch in which policymakers will abort the mission unless every factor is just right.

So, what do we do in the absence of a policy cycle?

Policy entrepreneurs’ know how to respond. They use persuasion to frame problems, help develop feasible solutions, wait for the right time to present them, and know how to adapt to their environment to exploit ‘windows of opportunity’.

Take home message for students. It is easy to dismiss the policy cycle, and find better explanations, but don’t stop there. Consider how to turn this insight into action. If policymaking is so messy, how should people respond? Studying ‘entrepreneurs’ helps us identify strategies to influence the policy process, but how could elected policymakers justify such a weird-looking process? Finally, look at many case studies to see how scholars describe MSA. It’s a flexible metaphor, but is there a coherent literature with common themes?

Next steps for reading:

Policy Concepts in 1000 Words: Multiple Streams Analysis – an expanded version of this introductory post

What is a policy entrepreneur? – describes various ways in which policy scholars define ‘entrepreneur’

Three habits of successful policy entrepreneurs – a blog post and paper on how entrepreneurs deal with ‘organized anarchy’

Whatever happened to multiple streams analysis? – introduces an article by Michael Jones and me on MSA studies

Paul Cairney and Michael Jones (2016) ‘Kingdon’s Multiple Streams Approach: What Is the Empirical Impact of this Universal Theory?’ Policy Studies Journal, 44, 1, 37-58 PDF (Annex to Cairney Jones 2016) (special issue of PSJ)

Paul Cairney and Nikos Zahariadis (2016) ‘Multiple streams analysis’ in Zahariadis, N. (eds) Handbook of Public Policy Agenda-Setting (Cheltenham: Edward Elgar) PDF see also

There is also a chapter on MSA and ideas in Understanding Public Policy.

If you are feeling really energetic, you can read the source texts:

Kingdon J (1984) Agendas, Alternatives and Public Policies New York: Harper Collins

Cohen, M., March, J. and Olsen, J. (1972) ‘A Garbage Can Model of Organizational Choice’, Administrative Science Quarterly, 17, 1, 1-25

Picture source: NASA Deep Space Gateway to Open Opportunities for Distant Destinations

 

 

 

 

 

22 Comments

Filed under 500 words, agenda setting, public policy

Three habits of successful policy entrepreneurs

This post is one part of a series – called Practical Lessons from Policy Theories and it summarizes this article (PDF).

Policy entrepreneurs’ invest their time wisely for future reward, and possess key skills that help them adapt particularly well to their environments. They are the agents for policy change who possess the knowledge, power, tenacity, and luck to be able to exploit key opportunities. They draw on three strategies:

1. Don’t focus on bombarding policymakers with evidence.

Scientists focus on making more evidence to reduce uncertainty, but put people off with too much information. Entrepreneurs tell a good story, grab the audience’s interest, and the audience demands information.

Table 1

2. By the time people pay attention to a problem it’s too late to produce a solution.

So, you produce your solution then chase problems.

Table 2

3. When your environment changes, your strategy changes.

For example, in the US federal level, you’re in the sea, and you’re a surfer waiting for the big wave. In the smaller subnational level, on a low attention and low budget issue, you can be Poseidon moving the ‘streams’. In the US federal level, you need to ‘soften’ up solutions over a long time to generate support. In subnational or other countries, you have more opportunity to import and adapt ready-made solutions.

Table 3

It all adds up to one simple piece of advice – timing and luck matters when making a policy case – but policy entrepreneurs know how to influence timing and help create their own luck.

Full paper: Three habits of successful policy entrepreneurs

(Note: the previous version was friendlier and more focused on entrepreneurs)

For more on ‘multiple streams’ see:

Paul Cairney and Michael Jones (2016) ‘Kingdon’s Multiple Streams Approach: What Is the Empirical Impact of this Universal Theory?’ Policy Studies Journal, 44, 1, 37-58 PDF (Annex to Cairney Jones 2016) (special issue of PSJ)

Paul Cairney and Nikos Zahariadis (2016) ‘Multiple streams analysis: A flexible metaphor presents an opportunity to operationalize agenda setting processes’ in Zahariadis, N. (eds) Handbook of Public Policy Agenda-Setting (Cheltenham: Edward Elgar) PDF see also

I use a space launch metaphor in the paper. If you prefer different images, have a look at 5 images of the policy process. If you prefer a watery metaphor (it’s your life, I suppose), click Policy Concepts in 1000 Words: Multiple Streams Analysis

For more on entrepreneurs:

18 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), Folksy wisdom, public policy, Storytelling

Writing for Impact: what you need to know, and 5 ways to know it

This is a post for my talk at the ‘Politheor: European Policy Network’ event Write For Impact: Training In Op-Ed Writing For Policy Advocacy. There are other speakers with more experience of, and advice on, ‘op-ed’ writing. My aim is to describe key aspects of politics and policymaking to help the audience learn why they should write op-eds in a particular way for particular audiences.

A key rule in writing is to ‘know your audience’, but it’s easier said than done if you seek many sympathetic audiences in many parts of a complex policy process. Two simple rules should help make this process somewhat clearer:

  1. Learn how policymakers simplify their world, and
  2. Learn how policy environments influence their attention and choices.

We can use the same broad concepts to help explain both processes, in which many policymakers and influencers interact across many levels and types of government to produce what we call ‘policy’:

  1. Policymaker psychology: tell an evidence-informed story

Policymakers receive too much information, and seek ways to ignore most of it while making decisions. To do so, they use ‘rational’ and ‘irrational’ means: selecting a limited number of regular sources of information, and relying on emotion, gut instinct, habit, and familiarity with information. In other words, your audience combines cognition and emotion to deal with information, and they can ignore information for long periods then quickly shift their attention towards it, even if that information has not really changed.

Consequently, an op-ed focusing solely ‘the facts’ can be relatively ineffective compared to an evidence-informed story, perhaps with a notional setting, plot, hero, and moral. Your aim shifts from providing more and more evidence to reduce uncertainty about a problem, to providing a persuasive reason to reduce ambiguity. Ambiguity relates to the fact that policymakers can understand a policy problem in many different ways – such as tobacco as an economic good, issue of civil liberties, or public health epidemic – but often pay exclusive attention to one.

So, your aim may be to influence the simple ways in which people understand the world, to influence their demand for more information. An emotional appeal can transform a factual case, but only if you know how people engage emotionally with information. Sometimes, the same story can succeed with one audience but fail with another.

  1. Institutions: learn the ‘rules of the game’

Institutions are the rules people use in policymaking, including the formal, written down, and well understood rules setting out who is responsible for certain issues, and the informal, unwritten, and unclear rules informing action. The rules used by policymakers can help define the nature of a policy problem, who is best placed to solve it, who should be consulted routinely, and who can safely be ignored. These rules can endure for long periods and become like habits, particularly if policymakers pay little attention to a problem or why they define it in a particular way.

  1. Networks and coalitions: build coalitions and establish trust

Such informal rules, about how to understand a problem and who to speak with about it, can be reinforced in networks of policymakers and influencers.

‘Policy community’ partly describes a sense that most policymaking is processed out of the public spotlight, often despite minimal high level policymaker interest. Senior policymakers delegate responsibility for policymaking to bureaucrats, who seek information and advice from groups. Groups exchange information for access to, and potential influence within, government, and policymakers have ‘standard operating procedures’ that favour particular sources of evidence and some participants over others

‘Policy community’ also describes a sense that the network seems fairly stable, built on high levels of trust between participants, based on factors such as reliability (the participant was a good source of information, and did not complain too much in public about decisions), a common aim or shared understanding of the problem, or the sense that influencers represent important groups.

So, the same policy case can have a greater impact if told by a well trusted actor in a policy community. Or, that community member may use networks to build key coalitions behind a case, use information from the network to understand which cases will have most impact, or know which audiences to seek.

  1. Ideas: learn the ‘currency’ of policy argument

This use of networks relates partly to learning the language of policy debate in particular ‘venues’, to learn what makes a convincing case. This language partly reflects a well-established ‘world view’ or the ‘core beliefs’ shared by participants. For example, a very specific ‘evidence-based’ language is used frequently in public health, while treasury departments look for some recognition of ‘value for money’ (according to a particular understanding of how you determine VFM). So, knowing your audience is knowing the terms of debate that are often so central to their worldview that they take them for granted and, in contrast, the forms of argument that are more difficult to pursue because they are challenging or unfamiliar to some audiences. Imagine a case that challenges completely someone’s world view, or one which is entirely consistent with it.

  1. Socioeconomic factors and events: influence how policymakers see the outside world

Some worldviews can be shattered by external events or crises, but this is a rare occurrence. It may be possible to generate a sense of crisis with reference to socioeconomic changes or events, but people will interpret these developments through the ‘lens’ of their own beliefs. In some cases, events seem impossible to ignore but we may not agree on their implications for action. In others, an external event only matters if policymakers pay attention to them. Indeed, we began this discussion with the insight that policymakers have to ignore almost all such information available to them.

Know your audience revisited: practical lessons from policy theories

To take into account all of these factors, while trying to make a very short and persuasive case, may seem impossible. Instead, we might pick up some basic rules of thumb from particular theories or approaches. We can discuss a few examples from ongoing work on ‘practical lessons from policy theories’.

Storytelling for policy impact

If you are telling a story with a setting, plot, hero, and moral, it may be more effective to focus on a hero than villain. More importantly, imagine two contrasting audiences: one is moved by your personal and story told to highlight some structural barriers to the wellbeing of key populations; another is unmoved, judges that person harshly, and thinks they would have done better in their shoes (perhaps they prefer to build policy on stereotypes of target populations). ‘Knowing your audience’ may involve some trial-and-error to determine which stories work under which circumstances.

Appealing to coalitions

Or, you may decide that it is impossible to write anything to appeal to all relevant audiences. Instead, you might tailor it to one, to reinforce its beliefs and encourage people to act. The ‘advocacy coalition framework’ describes such activities as routine: people go into politics to translate their beliefs into policy, they interpret the world through those beliefs, and they romanticise their own cause while demonising their opponents. If so, would a bland op-ed have much effect on any audience?

Learning from entrepreneurs

Policy entrepreneurs’ draw on three rules, two of which seem counterintuitive:

  1. Don’t focus on bombarding policymakers with evidence. Scientists focus on making more evidence to reduce uncertainty, but put people off with too much information. Entrepreneurs tell a good story, grab the audience’s interest, and the audience demands information.
  2. By the time people pay attention to a problem it’s too late to produce a solution. So, you produce your solution then chase problems.
  3. When your environment changes, your strategy changes. For example, in the US federal level, you’re in the sea, and you’re a surfer waiting for the big wave. In the smaller subnational level, on a low attention and low budget issue, you can be Poseidon moving the ‘streams’. In the US federal level, you need to ‘soften’ up solutions over a long time to generate support. In subnational or other countries, you have more opportunity to import and adapt ready-made solutions.

It all adds up to one simple piece of advice – timing and luck matters when making a policy case – but policy entrepreneurs know how to influence timing and help create their own luck.

On the day, we can use such concepts to help us think through the factors that you might think about while writing op-eds, even though it is very unlikely that you would mention them in your written work.

3 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), public policy, Storytelling

Practical Lessons from Policy Theories

These links to blog posts (the underlined headings) and tweets (with links to their full article) describe a new special issue of Policy and Politics, published in April 2018 and free to access until the end of May.

Weible Cairney abstract

Three habits of successful policy entrepreneurs

Telling stories that shape public policy

How to design ‘maps’ for policymakers relying on their ‘internal compass’

Three ways to encourage policy learning

How can governments better collaborate to address complex problems?

How do we get governments to make better decisions?

How to navigate complex policy designs

Why advocacy coalitions matter and how to think about them

None of these abstract theories provide a ‘blueprint’ for action (they were designed primarily to examine the policy process scientifically). Instead, they offer one simple insight: you’ll save a lot of energy if you engage with the policy process that exists, not the one you want to see.

Then, they describe variations on the same themes, including:

  1. There are profound limits to the power of individual policymakers: they can only process so much information, have to ignore almost all issues, and therefore tend to share policymaking with many other actors.
  2. You can increase your chances of success if you work with that insight: identify the right policymakers, the ‘venues’ in which they operate, and the ‘rules of the game’ in each venue; build networks and form coalitions to engage in those venues; shape agendas by framing problems and telling good stories, design politically feasible solutions, and learn how to exploit ‘windows of opportunity’ for their selection.

Background to the special issue

Chris Weible and I asked a group of policy theory experts to describe the ‘state of the art’ in their field and the practical lessons that they offer.

Our next presentation was at the ECPR in Oslo:

The final articles in this series are now complete, but our introduction discusses the potential for more useful contributions

Weible Cairney next steps pic

20 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), public policy

I know my audience, but does my other audience know I know my audience?

‘Know your audience’ is a key phrase for anyone trying to convey a message successfully. To ‘know your audience’ is to understand the rules they use to make sense of your message, and therefore the adjustments you have to make to produce an effective message. Simple examples include:

  • The sarcasm rules. The first rule is fairly explicit. If you want to insult someone’s shirt, you (a) say ‘nice shirt, pal’, but also (b) use facial expressions or unusual speech patterns to signal that you mean the opposite of what you are saying. Otherwise, you’ve inadvertently paid someone a compliment, which is just not on. The second rule is implicit. Sarcasm is sometimes OK – as a joke or as some nice passive aggression – and a direct insult (‘that shirt is shite, pal’) as a joke is harder to pull off.
  • The joke rule. If you say that you went to the doctor because a strawberry was growing out of your arse and the doctor gave you some cream for it, you’d expect your audience to know you were joking because it’s such a ridiculous scenario and there’s a pun. Still, there’s a chance that, if you say it quickly, with a straight face, your audience is not expecting a joke, and/ or your audience’s first language is not English, your audience will take you seriously, if only for a second. It’s hilarious if your audience goes along with you, and a bit awkward if your audience asks kindly about your welfare.
  • Keep it simple stupid. If someone says KISS, or some modern equivalent – ‘it’s the economy, stupid’, the rule is that, generally, they are not calling you stupid (even though the insertion of the comma, in modern phrases, makes it look like they are). They are referring to the value of a simple design or explanation that as many people as possible can understand. If your audience doesn’t know the phrase, they may think you’re calling them stupid, stupid.

These rules can be analysed from various perspectives: linguistics, focusing on how and why rules of language develop; and philosophy, to help articulate how and why rules matter in sense making.

There is also a key role for psychological insights, since – for example – a lot of these rules relate to the routine ways in which people engage emotionally with the ‘signals’ or information they receive.

Think of the simple example of twitter engagement, in which people with emotional attachments to one position over another (say, pro- or anti- Brexit), respond instantly to a message (say, pro- or anti- Brexit). While some really let themselves down when they reply with their own tweet, and others don’t say a word, neither audience is immune from that emotional engagement with information. So, to ‘know your audience’ is to anticipate and adapt to the ways in which they will inevitably engage ‘rationally’ and ‘irrationally’ with your message.

I say this partly because I’ve been messing around with some simple ‘heuristics’ built on insights from psychology, including Psychology Based Policy Studies: 5 heuristics to maximise the use of evidence in policymaking .

Two audiences in the study of ‘evidence based policymaking’

I also say it because I’ve started to notice a big unintended consequence of knowing my audience: my one audience doesn’t like the message I’m giving the other. It’s a bit like gossip: maybe you only get away with it if only one audience is listening. If they are both listening, one audience seems to appreciate some new insights, while the other wonders if I’ve ever read a political science book.

The problem here is that two audiences have different rules to understand the messages that I help send. Let’s call them ‘science’ and ‘political science’ (please humour me – you’ve come this far). Then, let’s make some heroic binary distinctions in the rules each audience would use to interpret similar issues in a very different way.

I could go on with these provocative distinctions, but you get the idea. A belief taken for granted in one field will be treated as controversial in another. In one day, you can go to one workshop and hear the story of objective evidence, post-truth politics, and irrational politicians with low political will to select evidence-based policies, then go to another workshop and hear the story of subjective knowledge claims.

Or, I can give the same presentation and get two very different reactions. If these are the expectations of each audience, they will interpret and respond to my messages in very different ways.

So, imagine I use some psychology insights to appeal to the ‘science’ audience. I know that,  to keep it on side and receptive to my ideas, I should begin by being sympathetic to its aims. So, my implicit story is along the lines of, ‘if you believe in the primacy of science and seek evidence-based policy, here is what you need to do: adapt to irrational policymaking and find out where the action is in a complex policymaking system’. Then, if I’m feeling energetic and provocative, I’ll slip in some discussion about knowledge claims by saying something like, ‘politicians (and, by the way, some other scholars) don’t share your views on the hierarchy of evidence’, or inviting my audience to reflect on how far they’d go to override the beliefs of other people (such as the local communities or service users most affected by the evidence-based policies that seem most effective).

The problem with this story is that key parts are implicit and, by appearing to go along with my audience, I provoke a reaction in another audience: don’t you know that many people have valid knowledge claims? Politics is about values and power, don’t you know?

So, that’s where I am right now. I feel like I ‘know my audience’ but I am struggling to explain to my original political science audience that I need to describe its insights in a very particular way to have any traction in my other science audience. ‘Know your audience’ can only take you so far unless your other audience knows that you are engaged in knowing your audience.

If you want to know more, see:

Kathryn Oliver and I have just published an article on the relationship between evidence and policy

How far should you go to secure academic ‘impact’ in policymaking? From ‘honest brokers’ to ‘research purists’ and Machiavellian manipulators

Why doesn’t evidence win the day in policy and policymaking?

The Science of Evidence-based Policymaking: How to Be Heard

When presenting evidence to policymakers, engage with the policy process that exists, not the process you wish existed

 

 

5 Comments

Filed under Academic innovation or navel gazing, agenda setting, Evidence Based Policymaking (EBPM), Psychology Based Policy Studies, public policy, Storytelling

Who are the most deserving and entitled to government benefits?

desrving-and-entitled-book-cover

Schneider and Ingram’s work on Social Construction and Policy Design identifies the populations deemed by policymakers to deserve the benefits or punishments of government policy.

The most concerted attempt to use this theoretical framework empirically is in Schneider and Ingram’s (2005b) edited book Deserving and Entitled, which focuses primarily on the social construction of particular target populations and its largely degenerative effect on US policy design (introduction here).

In this post, I summarise their insights in about 3000 words, splitting their five parts into seven main insights (that way, I can provide a far shorter summary in a journal article, and refer to this page for more discussion).

Insight 1: Policy designs that we take for granted now were built on social constructions of selective deservingness in the (often very distant) past, with key choices setting a precedent for policy design expansion.

Selective group entitlement to state benefits can be traced to the rules created in 1818, initially for very specific ‘veterans of the American Revolution’, providing benefits if they were: men who had travelled extensively to fight for at least nine months and were now ‘poor or unable to work’ (Jensen, 2005: 37). This excluded women, people who were otherwise poor, and veteran militia who fought locally. Officers were paid $20 per month but soldiers $8 (2005: 57). By 1832, pension entitlement extended to all who served for two years, and in 1836 it was expanded to their dependents if they died from war injuries. In 1871, Congress approved a non-means tested pension for those who served in the War of 1812 (and their widows) if they had demonstrated their patriotism by remaining ‘loyal to the United Stated during the Civil War’ and swearing an ‘oath to support the Constitution’, which excluded ‘aged southerners’ (2005: 58). Each expansion encouraged veteran participation: sending a signal about the entitlements that they would receive, or could campaign for if they didn’t.

Selective group exclusion from the ‘universal’ electoral franchise can be traced to a similar time, when many politicians equated popular participation with ‘mob rule’ and sought to limit inclusion. Schriner (2005: 66-7; 73; 76) identifies attempts to exclude ‘idiots’ and the ‘insane’ as the ‘antithesis of the democratic citizen’. For many politicians, they lacked the ‘intellectual competence necessary for political participation’ and/or were mistrusted by the politicians who equated incompetence with deviance, ‘moral degeneracy, dependency, and crime’ (2005: 76). These exclusions only came under serious challenge when social reformers had some success in shifting wider social constructions on mental capacity, highlighting the cruelty of asylums and challenging the link between intellectual incompetence and moral deviance (2005: 76).

Insight 2: Successful challenges to the social construction of target populations are possible but seem exceptional (and, can be at the expense of other minority groups).

Dialto (2005: 81) suggests that the image of Japanese-Americans shifted profoundly between 1941-61, from the hostility and fear of immigrant aliens underpinning WW2 relocation and interment (preceded by late 19th and early 20th century legal rulings which excluded naturalisation and the ability to buy land) to the tag of ‘model minority’ and good citizen. Strategies to change this image did not challenge discrimination directly: finding legal loopholes on land ownership through American-born children; internalising ‘their negative social construction’ and focusing on earning US citizenship (both before WW2); and, crucially, forming a Japanese-American US Army unit which earned war hero status (2005: 90-8). These efforts were reinforced by shifting contexts: internally, Japanese Americans were seen as less of an economic threat (internment left them ‘economically devastated’), and externally, Japan shifted from a threat to a common force against communism and new source of trade (2005: 98).

Court cases in the 1950s helped shift the rules of naturalisation and land ownership. Media coverage in the 1960s used positive portrayals of Japanese Americans, as working hard to overcome hardship, to contrast them with negative portrayals of others and ‘keep other minorities, such as blacks and Chicanos, in their place’ (2005: 99). The US President effectively apologised for internment in 1976, and Congress approved one-off compensation for internment survivors in 1988, and key federal court cases in the late 1980s challenged the War Department’s basis for discriminatory behaviour and convictions (2005: 100). Still, the ‘model minority’ tag has profound drawbacks, in which Japanese Americans perceive the need to live up to a stereotype to gain acceptance, while still facing social discrimination and violence (2005: 100). In other words, the shifting social construction of a social group does not necessarily indicate its growing power.

Insight 3: Advocates have used national policymaking institutions as a source of major change in policy design, but often with limited effects on delivery and/ or profound unintended consequences.

Although often contributing to degenerative policy design, Congress has proved a key organisation to challenging negative social constructions based on race. However, advocates of housing reform to reduce desegregation addressed major opposition (including public, congressional, and business) by assigning deserving/ undeserving status within racial minorities. The narrative underpinning legislative change on fair housing in 1968 was that only some black people deserved to benefit from policy: advocates told individual stories of the black middle class rejecting the life of the black rioter, who ‘deserved to escape the ghettos’, which helped create a race-based underclass (Sidney, 2005: 114-5). Further, this strategy secured very limited policy and social change, partly because few black people could afford the market price of homes in white areas (2005: 118). As predicted by Schneider and Ingram, the ‘benefits to weak groups’ are ‘undersubscribed and conferred at levels insufficient to change behaviour to an extent that it would remediate the problem’ (2005: 119). Policy changes in the late 70s and 80s were based on a ‘diluted’ message on race, focusing for example on the negative role of financial institutions, wider sources (such as class and age) of potential discrimination in housing, or on specific regions (not populations) that might benefit from community investment: the long term benefits of policy to reduce racial desegregation ‘remain fragile and open to dilution or contestation’ (2005: 137).

Congress also acted positively on immigration in 1965, making it ‘no longer acceptable’ for policies to ‘single out groups for exclusion on the base of race or national origin’. In 1986, it gave amnesty to illegal immigrants working in the US for five years, and punished employers not individuals for illegal immigration (Netwon, 2005: 139; 143). Yet, there remain fluctuations in social constructions of which target populations are willing and able to assimilate to achieve legitimate American citizenship, measured according to: legal citizenship, positive values regarding ‘hard work and self sufficiency’, and negative connotations of foreigners committing crimes, procreating excessively, claiming welfare and, during times of economic uncertainty, taking ‘scarce jobs from Americans’ (2005: 139-40; 142).

These negative constructions informed legislation from 1996 which increased southern border patrol staff, equipment, barriers, and detention facilities, made it easier to deport, raised the bar for employers to demonstrate continuously and adequately paid jobs, and reduced entitlement to social security, housing, and higher education subsidies to ‘remove the lure of public benefits’ for ‘problem’ immigrants (2005: 145-6; 151). In congressional debate, ways to define ‘unworthy’ immigrants related to arguments such as: they are freeloaders not paying taxes, but their children get access to free schools; if they break the law to get here, they’ll break the law while here; immigrants are causing a ‘lawless border’ that local law enforcement can’t control; and, an ineffective federal government is hindering border enforcement and blaming employers rather than the criminals (2005: 151-7). Further, rather than challenging such positions with positive counter narratives, opponents used a ‘cure worse than disease’ argument which pointed to the major unintended consequences of policy (for example, removing school entitlement will worsen crime) (2005: 163). Overall, congressional debates signal ‘who matters’ and which people (primarily Mexican) are the ‘wrong kinds of immigrants’ (2005: 166).

Insight 4: public and non-governmental bodies have some discretion to make policy when they deliver. The evidence is mixed on the extent to which those actors use that chance to reframe target populations more positively.

The social and welfare organisations funded to help target populations are often the bodies identifying many as undeserving. For example, Jurik and Cowgill (2005: 173) describe the difference between a potentially positive rationale and negative experience of ‘microenterprise development programs’. MDPs are often initially designed to reduce poverty by helping low-income people of colour with ‘business loans and training’, avoid the stigmatising and unintended consequences of ‘welfare schemes’, and providing programmes to encourage ‘women’s entrepreneurship’ and challenge male conceptions of business strategy and success (2005: 175). However, as philanthropic funding reduces and bodies become more reliant on government and other highly-conditional funding, MDP bodies feel they must demonstrate high performance and distinguish their work from ‘welfare programmes’, and staff become more risk averse to reduce financial losses (2005: 185). They make value judgements based on ‘white, middle-class, male centred standards of business’ to ‘evaluate client worthiness and attempt to resocialize them as well as shape access to further services’ (2005: 174). Worthiness is linked to likely success, and bodies engage in ‘creaming’ (focusing on the easiest, cheapest cases), use implicit moral assessments of character to identify a good attitude and reject ‘entitlement seekers’ and ‘dreamers’, with assessments often ‘class-laden, gendered, and racialized’ (2005: 177; 183). Consequently, people with the lowest incomes and levels of education are less likely to receive significant loans, and most likely to continue to be viewed as ‘helpless deviants … scorned by policymakers’ when they fail to “’lift’ themselves out of poverty” (2005: 196).

Camou (2005: 197-8) identifies a more positive story based on an unusual mix, in small ‘neighborhood organizations’ (non-profit, advocacy, and resident organizations), of constructing often-marginalised groups positively but to ‘resocialize them into an acceptable way of life’ containing the “‘correct’ ‘norms and values”. Most of the studied programs combine a focus on the ‘deficits’ of target populations (such as drug addiction, and low employability and education) and belief that they can’t ‘take control of their lives without interventions’ with the hope that they ‘can reject street values in favour of mainstream ones, including employment, nonviolence, family cohesion, delayed gratification, and respect for property’ (2005: 203). They combine direct interventions, such as programs to enhance employability, with indirect strategies based on some individuals leading by example and encouraging new rules (2005: 204-5). There is minimal emphasis on welfare and structural/ environmental factors such as housing supply, or on challenging a small number of deviants ruining the neighbourhood while helping the upwardly mobile or middle class population (2005: 206-13). Programs ‘tend not to “cream”’, particularly when staff live or recently lived in the same area: their focus is on challenging highly visible and vivid examples of ‘street values’ rather than encouraging the less visible and already positive behaviour (2005: 213-5). Camou (2005: 215) identifies a commitment to a form of ‘tough love’ that ‘would assume a different connotation if pursued by outsiders’; it might punish ‘deviants’ but it also ‘rewards the successful’ users of programs designed to help disadvantaged target populations.

Insight 5: only a small proportion of the many potentially demonised groups receive sustained negative policymaker attention, aided by the ‘moral entrepreneurs’, policy analysts, and ‘policy champions’ who translate specific social constructions into policy design.

Policymakers, the media, and public can only pay attention to a tiny proportion of the issues that could be described as policy problems. So, sustained public anxiety and political attention to a target population requires three conditions (Nicholson-Crotty and Meier, 2005: 224). First, a group is already perceived negatively by many, perhaps with the additional sense that their behaviour is ‘out of control’. Second, a ‘moral entrepreneur’ draws attention to the group successfully: the public, or powerful sections or institutions within it, perceive the entrepreneur as an expert on the social value and group’s deviance; and, those powerful actors perceive a threat that extends from the deviant group to ‘the rest of society’ (2005: 225-7). Moral entrepreneurs use ‘typification’, using the behaviour visible in certain individuals to represent behaviour of a larger group, to identify a ‘dangerous class’ that can be blamed for many more social problems than linked to initial behaviour (2005: 227). Third, there must be ‘sufficient political profit’ to entice a ‘policy champion’ to set the agenda and pursue policy change (2005: 227).

For example, moral entrepreneurs such as Jacob Riis turned broad and widespread suspicion of Chinese immigrants, in the mid to late 19th century, into a specific concern about the spillover effects of their opium smoking: opium addiction spreading to the white population, with the danger of Chinese men using ‘cruel cunning’ to encourage young female ‘promiscuity’ in ‘opium dens’ (2005: 230). Policy champions, such as State Department Secretary Root, proposed legislation to prohibit the import of opium for smoking, thus restricting its effect largely to the Chinese population without prompting opposition from, for example, doctors using opiates for therapeutic purposes (2005: 233).

By the 1980s, a rapid increase in public fear of crime became linked closely to racially-charged fears of social change (2005: 235). Moral entrepreneurs such as James Q Wilson encouraged then used such concerns to generate support for a major shift in policy rationale, from a focus on the structural causes of crime (such as poverty), to a focus on individual responsibility or out of control gangs. Policy champions such as President Reagan and Representative Dan Lundgren pursued legislation (the Comprehensive Crime Act 1984) on the back of ‘organised crime’, ‘the drug racketeers who are poisoning our young people’, and testimony of victims (2005: 239). The effect of such policies is to identify approximately five million people as members of a ‘dangerous class’ (2005: 242).

Insight 6: well-intentioned but problematic social constructions produce unintended consequences if policy designs fail to address major socio-economic inequalities.

Bensonsmith (2005: 244-5) describes the ‘War on Poverty’ programs fostered by President Johnson from 1965, describing key actors such as Daniel Moynihan – author of The Negro Family (‘Moynihan Report’) – as moral entrepreneur and policy champion. Moynihan was part of a ‘new breed of public servants’ with social scientific training and seeking to define as well as solve policy problems (2005: 246). His report identified lower poverty in white and higher in black populations, arguing that the main cause was the ‘disintegration of the African-American family’, in which racism undermined the role of black men as breadwinners, contributing to family breakdown and deviant behaviour (2005: 246). Bensonsmith (2005: 247) argues that the report contributes to black stereotypes that endured from times of slavery, including the ‘shiftless black male’ and the ‘overly fertile and lazy welfare mother’, and that such stereotypes contributed to the idea that welfare dependency helped cause ‘pathological behavior’: ‘broken homes’ have a ‘matriarchal structure’ (aided by female social workers) with no ‘strong father figure’, and a key solution for black men is the ‘masculine world’ of military service (2005: 251). This social construction informed key welfare policies in two main steps. First, black mothers were often initially excluded from social security measures designed for the widows and their dependent children: the state would take on the male role if he died, not if he chose to leave. Second, as entitlement became defined more widely, it became more closely associated with the often-incorrect idea that a disproportionate number of black families received welfare benefits. The association between welfare and race, along with a greater focus on individual responsibility for a culture of dependency, helps cause victim blaming and program retrenchment (2005: 255-6).

Schram (2005: 261) provides an often-different interpretation, arguing that the failure to recognise major ‘racial disparities in the US economy’ helps ‘whitewash’ them rather than asking the ‘hard questions’ in the pursuit of ‘racial justice’. Going too far, to argue that welfare is used more by white than black families – as argued by the Welfare Law Centre’s Welfare Myths in 1996 – can be misleading (from 1985, only the absolute figures on first time use, but not more regular use, and not proportionate calculations, support its position). It is also counterproductive because ‘racial minorities’ were ‘more likely to be living in poverty and in need of public assistance at higher rates’ (2005: 267-70). Similarly, it is tempting for advocacy groups to generate support for welfare rights by associating them with white, middle class and recently divorced mothers as ‘job ready’ and in transition from welfare to work, but problematic because it helps produce insufficient public support for other groups (2005: 281). Programs become ‘bifurcated’, with the predominantly-white male ‘breadwinner’ able to benefit from social insurance based on their previous employment, and others reliant on less generous ‘public assistance’ (2005: 282).

Insight 7: social constructions of (a) deservedness and entitlement create ‘positive identities and participation’ in politics, (b) unworthiness create ‘alienation, resignation, and failure to participate politically’ (Ingram and Schneider, 2005c: 289).

Soss (2005: 291) examines the signals that a bifurcated US social welfare system sends to specific target populations, and how they feel and respond. He examines the idea that US welfare programs have become influenced by ‘new paternalism’, focusing on the ways in which programs are designed ostensibly to encourage good citizenship. In paternalist regimes, they may be associated with ‘civility, self-restraint, and compliance with social expectations’, reflecting the pursuit of ‘social order over social justice and civic compliance over political engagement’ (2005: 292). Different programs also provide profoundly different signals about citizenship, and the signal that a person gets from one key public organization affects their views of government as a whole (2005: 309).

The ‘superior tier’ consists of depersonalised federal ‘social security’ programs, primarily for the ‘elderly or disabled’, pegged to inflation and previous earnings, and producing a positive signal to ‘rights bearing’ recipients (2005: 294-5). The rules are stringent but recipients are encouraged and rewarded, to ‘come away feeling that their claims are welcome’ (2005: 297). Recipients often receive the financial security that allows them to participate in public life as much as the general population, a signal from government that their participation efforts would have a tangible effect, and do not generate feelings of stigma that undermine collective action (2005: 308).

The ‘lower tier’ consists of less generous public assistance ‘that disproportionately serve disadvantaged groups such as people of color, women, and people who have lived in poverty’ (2005: 295). They send negative signals, with recipients attached to state or local case workers and following convoluted and dispiriting rules to establish limited entitlement. Recipients become ‘clients’, vulnerable to bureaucratic rules, entitled to very little privacy (following invasive questions, fingerprinting, and/ or drug testing in some cases), made to attend in person despite limited access to transport, and subject to ‘unpleasant procedures’ in unwelcoming offices (‘authoritarian’ in design and appearance). Many are left vulnerable, with the feeling that ‘they are not valued and their claims unwanted’ and some ‘feeling exposed and humiliated’, powerless, or ‘like cattle’ (2005: 296-9). Recipients are living in poverty and/or fleeing abusive relationships, with minimal choice but to accept these rules without complaint, left with the felling that their feedback would not alter government staff behaviour, and feeling the stigma and lack of self-worth that undermines collective action (2005: 306-7; 310).

Yet, these perceptions are not inevitable. Rather, the key example of Head Start, in which parents participate in collective decisions about the running of local services for their children, ‘provide citizens with evidence that participation can be effective and fulfilling’ (2005: 305), and expose other programs for the minimal extent to which they encourage civic engagement (2005: 321).

If you want to read part two, here it is: scpd-more-empirical-insights-section-31-12-16

3 Comments

Filed under agenda setting, public policy, Storytelling

We need better descriptions than ‘evidence-based policy’ and ‘policy-based evidence’: the case of UK government ‘troubled families’ policy

Here is the dilemma for ‘evidence-based’ ‘troubled families’ policy: there are many indicators of ‘policy based evidence’ but few (if any) feasible and ‘evidence based’ alternatives.

Viewed from the outside, TF looks like a cynical attempt to produce a quick fix to the London riots, stigmatise vulnerable populations, and hoodwink the public into thinking that the central government is controlling local outcomes and generating success.

Viewed from the inside, it is a pragmatic policy solution, informed by promising evidence which needs to be sold in the right way. For the UK government there may seem to be little alternative to this policy, given the available evidence, the need to do something for the long term and to account for itself in a Westminster system in the short term.

So, in this draft paper, I outline this disconnect between interpretations of ‘evidence based policy’ and ‘policy based evidence’ to help provide some clarity on the pragmatic use of evidence in politics:

cairney-offshoot-troubled-families-ebpm-5-9-16

See also:

Governments think it’s OK to use bad evidence to make good policy: the case of the UK Government’s ‘troubled families’

Early intervention policy, from ‘troubled families’ to ‘named persons’: problems with evidence and framing ‘valence’ issues

In each of these posts, I note that it is difficult to know how, for example, social policy scholars should respond to these issues – but that policy studies help us identify a choice between strategies. In general, pragmatic strategies to influence the use of evidence in policy include: framing issues to catch the attention or manipulate policymaker biases, identifying where the ‘action’ is in multi-level policymaking systems, and forming coalitions with like-minded and well-connected actors. In other words, to influence rather than just comment on policy, we need to understand how policymakers would respond to external evaluation. So, a greater understanding the routine motives of policymakers can help produce more effective criticism of its problematic use of evidence. In social policy, there is an acute dilemma about the choice between engagement, to influence and be influenced by policymakers, and detachment to ensure critical distance. If choosing the latter, we need to think harder about how criticism of PBE makes a difference.

4 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), Prevention policy, public policy, UK politics and policy

Early intervention policy, from ‘troubled families’ to ‘named persons’: problems with evidence and framing ‘valence’ issues

Imagine this as your ‘early intervention’ policy choice: (a) a universal and non-stigmatising programme for all parents/ children, with minimal evidence of effectiveness, high cost, and potential public opposition about the state intervening in family life; or (b) a targeted, stigmatising programme for a small number, with more evidence, less cost, but the sense that you are not really intervening ‘early’ (instead, you are waiting for problems to arise before you intervene). What would you do, and how would you sell your choice to the public?

I ask this question because ‘early intervention’ seems to the classic valence issue with a twist. Most people seem to want it in the abstract: isn’t it best to intervene as early as possible in a child’s life to protect them or improve their life chances?

However, profound problems or controversies arise when governments try to pursue it. There are many more choices than I presented, but the same basic trade-offs arise in each case. So, at the start, it looks like you have lucked onto a policy that almost everyone loves. At the end, you realise that you can’t win. There is no such thing as a valence issue at the point of policy choice and delivery.

To expand on these dilemmas in more depth, I compare cases of Scottish and UK Government ‘families policies’. In previous posts, I portrayed their differences – at least in the field of prevention and early intervention policies – as more difficult to pin down than you might think. Often, they either say the same things but ‘operationalise’ them in very different ways, or describe very different problems then select very similar solutions.

This basic description sums up very similar waves of key ‘families policies’ since devolution: an initial focus on social inclusion, then anti-social behaviour, followed by a contemporary focus on ‘whole family’ approaches and early intervention. I will show how they often go their own ways, but note the same basic context for choice, and similar choices, which help qualify that picture.

Early intervention & prevention policies are valence issues …

A valence (or ‘motherhood and apple pie’) issue is one in which you can generate huge support because the aim seems, to most people, to be obviously good. Broad aims include ‘freedom’ and ‘democracy’. In the UK specific aims include a national health service free at the point of use. We often focus on valence issues to highlight the importance of a political party’s or leader’s image of governing competence: it is not so much what we want (when the main parties support very similar things), but who you trust to get it.

Early intervention seems to fit the bill: who would want you to intervene late or too late in someone’s life when you can intervene early, to boost their life chances at an early stage as possible? All we have to do is work out how to do it well, with reference to some good evidence. Yet, as I discuss below, things get complicated as soon as we consider the types of early intervention available, generally described roughly as a spectrum from primary (stop a problem occurring and focus on the whole population – like a virus inoculation) to secondary (address a problem at an early stage, using proxy indicators to identify high-risk groups), and tertiary (stop a problem getting worse in already affected groups).

Similarly, look at how Emily St Denny and I describe prevention policy. Would many people object to the basic principles?

“In the name of prevention, the UK and Scottish Governments propose to radically change policy and policymaking across the whole of government. Their deceptively simple definition of ‘prevention policy’ is: a major shift in resources, from the delivery of reactive public services to solve acute problems, to the prevention of those problems before they occur. The results they promise are transformative, to address three crises in politics simultaneously: a major reduction in socioeconomic equalities by focusing on their ‘root causes’; a solution to unsustainable public spending which is pushing public services to breaking point; and, new forms of localised policymaking, built on community and service user engagement, to restore trust in politics”.

… but the evidence on their effectiveness is inconvenient …

A good simple rule about ‘evidence-based policymaking’ is that there is never a ‘magic bullet’ to tell you what to do or take the place of judgement. Politics is about making choices which benefit some people while others lose out. You can use evidence to help clarify those choices, but not produce a ‘technical’ solution. A further rule with ‘wicked’ problems is that the evidence is not good enough even to generate clarity about the cause of the problem. Or, you simply find out things you don’t want to hear.

Early intervention seems to be a good candidate for the latter, for three main reasons:

  1. Very few interventions live up to high evidence standards

There are two main types of relevant ‘evidence based’ interventions in this field. The first are ‘family intervention projects’ (FIPs). They generally focus on low income, often lone parent, families at risk of eviction linked to factors such as antisocial behaviour, and provide two forms of intervention: intensive 24/7 support, including after school clubs for children and parenting skills classes, and treatment for addiction or depression in some cases, in dedicated core accommodation with strict rules on access and behaviour; and an outreach model of support and training. The evidence of success comes from evaluation and a counterfactual: this intervention is expensive, but we think that it would have cost far more money and heartache if we had not intervened to prevent (for example) family homelessness. There is generally no randomised control trial (RCT) to establish the cause of improved outcomes, or demonstrate that those outcomes would not have happened without an intervention of this sort.

The second are projects imported from other countries (primarily the US and Australia) based on their reputation for success. This reputation has been generated according to evidential rules associated with ‘evidence based medicine’ (EBM), in which there is relatively strong adherence to a hierarchy of evidence, with RCTs and their systematic review at the top, and the belief that there should be ‘fidelity’ to programmes to make sure that the ‘dosage’ of the intervention is delivered properly and its effect measured. Key examples include the Family Nurse Partnership (although its first UK RCT evaluation was not promising), Triple P (although James Coyne has his doubts!), and Incredible Years (but note the importance of ‘indicated’ versus ‘selective’ programmes, below). In this approach, there may be more quantitative evidence of success, but it is still difficult to know if the project can be transferred effectively and if its success can be replicated in another country with a very different political drivers, problems, and levels of existing services. We know that some interventions are associated with positive outcomes, but we struggle to establish definitively that they caused them (solely, separate from their context).

  1. The evidence on ‘scaling up’ for primary prevention is relatively weak

Kenneth Dodge (2009) sums up a general problem with primary prevention in this field. It is difficult to see much evidence of success because: there are few examples of taking effective specialist projects ‘to scale’; there are major issues around ‘fidelity’ to the original project when you scale up (including the need to oversee a major expansion in well-trained practitioners); and, it is difficult to predict the effect of a programme, which showed promise when applied to one population, to a new and different population.

  1. The evidence on secondary early intervention is also weak

This point about different populations with different motivations is demonstrated in a more recent (published 2014) study by Stephen Scott et al of two Incredible Years interventions – to address ‘oppositional defiant disorder symptoms and antisocial personality character traits’ in children aged 3-7 (for a wider discussion of such programmes see the Early Intervention Foundation’s Foundations for life: what works to support parent child interaction in the early years?).

They highlight a classic dilemma in early intervention: the evidence of effectiveness is only clear when children have been clinically referred (‘indicated approach’), but unclear when children have been identified as high risk using socioeconomic predictors (‘selective approach’):

An indicated approach is simpler to administer, as there are fewer children with severe problems, they are easier to identify, and their parents are usually prepared to engage in treatment; however, the problems may already be too entrenched to treat. In contrast, a selective approach targets milder cases, but because problems are less established, whole populations have to be screened and fewer cases will go on to develop serious problems.

For our purposes, this may represent the most inconvenient form of evidence on early intervention: you can intervene early on the back of very limited evidence of likely success, or have a far higher likelihood of success when you intervene later, when you are running out of time to call it ‘early intervention’.

… so governments have to make and defend highly ‘political’ choices …

I think this is key context in which we can try to understand the often-different choices by the UK and Scottish Governments. Faced with the same broad aim, to intervene early to prevent poor outcomes, the same uncertainty and lack of evidence that their interventions will produce the desired effect, and the same need to DO SOMETHING rather than wait for the evidence that may never arise, what do they do?

Both governments often did remarkably similar things before they did different things

From the late 1990s, both governments placed primary emphasis initially on a positive social inclusion agenda, followed by a relatively negative focus on anti-social behaviour (ASB), before a renewed focus on the social determinants of inequalities and the use of early intervention to prevent poor outcomes.

Both governments link families policies strongly to parenting skills, reinforcing the idea that parents are primarily responsible for the life chances of their children.

Both governments talk about getting away from deficit models of intervention (the Scottish Government in particular focuses on the ‘assets’ of individuals, families, and communities) but use deficit-model proxies to identify families in need of support, including: lone parenthood, debt problems, ill health (including disability and depression), and at least one member subject to domestic abuse or intergenerational violence, as well as professional judgements on the ‘chaotic’ or ‘dysfunctional’ nature of family life and of the likelihood of ‘family breakdown’ when, for example, a child it taken into care.

So, when we consider their headline-grabbing differences, note this common set of problems and drivers, and similar responses.

… and selling their early intervention choices is remarkably difficult …

Although our starting point was valence politics, prevention and early intervention policies are incredibly hard to get off the ground. As Emily St Denny and I describe elsewhere, when policymakers ‘make a sincere commitment to prevention, they do not know what it means or appreciate the scale of their task. They soon find a set of policymaking constraints that will always be present. When they ‘operationalise’ prevention, they face several fundamental problems, including: the identification of ‘wicked’ problems which are difficult to define and seem impossible to solve; inescapable choices on how far they should go to redistribute income, distribute public resources, and intervene in people’s lives; major competition from more salient policy aims which prompt them to maintain existing public services; and, a democratic system which limits their ability to reform the ways in which they make policy. These problems may never be overcome. More importantly, policymakers soon think that their task is impossible. Therefore, there is high potential for an initial period of enthusiasm and activity to be replaced by disenchantment and inactivity, and for this cycle to be repeated without resolution’.

These constraints refer to the broad idea of prevention policy, while specific policies can involve different drivers and constraints. With general prevention policy, it is difficult to know what government policy is and how you measure its success. ‘Prevention’ is vague, plus governments encourage local discretion to adapt the evidence of ‘what works’ to local circumstances.

Governments don’t get away with this regarding specific policies. Instead, Westminster politics is built on a simple idea of accountability in which you know who is in charge and therefore to blame. UK central governments have to maintain some semblance of control because they know that people will try to hold them to account in elections and general debate. This ‘top down’ perspective has an enduring effect, particularly in the UK, but also the Scottish, government.

… so the UK Government goes for it and faces the consequences ….

‘Troubled Families’ in England: the massive expansion of secondary prevention?

So, although prevention policy is vague, individual programmes such as ‘troubled families’ contain enough detail to generate intense debate on central government policy and performance and contain elements which emphasise high central direction, including sustained ministerial commitment, a determination to demonstrate early success to justify a further rollout of policy, and performance management geared towards specific measurable short term outcomes – even if the broader aim is to encourage local discretion and successful long term outcomes.

In the absence of unequivocally supportive evidence (which may never appear), the UK government relied on a crisis (the London riots in 2011) to sell policy, and ridiculous processes of estimation of the size of the problem and performance measurement to sell the success of its solution. In this system, ministers perceive the need to display strength, show certainty that they have correctly diagnosed a problem and its solution, and claim success using the ‘currency’ of Westminster politics – and to do these things far more quickly than the people gathering evidence of more substantive success. There is a lot of criticism of the programme in terms of its lack, or cynical use, of evidence but little of it considers policy from an elected government’s perspective.

…while the Scottish Government is more careful, but faces unintended consequences

This particular UK Government response has no parallel in Scotland. The UK Government is far more likely than its Scottish counterpart to link families policies to a moral agenda in response to crisis, and there is no Scottish Government equivalent to ‘payment by results’ and massive programme expansion. Instead, it continued more modest roll-outs in partnership with local public bodies. Indeed, if we ‘zoom in’ to this one example, at this point in time, the comparison confirms the idea of a ‘Scottish Approach’ to policy and policymaking.

Yet, the Scottish Government has not solved the problems I describe in this post: it has not found an alternative ‘evidence based’ way to ‘scale up’ early intervention significantly and move from secondary/ tertiary forms of prevention to the more universal/ primary initiatives that you might associate intuitively with prevention policy.

Instead, its different experiences have highlighted different issues. For example, its key vehicle for early intervention and prevention is the ‘collaborative’ approach, such as in the Early Years Collaborative. Possibly, it represents the opposite of the UK’s attempt to centralise and performance-manage-the-hell-out-of the direction of major expansion.

Table 1 Three ideal types EBBP

Certainty, with this approach, your main aim is not to generate evidence of the success of interventions – at least not in the way we associate with ‘evidence based medicine’, randomised control trials, and the star ratings developed by the Early Intervention Foundation. Rather, the aim is to train local practitioners to use existing evidence and adapt it to local circumstances, experimenting as you go, and gathering/using data on progress in ways not associated with, for example, the family nurse partnership.

So, in terms of the discussion so far, perhaps its main advantage is that a government does not have to sell its political choices (it is more of a delivery system than a specific intervention) or back them up with evidence of success elsewhere. In the absence of much public, media, or political party attention, maybe it’s a nice pragmatic political solution built more on governance principles than specific evidence.

Yet, despite our fixation with the constitution, some policy issues do occasionally get discussed. For our purposes, the most relevant is the ‘named person’ scheme because it looks like a way to ‘scale up’ an initiative to support a universal or primary prevention approach and avoid stigmatising some groups by offering a service to everyone (in this respect, it is the antithesis to ‘troubled families’). In this case, all children in Scotland (and their parents or guardians) get access to a senior member of a public service, and that person acts as a way to ‘join up’ a public sector response to a child’s problems.

Interestingly, this universal approach has its own problems. ‘Troubled families’ sets up a distinction between troubled/ untroubled to limit its proposed intervention in family life. Its problem is the potential to stigmatise and demoralise ‘troubled’ families. ‘Named person’ shows the potential for greater outcry when governments try to not identify and stigmatise specific families. The scheme is largely a response to the continuous suggestion – made after high profile cases of child abuse or neglect – that children can suffer when no agency takes overall responsibility for their care, but has been opposed as excessive infringement on normal family life and data protection, successfully enough to delay its implementation.

[Update 20.9.19: Named person scheme scrapped by Scottish Government]

The punchline to early intervention as a valence issue

Problems arise almost instantly when you try to turn a valence issue into something concrete. A vague and widely-supported policy, to intervene early to prevent bad outcomes, becomes a set of policy choices based on how governments frame the balance between ideology, stigma, and the evidence of the impact and cost-effectiveness of key interventions (which is often very limited).

Their experiences are not always directly comparable, but the UK and Scottish Governments have helped show us the pitfalls of concrete approaches to prevention and early intervention. They help us show that your basic policy choices include: (a) targeted programmes which increase stigma, (b) ‘indicated’ approaches which don’t always look like early intervention; (c) ‘selective’ approaches which seem to be less effective despite intervening at an earlier stage, (d) universal programmes which might cross a notional line between the state and the family, and (e) approaches which focus primarily on local experimentation with uncertain outcomes.

None of these approaches provide a solution to the early intervention dilemmas that all governments face, and there is no easy way to choose between approaches. We can make these choices more informed and systematic, by highlighting how all of the pieces of the jigsaw fit together, and somehow comparing their intended and unintended consequences. However, this process does not replace political judgement – and quite right too – because there is no such thing as a valence issue at the point of policy choice and delivery.

See also:

Paul Cairney (2019) ‘The UK government’s imaginative use of evidence to make policy’, British Politics, 14, 1, 1-22 Open Access PDF

Paul Cairney and Emily St Denny (in press, January 2020) Why Isn’t Government Policy More Preventive? (Oxford: Oxford University Press) Preview Introduction Preview Conclusion

 

 

 

 

12 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), Prevention policy, public policy, Scottish politics, UK politics and policy

Idealism versus pragmatism in politics and policymaking: Labour, Brexit, and evidence-based policymaking

In a series of heroic leaps of logic, I aim to highlight some important links between three current concerns: Labour’s leadership contest, the Brexit vote built on emotion over facts, and the insufficient use of evidence in policy. In each case, there is a notional competition between ‘idealism’ and ‘pragmatism’ (as defined in common use, not philosophy); the often-unrealistic pursuit of a long term ideal versus the focus on solving more immediate problems often by compromising ideals and getting your hands dirty. We know what this looks like in party politics, including the compromises that politicians make to win elections and the consequences for their image, but do we know how to make the same compromises when we appeal for a more deliberative referendum or more evidence-informed policymaking?

I searched Google for a few minutes until I found a decent hook for this post. It is a short Forbes article by Susan Gunelius advocating a good mix of pragmatic and idealistic team members:

Pragmatic leaders focus on the practical, “how do we get this done,” side of any task, initiative or goal.  They can erroneously be viewed as negative in their approach when in fact they simply view the entire picture (roadblocks included) to get to the end result.  It’s a linear, practical way of thinking and “doing.”

Idealist leaders focus on the visionary, big ideas.  It could be argued that they focus more on the end result than the path to get there, and they can erroneously be viewed as looking through rose-colored glasses when, in fact, they simply “see” the end goal and truly believe there is a way to get there”.

On the surface, it’s a neat description of the current battle to win the Labour party, with Jeremy Corbyn representing the idealist willing to lose elections to stay true to the pure ideal, and Owen Smith representing the pragmatist willing to compromise on the ideal to win an election.

In this context, pragmatic politicians face a dilemma that we often take for granted in party politics: they want to look flexible enough to command the ‘centre’ ground, but also appear principled and unwilling to give up completely on their values to secure office. Perhaps pragmatists also accept to a large extent that the means can justify the ends: they can compromise their integrity and break a few rules to win office if it means that they serve the long term greater good as a result (in this case, better a compromised socialist than a Tory government). So, politicians accept that a slightly tarnished image is the price you pay to get what you want.

For current purposes, let us assume that you are the kind of person drawn more to the pragmatist rather than the idealist politician; you despair at the naiveté of the idealist politician, and expect to see them fail rather than gain office.

If so, how might we draw comparisons with other areas in politics and policymaking?

Referendums should be driven by facts and an intelligent public, not lies and emotions

Many people either joke or complain seriously about most of the public being too stupid to engage effectively in elections and referendums. I will use this joke about Trump because I saw it as a meme, and on Facebook it has 49000 smiley faces already:

Borowitz

An more serious idealistic argument about the Brexit vote goes something like this:

  • the case for Remain was relatively strong and backed by most of the best experts
  • most Leave voters ignored or mistrusted the experts
  • the Leave campaign was riddled with lies and exaggerations; and,
  • a large chunk of the public was not intelligent enough to separate the lies from the facts.

You often have to read between the lines and piece together this argument, but Dame Liz Forgan recently did me a favour by spelling out a key part in a speech to the British Academy:

Democracies require not just literate and numerate electorates. They need people who cannot be sold snake oil by every passing shyster because their critical faculties have been properly honed. Whose popular culture has not degenerated so completely that every shopping channel hostess is classed as a celebrity. Where post-modern irony doesn’t undermine both honest relaxation and serious endeavour. Where the idea of a post-factual age is seen as an acute peril not an amusing cultural meme. If the events of June have taught us anything it is that we need to put the rigour back in our education, the search for truth back in our media.

Of course, I have cherry picked the juiciest part to highlight a sense of idealism that I have seen in many places. Let’s link it back to our despair at the naïvely idealist politician: doesn’t this look quite similar? If we took this line, and pursued public education as our main solution to Brexit, wouldn’t people think that we are doomed to fail in the long term and lose a lot of other votes on the way?

Another (albeit quicker and less idealistic) solution, proposed largely by academics (many of whom are highly critical of the campaigns) is largely institutional: let’s investigate the abuse of facts during the referendum to help us produce new rules of engagement. Yet, if the problem is that people are too stupid or emotional to process facts, it doesn’t seem that much more effective.

At this stage, I’d like to say: instead of clinging to idealism, let’s be pragmatic about this. If you despair of the world, get your hands dirty to win key votes rather than hope that people will do the right thing or wait for a sufficiently ‘rational’ public.

Yet, I don’t think we yet know enough about how to do it and how far ‘experts’ should go, particularly since many experts are funded – directly or indirectly – by the state and are subject to different (albeit often unwritten) rules than politicians. So, in a separate post, I provide some bland advice that might apply to all:

  • Don’t simply supply people with more information when you think they are not paying enough attention to it. Instead, try to work out how they think, to examine how they are likely to demand and interpret information.
  • Don’t just bemoan the tendency of people to accept simple stories that reinforce their biases. Instead, try to work out how to produce evidence-based stories that can compete for attention with those of campaigners.
  • Don’t stop at providing simpler and more accessible information. People might be more likely to read a blog post than a book or lengthy report, but most people are likely to remain blissfully unaware of most academic blogs.

Yet, if we think that other referendum participants are winning because they are lying and cheating, we might also think that honourable strategies won’t tip the balance. We know that, like pragmatic politicians, we might need to go a bit further to win key debates. Anything else is idealism, right?

Policy should be based on evidence, not electoral politics, ideology and emotion

The same can be said for many scientists bemoaning the lack of ‘evidence-based policymaking’ (EBPM). Some express the naïve hope that politicians become trained to think like scientists and/ or the view that evidence-based policymaking should be more like the idea of evidence-based medicine in which there is a hierarchy of evidence. Others try to work out how they can improve the supply of evidence or set up new institutions to get policymakers to pay more attention to facts. This seems to be EBPM’s equivalent of idealism, in which you largely wish for something that won’t exist rather than trying to produce pragmatic strategies for the real world.

A more pragmatic two-step solution is to:

(1) work out how and why policymakers demand information, and the policymaking context in which they operate (which I describe in The Politics of Evidence-Based Policymaking, and with Kathryn Oliver and Adam Wellstead in PAR).

(2) draw on as many interdisciplinary insights to explore how to do something about it, such as to establish the psychology of policymakers and identify good ways to tell simple stories to generate an emotional connection to your evidence (which I describe in a forthcoming special issue in Palgrave Communications).

Should academics remain idealists rather than pragmatists?

Of course, it is legitimate to take what I am calling an idealistic approach. In politics, Corbyn’s idealism is certainly capturing a part of the public imagination (while another part of the public watches on, sniggering or aghast). In the Academy, it may be a part of a legitimate attempt to maintain your integrity by not engaging directly in politics or policymaking, and/or accepting that academics largely contribute to a very long term enlightenment function rather than enjoy immediate impact. All I am saying is that you need to choose and, if you seek more direct impact, you need to forego idealism and start thinking about what it means to be pragmatic while pursuing ‘evidence informed’ politics.

2 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), public policy, UK politics and policy

What we know about consultation and policy-making

Here are some notes for today’s workshop on ‘consultation’, at the Law Reform and Public Policy Group, School of Law, University of Glasgow. I discuss key insights from policy theory, the idea of a ‘draft Act’, and how the ‘Scottish policy style’ fits into this discussion.

My second favourite phrase, as an undergraduate at Glasgow’s top University, was Brian Hogwood’s: ‘if consultation means everything, then maybe it means nothing’. It was a call to ‘unpack’ the term, whose meaning could range from cosmetic consultation in public to crucial interventions in private.

My first favourite was Grant Jordan and Jeremy Richardson’s ‘policy community’, which describes an important relationship between policymakers and some of the actors they consult (see also ‘informal governance’, including Alison Woodward’s example of the ‘velvet triangle’). The logic is as follows:

1. Policymakers are subject tobounded rationality’: they cannot process issues comprehensively. By necessity, they have to make decisions in the face of uncertainty and ambiguity. Uncertainty relates to the amount of information we have to inform policy and policymaking. Ambiguity relates to the way in which we understand policy problems. The policy process is therefore about (1) the short cuts that policymakers use to gather information and understand complex issues, and (2) the ways in which policy participants compete to determine which information is used and how policymakers understand problems.

2. Policymakers and key actors form policy networks or communities. To deal with bounded rationality, they delegate responsibility to civil servants who, in turn, rely on specialist organisations for information and advice. Those organisations trade information for access to government. This process often becomes routine: civil servants begin to trust and rely on certain organisations and they form meaningful relationships. If so, most public policy is conducted primarily through small and specialist ‘policy communities’ that process issues at a level of government not particularly visible to the public, and with minimal senior policymaker involvement. Network theories tend to consider the key implications, including a tendency for governments to contain ‘silos’ and struggle to ‘join up’ government when policy is made in so many different places.

3. Note the relevance to our current focus on ‘evidence-based policymaking. In most cases, policymakers use consultation to reduce uncertainty: they gather information to help identify the size of a problem on which they already have a view. In some, they use it to reduce ambiguity: only some actors will influence how they understand and try to solve a policy problem, and therefore what further information they will seek.

4. Note the importance of pluralist democracy to some policymakers. Consider the implications of bounded rationality to democracy: we put our faith in representative democracy, but ministers can only pay attention to a tiny proportion of their responsibilities. A lot of practical responsibility is in the hands of civil servants, who partly seek legitimacy by consulting far and wide, to generate the ‘ownership’ of policy among key actors, professional groups, and perhaps ‘civil society’. This takes place well before, for example, a government presents a draft Act to Parliament and, in many cases, it produces policy without subsequent reference to Parliament.

If you put all these things together, you can see the importance of different kinds of consultation.

Consider a simple spectrum of consultation. At one end is cosmetic consultation: policymakers are paying high attention and they already know how they feel about a problem. So, they consult as part of a ‘standard operating procedure’ in which governments seek legitimacy, but the consultation will not influence their decision. Perhaps it comes towards the end of their deliberations.

At the other end is super-meaningful consultation, but perhaps with a small number of key people: they get together regularly, identify the issues that deserve most attention, and agree on how to ‘frame’ or understand the problem.

So, in our discussions, we might discuss how a range of activities fit in: the ‘trawling’ exercises to gain as many views as possible; working groups to generate questions for consultation; commissions to process rather technical looking issues, generally out of the public spotlight; other ‘pre-consultation’ with key actors before public consultation; and so on.

You can also see how the ‘Scottish Policy Style’ or ‘Scottish Approach’ fits in

We talk about a distinctive Scottish style or approach or system, but within the context that I describe above. There are three key reference points:

  1. Cultural. In the olden days we talked of new Scottish politics versus old Westminster: Scottish policymaking would be more consensual and participative; the Scottish Government would share power with the Scottish Parliament; the Parliament would be the hub for much consultation, or at least oversee the Scottish Government process; the Parliament would have a better gender balance; and, policymakers would not simply consult the ‘usual suspects’.
  2. Practical. In reality, most explanations for a Scottish policymaking culture relate to size and scale: it is easier for senior policymakers to form personal networks with key actors in interest and professional groups and with leaders of public bodies; and, a government with relatively low capacity relies relatively highly on external sources of information and advice.
  3. Storytelling. Note that the Scottish Government tells a particular story about its approach, built on high consultation and trust in public bodies, stakeholders and service users, which informs its approach to evidence gathering and policy delivery: the Scottish Approach to Policymaking.

An agenda for the study of consultation in Scotland

It is useful to talk of a Scottish style of consultation, but investigate rather than assume its distinctiveness, and to interpret that distinctiveness rather than assume it relates broadly to a Scottish political culture.

This is true even before we consider consultation in a multi-level system, of which the Scottish Government is one of many governments that groups may want to consult.

I usually do this research through interviews with pressure participants and civil servants, which often involves trying to separate the story we tell about Scotland (where everyone knows everyone else) from the other drivers for policy and policymaking (education, mental health legislation, general, more general, even more general).

Other methods worth discussing include consultation analysis (Darren Halpin used to analyse the number and types of responses to open consultations), networks analysis to gauge the interaction between policy makers and participants, and comparative analyses in which we examine, for example, the extent to which consultation on legal reform resembles that of other sectors. If not, what makes ‘the law’ distinctive?

1 Comment

Filed under agenda setting, Evidence Based Policymaking (EBPM), public policy, Scottish politics

Policy bubbles and emotional policymaking

I am at a workshop today on policy ‘bubbles’, or (real and perceived) disproportionate policy responses. For Moshe Maor, a bubble describes an over-reaction to a problem, and a negative policy bubble describes under-reaction.

For Maor, this focus on bubbles is one way into our increasing focus on the role of emotion in policymaking: we pay disproportionate attention to problems, and try to solve some but not others, based on the ways in which we engage emotionally with information.

This focus on psychology is, I think, gaining a lot of traction in political science now, and I think it is crucial to explaining, for example, processes associated with ‘evidence-based policymaking’.

In taking this agenda forward, there remain some outstanding issues:

How much of the psychology literature is already reflected in policy studies? For example, see the social construction of target populations (emotion-driven treatment of social groups), ACF (on loss aversion and the devil shift), and the NPF (telling stories to exploit cognitive biases).

What insights remain untapped from key fields such as organisational psychology? I’ll say more about this in a forthcoming post.

How can we study the psychology of policymaking? Most policy theory begins with some reference to bounded rationality, including PET and the identification of disproportionate information processing (policymakers pay disproportionate attention to some issues and ignore the rest). It is largely deductive then empirical: we make some logical steps about the implications of bounded rationality, then study the process in that light.

Similarly, I think most studies of emotion/ policymaking take insights from psychology (e.g. people value losses more than gains, or they make moral judgements then seek evidence to justify them) and then apply them indirectly to policymaking (asking, for example, what is the effect of prospect theory on the behaviour of coalitions).

Can we do more, by studying more directly the actions of policymakers rather than merely interpreting their actions? The problem, of course, is that few policymakers may be keen on engaging in the types of study (e.g. experiments with control groups) that psychologists have used to establish things like fluency effects.

How does policymaker psychology fit into broader explanations of policymaking? The psychology of policymakers is one part of the story. The other is the system or environment in which they operate. So, we have some choices to make about future studies. Some might ‘zoom in’ to focus on emotionally-driven policymaking in key actors, perhaps at the centre of government.

Others may ‘zoom out’. The latter may involve ascribing the same basic thought processes to a large number of actors, examining that process at a relatively abstract level. This is the necessary consequence of trying to account for the effects of a very large number of actors, and to take into account the role of a policymaking environment, only some of which is in the control of policymakers.

Can we really demonstrate disproportionate policy action? The idea of a proportionate policy response interests me, because I think it is always in the eye of the beholder. We make moral and other personal evaluative statements when we describe a proportionate solution in relation to the size of the problem.

For example, in tobacco policy, a well-established argument in public health is that a proportionate policy response to the health effects of smoking and passive smoking (a) has been 20-30 years behind the evidence in ‘leading countries’, and (b) has yet to happen in ‘laggard’ countries. The counterargument is that the identification of a problem does not necessitate the favoured public health solution (comprehensive tobacco control, towards the ‘endgame’ of zero smoking) because it involves major limits to personal liberties and choice.

Is emotion-driven policymaking necessarily a bad thing?

[excerpt from my 2014 PSA paper ] This is partly the focus of Alter and Oppenheimer (2008) when they argue that policymakers spend disproportionate amounts of money on risks with which they are familiar, at the expense of spending money on things with more negative effects, producing a ‘dramatic misallocation of funds’. They draw on Sunstein (2002), who suggests that emotional bases for attention to environmental problems from the 1970s prompted many regulations to be disproportionate to the risk involved. Further, Slovic’s work suggest that people’s feelings towards risk may even be influenced by the way in which it is described, for example as a percentage versus a 1 in X probability (Slovic, P. 2010: xxii).

Haidt (2001: 815) argues that a focus on psychology can be used to improve policymaking: the identification of the ‘intuitive basis of moral judgment’ can be used to help policymakers ‘avoid mistakes’ or allow people to develop ‘programs’ or an ‘environment’ to ‘improve the quality of moral judgment and behavior’. Similarly, Alter and Oppenheimer (2009: 232) worry about medical and legal judgements swayed by fluid diagnoses and stories.

These studies compare with arguments focusing on the positive role of emotions of decision-making, either individually (see Constantinescu, 2012, drawing on Frank, 1988 and Elster, 2000 on the decisions of judges) or as part of social groups, with emotional responses providing useful information in the form of social cues (Van Kleef et al, 2010).

Policy theory does not shy away from these issues. For example, Schneider and Ingram (2014) argue that the outcomes of social construction are often dysfunctional and not based on a well-reasoned, goal-oriented strategy: ‘Studies have shown that rules, tools, rationales and implementation structures inspired by social constructions send dysfunctional messages and poor choices may hamper the effectiveness of policy’. However, part of the value of policy theory is to show that policy results from the interaction of large numbers of people and institutions. So, the poor actions of one policymaker would not be the issue; we need to know more about the cumulative effect of individual emotional decision making in collective decision-making – not only in discrete organisations, but also networks and systems.

And finally: if it is a bad thing, should we do something about it?

Our choice is to find it interesting then go home (this might appeal to the academics) or try to limit the damage/ maximise the benefits of policymaker psychology to policy and society (this might appeal to practitioners). There is no obvious way to do something, though, is there?

2 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), public policy

Brexit and the inevitability of Scottish Independence

image for POLU9SP

My gut says that there will be a second referendum on Scottish independence and that Yes will win comfortably. Yet, predicting political events and outcomes right now is like predicting the weather. The result is not inevitable, largely because the key factors prompting people to vote No have not gone away – and, in some ways, the No case is now stronger. I’ll explain this by (a) comparing the likely Yes and No stories during the next campaign, and (b) speculating wildly about the extent to which key parties will campaign as hard for No in the second referendum.

Brexit is a Godsend for the strongest Yes stories: 1. only independence can remove the democratic deficit and guarantee that we make our own decisions.

It sounds like the Brexit ‘take our country back’ story, in which a remote government in a remote city makes decisions on our behalf without us having any say (it’s ‘London’ for Scotland, whereas in England/ Wales it can be London or ‘Brussels’).

Yet, there are key differences: the SNP is pro-immigration (its nationalism is ‘civic’, not ‘ethnic’) and the ‘democratic deficit’ means something else. When applied to the EU, it means that (a) few people know how it works and who, if anyone, is accountable, and (b) that it is difficult to vote for EU policymakers in the same way as we vote for national governments.

When applied to Scotland, it means that most voters in Scotland have tended to vote Labour or SNP in Westminster elections, but they often get a UK Tory government. So, a government with no legitimacy in Scotland makes decisions on our behalf, and there is nothing the Scottish Government can do about it.

In the campaign for devolution, this story developed in opposition to the Thatcherite imposition of things like the poll tax. In the campaign for independence, the poll tax became the bedroom tax.

In the next campaign for independence, the Brexit vote will become an important symbol for this argument: we voted overwhelmingly to stay in the EU and we are being dragged out against our will by England (and Wales).

I think this argument will win the day, for two reasons. First, most of the 45% who voted Yes in 2014 seem like a sure bet for the next vote. Second, there are some people who voted No on the assumption of remaining in the UK in the EU. They now have to choose between (a) in the UK and out of the EU, or (b) in the EU and out of the UK.

  1. Scottish independence is the cosmopolitan choice

Crucially, the Brexit is a godsend for the argument that Scottish independence is the cosmopolitan choice. It was too easy for opponents to argue in 2014 that nationalism was parochialism: by focusing on Scotland, you are removing yourself from the world. The counter-argument – let’s become independent to play a more positive role in that world – was relatively difficult to make.

Now, the door is open to argue that the Brexit vote reflects a Little England mentality, and that only Scottish independence offers the chance to cooperate fully with our European partners. In Scotland, cosmopolitan voters will share a campaign with nationalist voters.

Put these parts together and you have this story: independence is the only solution to being ruled from afar by the Tories who are determined (with the help of UKIP) to turn us into a Little England state which blames immigrants or the rest of the world for its problems.

Yet, the No story remains powerful too, for two original reasons and one new reason.

The No story: 1. Economic damage, uncertainty, and the currency issue

‘Better Together’ campaigned hard on the idea that a Yes vote will be economically damaging, producing a major government deficit in the short term with no guarantee of improvement in the long term (note that the rest of the UK is Scotland’s biggest trading partner, and we need their partnership). They also argued successfully that Scotland could no longer use Sterling if independent (which really meant that the Scottish Government would no longer enjoy the same crucial relationship with the Bank of England).

Most No voters will have felt good about their decision because the price of a barrel of oil plummeted after 2014, giving the impression that Scotland’s short term economic deficit would have been even higher. Further, the currency issue remains unresolved, and the main alternatives to using the pound with the UK Government/ BoE’s blessing (Sterlingisation, a Scottish pound, joining the Euro) still won’t seem like brilliant prospects to undecided voters.

The No story: 2. The Yes vote meant all things to all people.

A further No argument related to the idea that all sorts of people were making all sorts of claims about a future independent Scotland, and that they couldn’t all be right. The Scottish Government’s ‘White Paper’ was more sensible, but was still built on hope more than expectation. So, if you don’t share that optimism, it just looks like a long document designed to look professional and reassuring without really providing a blueprint for action or a measured set of expectations.

A third new part of the story: we now see what happens when you vote to leave (and it’s bad)

The biggest effect of the Brexit on the No story is that we can already see what happens when people vote to leave a political union:

(1) We immediately see that people were making all sorts of promises that they couldn’t keep, and/ or that key people backtrack very quickly (examples after the Brexit vote include the ridiculous £350m for the NHS claim, and the now more modest claims about immigration). It’s easy to say what you are leaving behind, but not what you will do instead.

(2) We immediately see some frightening economic consequences.

(3) We are about to discover how our former political partners will react, and it doesn’t look like they’ll simply hug us and wish us all the best.

So, (4) the No campaign will be about emphasising this uncertainty and the poor consequences of political divorce as they are happening in real time.

In the end, it comes down to who will tell these Yes/ No stories and how well they do it

The main problem for a new No campaign is that I don’t think it will have the same backing. In the first campaign, almost all of the main parties against independence signed up to a common project. Yet, it was damaging to Scottish Labour and I doubt they’ll sign up a second time to represent the ‘Red Tories’, particularly since many members will vote Yes next time.

It will be largely down to Ruth Davidson and the Scottish Conservatives, who campaigned in 2016 as the SNP’s main opposition and the defenders of the UK. Although they did pretty well in the Holyrood elections, pretty well means 23% of the vote.

In contrast, the SNP is a highly professional outfit, which lost a referendum but gained a huge membership, has a very popular leadership, and still enjoys an incredibly strong image of governing competence (particularly for a party in government for 9 years).

If you want to put it more simply and to personalise the next campaign, I simply say this:

Nicola Sturgeon has already perfected the look of someone pissed off with UK Government incompetence, reluctantly proposing a second referendum to deal with the mess, and able to reject most arguments about economic and political uncertainty as bloody rich coming from the people who just voted to leave the EU. Salmond might have looked too (‘I told you so’) smug to pull it off, but Sturgeon looks genuinely annoyed rather than opportunistic.

Who can perform the same function for the No side? There are almost no London-based politicians that could generate the same kind of respect that Sturgeon enjoys. Ruth Davidson is the next best thing, but she will spend a fair amount of each debate being a bit embarrassed about the situation in which she finds herself, through no fault of her own.

So, the irony may be that No has, in some ways, a stronger case in the second referendum but a far lower chance of success: it will lose because there will be no-one out there able to tell the No story.

This emphasis on telling simple stories well matters more than we would like to admit. The facts don’t speak for themselves: you turn them into a story to engage with people’s existing biases and tendency to base decisions on very little information.  So, who will tell and listen to the No story the next time around?

See also:

We are in danger of repeating the same mistakes if we bemoan low attention to ‘facts’

Heresthetics and referendums

I also wrote a million posts on the last Scottish referendum

 

 

11 Comments

Filed under agenda setting, Scottish independence, Scottish politics

We are in danger of repeating the same mistakes if we bemoan low attention to ‘facts’

A key theme of some of the early analysis of Brexit is that many voters followed their feelings rather than paying attention to facts*.

For some people, this is just a part of life: to describe decision-making as ‘rational’ is to deny the inevitable use of heuristics, gut feelings, emotions, and deeply held beliefs.

For others, it is indicative of a worrying ‘post-truth politics’, or a new world in which campaigners play fast and loose with evidence and say anything to win, while experts are mistrusted and ignored or excluded from debates, and voters don’t get the facts they need to make informed decisions.

One solution, proposed largely by academics (many of whom are highly critical of the campaigns) is largely institutional: let’s investigate the abuse of facts during the referendum to help us produce new rules of engagement.

Another is more pragmatic: let’s work out how to maximise the effectiveness of experts and evidence in political debate. So far, we know more about what doesn’t work. For example:

  • Don’t simply supply people with more information when you think they are not paying enough attention to it. Instead, try to work out how they think, to examine how they are likely to demand and interpret information.
  • Don’t just bemoan the tendency of people to accept simple stories that reinforce their biases. Instead, try to work out how to produce evidence-based stories that can compete for attention with those of campaigners.
  • Don’t stop at providing simpler and more accessible information. People might be more likely to read a blog post than a book or lengthy report, but most people are likely to remain blissfully unaware of most academic blogs.

I’m honestly not sure how to tell good stories to capture the public imagination (beyond that time I put the word ‘shite’ in a title) but, for example, we have a lot to learn from traditional media (and from some of the most effective academics who write for them) and from scholars who study story-telling and discourse (although, ironically, discourse analysis is often one of the most jargon-filled areas in the Academy).

We have been here before (in policy studies)

This issue of agenda setting is a key feature in current discussions of (the alleged lack of) evidence-based policymaking. Many academics, in areas such as health and environmental policy, bemoan the inevitability of ‘policy based evidence’. Some express the naïve view that policymakers should think like scientists and/ or that evidence-based policymaking should be more like the idea of evidence-based medicine in which there is a hierarchy of evidence. Others try to work out how they can improve the supply of evidence or set up new institutions to get policymakers to pay more attention to facts.

Yet, a more pragmatic solution is to work out how and why policymakers demand information, and the policymaking context in which they operate. Only then can we produce evidence-based strategies based on how the world works rather than how we would like it to work.

See also:

The Politics of Evidence Based Policymaking:3 messages

Evidence-based policymaking: lecture and Q&A

‘Evidence-based Policymaking’ and the Study of Public Policy

Paul Cairney (2016) The Politics of Evidence-based Policymaking (London: Palgrave Pivot) PDF

Paul Cairney, Kathryn Oliver, and Adam Wellstead (2016) ‘To Bridge the Divide between Evidence and Policy: Reduce Ambiguity as Much as Uncertainty’, Public Administration Review, Early View (forthcoming) DOI:10.1111/puar.12555 PDF

* Then, many people on twitter vented their negative feelings about other people expressing their feelings.

13 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), public policy, UK politics and policy