Tag Archives: Policy studies

Tips for policy success: learn from ‘policy entrepreneurs’ and exploit ‘windows of opportunity’

Policy influence is impossible to find if you don’t know where to look. Policies theories can help you look in the right places, but they take time to understand.

It’s not realistic to expect people with their own day jobs – such as scientists producing policy-relevant knowledge in other fields – to take the time to use the insights it takes my colleagues a full-time career to appreciate.

So, we need a way to explain those insights in a way that people can pick up and use when they engage in the policy process for the first time. That’s why Chris Weible and I asked a group of policy theory experts to describe the ‘state of the art’ in their field and the practical lessons that they offer.

None of these abstract theories provide a ‘blueprint’ for action (they were designed primarily to examine the policy process scientifically). Instead, they offer one simple insight: you’ll save a lot of energy if you engage with the policy process that exists, not the one you want to see.

Then, they describe variations on the same themes, including:

  1. There are profound limits to the power of individual policymakers: they can only process so much information, have to ignore almost all issues, and therefore tend to share policymaking with many other actors.
  2. You can increase your chances of success if you work with that insight: identify the right policymakers, the ‘venues’ in which they operate, and the ‘rules of the game’ in each venue; build networks and form coalitions to engage in those venues; shape agendas by framing problems and telling good stories, design politically feasible solutions, and learn how to exploit ‘windows of opportunity’ for their selection.

Learn from ‘multiple streams’ analysis

Streams pic and textMy paper on the ‘multiple streams approach’ shows what happens in the absence of two things you might want to see: ‘rational’ and ‘evidence based’ policymaking which takes place in a policy cycle with linear stages. If you act according to that hope, you’ll likely say the wrong thing to the wrong people at the wrong time. It would be better to adapt to the following implications of an agenda setting process in which framing is more important than evidence, and solutions chase problems (table 1).

Tablle 1 MSA

Learn the meaning of timing and windows of opportunity

Most people would associate ‘timing’ with the idiom ‘be in the right place at the right time’. In agenda setting it means two more important things:

  1. Learning the right time to exploit emotional thinking in policymakers to help generate attention to a policy problem, not waiting for their attention to shift naturally.
  2. Producing policy solutions first, then waiting for the right time to attach them to problems. If a policy cycle existed, policymakers would identify a problem then spark of a series of stages, to select a solution, implement, and evaluate it. In the real world, policymaker attention often shifts before a feasible solution can be developed.

Learn from ‘policy entrepreneurs’

So, successful ‘policy entrepreneurs’ ‘lie in wait in and around government with their solutions at hand, waiting for problems to float by to which they can attach their solutions, waiting for a development in the political stream they can use to their advantage’ (Kingdon 1984: 165–6). Entrepreneurs are the elected policymakers or unelected influencers with the knowledge, power, tenacity and luck to be able to exploit ‘windows of opportunity’ when: attention rises to a problem, a feasible solution is available, and policymakers have the motive to select it.

Learn if you can be the Queen of Makaha, Poseidon, or Cnut

Policy entrepreneurs seem to have particular skills or strategies, to frame issues well, build networks, and lead coalitions. However, Kingdon described them as ‘surfers waiting for the big wave’, which suggests that their environment is more important than their action. He was describing a large US political system in which different actors tended to be involved in different ‘streams’ or parts of policymaking (such as a President raising problems, and a bureaucracy coordinating solutions), no one was powerful enough to bring them together, and it took a lot of time for policy solutions to ‘soften’ or change enough to become acceptable to many actors in the system.

In modern studies, we can see some key differences: policymaking at a smaller scale seems to allow ‘entrepreneurs’ more opportunities to propose solutions and generate attention to problems; and, it seems possible to short-circuit the need to ‘soften’ policies by finding sympathetic audiences in different ‘venues’ or importing solutions that have a reputation for working elsewhere. Yet, most of MSA’s abstract insights remain ‘universal’, inviting us to  adopt a counterintuitive strategy of producing solutions then chasing problems, and focus on framing and persuasion to reduce ambiguity and generate demand for evidence, rather than producing more and more evidence to reduce uncertainty in the hope that scientific evidence will win the day or speak for itself.

The full draft paper is here: Practical lessons from the study of agenda setting: combine evidence with emotional appeals to exploit ‘windows of opportunity’

See also: Three ways to explain the politics of evidence-based policymaking

Original streams pic

1 Comment

Filed under agenda setting, Evidence Based Policymaking (EBPM), public policy

‘Co-producing’ comparative policy research: how far should we go to secure policy impact?

Two recent articles explore the role of academics in the ‘co-production’ of policy and/or knowledge.

Both papers suggest (I think) that academic engagement in the ‘real world’ is highly valuable, and that we should not pretend that we can remain aloof from politics when producing new knowledge (research production is political even if it is not overtly party political). They also suggest that it is fraught with difficulty and, perhaps, an often-thankless task with no guarantee of professional or policy payoffs (intrinsic motivation still trumps extrinsic motivation).

So, what should we do?

I plan to experiment a little bit while conducting some new research over the next 4 years. For example, I am part of a new project called IMAJINE, and plan to speak with policymakers, from the start to the end, about what they want from the research and how they’ll use it. My working assumption is that it will help boost the academic value and policy relevance of the research.

I have mocked up a paper abstract to describe this kind of work:

In this paper, we use policy theory to explain why the ‘co-production’ of comparative research with policymakers makes it more policy relevant: it allows researchers to frame their policy analysis with reference to the ways in which policymakers frame policy problems; and, it helps them identify which policymaking venues matter, and the rules of engagement within them.  In other words, theoretically-informed researchers can, to some extent, emulate the strategies of interest groups when they work out ‘where the action is’ and how to adapt to policy agendas to maximise their influence. Successful groups identify their audience and work out what it wants, rather than present their own fixed views to anyone who will listen.

Yet, when described so provocatively, our argument raises several practical and ethical dilemmas about the role of academic research. In abstract discussions, they include questions such as: should you engage this much with politics and policymakers, or maintain a critical distance; and, if you engage, should you simply reflect or seek to influence the policy agenda? In practice, such binary choices are artificial, prompting us to explore how to manage our engagement in politics and reflect on our potential influence.

We explore these issues with reference to a new Horizon 2020 funded project IMAJINE, which includes a work package – led by Cairney – on the use of evidence and learning from the many ways in which EU, national, and regional policymakers have tried to reduce territorial inequalities.

So, in the paper we (my future research partner and I), would:

  • Outline the payoffs to this engage-early approach. Early engagement will inform the research questions you ask, how you ask them, and how you ‘frame’ the results. It should also help produce more academic publications (which is still the key consideration for many academics), partly because this early approach will help us speak with some authority about policy and policymaking in many countries.
  • Describe the complications of engaging with different policymakers in many ‘venues’ in different countries: you would expect very different questions to arise, and perhaps struggle to manage competing audience demands.
  • Raise practical questions about the research audience, including: should we interview key advocacy groups and private sources of funding for applied research, as well as policymakers, when refining questions? I ask this question partly because it can be more effective to communicate evidence via policy influencers rather than try to engage directly with policymakers.
  • Raise ethical questions, including: what if policymaker interviewees want the ‘wrong’ questions answered? What if they are only interested in policy solutions that we think are misguided, either because the evidence-base is limited (and yet they seek a magic bullet) or their aims are based primarily on ideology (an allegedly typical dilemma regards left-wing academics providing research for right-wing governments)?

Overall, you can see the potential problems: you ‘enter’ the political arena to find that it is highly political! You find that policymakers are mostly interested in (what you believe are) ineffective or inappropriate solutions and/ or they think about the problem in ways that make you, say, uncomfortable. So, should you engage in a critical way, risking exclusion from the ‘coproduction’ of policy, or in a pragmatic way, to ‘coproduce’ knowledge and maximise your chances of their impact in government?

The case study of territorial inequalities is a key source of such dilemmas …

…partly because it is difficult to tell how policymakers define and want to solve such policy problems. When defining ‘territorial inequalities’, they can refer broadly to geographical spread, such as within the EU Member States, or even within regions of states. They can focus on economic inequalities, inequalities linked strongly to gender, race or ethnicity, mental health, disability, and/ or inequalities spread across generations. They can focus on indicators of inequalities in areas such as health and education outcomes, housing tenure and quality, transport, and engagement with social work and criminal justice. While policymakers might want to address all such issues, they also prioritise the problems they want to solve and the policy instruments they are prepared to use.

When considering solutions, they can choose from three basic categories:

  1. Tax and spending to redistribute income and wealth, perhaps treating economic inequalities as the source of most others (such as health and education inequalities).
  2. The provision of public services to help mitigate the effects of economic and other inequalities (such as free healthcare and education, and public transport in urban and rural areas).
  3. The adoption of ‘prevention’ strategies to engage as early as possible in people’s lives, on the assumption that key inequalities are well-established by the time children are three years old.

Based on my previous work with Emily St Denny, I’d expect that many governments express a high commitment to reduce inequalities – and it is often sincere – but without wanting to use tax/ spending as the primary means, and faced with limited evidence on the effectiveness of public services and prevention. Or, many will prefer to identify ‘evidence-based’ solutions for individuals rather than to address ‘structural’ factors linked to factors such as gender, ethnicity, and class. This is when the production and use of evidence becomes overtly ‘political’, because at the heart of many of these discussions is the extent to which individuals or their environments are to blame for unequal outcomes, and if richer regions should compensate poorer regions.

‘The evidence’ will not ‘win the day’ in such debates. Rather, the choice will be between, for example: (a) pragmatism, to frame evidence to contribute to well-established beliefs, about policy problems and solutions, held by the dominant actors in each political system; and, (b) critical distance, to produce what you feel to be the best evidence generated in the right way, and challenge policymakers to explain why they won’t use it. I suspect that (a) is more effective, but (b) better reflects what most academics thought they were signing up to.

For more on IMAJINE, see New EU study looks at gap between rich and poor and The theory and practice of evidence-based policy transfer: can we learn how to reduce territorial inequalities?

For more on evidence/ policy dilemmas, see Kathryn Oliver and I have just published an article on the relationship between evidence and policy

 

1 Comment

Filed under Evidence Based Policymaking (EBPM), IMAJINE, public policy

Kathryn Oliver and I have just published an article on the relationship between evidence and policy

Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy?

“There is extensive health and public health literature on the ‘evidence-policy gap’, exploring the frustrating experiences of scientists trying to secure a response to the problems and solutions they raise and identifying the need for better evidence to reduce policymaker uncertainty. We offer a new perspective by using policy theory to propose research with greater impact, identifying the need to use persuasion to reduce ambiguity, and to adapt to multi-level policymaking systems”.

We use this table to describe how the policy process works, how effective actors respond, and the dilemmas that arise for advocates of scientific evidence: should they act this way too?

We summarise this argument in two posts for:

The Guardian If scientists want to influence policymaking, they need to understand it

Sax Institute The evidence policy gap: changing the research mindset is only the beginning

The article is part of a wider body of work in which one or both of us considers the relationship between evidence and policy in different ways, including:

Paul Cairney, Kathryn Oliver, and Adam Wellstead (2016) ‘To Bridge the Divide between Evidence and Policy: Reduce Ambiguity as Much as Uncertainty’, Public Administration Review PDF

Paul Cairney (2016) The Politics of Evidence-Based Policy Making (PDF)

Oliver, K., Innvar, S., Lorenc, T., Woodman, J. and Thomas, J. (2014a) ‘A systematic review of barriers to and facilitators of the use of evidence by policymakers’ BMC health services research, 14 (1), 2. http://www.biomedcentral.com/1472-6963/14/2

Oliver, K., Lorenc, T., & Innvær, S. (2014b) ‘New directions in evidence-based policy research: a critical analysis of the literature’, Health Research Policy and Systems, 12, 34 http://www.biomedcentral.com/content/pdf/1478-4505-12-34.pdf

Paul Cairney (2016) Evidence-based best practice is more political than it looks in Evidence and Policy

Many of my blog posts explore how people like scientists or researchers might understand and respond to the policy process:

The Science of Evidence-based Policymaking: How to Be Heard

When presenting evidence to policymakers, engage with the policy process that exists, not the process you wish existed

Policy Concepts in 1000 Words: ‘Evidence Based Policymaking’

‘Evidence-based Policymaking’ and the Study of Public Policy

How far should you go to secure academic ‘impact’ in policymaking?

Political science improves our understanding of evidence-based policymaking, but does it produce better advice?

Psychology Based Policy Studies: 5 heuristics to maximise the use of evidence in policymaking

What 10 questions should we put to evidence for policy experts?

Why doesn’t evidence win the day in policy and policymaking?

We all want ‘evidence based policy making’ but how do we do it?

How can political actors take into account the limitations of evidence-based policy-making? 5 key points

The Politics of Evidence Based Policymaking:3 messages

The politics of evidence-based best practice: 4 messages

The politics of implementing evidence-based policies

There are more posts like this on my EBPM page

I am also guest editing a series of articles for the Open Access journal Palgrave Communications on the ‘politics of evidence-based policymaking’ and we are inviting submissions throughout 2017.

There are more details on that series here.

And finally ..

… if you’d like to read about the policy theories underpinning these arguments, see Key policy theories and concepts in 1000 words and 500 words.

 

 

2 Comments

Filed under Evidence Based Policymaking (EBPM), public policy

No one will understand British politics and policymaking after Brexit

Let’s be optimistic for a few seconds, and focus on the idea that a vote for the UK to leave the European Union was a vote for UK sovereignty and ‘taking back control’ of policy and policymaking. The comparison is between an EU process that is distant and undemocratic and a UK process we can all understand and influence, following the simple phrase ‘if you know who is in charge, you know who to blame’.

The down side is that we don’t know who is in charge, and it’s often futile to try to find a named individual or role to blame. The EU certainly complicates the picture, but don’t be fooled into thinking that we will eventually produce a UK political system that anyone understands.

If giving a lecture, this is the point at which I’d pause for effect and restate the idea that no-one understands the UK policymaking system as a whole [insert meaningful looks here]. Many people know about many parts of the system, but it’s not like a jigsaw puzzle that we’ve completed by working together. At best, it’s like that Dalmatian jigsaw that we started at Christmas before getting drunk and falling out.

Top-10-Almost-Unsolvable-Worlds-Hardest-Jigsaw-Puzzles-9

Instead, policymakers and commentators tell simple stories about British politics

The dominant story of British politics relates initially to the idea of parliamentary sovereignty: we vote in constituencies to elect MPs as our representatives, and MPs as a whole represent the final arbiters on policy in the UK. This idea connects strongly to elements of the ‘Westminster model’ (WM), a shorthand phrase to describe key ways in which the UK political system is perhaps designed to work. Perhaps policymaking should reflect strongly the wishes of the public. In representative democracies, political parties engage each other in a battle of ideas, to attract the attention and support of the voting public; the public votes every 4-5 years; the winner forms a government; the government turns its manifesto into policy; and, policy choices are carried out by civil servants and other bodies. In other words, there should be a clear link between public preferences, the strategies and ideas of parties and the final result.

The WM serves this purpose in a particular way: the UK has a plurality (‘first past the post’) voting system which tends to exaggerate support for, and give a majority in Parliament to, the winning party. It has an adversarial (and majoritarian?) style of politics and a ‘winner takes all’ mentality which tends to exclude opposition parties. The executive resides in the legislature and power tends to be concentrated within government – in ministers that head government departments and the Prime Minister who heads (and determines the members of) Cabinet. The government is responsible for the vast majority of public policy and it uses its governing majority, combined with a strong party ‘whip’, to make sure that its legislation is passed by Parliament.

In other words, the ‘take home message’ of this story is that the UK policy process is centralised and that the arrangement reflects a ‘British political tradition’: the government is accountable to public on the assumption that it is powerful and responsible. So, you know who is in charge and therefore who to praise or blame, and elections every 4-5 years are supplemented by parliamentary scrutiny built on holding ministers directly to account.

These stories are more useful for our entertainment than enlightenment

Consider these five factors which challenge the ability of elected policymakers to control the policy process.

  1. Bounded rationality. Ministers only have the ability to pay attention to a tiny proportion of the issues over which have formal responsibility. So, how can they control issues if they have to ignore almost all of them?
  2. Policy communities. Ministers delegate responsibility to civil servants at a quite-low level of government. Civil servants make policy in consultation with interest groups and other participants with the ability to trade resources (such as information) for access or influence. Such relationships can endure long after particular ministers or elected governments have come and gone.
  3. Multi-level governance. The UK government shares policymaking ‘vertically’ (with international, EU, devolved, and local governments) and ‘horizontally’ (with non-governmental and quasi-non-governmental organisations).
  4. Complex government. Policymaking ‘emerges’ from the interaction between many actors, institutions, and regulations. In complex policymaking systems, people act without full knowledge of how other people act elsewhere in the system.
  5. Policy environments. Many policy conditions and events are out of policymakers’ control (including demographic, technological, and economic change)

So, for example, the UK government has to juggle two stories of British politics – on the need to be pragmatic in the face of these five challenges to their power and sense of control, versus the need to construct a strong image of governing competence with reference to control – in the knowledge that one of them is a tall tale.

Brexit will change only one part of that story

None of these factors should prompt us to minimise the influence of the EU on the UK. Rather, they should prompt us to think harder about the impact of Brexit on ‘parliamentary sovereignty’ and ministerial accountability via UK central government control. The phrase ‘you know who is in charge, and who to blame’ will become a more important rallying cry in British politics (when we can no longer blame the EU for British policy), but let’s focus on what actually happens in British politics and recognise how little of it we understand before we decide who to blame.

This post is an amended version of the introductory post for the course POLU9UK: Policy and Policymaking in the UK which draws on this ‘1000 Words’ series on public policy.

 

1 Comment

Filed under POLU9UK, public policy

Writing an essay on politics, policymaking, and policy change

I tend to set this simple-looking question for coursework in policy modules: what is policy, how much has it changed, and why? Students get to choose the policy issue, timeframe (and sometimes the political system), and relevant explanatory concepts.

On the face of it, it looks super-simple: A+ for everyone!

Give it a few more seconds, and you can see the difficulties:

  1. We spent a lot of time agreeing that it seems almost impossible to define policy (explained in 1000 Words and 500 Words)
  2. There are a gazillion possible measures of policy change (1000 Words and 500 Words)
  3. There is an almost unmanageable number of models, concepts, and theories to use to explain policy dynamics (I describe about 25 in 1000 Words each)

I try to encourage some creativity when solving this problem, but also advise students to keep their discussion as simple and jargon-free as possible (often by stretching an analogy with diving, in which a well-executed simple essay can score higher than a belly-flopped hard essay).

Choosing a format: the initial advice

  1. Choose a policy area (such as health) or issue (such as alcohol policy).
  2. Describe the nature of policy, and the extent of policy change, in a particular time period (such as in the post-war era, since UK devolution, or since a change in government).
  3. Select one or more policy concept or theory to help structure your discussion and help explain how and why policy has changed.

For example, a question might be: What is tobacco policy in the UK, how much has it changed since the 1980s, and why? I use this example because I try to answer that – UK and global – question myself, even though my 2007 article on the UK is too theory-packed to be a good model for an undergraduate essay.

Choosing a format: the cautionary advice

You may be surprised about how difficult it is to answer a simple question like ‘what is policy?’ and I will give you considerable credit for considering how to define and measure it, by identifying, for example, the use of legislation/ regulation, funding, staff, and ‘nodality’ and/ or by considering the difference between, say, policy as a statement of intent or a long term outcome. In turn, a good description and explanation of policy change is difficult. If you are feeling ambitious, you can go further, to compare, say, two issues (such as tobacco and alcohol) or places (such UK Government policy and the policy of another country), but sometimes a simple and narrow discussion can be as, or more, effective. Similarly, you can use many theories or concepts to aid explanation, but often one theory will do. Note that (a) your description of your research question, and your essay structure, is more important than (b) your decision on what topic to focus or concepts to use.

Choosing a topic: the ‘joined up’ advice

The wider aim is to encourage students to think about the relationship between different perspectives on policy theory and analysis. For example, in a blog and policy analysis paper they try to generate attention to a policy problem and advocate a solution. Then, they draw on policy theories and concepts to reflect on their papers, highlighting (say): the need to identify the most important audience; the importance of framing issues with a mixture of evidence and emotional appeals; and, the need to present ‘feasible’ solutions.

The reflection can provide a useful segue to the essay, since we’re already identifying important policy problems, advocating change, reflecting on how best to encourage it – such as by presenting modest objectives – and then, in the essay, trying to explain (say) why governments have not taken that advice in the past. Their interest in the policy issue can prompt interest in researching the issue further; their knowledge of the issue and the policy process can help them develop politically-aware policy analysis. All going well, it produces a virtuous circle.

Some examples from my pet subject

Let me outline how I would begin to answer the three questions with reference to UK tobacco policy. I’m offering a brief summary of each section rather than presenting a full essay with more detail (partly to hold on to that idea of creativity – I don’t want students to use this description as a blueprint).

What is modern UK tobacco policy?

Tobacco policy in the UK is now one of the most restrictive in the world. The UK government has introduced a large number of policy instruments to encourage a major reduction of smoking in the population. They include: legislation to ban smoking in public places; legislation to limit tobacco advertising, promotion, and sponsorship; high taxes on tobacco products; unequivocal health education; regulations on tobacco ingredients; significant spending on customs and enforcement measures; and, plain packaging measures.

[Note that I selected only a few key measures to define policy. A fuller analysis might expand on why I chose them and why they are so important].

How much has policy changed since the 1980s?

Policy has changed radically since the post-war period, and most policy change began from the 1980s, but it was not until the 2000s onwards that the UK cemented its place as one of the most restrictive countries. The shift from the 1980s relates strongly to the replacement of voluntary agreements and limited measures with limited enforcement with legislative measures and stronger enforcement. The legislation to ban tobacco advertising, passed in 2002, replaced limited bans combined with voluntary agreements to (for example) keep billboards a certain distance from schools. The legislation to ban smoking in public places, passed in 2006 (2005 in Scotland), replaced voluntary measures which allowed smoking in most pubs and restaurants. Plain packaging measures, combined with large and graphic health warnings, replace branded packets which once had no warnings. Health education warnings have gone from stating the facts and inviting smokers to decide, and the promotion of harm reduction (smoke ‘low tar’), to an unequivocal message on the harms of smoking and passive smoking.

[Note that I describe these changes in broad terms. Other articles might ‘zoom’ in on specific instruments to show how exactly they changed]

Why has it changed?

This is the section of the essay in which we have to make a judgement about the type of explanation: should you choose one or many concepts; if many, do you focus on their competing or complementary insights; should you provide an extensive discussion of your chosen theory?

I normally recommend a very small number of concepts or simple discussion, largely because there is only so much you can say in an essay of 2-3000 words.

For example, a simple ‘hook’ is to ask if the main driver was the scientific evidence: did policy change as the evidence on smoking (and then passive smoking) related harm became more apparent? Is it a good case of ‘evidence based policymaking’? The answer may then note that policy change seemed to be 20-30 years behind the evidence [although I’d have to explain that statement in more depth] and set out the conditions in which this driver would have an effect.

In short, one might identify the need for a ‘policy environment’, shaped by policymakers, and conducive to a strong policy response based on the evidence of harm and a political choice to restrict tobacco use. It would relate to decisions by policymakers to: frame tobacco as a public health epidemic requiring a major government response (rather than primarily as an economic good or issue of civil liberties); place health departments or organisations at the heart of policy development; form networks with medical and public health groups at the expense of tobacco companies; and respond to greater public support for control, reduced smoking prevalence, and the diminishing economic value of tobacco.

This discussion can proceed conceptually, in a relatively straightforward way, or with the further aid of policy theories which ask further questions and help structure the answers.

For example, one might draw on punctuated equilibrium theory to help describe and explain shifts of public/media/ policymaker attention to tobacco, from low and positive in the 1950s to high and negative from the 1980s.

Or, one might draw on the ACF to explain how pro-tobacco coalitions helped slow down policy change by interpreting new scientific evidence though the ‘lens’ of well-established beliefs or approaches (examples from the 1950s include filter tips, low tar brands, and ventilation as alternatives to greater restrictions on smoking).

One might even draw on multiple streams analysis to identify a ‘window of opportunity for change (as I did when examining the adoption of bans on smoking in public places).

Any of these approaches will do, as long as you describe and justify your choice well. One cannot explain everything, so it may be better to try to explain one thing well.

Leave a comment

Filed under 1000 words, 500 words, POLU9UK, tobacco, tobacco policy, UK politics and policy

How can political actors take into account the limitations of evidence-based policy-making? 5 key points

These notes are for my brief panel talk at the European Parliament-European University Institute ‘Policy Roundtable’: Evidence and Analysis in EU Policy-Making: Concepts, Practice and Governance. As you can see from the programme description, the broader theme is about how EU institutions demonstrate their legitimacy through initiatives such as stakeholder participation and evidence-based policymaking (EBPM). So, part of my talk is about what happens when EBPM does not exist.

The post is a slightly modified version of my (recorded) talk for Open Society Foundations (New York) but different audiences make sense of these same basic points in very different ways.

  1. Recognise that the phrase ‘evidence-based policy-making’ means everything and nothing

The main limitation to ‘evidence-based policy-making’ is that no-one really knows what it is or what the phrase means. So, each actor makes sense of EBPM in different ways and you can tell a lot about each actor by the way in which they answer these questions:

  • Should you use restrictive criteria to determine what counts as ‘evidence’? Some actors equate evidence with scientific evidence and adhere to specific criteria – such as evidence-based medicine’s hierarchy of evidence – to determine what is scientific. Others have more respect for expertise, professional experience, and stakeholder and service user feedback as sources of evidence.
  • Which metaphor, evidence based or informed is best? ‘Evidence based’ is often rejected by experienced policy participants as unrealistic, preferring ‘informed’ to reflect pragmatism about mixing evidence and political calculations.
  • How far do you go to pursue EBPM? It is unrealistic to treat ‘policy’ as a one-off statement of intent by a single authoritative actor. Instead, it is made and delivered by many actors in a continuous policymaking process within a complicated policy environment (outlined in point 3). This is relevant to EU institutions with limited resources: the Commission often makes key decisions but relies on Member States to make and deliver, and the Parliament may only have the ability to monitor ‘key decisions’. It is also relevant to stakeholders trying to ensure the use of evidence throughout the process, from supranational to local action.
  • Which actors count as policymakers? Policymaking is done by ‘policymakers’, but many are unelected and the division between policymaker/ influencer is often unclear. The study of policymaking involves identifying networks of decision-making by elected and unelected policymakers and their stakeholders, while the actual practice is about deciding where to draw the line between influence and action.
  1. Respond to ‘rational’ and ‘irrational’ thought.

Comprehensive rationality’ describes the absence of ambiguity and uncertainty when policymakers know what problem they want to solve and how to solve it, partly because they can gather and understand all information required to measure the problem and determine the effectiveness of solutions.

Instead, we talk of ‘bounded rationality’ and how policymakers deal with it. They employ two kinds of shortcut: ‘rational’, by pursuing clear goals and prioritizing certain kinds and sources of information, and ‘irrational’, by drawing on emotions, gut feelings, deeply held beliefs, habits, and familiarity, make decisions quickly.

I say ‘irrational’ provocatively, to raise a key strategic question: do you criticise emotional policymaking (describing it as ‘policy based evidence’) and try somehow to minimise it, adapt pragmatically to it, or see ‘fast thinking’ more positively in terms of ‘fast and frugal heuristics’? Regardless, policymakers will think that their heuristics make sense to them, and it can be counterproductive to simply criticise their alleged irrationality.

  1. Think about how to engage in complex systems or policy environments.

Policy cycle’ describes the idea that there is a core group of policymakers at the ‘centre’, making policy from the ‘top down’, and pursuing their goals in a series of clearly defined and well-ordered stages, such as: agenda setting, policy formulation, legitimation, implementation, and evaluation. In this context, one might identify how to influence a singular point of central government decision.

However, a cycle model does not describe policymaking well. Instead, we tend to identify the role of less ordered and more unpredictable complex systems, or policy environments containing:

  • A wide range of actors (individuals and organisations) influencing policy at many levels of government. Scientists and practitioners are competing with many actors to present evidence in a particular way to secure a policymaker audience.
  • A proliferation of rules and norms maintained by different levels or types of government. Support for particular ‘evidence based’ solutions varies according to which organisation takes the lead and how it understands the problem.
  • Important relationships (‘networks’) between policymakers and powerful actors. Some networks are close-knit and difficult to access because bureaucracies have operating procedures that favour particular sources of evidence and some participants over others, and there is a language – indicating what ways of thinking are in good ‘currency’ – that takes time to learn.
  • A tendency for certain ‘core beliefs’ or ‘paradigms’ to dominate discussion. Well-established beliefs provide the context for policymaking: new evidence on the effectiveness of a policy solution has to be accompanied by a shift of attention and successful persuasion.
  • Policy conditions and events that can reinforce stability or prompt policymaker attention to lurch at short notice. In some cases, social or economic ‘crises’ can prompt lurches of attention from one issue to another, and some forms of evidence can be used to encourage that shift, but major policy change is rare.

For stakeholders, an effective engagement strategy is not straightforward: it takes time to know ‘where the action is’, how and where to engage with policymakers, and with whom to form coalitions. For the Commission, it is difficult to know what will happen to policy after it is made (although we know the end point will not resemble the starting point). For the Parliament, it is difficult even to know where to look.

  1. Recognise that EBPM is only one of many legitimate ‘good governance’ principles.

There are several principles of ‘good’ policymaking and only one is EBPM. Others relate to the value of pragmatism and consensus building, combining science advice with public values, improving policy delivery by generating ‘ownership’ of policy among key stakeholders, and sharing responsibility with elected national and local policymakers.

Our choice of which principle and forms of evidence to privilege are inextricably linked. For example, some forms of evidence gathering seem to require uniform models and limited local or stakeholder discretion to modify policy delivery. The classic example is a programme whose value is established using randomised control trials (RCTs). Others begin with local discretion, seeking evidence from stakeholders, professional groups, service user and local practitioner experience. This principle seems to rule out the use of RCTs, at least as a source of a uniform model to be rolled out and evaluated. Of course, one can try to pursue both approaches and a compromise between them, but the outcome may not satisfy advocates of either approach to EBPM or help produce the evidence that they favour.

  1. Decide how far you’ll go to achieve EBPM.

These insights should prompt us to see how far we are willing, and should, go to promote the use of certain forms of evidence in policymaking

  • If policymakers and the public are emotional decision-makers, should we seek to manipulate their thought processes by using simple stories with heroes, villains, and clear but rather simplistic morals?
  • If policymaking systems are so complex, should stakeholders devote huge amounts of resources to make sure they’re effective at each stage?
  • Should proponents of scientific evidence go to great lengths to make sure that EBPM is based on a hierarch of evidence? There is a live debate on science advice to government on the extent to which scientists should be more than ‘honest brokers’.
  • Should policymakers try to direct the use of evidence in policy as well as policy itself?

Where we go from there is up to you

The value of policy theory to this topic is to help us reject simplistic models of EBPM and think through the implications of more sophisticated and complicated processes. It does not provide a blueprint for action (how could it?), but instead a series of questions that you should answer when you seek to use evidence to get what you want. They are political choices based on value judgements, not issues that can be resolved by producing more evidence.

 

4 Comments

Filed under Evidence Based Policymaking (EBPM), public policy

Storytelling for Policy Change: promise and problems

I went to a fantastic workshop on storytelling for policy change. It was hosted by Open Society Foundations New York (25/6 October), and brought together a wide range of people from different backgrounds: Narativ, people experienced in telling their own story, advocacy and professional groups using stories to promote social or policy change, major funders, journalists, and academics. There was already a lot of goodwill in the room at the beginning, and by the end there was more than a lot!

The OSF plans to write up a summary of the whole discussion, so my aim is to highlight the relevance for ‘evidence-based policymaking’ and for scientists and academics seeking more ‘impact’ for their research. In short, although I recommend that scientists ‘turn a large amount of scientific evidence into simple and effective stories that appeal to the biases of policymakers’, it’s easier said than done, and not something scientists are trained in. Good storytellers might enthuse people already committed to the idea of storytelling for policy, but what about scientists more committed to the language of scientific evidence and perhaps sceptical about the need to develop this new skill (particularly those who describe stories pejoratively as ‘anecdata’)? What would make them take a leap in the dark, to give up precious research time to develop skills in storytelling?

So, let me tell you why I thought the workshop was brilliant – including outlining its key insights – and why you might not!

Why I thought it was brilliant

Academic conferences can be horrible: a seemingly never-ending list of panels with 4-5 paper givers and a discussant, taking up almost all of the talking time with too-long and often-self-indulgent and long-winded PowerPoint presentations and little time for meaningful discussion. It’s a test of meeting deadlines for the presenter and an endurance test for the listener.

This workshop was different: the organisers thought about what it means to talk and listen, and therefore how to encourage people to talk in an interesting way and encourage high attention and engagement.

There were three main ‘listening exercises’: a personal exercise in which you closed your eyes and thought about the obstacles to listening (I confess that I cheated on that one); a paired exercise in which one person listened and thought of three poses to sum up the other’s short story; and a group exercise in which people paired up, told and then summarised each other’s stories, and spoke as a group about the implications.

This final exercise was powerful: we told often-revealing stories to strangers, built up trust very quickly, and became emotionally involved in each other’s accounts. It was interesting to watch how quickly we could become personally invested in each other’s discussion, form networks, and listen intently to each other.

For me, it was a good exercise in demonstrating what you need in a policymaker audience: ideally, they should care about the problem you raise, be personally invested in trying to solve it, and trust you and therefore your description of the most feasible solutions. If it helps recreate these conditions, a storytelling scientist may be more effective than an ‘honest broker’. Without a good story to engage your audience, your evidence will be like a drop in the ocean and your audience might be checking its email or playing Pokemon Go while you present.

Key insights and impressions

Most participants expressed strong optimism about the effect of stories on society and policy, particularly when the aim is more expressive than instrumental: the act itself of telling one’s story and being heard can be empowering, particularly within marginalised groups from which we hear few voices. It can also be remarkably powerful, remarkably quickly: most of us were crying or laughing instantly and frequently as we heard many moving stories about many issues. It’s hard to overstate just how effective many of these stories were when you heard them in person.

When discussing more instrumental concerns – can we use a story to get what we want? – the optimism was more cautious and qualified. Key themes included:

  • The balance between effective and ethical storytelling, accepting that if a story is a commodity it can be used by people less sympathetic to our aims, and exploring the ethics of using the lens of sympathetic characters (e.g. white grandparents) to make the case for marginalised groups.
  • This ethical dimension was reinforced continuously by stories of vulnerable storytellers and the balance between telling their story and protecting their safety.
  • The importance of context: the same story may have more or less impact depending on the nature of the audience; many examples conveyed the sense that a story with huge impact now would have failed 10 or 20 years ago and/ or in a different region.
  • The importance of tailoring stories to the biases of audiences and trying to reframe the implications of your audience’s beliefs (one particularly interesting example was of portraying equal marriage in Ireland as the Christian thing to do).
  • Many campaigns used humour and positive stories with heroes, based on the assumption that audiences would be put off by depressing stories of problems with no obvious solution but energised by a message of new possibilities.
  • Many warned against stories that were too personal, identifying the potential for an audience to want to criticise or fix an individual’s life rather than solve a systemic problem (and this individualistic interpretation was most pronounced among people identifying with right-wing parties).
  • Many described the need to experiment/ engage in trial-and-error to identify what works with each audience, including the length of written messages, choice of media, and choice of ‘thin’ stories with clear messages to generate quick attention or ‘thick’ stories which might be more memorable if we have the resources to tell them and people take the time to listen.

Many of these points will seem familiar if you study psychology or the psychology of policymaking. So, the benefit of these experiences is that they tell us how people have applied such insights and how it has helped their cause. Most speakers were confident that they were making an impact.

Why you may not be as impressed: two reasons

The first barrier to getting you enthusiastic is that you weren’t there. If emotional engagement is such a key part of storytelling, and you weren’t there to hear it, why would you care? So, a key barrier to making an ‘impact’ with storytelling is that it is difficult to increase its scale. You might persuade someone if they spent enough time with you, but what if you only had a few seconds in which to impress them or, worse still, you couldn’t impress them because they weren’t interested in the first place? Our worry may be that we can only influence people who are already open to our idea. This isn’t the end of the world, since a key political aim may be to enthuse people who share your beliefs and get them to act (for example, to spread the word to their friends). However, it prompts us to wonder about the varying effect of the same message and the extent to which our message’s power comes from our audience rather than our story.

The second barrier is that, when the question is framed for an academic audience – what is the scientific evidence on the impact of stories? – the answer is not clear.

On the panel devoted to this question (and in a previous session), there were some convincing accounts of the impact of initiatives such as: the Women’s Policy Institute ‘grass roots’ training in California (leading to advocacy prompting 2 dozen bills to be signed over 13 years); Purpose’s branding campaign for the White Helmets (including the miracle baby video which has received tens of millions of views); and, the Frame Works Institute’s ability to change minds with very brief interventions (for example, getting people to think in terms of problem systems more than problem individuals in areas like criminal justice).

However, the academic analysis – with contributions from Francesca Polletta, Jeff Niederdeppe, Douglas Storey, Michael Jones – tended to stress caution or note limited effects:

  • Stories work when they create an empathic reaction, but intensely personal experiences are difficult to recreate when mediated (soap operas and ‘edutainment’ come closest).
  • Randomised control trials suggest that stories have a measurable effect, but it’s small and compared to no intervention at all rather than a competing approach such as providing evidence in reports (note that most experimental studies do not draw on skilful storytellers)
  • Stories work most clearly when they reinforce the beliefs of your allies (and when you refer to heroes, not villains), but the effects are indirect at best with your opponents.

More research required?!

So, you might want more convincing evidence before you take that giant leap to train to become a skilful storyteller: why go for it when its effects are so unclear and difficult to measure?

For me, that response might seem sensible but is also a cop out: unequivocal evidence may never arrive and good science often involves researching as you go. A key insight into policymaking regards a continuous sense of urgency to solve problems: policymakers don’t wait for the evidence to become unequivocal before they act, partly because that sense of clarity may never happen. They feel the need to act on the basis of available evidence. Perhaps scientists should at least think about doing the same when they seek to act on research rather than simply do the research: how long should you postpone potentially valuable action with the old cliché ‘more research required’?

listening-new-york-1-11-16

4 Comments

Filed under Evidence Based Policymaking (EBPM), public policy, Storytelling