What 10 questions should we put to evidence for policy experts?

The European Commission’s Joint Research Centre’s Science Hub is making some videos about evidence and policy, asking 10 questions. Here are my answers (the video will come later):

  1. Who are you?

Paul Cairney, Professor of Politics and Public Policy, University of Stirling. I write about public policy, applying theoretical insight to issues such as ‘the politics of EBPM’.

  1. How did you become interested in evidence for policy?

It was always in the back of my mind because it is the latest version of a long-standing interest (in policy studies) about the absence of ‘comprehensive rationality’: what do policymakers do when they can’t consider all information, and what are the consequences for politics and policy? Do they use ‘irrational’ shortcuts? Does their attention tend to lurch? Does policy become incremental or ‘punctuated’? There are many different answers, explored in this ‘1000 Words series’.

  1. Why is evidence-informed policy important?

It’s part of the broader importance of inclusive policymaking based on a diversity of voices and the generation of knowledge about how the world works (alongside a debate about how it should work).

  1. What is the most common misconception about evidence-informed policy?

I think that many scientists are too quick to dismiss politics – and identify ‘policy based evidence’ driven by ideological and emotional politicians – rather than understand the ever-present limits to the use of evidence in policy. I think many also exaggerate the lack of scientific influence on policy by focusing on the most salient issues.

  1. What are the most common mistakes made by researchers or policymakers?

The classic mistake by researchers is to think that you make a good argument by bombarding people with a lot of information without thinking about how they’ll receive it. An important mistake that policymakers can make is to rely too much on the experts they know and trust, rather than seeking ways to identify diverse and ‘state of the art’ sources of information.

  1. What is the single most important advice to researchers/scientists who want to have policy impact?

Think about your audience and how they demand information: get their attention with a simple story, describe the problem in ways they understand (and think about the world), and show that your solution is technically and politically feasible.

  1. How do you change minds with facts and evidence?

Engage for the long term, recognising your ‘enlightenment’ role. Something dramatic would have to happen to change minds immediately and dramatically – it would be akin to a religious conversion. Or, in politics, it’s about finding a sympathetic audience (different minds) in another policymaking venue or hoping for a change of government. In other words, this is about the power of participant as much as the power of evidence and ideas.

  1. How should you communicate uncertainty about the evidence?

Since I study politics, I’d focus on the political choices here. You can communicate uncertainty in academic journals via ‘limitations’ sections and expect robust challenge on your evidence from your peers. In politics, if you show uncertainty – and your competitor does not – you may be at a disadvantage, and may need to do some soul searching about how much uncertainty you hold back. As soon as you become a scientist and advocate, the rules change.

  1. How do you measure the policy impact of evidence?

In ways that are not conducive to ‘impact’ measurement by research bodies! For example, with colleagues, I tracked the influence of evidence on smoking harms on policy. In ‘leading countries’ it took 2-3 decades, and depended on three conditions: (1) key actors ‘frame’ the evidence to set a policy agenda; (2) the policy environment is generally conducive to evidence-informed change; and (3) key actors exploit ‘windows of opportunity’ for each policy change. In most countries, policy change of this scale has not happened. In such cases, we can never say that evidence simply wins the day.

  1. Who or What are your “must-reads”?

I partly took more notice of this topic after reading two articles by Kathryn Oliver and colleagues:

Oliver, K., Innvar, S., Lorenc, T., Woodman, J. and Thomas, J. (2014a) ‘A systematic review of barriers to and facilitators of the use of evidence by policymakers’ BMC health services research, 14 (1), 2. http://www.biomedcentral.com/1472-6963/14/2

Oliver, K., Lorenc, T., & Innvær, S. (2014b) ‘New directions in evidence-based policy research: a critical analysis of the literature’, Health Research Policy and Systems, 12, 34 http://www.biomedcentral.com/content/pdf/1478-4505-12-34.pdf

I was struck by the argument here, that policymakers often fund sophisticated models for evidence-based policymaking but don’t understand or use them:

Nilsson, M., Jordan, A., Turnpenny, J., Hertin, J., Nykvist, B. and Russel, D. (2008) ‘The use and non-use of policy appraisal tools in public policy making: an analysis of three European countries and the European Union’, Policy Sciences, 41, 4, 335-55

It’s also worth reading this account, which shows that policymakers don’t have the same respect for a ‘hierarchy’ of evidence/ methods as many scientists:

Bédard, P. and Ouimet, M. (2012) ‘Cognizance and consultation of randomized controlled trials among ministerial policy analysts’ Review of Policy Research, 29, 5, 625-644

 

 

For more information, start with my EBPM page

Leave a comment

Filed under Evidence Based Policymaking (EBPM), Uncategorized

Scottish Politics in Brexit Britain: is independence inevitable?

CAirney Scottish Politics Brexit Dundee CAfe 2017

This is an updated and shortened version of previous posts, designed for my talk at the Dundee Arts Café tonight. I’d like to thank First Minster Nicola Sturgeon for making me look like the best scheduler of a talk ever:

We don’t know much about the second referendum on Scottish independence, but we can be guided by three basic insights:

  1. Most people make up their mind fairly quickly and may not be swayed too much by the campaign, but there are enough undecided voters to tip the result

2. The campaign will come down to who can tell the best story (to stir the emotions, perhaps with a convincing hero and moral) rather than simply command the facts.

3. Brexit has changed the independence story dramatically, but it could support either Yes/ No campaign.

The rest is mostly gut-driven speculation: I think Yes will win, partly because it has a new way to present its case, and a better campaigner to do so, while (as ridiculous as this sounds) No may look like it is banging on about the same old arguments, and it’s less clear who will do it.

Let’s start with an updated summary of why Brexit is good for Yes:

  1. It reinforces a well-established argument for constitutional change: we voted for X but got Y because we are outnumbered by voters in England. Voting Remain but getting Leave is the latest version of voting Labour or SNP in Scotland but getting a Conservative UK government.
  2. It reinforces the same argument about the effect of that ‘democratic deficit’: ‘London’/’Westminster’ is forcing us to accept policies we did not choose. Voting Leave is the latest version of the ‘bedroom tax’ (and, for older readers, the ‘poll tax’).
  3. It helps challenge the idea that the Scottish independence aim is nationalist and parochial. Suddenly, independence is the cosmopolitan choice if we are rejecting a ‘Little England’ mentality.
  4. Some people who voted to stay in the UK and EU will prefer the EU to the UK (and think an independence vote is the best way to achieve it), or perhaps feel let down by the claim that a No vote in 2014 was to stay in the UK and EU.

Historically, the main response to 1 & 2 came from the Conservative Party, offering concessions in areas such as spending, levels of representation in Westminster, and in Scotland’s status in UK-devolved relations.

Recently, UKIP has been more critical of Scotland’s privileged position in the UK, and even the Conservative party qualifies its support of Scotland’s place in the Union.

Labour’s more recent response has been more interesting, and not what I expected. I figured Scottish Labour would encourage the equivalent of a free vote of its members. Instead, it has rejected indyref2 in favour of a ‘federal’ solution and two anti-referendum strategies:

  1. To describe indyref2 as yet another divisive and destabilising event like Brexit and the election of Trump.
  2. To challenge the idea that Scottish independence is the cosmopolitan choice. Sadiq Khan seemed to link Scottish nationalism strongly with the divisiveness of Trump and Leave campaigns, prompting some debate about how far he went to equate it with bigotry and racism.

This strategy was generally received badly among people already committed to Yes. It’s too early to gauge its durability or long term effect on the voters thinking about switching, but we already know that the SNP campaigned in indyref1 with a message – for example, ‘to make life better for the people who live here’ – that contrasts heavily with the anti-immigrant rhetoric in some parts of the Leave campaign. Indeed, I’d expect it to reinforce a pro-immigration (or, rather, a very pro-EU citizen) message to provide a deliberate contrast to parts of the Brexit campaign, making it relatively difficult for Labour to maintain an if-you-vote-Yes-you-share-the-same-aim-as-bigots argument (which didn’t work well during the Brexit debate anyway).

Let’s continue with an updated summary of why Brexit is good for No

  1. The No campaign was based on the economic harms of independence, and key symbols (like oil price volatility) have reinforced the message.
  2. We still don’t know what currency an independent Scotland would use.
  3. The Yes vote meant all things to all people, with no sense of what would be realistic.
  4. Brexit shows you that a transition to independence would be far tougher than advertised.

Point 4 is still unfolding. We’ve already seen that the £350m-for-the-NHS argument was misleading, witnessed a reduction in the value of the pound, and seen some hard talking from likely EU negotiators that might be emulated in Scotland-UK discussions (UK hard-talking was a key theme of indyref1). Yet, the effects of such developments are still open to debate (see for example the sterling issue).

More importantly, it’s hard to know how to relate these events to Scotland:

One the one hand, Yes needs a disastrous Brexit to show that it is powerless to ward off disaster. Ideally, it would wait long enough to argue that (a) Brexit is starting to ‘bite’, (b) the UK Government is stiffing Scotland in its negotiations of future devolved powers, but not so long that (c) it disrupts the (not guaranteed) continuation of its EU membership. This time may not arrive, and the date is not in the SNP’s gift.

On the other, No needs a partly-disastrous Brexit to show that separation is painful.

Who will have the best story?

If recent events have taught us anything, it’s that people are driven strongly by emotion, and might put ‘feelings over facts’. I still think that the result itself will come down to who tells the Yes/ No stories and how well they do it, and that Yes has a far better hero (Nicola Sturgeon)/villain (Nigel Farage, Boris Johnson, Theresa May?) story now than in 2014, while No has the same old boring story of economic disaster and can no longer rely on those leaflets with Salmond’s face on a pound coin. Who will become the face of No (I reckon it will be Davidson), and how can they repackage the same arguments (who knows)?

 

 

2 Comments

Filed under Scottish independence, Scottish politics

Racism and Stories in Scottish Politics

Brexit boosts the case for Scottish independence because it can now be framed more easily as the cosmopolitan choice: vote Yes to get away from a ‘little England’ mentality. This possibility was perhaps in Labour’s mind when it described Scottish nationalism as anything but cosmopolitan. For example, Sadiq Khan seemed to link Scottish nationalism strongly with the divisiveness of Trump and Leave campaigns, prompting some debate about how far he went to equate it with bigotry and racism, while former Labour minister Douglas Alexander left less room for doubt.

These claims prompted a small number of commentary pieces supporting or rejecting the idea that Scottish nationalism fosters racism, bigotry, and/ or social division:

  • Claire Heuchan welcomed Khan’s intervention cautiously, highlighting a tendency of Scottish actors to assert their superiority over their English counterparts, and using the opportunity to expand the debate, to highlight important issues that we often ignore, from personal stories of racist abuse and examples of more limited education and employment opportunities for people of colour, to the role of Scotland in the British empire’s colonial past built on slavery and exploitation.
  • In contrast, Robert Somynne identified a civic Scottish nationalism far apart from a ‘western trend towards populism based on tribal and ethnic divisions’, arguing that Khan’s description ‘doesn’t bear out the experience of so many people of colour in Scotland who campaigned in the grassroots’.
  • Kevin McKenna ridiculed Khan’s argument, rejecting the idea of nationalism underpinned by anti-Englishness, identifying a more divisive UK politics of which Labour is a key part, and dismissing Khan and others as part of ‘the leftwing intelligentsia’. John McKee argued that current Scottish nationalism is more about rejecting the British state than British people, while Eric Joyce links it more to rejecting more worrying forms of nationalism pursued by parties like UKIP.

The debate rages on in twitter, but the discussion has not been driven primarily by a willingness to listen, engage constructively, or talk about issues that challenge our beliefs. I don’t suppose you need me to explain why, but it’s worth highlighting three analytically-separate explanations that will likely be present throughout all debates like this:

  1. The devil shift undermines debate

People form coalitions with the people who share their beliefs, and they compete with people who don’t. The ‘devil shift’ describes a form of ‘groupthink’; a tendency of actors in those coalitions to romanticise their own cause and demonise the cause of their opponents: ‘anyone who disagrees with them must be mistaken about the facts, operating from the wrong value premises, or acting from evil motive’ (Sabatier, Hunter and McLaughlin). They question the motives of their opponents but not their allies, subject only their opponent’s arguments to criticism, and think their opponents are more powerful than they are.

If all debates are interpreted through this lens of Scottish independence, you can predict the results: Yes groups will see Khan’s intervention as threatening to their beliefs and aims, No groups will embrace it because it supports their beliefs and aims, and there is little scope for conflict resolution. Indeed, during such debates, we’ll put up with some sketchy characters if they support our cause while denouncing their equivalents in the other coalition.

This dynamic will be apparent each time we interpret each issue. For example, imagine a semi-honest and open discussion about Scotland and the colonial empire. One side might combine two points – Scotland ‘punches above its weight’ in every endeavour, and Scotland was part of the empire – to argue that Scotland played a disproportionate role in colonialism and slavery. The other will remind us of the unequal and coerced alliance of 1707 and/ or blame an unelected elite in Scotland for the Union and empire, to argue that Scottish independence is the best way to reject the colonial past. The same history has a very different villain and moral.

  1. There are two colliding roles of storytelling

We can identify two main roles of personal storytelling: (1) to empower an individual, when they share their experiences of life and feel listened to, and (2) to take forward a political agenda, when they identify a hero/ villain and moral that suits one coalition’s beliefs and aims.

In our case, it is difficult to separate the two, and most people are not willing or able to do so. This may be understandable with Khan’s recent intervention since, although he generally has an important story to tell from his perspective, this specific speech seemed designed to bolster the position of Scottish Labour at its party conference. Heuchan’s experience is more worrying, her motives seemed far less instrumental and her personal story was worth listening to, but almost no-one has simply said ‘thank you for your story’ without adding conditions or objections. There are very few spaces in which people will listen rather than judge.

  1. Some topics are unusually personal

In part, this is because few people like to think that they are racist or bigoted. Some won’t think about the ways in which they benefit from the systematic effects of racism – in which some groups benefit disproportionately from education/ employment opportunities and face a smaller risk of personal abuse – partly because they don’t have to, and it’s generally not an enjoyable experience. Others will flinch at the idea that they are privileged because they are white, often because they have vivid memories of personal experiences of abuse or disadvantage linked to another part of their background, such as their gender, class, religion, or disability.

So, we often want to tell our stories without listing to those of others. If there is no such space in which to exchange the details of such stories, we soon end up with heated, futile debates based on the sense that you don’t understand my experience or perspective before you criticise it. We can’t solve this problem, but we should at least be aware of it, and perhaps be aware that, although the Scottish debate has some unusual features, it is one of many examples of routine divisive politics.

 

2 Comments

Filed under Folksy wisdom, Scottish independence, Scottish politics, Storytelling

Five advantages of blogging

This is my third ‘hey, let’s blog’ event, so it finally dawned on me to write a blog post about it. See also Fiona Miller’s account of the Stirling event.

I don’t know much about blogging research, so will focus on my personal experience of its advantages. One frequent academic argument against blogging is that it takes you away from more important parts of the job, such as teaching and research. My argument is that it helps you do both things more effectively.

See also the accounts of the disadvantages, which often relate to the ways in which they make you vulnerable to personal abuse on social media (examples 1, 2, 3).

Advantage 1: Clarity

Writing a blog has improved my academic writing. When you blog, you write for a non-specialist audience. You use less jargon or explain its meaning and value. You assume that people will not read your work unless you front-load the ‘reveal’. You need a catchy and tweetable title, to provide a ‘hook’ in the first sentence, and to show your work in a few hundred words (perhaps to encourage people to read more of your work). When you develop these skills, you can use them while writing journal article titles, abstracts, and introductions.

If you like, you can also write a blog post instead of relying on the paper/ powerpoint combo for workshops and conferences, since a 4-paper panel at conferences is usually an endurance test, and a blog post reminds you to say why people should be interested in the paper (e.g. recent examples on evidence/ policy and Scottish independence).

Advantage 2: Timeliness

It can take years for people to read an article you publish in a top journal. Sometimes the article is worth the wait. In other cases, I think it’s best to see this work as part of a package in which the article is one of the last things to appear. There is a good case to be made for taking your time to get articles right, but a less good case to keep it a secret while you do so.

Advantage 3: Exposure

It’s now common to say that we make better links with practitioners and policymakers by making our writing more accessible (short, punchy, and one click away). In my experience, the biggest payoff has been with other academics. Politics colleagues will mention my blog (and textbook) more than my articles. I can also use introductory blog posts to communicate ideas with colleagues in other disciplines – and/ or in other countries – without expecting them to do weeks of homework on the foundational texts. In each case, it works partly because we struggle to find the time to read, and appreciate a short story. Indeed, my articles are one click away on my website, but very, very, very, very few people read them.

However, you don’t need a personal blog. In fact, my most exposureyish posts have been elsewhere, including two in the Guardian’s political science blog (on evidence-based policymaking, and (with Kathryn Oliver) the dilemmas that arise when we seek it), some on the LSE blog (I tried really hard to compare tobacco and alcohol policy – look! There’s a video!), and many in The Conversation.

Advantage 4: Teaching and Learning

Teaching. The most-used page of my website hosts a series of 1000 Word summaries of policy concepts (the ‘policy cycle’ got 26000 hits in 2016). I use them, like a gateway drug, to teach undergraduate and MPP modules: they can get a feel for the concept quickly then do further reading. They now come with podcasts, which I use instead of lectures (for workshops). Other academics also use the podcasts, particularly when their students are new to policy studies (e.g. David P. Carter).

Learning. I also ask my students to write blog posts as part of their coursework, to help them learn how to write in a concise and punchy way for a non-academic audience. In most cases, students excel at this kind of work, as part of a package of assessment in which they learn how to communicate the same insights in many different ways.

Advantage 5: Unexpected benefits

When I started blogging I didn’t really know what it was for. I used to copy and paste my article abstracts, or complain about David Cameron’s handling of Scottish independence. This was at a time in which colleagues at my former University were reticent about self-publicity, and sending round a link to a new journal article via the departmental email was pushing it a bit. Now, self-promotion seems to be part of the job, and we might expect some benefits without really knowing what they’ll be. For example, my links with some very interesting people in places like the European Commission and Alliance for Useful Evidence have arisen largely from blogging.

We all have different things that tickle us in life. For me, the most tickling part of the unexpected benefit of blogging is that I now (almost!) top the following google searches: policy cycle, multiple streams, advocacy coalition framework, punctuated equilibrium theory, the politics of evidence based policymaking, and the psychology of policymaking. I’m also doing my best to push out the other Paul Cairney from the first page of google, but Wikipedia is getting in the way. The more serious point is that a personal blog might need to generate attention through social media first, before it catches fire and rises up the search engine pages.

Leave a comment

Filed under Academic innovation or navel gazing, Folksy wisdom, Uncategorized

There may never be a good time to call #indyref2, but …

This is a blog post for one of the two talks I offered to give, first at Warwick then Dundee.

I’ll start off with the same admission of hubris each time: these talks take a while to arrange, I suggested the topic in the late summer, and I assumed I’d know something about the effect of Brexit on the future of Scottish politics by now. Instead, I don’t know much more than the stuff I described in June (‘Brexit and the inevitability of Scottish Independence’). So, I’ll focus on what we know now, and speculate wildly (hand gestures at 100%) about what might happen. My gut still tells me that there will be an indyref2 and that Yes will win, but my gut is pretty crap.

Let’s start with an updated summary of why Brexit is good for Yes:

  1. It reinforces a well-established argument for constitutional change: we voted for X but got Y because we are outnumbered by voters in England. Voting Remain but getting Leave is the latest version of voting Labour (and now SNP) in Scotland but getting a Conservative UK government.
  2. It reinforces the same argument about the effect of that ‘democratic deficit’: ‘London’/’Westminster’ is forcing us to accept policies we did not choose. Voting Leave is the latest version of the ‘bedroom tax’ (and, for older readers, the ‘poll tax’).
  3. It helps reframe the idea that the Scottish independence aim is nationalist and parochial. Suddenly, independence is the cosmopolitan choice if we are rejecting a ‘Little England’ mentality.
  4. Some people who voted to stay in the UK and EU will prefer the EU to the UK (and think an independence vote is the best way to achieve it).

Traditionally, the main response to 1 & 2 has come from the Conservative Party, offering concessions in areas such as spending, levels of representation in Westminster, and in Scotland’s status in UK-devolved relations.

Recently, UKIP has been more critical of Scotland’s privileged position in the UK, and even the Conservative party qualifies its support of Scotland’s place in the Union.

Labour’s more recent response has been more interesting, and not what I expected. I figured Scottish Labour would encourage the equivalent of a free vote of its members. Instead, it has rejected indyref2 in favour of a ‘federal’ solution and two anti-referendum strategies:

  1. To describe indyref2 as yet another divisive and destabilising event like Brexit and the election of Trump.
  2. To challenge the idea that Scottish independence is the cosmopolitan choice. Sadiq Khan seemed to link Scottish nationalism strongly with the divisiveness of Trump and Leave campaigns, prompting some debate about how far he went to equate it with bigotry and racism. Although Khan is reported to have backtracked a bit, former Labour minister Douglas Alexander doubled down:

This strategy has gone down like a fart in a lift among people already committed to Yes. It’s too early to gauge its durability or long term effect on the voters thinking about switching, but we already know that the SNP campaigned in indyref1 with a message – for example, ‘to make life better for the people who live here’ – that contrasts heavily with the anti-immigrant rhetoric in some parts of the Leave campaign. Indeed, I’d expect it to reinforce a pro-immigration (or, a very pro-EU citizen) message to provide a deliberate contrast to parts of the Brexit campaign, making it relatively difficult for Labour to maintain an if-you-vote-Yes-you-share-the-same-aim-as-bigots argument (which didn’t work well during the Brexit debate anyway).

Let’s continue with an updated summary of why Brexit is good for No

  1. The No campaign was based on the economic harms of independence, and key symbols (like oil price volatility) have reinforced the message.
  2. We still don’t know what currency an independent Scotland would use.
  3. The Yes vote meant all things to all people, with no sense of what would be realistic.
  4. Brexit shows you that a transition to independence would be far tougher than advertised.

Point 4 is still unfolding. We’ve already seen that the £350m-for-the-NHS argument was mince, a reduction in the value of the pound, and some hard talking from likely EU negotiators (UK hard-talking was a key theme of indyref1). Yet, the effects of such developments are still open to debate (the £ issue is bad for the consumer but good for the exporter).

More importantly, it’s hard to know how to relate these events to Scotland. One the one hand, Yes needs a disastrous Brexit to show that it is powerless to ward off disaster. On the other, No needs a partly-disastrous Brexit to show that separation is painful.

Can there be a ‘rational’ calculation of when/if to call indyref2?

If we focus on the idea of a rational calculating Nicola Sturgeon, developing a formula to determine the right time to hold indyref2, the timing would involve: (a) waiting long enough for Brexit to ‘bite’ and prompt voters to feel its effects and shift to Yes, and (b) waiting for the UK Government to stiff Scotland in its negotiations of future Scottish devolved powers, but (c) not waiting too long to disrupt the (not guaranteed) continuation of its EU membership. This time has not arrived and, as John Curtice suggests, may not arrive.

Or, will it come down to passion and emotion?

Yet, if recent events have taught us anything, it’s that people are driven strongly by emotion, and might put ‘feelings over facts’. So, why should leaders of the SNP be exempt from a bout of passion, especially if loads of their supporters are keen, see it as a last opportunity for decades, and hope that they can change some minds during the next campaign? The fact that I argue the very opposite in another post is neither here nor there!

I still think that the result itself will comes down to who tells the Yes/ No stories and how well they do it, and that Yes has a far better hero/villain story now than in 2014, while No has the same old boring story of economic disaster and can no longer rely on those leaflets with Salmond’s face on a pound coin.

See also: a gazillion posts on the last Scottish referendum (scroll)

 

 

4 Comments

Filed under Scottish independence, Scottish politics, Storytelling

Why doesn’t evidence win the day in policy and policymaking?

cairney-southampton-evidence-win-the-dayPolitics has a profound influence on the use of evidence in policy, but we need to look ‘beyond the headlines’ for a sense of perspective on its impact.

It is tempting for scientists to identify the pathological effect of politics on policymaking, particularly after high profile events such as the ‘Brexit’ vote in the UK and the election of Donald Trump as US President. We have allegedly entered an era of ‘post-truth politics’ in which ideology and emotion trumps evidence and expertise (a story told many times at events like this), particularly when issues are salient.

Yet, most policy is processed out of this public spotlight, because the flip side of high attention to one issue is minimal attention to most others. Science has a crucial role in this more humdrum day-to-day business of policymaking which is far more important than visible. Indeed, this lack of public visibility can help many actors secure a privileged position in the policy process (and further exclude citizens).

In some cases, experts are consulted routinely. There is often a ‘logic’ of consultation with the ‘usual suspects’, including the actors most able to provide evidence-informed advice. In others, scientific evidence is often so taken for granted that it is part of the language in which policymakers identify problems and solutions.

In that context, we need better explanations of an ‘evidence-policy’ gap than the pathologies of politics and egregious biases of politicians.

To understand this process, and appearance of contradiction between excluded versus privileged experts, consider the role of evidence in politics and policymaking from three different perspectives.

The perspective of scientists involved primarily in the supply of evidence

Scientists produce high quality evidence only for politicians often ignore it or, even worse, distort its message to support their ideologically-driven policies. If they expect ‘evidence-based policymaking’ they soon become disenchanted and conclude that ‘policy-based evidence’ is more likely. This perspective has long been expressed in scientific journals and commentaries, but has taken on new significance following ‘Brexit’ and Trump.

The perspective of elected politicians

Elected politicians are involved primarily in managing government and maximising public and organisational support for policies. So, scientific evidence is one piece of a large puzzle. They may begin with a manifesto for government and, if elected, feel an obligation to carry it out. Evidence may play a part in that process but the search for evidence on policy solutions is not necessarily prompted by evidence of policy problems.

Further, ‘evidence based policy’ is one of many governance principles that politicians should feel the need to juggle. For example, in Westminster systems, ministers may try to delegate policymaking to foster ‘localism’ and/ or pragmatic policymaking, but also intervene to appear to be in control of policy, to foster a sense of accountability built on an electoral imperative. The likely mix of delegation and intervention seems almost impossible to predict, and this dynamic has a knock-on effect for evidence-informed policy. In some cases, central governments roll out the same basic policy intervention and limit local discretion; in others, it identifies broad outcomes and invites other bodies to gather evidence on how best to meet them. These differences in approach can have profound consequences on the models of evidence-informed policy available to us (see the example of Scottish policymaking).

Political science and policy studies provide a third perspective

Policy theories help us identify the relationship between evidence and policy by showing that a modern focus on ‘evidence-based policymaking’ (EBPM) is one of many versions of the same fairy tale – about ‘rational’ policymaking – that have developed in the post-war period. We talk about ‘bounded rationality’ to identify key ways in which policymakers or organisations could not achieve ‘comprehensive rationality’:

  1. They cannot separate values and facts.
  2. They have multiple, often unclear, objectives which are difficult to rank in any meaningful way.
  3. They have to use major shortcuts to gather a limited amount of information in a limited time.
  4. They can’t make policy from the ‘top down’ in a cycle of ordered and linear stages.

Limits to ‘rational’ policymaking: two shortcuts to make decisions

We can sum up the first three bullet points with one statement: policymakers have to try to evaluate and solve many problems without the ability to understand what they are, how they feel about them as a whole, and what effect their actions will have.

To do so, they use two shortcuts: ‘rational’, by pursuing clear goals and prioritizing certain kinds and sources of information, and ‘irrational’, by drawing on emotions, gut feelings, deeply held beliefs, habits, and the familiar to make decisions quickly.

Consequently, the focus of policy theories is on the links between evidence, persuasion, and framing issues to produce or reinforce a dominant way to define policy problems. Successful actors combine evidence and emotional appeals or simple stories to capture policymaker attention, and/ or help policymakers interpret information through the lens of their strongly-held beliefs.

Scientific evidence plays its part, but scientists often make the mistake of trying to bombard policymakers with evidence when they should be trying to (a) understand how policymakers understand problems, so that they can anticipate their demand for evidence, and (b) frame their evidence according to the cognitive biases of their audience.

Policymaking in ‘complex systems’ or multi-level policymaking environments

Policymaking takes place in less ordered, less hierarchical, and less predictable environment than suggested by the image of the policy cycle. Such environments are made up of:

  1. a wide range of actors (individuals and organisations) influencing policy at many levels of government
  2. a proliferation of rules and norms followed by different levels or types of government
  3. close relationships (‘networks’) between policymakers and powerful actors
  4. a tendency for certain beliefs or ‘paradigms’ to dominate discussion
  5. shifting policy conditions and events that can prompt policymaker attention to lurch at short notice.

These five properties – plus a ‘model of the individual’ built on a discussion of ‘bounded rationality’ – make up the building blocks of policy theories (many of which I summarise in 1000 Word posts). I say this partly to aid interdisciplinary conversation: of course, each theory has its own literature and jargon, and it is difficult to compare and combine their insights, but if you are trained in a different discipline it’s unfair to ask you devote years of your life to studying policy theory to end up at this point.

To show that policy theories have a lot to offer, I have been trying to distil their collective insights into a handy guide – using this same basic format – that you can apply to a variety of different situations, from explaining painfully slow policy change in some areas but dramatic change in others, to highlighting ways in which you can respond effectively.

We can use this approach to help answer many kinds of questions. With my Southampton gig in mind, let’s use some examples from public health and prevention.

Why doesn’t evidence win the day in tobacco policy?

My colleagues and I try to explain why it takes so long for the evidence on smoking and health to have a proportionate impact on policy. Usually, at the back of my mind, is a public health professional audience trying to work out why policymakers don’t act quickly or effectively enough when presented with unequivocal scientific evidence. More recently, they wonder why there is such uneven implementation of a global agreement – the WHO Framework Convention on Tobacco Control – that almost every country in the world has signed.

We identify three conditions under which evidence will ‘win the day’:

  1. Actors are able to use scientific evidence to persuade policymakers to pay attention to, and shift their understanding of, policy problems. In leading countries, it took decades to command attention to the health effects of smoking, reframe tobacco primarily as a public health epidemic (not an economic good), and generate support for the most effective evidence-based solutions.
  2. The policy environment becomes conducive to policy change. A new and dominant frame helps give health departments (often in multiple venues) a greater role; health departments foster networks with public health and medical groups at the expense of the tobacco industry; and, they emphasise the socioeconomic conditions – reductions in smoking prevalence, opposition to tobacco control, and economic benefits to tobacco – supportive of tobacco control.
  3. Actors exploit ‘windows of opportunity’ successfully. A supportive frame and policy environment maximises the chances of high attention to a public health epidemic and provides the motive and opportunity of policymakers to select relatively restrictive policy instruments.

So, scientific evidence is a necessary but insufficient condition for major policy change. Key actors do not simply respond to new evidence: they use it as a resource to further their aims, to frame policy problems in ways that will generate policymaker attention, and underpin technically and politically feasible solutions that policymakers will have the motive and opportunity to select. This remains true even when the evidence seems unequivocal and when countries have signed up to an international agreement which commits them to major policy change. Such commitments can only be fulfilled over the long term, when actors help change the policy environment in which these decisions are made and implemented. So far, this change has not occurred in most countries (or, in other aspects of public health in the UK, such as alcohol policy).

Why doesn’t evidence win the day in prevention and early intervention policy?

UK and devolved governments draw on health and economic evidence to make a strong and highly visible commitment to preventive policymaking, in which the aim is to intervene earlier in people’s lives to improve wellbeing and reduce socioeconomic inequalities and/ or public sector costs. This agenda has existed in one form or another for decades without the same signs of progress we now associate with areas like tobacco control. Indeed, the comparison is instructive, since prevention policy rarely meets the three conditions outlined above:

  1. Prevention is a highly ambiguous term and many actors make sense of it in many different ways. There is no equivalent to a major shift in problem definition for prevention policy as a whole, and little agreement on how to determine the most effective or cost-effective solutions.
  2. A supportive policy environment is far harder to identify. Prevention policy cross-cuts many policymaking venues at many levels of government, with little evidence of ‘ownership’ by key venues. Consequently, there are many overlapping rules on how and from whom to seek evidence. Networks are diffuse and hard to manage. There is no dominant way of thinking across government (although the Treasury’s ‘value for money’ focus is key currency across departments). There are many socioeconomic indicators of policy problems but little agreement on how to measure or which measures to privilege (particularly when predicting future outcomes).
  3. The ‘window of opportunity’ was to adopt a vague solution to an ambiguous policy problem, providing a limited sense of policy direction. There have been several ‘windows’ for more specific initiatives, but their links to an overarching policy agenda are unclear.

These limitations help explain slow progress in key areas. The absence of an unequivocal frame, backed strongly by key actors, leaves policy change vulnerable to successful opposition, especially in areas where early intervention has major implications for redistribution (taking from existing services to invest in others) and personal freedom (encouraging or obliging behavioural change). The vagueness and long term nature of policy aims – to solve problems that often seem intractable – makes them uncompetitive, and often undermined by more specific short term aims with a measurable pay-off (as when, for example, funding for public health loses out to funding to shore up hospital management). It is too easy to reframe existing policy solutions as preventive if the definition of prevention remains slippery, and too difficult to demonstrate the population-wide success of measures generally applied to high risk groups.

What happens when attitudes to two key principles – evidence based policy and localism – play out at the same time?

A lot of discussion of the politics of EBPM assumes that there is something akin to a scientific consensus on which policymakers do not act proportionately. Yet, in many areas – such as social policy and social work – there is great disagreement on how to generate and evaluate the best evidence. Broadly speaking, a hierarchy of evidence built on ‘evidence based medicine’ – which has randomised control trials and their systematic review at the top, and practitioner knowledge and service user feedback at the bottom – may be completely subverted by other academics and practitioners. This disagreement helps produce a spectrum of ways in which we might roll-out evidence based interventions, from an RCT-driven roll-out of the same basic intervention to a storytelling driven pursuit of tailored responses built primarily on governance principles (such as to co-produce policy with users).

At the same time, governments may be wrestling with their own governance principles, including EBPM but also regarding the most appropriate balance between centralism and localism.

If you put both concerns together, you have a variety of possible outcomes (and a temptation to ‘let a thousand flowers bloom’) and a set of competing options (outlined in table 1), all under the banner of ‘evidence based’ policymaking.

Table 1 Three ideal types EBBP

What happens when a small amount of evidence goes a very long way?

So, even if you imagine a perfectly sincere policymaker committed to EBPM, you’d still not be quite sure what they took it to mean in practice. If you assume this commitment is a bit less sincere, and you add in the need to act quickly to use the available evidence and satisfy your electoral audience, you get all sorts of responses based in some part on a reference to evidence.

One fascinating case is of the UK Government’s ‘troubled families’ programme which combined bits and pieces of evidence with ideology and a Westminster-style-accountability imperative, to produce:

  • The argument that the London riots were caused by family breakdown and bad parenting.
  • The use of proxy measures to identify the most troubled families
  • The use of superficial performance management to justify notionally extra expenditure for local authorities
  • The use of evidence in a problematic way, from exaggerating the success of existing ‘family intervention projects’ to sensationalising neuroscientific images related to brain development in deprived children …

normal brain

…but also

In other words, some governments feel the need to dress up their evidence-informed policies in a language appropriate to Westminster politics. Unless we understand this language, and the incentives for elected policymakers to use it, we will fail to understand how to act effectively to influence those policymakers.

What can you do to maximise the use of evidence?

When you ask the generic question you can generate a set of transferable strategies to engage in policymaking:

how-to-be-heard

ebpm-5-things-to-do

Yet, as these case studies of public health and social policy suggest, the question lacks sufficient meaning when applied to real world settings. Would you expect the advice that I give to (primarily) natural scientists (primarily in the US) to be identical to advice for social scientists in specific fields (in, say, the UK)?

No, you’d expect me to end with a call for more research! See for example this special issue in which many scholars from many disciplines suggest insights on how to maximise the use of evidence in policy.

Palgrave C special

3 Comments

Filed under Evidence Based Policymaking (EBPM), Prevention policy, Public health, public policy, tobacco, tobacco policy

The Science of Evidence-based Policymaking: How to Be Heard

I was interviewed in Science, on the topic of evidence-based policymaking, and we discussed some top tips for people seeking to maximise the use of evidence in a complex policy process (or, perhaps, feel less dispirited about the lack of EBPM in many cases). If it sparks your interest, I have some other work on this topic:

I am editing a series of forthcoming articles on maximising the use of scientific evidence in policy, and the idea is that health and environmental scientists can learn from many other disciplines about how to, for example, anticipate policymaker psychology, find the right policymaking venue, understand its rules and ‘currency’ (the language people use, to reflect dominant ways of thinking about problems), and tell effective stories to the right people.

Palgrave C special

I have also completed a book, some journal articles (PAR, E&P), and some blog posts on the ‘politics of evidence-based policymaking’.

Pivot cover

Two posts appear in the Guardian political science blog (me, me and Kathryn Oliver).

One post, for practitioners, has ‘5 things you need to know’, and it links to presentations on the same theme to different audiences (Scotland, US, EU).

ebpm-5-things-to-do

In this post, I’m trying to think through in more detail what we do with such insights.

The insights I describe come from policy theory, and I have produced 25 posts which introduce each of them in 1000 words (or, if you are super busy, 500 words). For example, the Science interview mentions a spirograph of many cycles, which is a reference to the idea of a policy cycle. Also look out for the 1000-word posts on framing and narrative and think about how they relate to the use of storytelling in policy.

If you like what you see, and want to see more, have a look at my general list of offerings (home page) or list of books and articles with links to theirs PDFs (CV).

how-to-be-heard

2 Comments

Filed under Evidence Based Policymaking (EBPM), public policy, Storytelling