Writing an essay on politics, policymaking, and policy change

I tend to set this simple-looking question for coursework in policy modules: what is policy, how much has it changed, and why? Students get to choose the policy issue, timeframe (and sometimes the political system), and relevant explanatory concepts.

On the face of it, it looks super-simple: A+ for everyone!

Give it a few more seconds, and you can see the difficulties:

1. We spent a lot of time agreeing that it seems almost impossible to define policy (explained in 1000 Words and 500 Words)
2. There are a gazillion possible measures of policy change (1000 Words and 500 Words)
3. There is an almost unmanageable number of models, concepts, and theories to use to explain policy dynamics (I describe about 25 in 1000 Words each)

I try to encourage some creativity when solving this problem, but also advise students to keep their discussion as simple and jargon-free as possible (often by stretching an analogy with diving, in which a well-executed simple essay can score higher than a belly-flopped hard essay).

Choosing a format: the initial advice

1. Choose a policy area (such as health) or issue (such as alcohol policy).
2. Describe the nature of policy, and the extent of policy change, in a particular time period (such as in the post-war era, since UK devolution, or since a change in government).
3. Select one or more policy concept or theory to help structure your discussion and help explain how and why policy has changed.

For example, a question might be: What is tobacco policy in the UK, how much has it changed since the 1980s, and why? I use this example because I try to answer that – UK and global – question myself, even though my 2007 article on the UK is too theory-packed to be a good model for an undergraduate essay.

Choosing a format: the cautionary advice

You may be surprised about how difficult it is to answer a simple question like ‘what is policy?’ and I will give you considerable credit for considering how to define and measure it, by identifying, for example, the use of legislation/ regulation, funding, staff, and ‘nodality’ and/ or by considering the difference between, say, policy as a statement of intent or a long term outcome. In turn, a good description and explanation of policy change is difficult. If you are feeling ambitious, you can go further, to compare, say, two issues (such as tobacco and alcohol) or places (such UK Government policy and the policy of another country), but sometimes a simple and narrow discussion can be as, or more, effective. Similarly, you can use many theories or concepts to aid explanation, but often one theory will do. Note that (a) your description of your research question, and your essay structure, is more important than (b) your decision on what topic to focus or concepts to use.

Choosing a topic: the ‘joined up’ advice

The wider aim is to encourage students to think about the relationship between different perspectives on policy theory and analysis. For example, in a blog and policy analysis paper they try to generate attention to a policy problem and advocate a solution. Then, they draw on policy theories and concepts to reflect on their papers, highlighting (say): the need to identify the most important audience; the importance of framing issues with a mixture of evidence and emotional appeals; and, the need to present ‘feasible’ solutions.

The reflection can provide a useful segue to the essay, since we’re already identifying important policy problems, advocating change, reflecting on how best to encourage it – such as by presenting modest objectives – and then, in the essay, trying to explain (say) why governments have not taken that advice in the past. Their interest in the policy issue can prompt interest in researching the issue further; their knowledge of the issue and the policy process can help them develop politically-aware policy analysis. All going well, it produces a virtuous circle.

Some examples from my pet subject

Let me outline how I would begin to answer the three questions with reference to UK tobacco policy. I’m offering a brief summary of each section rather than presenting a full essay with more detail (partly to hold on to that idea of creativity – I don’t want students to use this description as a blueprint).

What is modern UK tobacco policy?

Tobacco policy in the UK is now one of the most restrictive in the world. The UK government has introduced a large number of policy instruments to encourage a major reduction of smoking in the population. They include: legislation to ban smoking in public places; legislation to limit tobacco advertising, promotion, and sponsorship; high taxes on tobacco products; unequivocal health education; regulations on tobacco ingredients; significant spending on customs and enforcement measures; and, plain packaging measures.

[Note that I selected only a few key measures to define policy. A fuller analysis might expand on why I chose them and why they are so important].

How much has policy changed since the 1980s?

Policy has changed radically since the post-war period, and most policy change began from the 1980s, but it was not until the 2000s onwards that the UK cemented its place as one of the most restrictive countries. The shift from the 1980s relates strongly to the replacement of voluntary agreements and limited measures with limited enforcement with legislative measures and stronger enforcement. The legislation to ban tobacco advertising, passed in 2002, replaced limited bans combined with voluntary agreements to (for example) keep billboards a certain distance from schools. The legislation to ban smoking in public places, passed in 2006 (2005 in Scotland), replaced voluntary measures which allowed smoking in most pubs and restaurants. Plain packaging measures, combined with large and graphic health warnings, replace branded packets which once had no warnings. Health education warnings have gone from stating the facts and inviting smokers to decide, and the promotion of harm reduction (smoke ‘low tar’), to an unequivocal message on the harms of smoking and passive smoking.

[Note that I describe these changes in broad terms. Other articles might ‘zoom’ in on specific instruments to show how exactly they changed]

Why has it changed?

This is the section of the essay in which we have to make a judgement about the type of explanation: should you choose one or many concepts; if many, do you focus on their competing or complementary insights; should you provide an extensive discussion of your chosen theory?

I normally recommend a very small number of concepts or simple discussion, largely because there is only so much you can say in an essay of 2-3000 words.

For example, a simple ‘hook’ is to ask if the main driver was the scientific evidence: did policy change as the evidence on smoking (and then passive smoking) related harm became more apparent? Is it a good case of ‘evidence based policymaking’? The answer may then note that policy change seemed to be 20-30 years behind the evidence [although I’d have to explain that statement in more depth] and set out the conditions in which this driver would have an effect.

In short, one might identify the need for a ‘policy environment’, shaped by policymakers, and conducive to a strong policy response based on the evidence of harm and a political choice to restrict tobacco use. It would relate to decisions by policymakers to: frame tobacco as a public health epidemic requiring a major government response (rather than primarily as an economic good or issue of civil liberties); place health departments or organisations at the heart of policy development; form networks with medical and public health groups at the expense of tobacco companies; and respond to greater public support for control, reduced smoking prevalence, and the diminishing economic value of tobacco.

This discussion can proceed conceptually, in a relatively straightforward way, or with the further aid of policy theories which ask further questions and help structure the answers.

For example, one might draw on punctuated equilibrium theory to help describe and explain shifts of public/media/ policymaker attention to tobacco, from low and positive in the 1950s to high and negative from the 1980s.

Or, one might draw on the ACF to explain how pro-tobacco coalitions helped slow down policy change by interpreting new scientific evidence though the ‘lens’ of well-established beliefs or approaches (examples from the 1950s include filter tips, low tar brands, and ventilation as alternatives to greater restrictions on smoking).

One might even draw on multiple streams analysis to identify a ‘window of opportunity for change (as I did when examining the adoption of bans on smoking in public places).

Any of these approaches will do, as long as you describe and justify your choice well. One cannot explain everything, so it may be better to try to explain one thing well.

What happens when UK Governments try to control and delegate policymaking? #POLU9UK

To celebrate Andy Murray becoming number 1, I have recorded the podcast in the style of him giving an interview:

 

British politics looks weird because UK governments have contradictory incentives: to look like they are in control, but delegate most, of policymaking; to take but shuffle off responsibility for policy outcomes; to hold on and let go.

These incompatible incentives reflect our incompatible stories of British politics:

• One stresses central control, the other stresses complexity and emergent outcomes despite central government intervention
• One stresses the need for central control to ensure clear lines of accountability, the other stresses the need for pragmatism and how ridiculous it is to hold people to account for things over which they have minimal control.
• One gets all the attention, despite being misleading, partly because it relates to a simple and comforting message on accountability and the exciting world of high politics. The other gets little attention, despite being more accurate, because its message is confusing and often boring.

So, when we discuss the big post-war developments in British politics, and their impact on policymaking and accountability, we should not expect to find a grand or consistent plan. Instead, post war government reforms reflect these contradictions, and prompt a tendency for elected policymakers to delegate or ‘shuffle off’ most responsibility but intervene in unpredictable and inconsistent ways.

What were these big changes? 1. A shift from state to market?

I say this not to diminish the argument that major changes from the 1970s, to alter the balance between the state and market in the UK, were often ideologically driven. Rather, don’t assume that the consistent/systematic application of that ideology is the main explanation. In some cases, governments:

• diluted their reformist beliefs, preferring pragmatism and realistic aims
• pursued reforms for simple aims such as to bolster their popularity
• accepted or reinforced the actions of their predecessors (even if from another party)
• pursued major reforms after key events and crises seemed to force their hand.

Overall, politics is often about telling a story about handling government or crises well, not actually controlling events and outcomes, and no single elected government can oversee a 10, 20, or 30-year plan to reform the state in the scale we witnessed.

Still, we can now see fundamental differences when we compare the UK state with that of the 1970s. Examples include:

• A ‘paradigm’ shift in economic policy, from ‘Keynesian’ to ‘monetarist’ economics (see Hall), prompted by economic crisis in the 1970s under Labour and the election of a Conservative government in 1979. For example, governments no longer promise to achieve ‘full employment’ via measures such as capital investment (indeed, the Thatcher government appeared to accept high unemployment while favouring inflation controls).
• Privatisation. The sale of public assets (including major nationalised utilities and local authority owned social housing), break up of state monopolies, injection of competition in the public sector, introduction of public–private partnerships for major capital projects, and charging for government services.

In both cases, you can see one form of this debate on central control playing out: for some advocates of economic reform and privatisation, this was about producing a ‘rejuvenated’ and ‘lean’ state, with ministers able to focus on core tasks – making strategic decisions and creating rules for others to follow – without having to pretend that they can control the economy or manage major industries. In this account, post-war developments were based on the idea of state planning and central control over the economy and most public services, while post-79 developments were driven by the belief that such planning had failed.

Although prompted by the Conservative government of 1979-97, the Labour government from 1997-2010 reinforced most measures (and privatised more services than Thatcher would have envisaged). It also extended the idea of limiting central government ministerial intervention in the economy by introducing Bank of England independence (making it primarily responsible for interest rates and strategies to manage inflation).

2. A shift from ‘rowing’ to ‘steering’?

This ‘lean’ theme is summed up in the metaphor (made famous by management consultants Osborne and Gaebler) of ‘steering, not rowing’, in which governments decide to provide direction to public services/ public servants rather than managing them directly. Also look out for the phrase ‘new public management’ (NPM) which mostly describes the application of private business methods to the public sector. Examples include:

• Civil service reforms to separate strategic ministerial/ operational decisions and make public servants more directly accountable for the latter.
• Quasi-markets. Public bodies like hospitals and schools are given greater operational independence. One part of the public sector competes with another for (say) the business of commissioning agencies and/ or to compete in league tables of performance.
• Quangos. The increased use of quasi-non-governmental bodies, sponsored by government departments but operating at ‘arms-length’ from elected policymakers.
• Public sector reforms in which non-governmental bodies play an increasing role in service delivery while subject to regulation, inspection, and performance management.

3. Constitutional

These reforms, often designed to give a sense of reinforced central control, are different from decisions by the UK government to shift power upwards, to the European Union, and downwards,(a) in 1999, to devolved governments in Scotland, Wales and Northern Ireland, and (b) through various experiments in regional government (in the early 2000s) and ‘localism’ (from 2010).

What is the overall effect of these reforms?

These reforms prompted several debates about the modern nature of the UK state, based on questions such as, Is it ‘hollowing’ or rejuvenated?

• Is UK central government now less able to influence policy outcomes, and more reliant on persuasion and cooperation from many actors in policy networks? Do we talk about multi-level governance, not government, because no single government can control policy? Is this the great irony of reform: they were designed to reinforce central control but they actually exacerbated the UK’s governance problem?
• Or, has central government shuffled off direct responsibility for the previously unmanageable parts of the public sector that took up a disproportionate amount of ministerial energy (major industries, local government, Scotland), and become more powerful via regulatory mechanisms or more able to shift blame?

When considering these questions, note how this UK-specific discussion can be supplemented by the ‘universal’ factors we discuss in POLU9UK and covered in the 1000 Words series, including: ministers are boundedly rational, operating in a policy environment with a huge number of actors, and apparently unable to control outcomes that ‘emerge’ from complex systems. In other words, the answer to the ‘hollowing’ question will not come only from an analysis of UK government policies.

What is the effect on ministerial accountability?

As in Scotland, the UK Government has experimented with many forms of accountability based on one of these two stories of central government:

1. Westminster-style democratic accountability, through periodic elections and more regular reports by ministers to Westminster. This requires a strong sense of central government and ministerial control – if you know who is in charge, you know who to hold to account or reward or punish in the next election.
2. Institutional accountability, through performance management measures applied to the chief executives of public bodies, such as elected local authorities and unelected agencies and quangos.
3. Accountability via pluralist democracy, fostering the shared ‘ownership’ of policy with stakeholders to produce choices that both support.
4. Localist democracy, encouraging a sense of collective responsibility between local authorities and their stakeholders.
5. User based notions of accountability, when a public body considers its added value to (and responds to the wishes of) service users, or public bodies and users ‘co-produce’ and share responsibility for the outcomes.

Yet, 2-5 generally seem incompatible with, or overshadowed by, 1. Ministers think that the public expects Westminster-style accountability, so they try these other measures but also:

• Try to show that they still control the direction of delegated services, often with reference to problematic proxies of their own success (see the example of Troubled Families)
• Intervene in an ad hoc way in the decisions of public bodies that they’d otherwise like to run themselves (see Gains and Stoker)
• Or, they seem to delegate power to public bodies but introduce so many regulations, budget limits, and performance measures that it is difficult for those bodies to exert their autonomy (see the example of ‘prevention policy’, in which central governments simultaneously support and scupper various forms of prevention and early intervention).

Group work

In groups we can discuss these major reforms and the extent to which they were driven by a grand plan or a series of unfortunate events.

We can discuss accountability and try to explain how and why ministers intervene in some areas but not others.

Since we focused on the two basic stories of (lack of) control in week 2, this week we can zoom in to discuss specific measures to demonstrate success in government or produce the appearance of control. What examples spring to mind?

Using psychological insights in politics: can we do it without calling our opponents mental, hysterical, or stupid?

One of the most dispiriting parts of fierce political debate is the casual use of mental illness or old and new psychiatric terms to undermine an opponent: she is mad, he is crazy, she is a nutter, they are wearing tin foil hats, get this guy a straitjacket and the men in white coats because he needs to lie down in a dark room, she is hysterical, his position is bipolar, and so on. This kind of statement reflects badly on the campaigner rather than their opponent.

I say this because, while doing some research on a paper on the psychology of politics and policymaking (this time with Richard Kwiatkowski, as part of this special collection), there are potentially useful concepts that seem difficult to insulate from such political posturing. There is great potential to use them cynically against opponents rather than benefit from their insights.

The obvious ‘live’ examples relate to ‘rational’ versus ‘irrational’ policymaking. For example, one might argue that, while scientists develop facts and evidence rationally, using tried and trusted and systematic methods, politicians act irrationally, based on their emotions, ideologies, and groupthink. So, we as scientists are the arbiters of good sense and they are part of a pathological political process that contributes to ‘post truth’ politics.

The obvious problem with such accounts is that we all combine cognitive and emotional processes to think and act. We are all subject to bias in the gathering and interpretation of evidence. So, the more positive, but less tempting, option is to consider how this process works – when both competing sides act ‘rationally’ and emotionally – and what we can realistically do to mitigate the worst excesses of such exchanges. Otherwise, we will not get beyond demonising our opponents and romanticising our own cause. It gives us the warm and fuzzies on twitter and in academic conferences but contributes little to political conversations.

A less obvious example comes from modern work on the links between genes and attitudes. There is now a research agenda which uses surveys of adult twins to compare the effect of genes and environment on political attitudes. For example, Oskarsson et al (2015: 650) argue that existing studies ‘report that genetic factors account for 30–50% of the variation in issue orientations, ideology, and party identification’. One potential mechanism is cognitive ability: put simply, and rather cautiously and speculatively, with a million caveats, people with lower cognitive ability are more likely to see ‘complexity, novelty, and ambiguity’ as threatening and to respond with fear, risk aversion, and conservatism (2015: 652).

My immediate thought, when reading this stuff, is about how people would use it cynically, even at this relatively speculative stage in testing and evidence gathering: my opponent’s genes make him stupid, which makes him fearful of uncertainty and ambiguity, and therefore anxious about change and conservative in politics (in other words, the Yoda hypothesis applied only to stupid people). It’s not his fault, but his stupidity is an obstacle to progressive politics. If you add in some psychological biases, in which people inflate their own sense of intelligence and underestimate that of their opponents, you have evidence-informed, really shit political debate! ‘My opponent is stupid’ seems a bit better than ‘my opponent is mental’ but only in the sense that eating a cup of cold sick is preferable to eating shit.

I say this as we try to produce some practical recommendations (for scientist and advocates of EBPM) to engage with politicians to improve the use of evidence in policy. I’ll let you know if it goes beyond a simple maxim: adapt to their emotional and cognitive biases, but don’t simply assume they’re stupid.

See also: the many commentaries on how stupid it is to treat your political opponents as stupid

A great response to those who blame "low info voters" and call for "epistocracy", from @clairlemon https://t.co/n6hoepr027 pic.twitter.com/x1IzklyqbK

— Jonathan Haidt (@JonHaidt) December 2, 2016

Stop Calling People “Low Information Voters”

Socioeconomic factors and events in British politics #POLU9UK

See also: Policy Concepts in 1000 Words: Context, Events, Structural and Socioeconomic Factors (you can hear my dulcet tones there, so I won’t do an extra podcast).

You get the idea by now. We began the course with a stylised Westminster model of centralised control and each week we use something new to chip away at that image. This week we focus on the sense that the UK government is (a) constantly responding to events rather than setting the political agenda, and (b) dealing with policy conditions and outcomes that seem to be out of its control.

Big E and small e events

You might get two impressions from the word ‘events’:

1. The really big ones that seem to shock half of the population (more or less, or much more). Take your pick from recent Events including Brexit and the Scottish independence referendum, or from economic crises including the global financial crises and shocks to the British Pound. Generally speaking, it is difficult to get a sense from these Events that the UK Government is in control of the agenda or outcome.
2. The day-to-day ones. You now get this other sense of events most strongly from social media: people’s attention lurches from issue to issue very quickly, and governments (or individual policymakers) often seem to struggle to respond effectively or set the agenda.

So, as we discussed in week 2, policymakers will often settle for the chance to portray themselves as decisive in responding and adapting to such events rather than controlling them (perhaps particularly in an age in which they struggle more and more to control the flow of information within populations).

Policy conditions: from funnels of causality to globalisation

A reference to policy ‘conditions’ or ‘environments’ is broader. It refers to the context in which policymakers make choices, including:

• Literally, the environment in which people live and the spread of populations in urban and rural areas.
• The demographic profile and trends in birth and ageing.
• Levels of economic activity.
• Social behaviour and attitudes.
• Technological changes which prompt social change, from mass road transit to information technology.
• Governing institutions with rules that constrain and facilitate behaviour.

One main connection between conditions and events is that the former often shape the latter: environmental crises prompt new forms of behaviour, ageing populations prompt a sense of crisis in health and social care, economic downturns panic governments, and so on. You get the idea: this is a far cry from our initial starting point in which we focus on what governments do. If we focus on what surrounds governments we get more of a sense of the limits to what governments can do, and a limited sense of their control of policymaking processes and outcomes.

As the language of ‘structure and agency’ suggests, we need to find a convincing way to describe this sense of limited choice. We (or, at least, I) want to maintain the sense that policymakers are actors making choices but that some choices are far more attractive or possible than others.

So, we have a choice about how to portray these choices. On the one hand, we have accounts which focus on the limits to choice:

• The classic (but now little-discussed) way of thinking about wider conditions is Hofferbert’s ‘funnel of causality’. Its usefulness is to expand our horizons to think about the wider (literal or metaphorical) environment of policymaking in which, for example, geographical conditions influence population concentrations and public behaviour and attitudes influence elite behaviour.
• The concept of ‘globalisation’ prompts us to think about the pressures on domestic governments to respond to global factors often outside of their control. In such cases, their choices about how to respond to external factors are not particularly attractive, such as when they are deciding how to set interest rates to deal with external fluctuations in demand for their currency (who do I mean by ‘they’ these days?), or how willing they are to reduce taxes and offer subsidies to attract foreign direct investment.

On the other hand is the sense that actors mediate such conditions and events: to a large extent they decide how to interpret events, the importance to attach to policy conditions, and which conditions produce events that seem the most urgent or important. In other words, many governments have shown an impressive ability to completely ignore events that other governments would treat as urgent crises.

The latter point is a nice segue to one of the recommended articles for this week, by Hindmoor and McConnell, in which they discuss the UK Government’s response to financial crisis. They remind us that we should understand the processing of events as they occurred, rather than via hindsight. This approach allows us to see that governments have highly imperfect ways to gather information and detect ‘warning signals’ effectively: what seems obvious now would not be obvious then. What now seems like a crisis to which governments inevitably had to respond would then seem very different.

Group exercises

In our groups we can identify and discuss key examples: how have UK governments dealt with demographic change or economic crisis? What kinds of factors are most likely to get their attention (and why?) and how are they likely to deal with them? What are the big events or conditions in UK politics that seem impossible to ignore? And what does our discussion tell us about the idea of a UK political system characterised by central government control and a centralisation of power.

Policy networks and communities #POLU9UK

As we discussed in week 2, if you start your study of British politics by describing the Westminster model, you get something like this:

Key parts of the Westminster political system help concentrate power in the executive. Representative democracy is the basis for most participation and accountability. The UK is a unitary state built on parliamentary sovereignty and a fusion of executive and legislature, not a delegation or division of powers. The plurality electoral system exaggerates single party majorities, the whip helps maintain party control of Parliament, the government holds the whip, and the Prime Minister controls membership of the government. So, you get centralised government and you know who is in charge and therefore to blame.

Yet, if you read the recommended reading, you get this:

Most contemporary analysts dwell on the shortcomings of the Westminster account and compare it with a more realistic framework based on modern discussions of governance … Britain has moved away from a distinctive Westminster model.

And, if you read this post on the pervasiveness of policy networks and communities, you get something like this:

‘Policy networks’ or ‘policy communities’ represent the building blocks of policy studies. Most policy theories situate them at the heart of the policy process.

So, you may want to know: ‘How did we get from the one case of affairs to the other case of affairs?’ (source). Here are some possible explanations to discuss.

One account is wrong

In our grumpy account, we pretty much complain that the incorrect story still wins because it sounds so good. The uncool academics have all agreed that the ‘governance’ story best sums up British politics, but the media and public don’t pay attention to it, politicians act as if it doesn’t exist, and cool Lijphart gets all the attention with his ‘majoritarian’ model of the UK which accentuates the adversarial and top-down nature compared to the utopian consensus democracies in which all politicans hold hands and sing together before agreeing all their policies.

One account is wrong most of the time

When less grumpy, we suggest that our account is correct most of the time. People pay attention to the exciting world of elected politics and governing politicians, but it represents the tip of the iceberg. Most policy is processed below the surface, away from the public spotlight, and this process does not match the UK’s majoritarian image. Instead, policymakers tend to work routinely with other policy participants to share information and advice and come to collective understandings of problems and feasible solutions.

What explains the shift from one image to the other?

If we go for the latter explanation, we need to know how this process works: what prompts a tiny number of issues to receive the excitement and attention and a huge number to receive almost none? I’ll give you some ideas below, but note that you can find the same basic explanation of this agenda setting/ framing process in many theories of the policy process. You should read as many as possible and, in particular, those on framing, punctuated equilibrium, and power/ideas. Combined, you get the sense of two scenarios: one in which people simply can’t pay attention to many policy issues and have to ignore most; and, one in which people exploit this limitation to make sure that some issues are ignored (for example, by framing issues as ‘solved’ by policymakers, with only experts required to oversee the implementation of key choices).

The general explanation: powerful people have limited attention

You’ll find this general explanation squirrelled away somewhere in almost everything I’ve written. In this case, it’s in the networks 1000 words post:

• The size and scope of the state is so large that it is in danger of becoming unmanageable. The same can be said of the crowded environment in which huge numbers of actors seek policy influence. Consequently, the state’s component parts are broken down into policy sectors and sub-sectors, with power spread across government.
• Elected policymakers can only pay attention to a tiny proportion of issues for which they are responsible. So, they pay attention to a small number and ignore the rest. In effect, they delegate policymaking responsibility to other actors such as bureaucrats, often at low levels of government.
• At this level of government and specialisation, bureaucrats rely on specialist organisations for information and advice.
• Those organisations trade that information/advice and other resources for access to, and influence within, the government (other resources may relate to who groups represent – such as a large, paying membership, an important profession, or a high status donor or corporation).
• Therefore, most public policy is conducted primarily through small and specialist policy communities that process issues at a level of government not particularly visible to the public, and with minimal senior policymaker involvement.

A specific explanation: even ‘majoriarian’ governments seek consensus even when issues become high profile

I like this story about Brent Spar as an example of ‘bureaucratic accommodation’. In a nutshell (from p577), they argue that we began with a high profile issue in which Greenpeace occupied a Shell oil rig that was due for disposal, got Shell to change its policy through high profile campaigning, but that they came to quieter agreement within government by agreeing on specific policies without shifting their basic principles. Many of us saw the conflict but few saw the consensus building that followed (and, in fact, preceded these events). There are many stories like this, in which relatively short periods of highly salient policymaking ‘punctuate’ much longer spells of humdrum activity.

Group activities

So, in our group work we can explore the key themes through examples. I’ll ask you to identify the conditions under which Westminster-model-style activity happens, and the conditions under which we’d expect policy communities to develop. I’ll ask you to compare issues in which there is high salience and conflict with issues that are low salience and/ or low conflict. I might even ask you to remember some high profile issues from the past then ask: where are they now?

 

 

 

 

Why the pollsters got it wrong

We have a new tradition in politics in which some people glory in the fact that the polls got it wrong. It might begin with ‘all these handsome experts with all their fancy laptops and they can’t even tell us exactly how an election will turn out’, and sometimes it ends with, ‘yet, I knew it all along’. I think that the people who say it most are the ones that are pleased with the result and want to stick it to the people who didn’t predict it: ‘if, like me, they’d looked up from their laptops and spoken to real people, they’d have seen what would happen’.

To my mind, it’s always surprising when so many polls seem to do so well. Think for a second about what ‘pollsters’ do: they know they can’t ask everyone how they will vote (and why), so they take a small sample and use it as a proxy for the real world. To make sure the sample isn’t biased by selection, they develop methods to generate respondents randomly. To try to make the most of their resources, and make sure that their knowledge is cumulative, they use what they think they know about the population to make sure that they get enough responses from a ‘representative’ sample of the population. In many cases, that knowledge comes from things like focus groups or one-to-one interviews to get richer (qualitative) information than we can achieve from asking everyone the same question, often super-quickly, in a larger survey.

This process involves all sorts of compromises and unintended consequences when we have a huge population but limited resources: we’d like to ask everyone in person, but it’s cheaper to (say) get a 4-figure response online or on the phone; and, if we need to do it quickly, our sample will be biased towards people willing to talk to us.* So, on top of a profound problem – the possibility of people not telling the truth in polls – we have a potentially less profound but more important problem: the people we need to talk to us aren’t talking to us. So, we get a misleading read because we’re asking an unrepresentative sample (although it is nothing like as unrepresentative as proxy polls from social media, the word ‘on the doorstep’, or asking your half-drunk mates how they’ll vote).

Sensible ‘pollsters’ deal with such problems by admitting that they might be a bit off: highlighting their estimated ‘margin of error’ from the size of their sample, then maybe crossing their fingers behind their backs if asked about the likelihood of more errors based on non-random sampling. So, ignore this possibility for error at your peril. Yet, people do ignore it despite the peril! Here are two reasons why.

1. Being sensible is boring.

In a really tight-looking two-horse race, the margin of error alone might suggest that either horse might win. So, a sensible interpretation of a poll might be (say), ‘either Clinton or Trump will get the most votes’. Who wants to hear or talk about that?! You can’t fill a 24-hour news cycle and keep up shite Twitter conversations by saying ‘who knows?’ and then being quiet. Nor will anyone pay much attention to a quietly sensible ‘pollster’ or academic telling them about the importance of embracing uncertainty. You’re in the studio to tell us what will happen, pal. Otherwise, get lost.

2. Recognising complexity and uncertainty is boring.

You can heroically/ stupidly break down the social scientific project into two competing ideas: (1) the world contains general and predictable patterns of behaviour that we can identify with the right tools; or (2) the world is too complex and unpredictable to produce general laws of behaviour, and maybe your best hope is to try to make sense of how other people try to make sense of it. Then, maybe (1) sounds quite exciting and comforting while (2) sounds like it is the mantra of a sandal-wearing beansprout-munching hippy academic. People seem to want a short, confidently stated, message that is easy to understand. You can stick your caveats.

Can we take life advice from this process?

These days I’m using almost every topic as a poorly-constructed segue into a discussion about the role of evidence in politics and policy. This time, the lesson is about using evidence correctly for the correct purpose. In our example, we can use polls effectively for their entertainment value. Or, campaigners can use them as the best-possible proxies during their campaigns: if their polls tell them they are lagging in one area, give it more attention; if they seem to have a big lead in another area; give it less attention. The evidence won’t be totally accurate, but it gives you enough to generate a simple campaigning strategy. Academics can also use the evidence before and after a campaign to talk about how it’s all going. Really, the only thing you don’t expect poll evidence to do is predict the result. For that, you need the Observers from Fringe.

The same goes for evidence in policymaking: people use rough and ready evidence because they need to act on what they think is going on. There will never be enough evidence to make the decision for you, or let you know exactly what will happen next. Instead, you combine good judgement with your values, sprinkle in some evidence, and off you go. It would be silly to expect a small sample of evidence – a snapshot of one part of the world – to tell you exactly what will happen in the much larger world. So, let’s not kid ourselves about the ability of science to tell us what’s what and what to do. It’s better, I think, to recognise life’s uncertainties and act accordingly. It’s better than blaming other people for not knowing what will happen next.

 

*I say ‘we’ and ‘us’ but I’ve never conducted a poll in my life. I interview elites in secret and promise them anonymity.

We all want ‘evidence based policy making’ but how do we do it?

Here are some notes for my talk to the Scottish Government on Thursday as part of its ‘inaugural ‘evidence in policy week’. The advertised abstract is as follows:

A key aim in government is to produce ‘evidence based’ (or ‘informed’) policy and policymaking, but it is easier said than done. It involves two key choices about (1) what evidence counts and how you should gather it, and (2) the extent to which central governments should encourage subnational policymakers to act on that evidence. Ideally, the principles we use to decide on the best evidence should be consistent with the governance principles we adopt to use evidence to make policy, but what happens when they seem to collide? Cairney provides three main ways in which to combine evidence and governance-based principles to help clarify those choices.

I plan to use the same basic structure of the talks I gave to the OSF (New York) and EUI-EP (Florence) in which I argue that every aspect of ‘evidence based policy making’ is riddled with the necessity to make political choices (even when we define EBPM):

I’ll then ‘zoom in’ on points 4 and 5 regarding the relationship between EBPM and governance principles. They are going to videotape the whole discussion to use for internal discussions, but I can post the initial talk here when it becomes available. Please don’t expect a TED talk (especially the E part of TED).

EBPM and good governance principles

The Scottish Government has a reputation for taking certain governance principles seriously, to promote high stakeholder ‘ownership’ and ‘localism’ on policy, and produce the image of a:

1. Consensual consultation style in which it works closely with interest groups, public bodies, local government organisations, voluntary sector and professional bodies, and unions when making policy.
2. Trust-based implementation style indicating a relative ability or willingness to devolve the delivery of policy to public bodies, including local authorities, in a meaningful way

Many aspects of this image were cultivated by former Permanent Secretaries: Sir John Elvidge described a ‘Scottish Model’ focused on joined-up government and outcomes-based approaches to policymaking and delivery, and Sir Peter Housden labelled the ‘Scottish Approach to Policymaking’ (SATP) as an alternative to the UK’s command-and-control model of government, focusing on the ‘co-production’ of policy with local communities and citizens.

The ‘Scottish Approach’ has implications for evidence based policy making

Note the major implication for our definition of EBPM. One possible definition, derived from ‘evidence based medicine’, refers to a hierarchy of evidence in which randomised control trials and their systematic review are at the top, while expertise, professional experience and service user feedback are close to the bottom. An uncompromising use of RCTs in policy requires that we maintain a uniform model, with the same basic intervention adopted and rolled out within many areas. The focus is on identifying an intervention’s ‘active ingredient’, applying the correct dosage, and evaluating its success continuously.

This approach seems to challenge the commitment to localism and ‘co-production’.

At the other end of the spectrum is a storytelling approach to the use of evidence in policy. In this case, we begin with key governance principles – such as valuing the ‘assets’ of individuals and communities – and inviting people to help make and deliver policy. Practitioners and service users share stories of their experiences and invite others to learn from them. There is no model of delivery and no ‘active ingredient’.

This approach seems to challenge the commitment to ‘evidence based policy’

The Goldilocks approach to evidence based policy making: the improvement method

We can understand the Scottish Government’s often-preferred method in that context. It has made a commitment to:

‘Service performance and improvement underpinned by data, evidence and the application of improvement methodologies’

So, policymakers use many sources of evidence to identify promising, make broad recommendations to practitioners about the outcomes they seek, and they train practitioners in the improvement method (a form of continuous learning summed up by a ‘Plan-Do-Study-Act’ cycle).

This approach appears to offer the best of both worlds; just the right mix of central direction and local discretion, with the promise of combining well-established evidence from sources including RCTs with evidence from local experimentation and experience.

Four unresolved issues in decentralised evidence-based policy making

Not surprisingly, our story does not end there. I think there are four unresolved issues in this process:

1. The Scottish Government often indicates a preference for improvement methods but actually supports all three of the methods I describe. This might reflect an explicit decision to ‘let a thousand flowers bloom’ or the inability to establish a favoured approach.
2. There is not a single way of understanding ‘improvement methodology’. I describe something akin to a localist model here, but other people describe a far more research-led and centrally coordinated process.
3. Anecdotally, I hear regularly that key stakeholders do not like the improvement method. One could interpret this as a temporary problem, before people really get it and it starts to work, or a fundamental difference between some people in government and many of the local stakeholders so important to the ‘Scottish approach’.

4. The spectre of democratic accountability and the politics of EBPM

The fourth unresolved issue is the biggest: it’s difficult to know how this approach connects with the most important reference in Scottish politics: the need to maintain Westminster-style democratic accountability, through periodic elections and more regular reports by ministers to the Scottish Parliament. This requires a strong sense of central government and ministerial control – if you know who is in charge, you know who to hold to account or reward or punish in the next election.

In principle, the ‘Scottish approach’ provides a way to bring together key aims into a single narrative. An open and accessible consultation style maximises the gathering of information and advice and fosters group ownership. A national strategic framework, with cross-cutting aims, reduces departmental silos and balances an image of democratic accountability with the pursuit of administrative devolution, through partnership agreements with local authorities, the formation of community planning partnerships, and the encouragement of community and user-driven design of public services. The formation of relationships with public bodies and other organisations delivering services, based on trust, fosters the production of common aims across the public sector, and reduces the need for top-down policymaking. An outcomes-focus provides space for evidence-based and continuous learning about what works.

In practice, a government often needs to appear to take quick and decisive action from the centre, demonstrate policy progress and its role in that progress, and intervene when things go wrong. So, alongside localism it maintains a legislative, financial, and performance management framework which limits localism.

How far do you go to ensure EBPM?

So, when I describe the ‘5 things to do’, usually the fifth element is about how far scientists may want to go, to insist on one model of EBPM when it has the potential to contradict important governance principles relating to consultation and localism. For a central government, the question is starker:

Do you have much choice about your model of EBPM when the democratic imperative is so striking?

I’ll leave it there on a cliff hanger, since these are largely questions to prompt discussion in specific workshops. If you can’t attend, there is further reading on the EBPM and EVIDENCE tabs on this blog, and specific papers on the Scottish dimension

The ‘Scottish Approach to Policy Making’: Implications for Public Service Delivery

Paul Cairney, Siabhainn Russell and Emily St Denny (2016) “The ‘Scottish approach’ to policy and policymaking: what issues are territorial and what are universal?” Policy and Politics, 44, 3, 333-50

The politics of evidence-based best practice: 4 messages

 

 

How can political actors take into account the limitations of evidence-based policy-making? 5 key points

These notes are for my brief panel talk at the European Parliament-European University Institute ‘Policy Roundtable’: Evidence and Analysis in EU Policy-Making: Concepts, Practice and Governance. As you can see from the programme description, the broader theme is about how EU institutions demonstrate their legitimacy through initiatives such as stakeholder participation and evidence-based policymaking (EBPM). So, part of my talk is about what happens when EBPM does not exist.

The post is a slightly modified version of my (recorded) talk for Open Society Foundations (New York) but different audiences make sense of these same basic points in very different ways.

1. Recognise that the phrase ‘evidence-based policy-making’ means everything and nothing

The main limitation to ‘evidence-based policy-making’ is that no-one really knows what it is or what the phrase means. So, each actor makes sense of EBPM in different ways and you can tell a lot about each actor by the way in which they answer these questions:

• Should you use restrictive criteria to determine what counts as ‘evidence’? Some actors equate evidence with scientific evidence and adhere to specific criteria – such as evidence-based medicine’s hierarchy of evidence – to determine what is scientific. Others have more respect for expertise, professional experience, and stakeholder and service user feedback as sources of evidence.
• Which metaphor, evidence based or informed is best? ‘Evidence based’ is often rejected by experienced policy participants as unrealistic, preferring ‘informed’ to reflect pragmatism about mixing evidence and political calculations.
• How far do you go to pursue EBPM? It is unrealistic to treat ‘policy’ as a one-off statement of intent by a single authoritative actor. Instead, it is made and delivered by many actors in a continuous policymaking process within a complicated policy environment (outlined in point 3). This is relevant to EU institutions with limited resources: the Commission often makes key decisions but relies on Member States to make and deliver, and the Parliament may only have the ability to monitor ‘key decisions’. It is also relevant to stakeholders trying to ensure the use of evidence throughout the process, from supranational to local action.
• Which actors count as policymakers? Policymaking is done by ‘policymakers’, but many are unelected and the division between policymaker/ influencer is often unclear. The study of policymaking involves identifying networks of decision-making by elected and unelected policymakers and their stakeholders, while the actual practice is about deciding where to draw the line between influence and action.

2. Respond to ‘rational’ and ‘irrational’ thought.

‘Comprehensive rationality’ describes the absence of ambiguity and uncertainty when policymakers know what problem they want to solve and how to solve it, partly because they can gather and understand all information required to measure the problem and determine the effectiveness of solutions.

Instead, we talk of ‘bounded rationality’ and how policymakers deal with it. They employ two kinds of shortcut: ‘rational’, by pursuing clear goals and prioritizing certain kinds and sources of information, and ‘irrational’, by drawing on emotions, gut feelings, deeply held beliefs, habits, and familiarity, make decisions quickly.

I say ‘irrational’ provocatively, to raise a key strategic question: do you criticise emotional policymaking (describing it as ‘policy based evidence’) and try somehow to minimise it, adapt pragmatically to it, or see ‘fast thinking’ more positively in terms of ‘fast and frugal heuristics’? Regardless, policymakers will think that their heuristics make sense to them, and it can be counterproductive to simply criticise their alleged irrationality.

3. Think about how to engage in complex systems or policy environments.

‘Policy cycle’ describes the idea that there is a core group of policymakers at the ‘centre’, making policy from the ‘top down’, and pursuing their goals in a series of clearly defined and well-ordered stages, such as: agenda setting, policy formulation, legitimation, implementation, and evaluation. In this context, one might identify how to influence a singular point of central government decision.

However, a cycle model does not describe policymaking well. Instead, we tend to identify the role of less ordered and more unpredictable complex systems, or policy environments containing:

• A wide range of actors (individuals and organisations) influencing policy at many levels of government. Scientists and practitioners are competing with many actors to present evidence in a particular way to secure a policymaker audience.
• A proliferation of rules and norms maintained by different levels or types of government. Support for particular ‘evidence based’ solutions varies according to which organisation takes the lead and how it understands the problem.
• Important relationships (‘networks’) between policymakers and powerful actors. Some networks are close-knit and difficult to access because bureaucracies have operating procedures that favour particular sources of evidence and some participants over others, and there is a language – indicating what ways of thinking are in good ‘currency’ – that takes time to learn.
• A tendency for certain ‘core beliefs’ or ‘paradigms’ to dominate discussion. Well-established beliefs provide the context for policymaking: new evidence on the effectiveness of a policy solution has to be accompanied by a shift of attention and successful persuasion.
• Policy conditions and events that can reinforce stability or prompt policymaker attention to lurch at short notice. In some cases, social or economic ‘crises’ can prompt lurches of attention from one issue to another, and some forms of evidence can be used to encourage that shift, but major policy change is rare.

For stakeholders, an effective engagement strategy is not straightforward: it takes time to know ‘where the action is’, how and where to engage with policymakers, and with whom to form coalitions. For the Commission, it is difficult to know what will happen to policy after it is made (although we know the end point will not resemble the starting point). For the Parliament, it is difficult even to know where to look.

4. Recognise that EBPM is only one of many legitimate ‘good governance’ principles.

There are several principles of ‘good’ policymaking and only one is EBPM. Others relate to the value of pragmatism and consensus building, combining science advice with public values, improving policy delivery by generating ‘ownership’ of policy among key stakeholders, and sharing responsibility with elected national and local policymakers.

Our choice of which principle and forms of evidence to privilege are inextricably linked. For example, some forms of evidence gathering seem to require uniform models and limited local or stakeholder discretion to modify policy delivery. The classic example is a programme whose value is established using randomised control trials (RCTs). Others begin with local discretion, seeking evidence from stakeholders, professional groups, service user and local practitioner experience. This principle seems to rule out the use of RCTs, at least as a source of a uniform model to be rolled out and evaluated. Of course, one can try to pursue both approaches and a compromise between them, but the outcome may not satisfy advocates of either approach to EBPM or help produce the evidence that they favour.

5. Decide how far you’ll go to achieve EBPM.

These insights should prompt us to see how far we are willing, and should, go to promote the use of certain forms of evidence in policymaking

• If policymakers and the public are emotional decision-makers, should we seek to manipulate their thought processes by using simple stories with heroes, villains, and clear but rather simplistic morals?
• If policymaking systems are so complex, should stakeholders devote huge amounts of resources to make sure they’re effective at each stage?
• Should proponents of scientific evidence go to great lengths to make sure that EBPM is based on a hierarch of evidence? There is a live debate on science advice to government on the extent to which scientists should be more than ‘honest brokers’.
• Should policymakers try to direct the use of evidence in policy as well as policy itself?

Where we go from there is up to you

The value of policy theory to this topic is to help us reject simplistic models of EBPM and think through the implications of more sophisticated and complicated processes. It does not provide a blueprint for action (how could it?), but instead a series of questions that you should answer when you seek to use evidence to get what you want. They are political choices based on value judgements, not issues that can be resolved by producing more evidence.

 

We are recruiting a Senior Lecturer/ Associate Professor in International Politics at the University of Stirling

The details are here, and they include this discussion of further particulars:

We seek to appoint a Senior Lecturer or Associate Professor (grade 9) in international politics with an emphasis on the politics and policy of the European Union. International Politics constitutes a core element of both our successful Masters Programme in International Conflict and Cooperation (ICC), our BA programmes in Politics and International Politics, and our professional doctorate in diplomacy. The appointee will contribute to both doctoral, Masters and undergraduate provisions and to our Centre for Policy, Conflict and Co-Operation Research. An ability to deliver the introductory module Introduction to International Politics (POLU9X3) at undergraduate level is essential. The appointee will also play a key role in delivering our ICC Masters and Doctor of Diplomacy programmes. A taught specialism in fields such as European Union politics and policymaking (preferably in the context of international politics), EU in the context of international organisations, and EU public policy – as well as its intersection with concepts such as gender, sexuality, and race – would be particularly welcome. As well as making a significant contribution to our Masters and undergraduate programmes, the appointee would be expected to pursue a programme of research, including research outputs and funding applications, to undertake postgraduate research supervision relevant to their expertise and to undertake administrative duties as prescribed by the Head of Division.

Why do we make reference to ‘gender, sexuality, and race’ in the FPs?

6 of our 8 permanent lecturers are men and 8 are white. We are not interested in simply reinforcing the imbalances that are already there. So, we worded the ‘further particulars’ to make sure that people know we have realistic hopes of producing a more diverse and gender-balanced short list. Usually, job adverts will have a pro-forma statement about equalities, but we are trying to go one step further to signal – albeit with rather subtle cues – that we have thought about this issue a bit more; that we’d like to expand our networks and the ways in which our staff approach the study of politics. We are trying to make sure that our current set up does not put off women or people of colour from applying, recruiting from a subject pool in which there is (I think) a relatively good gender balance, and signaling support for research topics that might help expand our current offering.

The more general advice

I am your pre-interview contact point and recommend that you get in touch with me before you apply. In the meantime, here are some tips on the application and interview processes.

The application process:

• At this stage, the main documents are the CV and the cover letter.
• You should keep the cover letter short to show your skills at concise writing. Focus on what you can offer the Division specifically, given the nature of our call and further particulars.
• Shortlisted candidates at this level will almost certainly be established lecturers with a strong record on publications, income, and leadership – so what makes you stand out? Note that you will have the chance to play an important part of a group which is small enough (about 9 in Politics, as part of a larger Division) to act collectively – to, for example, influence its research direction (as a group, we hold 6 x 90 minute research workshops per year for that purpose).
• Focus on what you have already done when discussing what you will promise to do over the next five years. Those plans seem more realistic if there is already some sort of track record.
• We take teaching very seriously. Within our division, we plan an overall curriculum together, discuss regularly if it is working, and come to agreements about how to teach and assess work. We pride ourselves on being a small and friendly bunch of people, open to regular student contact and, for example, committed to meaningful and regular feedback. You might think about how you would contribute in that context. In particular, you should think about how you would deliver large undergraduate courses (in which you may only be an expert on some of the material) as well as the smaller, more specialist and advanced, courses closer to your expertise.

The interview process

By the interview stage, you should almost certainly have a conversation with me to make sure that you are well prepared. For example, here are the things that you really should know at that stage:

1. The teaching and research specialisms of the division and their links to cross-divisional research.
2. The kinds of courses that the division would expect you to teach.
3. Perhaps most importantly, you need to be able to articulate why you want to come and work at Stirling.‘Why Stirling?’ or ‘Why this division?’ is usually the first question in an interview, so you should think about it in advance. We recommend doing some research on Stirling and the division/ faculty, to show in some detail that you have a considered reply (beyond ‘it is a beautiful campus’). We will see through a generic response in a heartbeat and, since it is the first question, your answer will set the tone for the rest of the interview. You might check, for example, who you might share interests with in the Division, and how you might  develop links beyond the division (for example, the Centre for Gender & Feminist Studies in our school) or faculty (such as the Faculty of Social Sciences) – since this is likely to be a featured question too.
4. Then you might think about what you would bring to the University in a wider sense, such as through well-established (domestic and international) links with other scholars in academic networks.
5. Further, since ‘impact’ is of rising importance, you might discuss your links with people and organisations outside of the University, and how you have pursued meaningful engagement with the public or practitioners to maximise the wider contribution of your research.

The presentation plus interview format

1. In our system there tend to be presentations to divisional (and other interested) staff in the morning, with interviews in the afternoon. The usual expectation is that if you can’t make the date, you can’t get the job (although we can make accommodations to help you apply).
2. We recommend keeping the presentation compact, to show that you can present complex information in a concise and clear way. Presentations are usually a mix of what you do in research and what you will contribute in a wider sense to the University.
3. The interview panel varies according to the seniority of the role. For senior lecturers, the panel will have five members: one subject specialist from the Division, one other member of the Faculty (not necessarily from our division), the Head of Faculty of Arts and Humanities, a senior manager of the University (in the chair), and a senior academic in another Faculty (by the time of interview you should know what these terms mean at Stirling).

• So, note that 1 member will be a subject specialist (in Politics). This means that (at the very least) you need to describe your success in a way that a wider audience will appreciate (for example, you would have to explain the significance of a single-author article in the APSR!). It sounds daunting, but we are a friendly bunch and want you to do well. You might struggle to retain all of our names (nerves), so focus on the types of question we ask – for example, the general question to get you started will be from the senior manager, and the research question from the divisional representative. There will be 4 men and 1 woman on the panel.

 

 

 

Storytelling for Policy Change: promise and problems

I went to a fantastic workshop on storytelling for policy change. It was hosted by Open Society Foundations New York (25/6 October), and brought together a wide range of people from different backgrounds: Narativ, people experienced in telling their own story, advocacy and professional groups using stories to promote social or policy change, major funders, journalists, and academics. There was already a lot of goodwill in the room at the beginning, and by the end there was more than a lot!

The OSF plans to write up a summary of the whole discussion, so my aim is to highlight the relevance for ‘evidence-based policymaking’ and for scientists and academics seeking more ‘impact’ for their research. In short, although I recommend that scientists ‘turn a large amount of scientific evidence into simple and effective stories that appeal to the biases of policymakers’, it’s easier said than done, and not something scientists are trained in. Good storytellers might enthuse people already committed to the idea of storytelling for policy, but what about scientists more committed to the language of scientific evidence and perhaps sceptical about the need to develop this new skill (particularly those who describe stories pejoratively as ‘anecdata’)? What would make them take a leap in the dark, to give up precious research time to develop skills in storytelling?

So, let me tell you why I thought the workshop was brilliant – including outlining its key insights – and why you might not!

Why I thought it was brilliant

Academic conferences can be horrible: a seemingly never-ending list of panels with 4-5 paper givers and a discussant, taking up almost all of the talking time with too-long and often-self-indulgent and long-winded PowerPoint presentations and little time for meaningful discussion. It’s a test of meeting deadlines for the presenter and an endurance test for the listener.

This workshop was different: the organisers thought about what it means to talk and listen, and therefore how to encourage people to talk in an interesting way and encourage high attention and engagement.

There were three main ‘listening exercises’: a personal exercise in which you closed your eyes and thought about the obstacles to listening (I confess that I cheated on that one); a paired exercise in which one person listened and thought of three poses to sum up the other’s short story; and a group exercise in which people paired up, told and then summarised each other’s stories, and spoke as a group about the implications.

This final exercise was powerful: we told often-revealing stories to strangers, built up trust very quickly, and became emotionally involved in each other’s accounts. It was interesting to watch how quickly we could become personally invested in each other’s discussion, form networks, and listen intently to each other.

For me, it was a good exercise in demonstrating what you need in a policymaker audience: ideally, they should care about the problem you raise, be personally invested in trying to solve it, and trust you and therefore your description of the most feasible solutions. If it helps recreate these conditions, a storytelling scientist may be more effective than an ‘honest broker’. Without a good story to engage your audience, your evidence will be like a drop in the ocean and your audience might be checking its email or playing Pokemon Go while you present.

Key insights and impressions

Most participants expressed strong optimism about the effect of stories on society and policy, particularly when the aim is more expressive than instrumental: the act itself of telling one’s story and being heard can be empowering, particularly within marginalised groups from which we hear few voices. It can also be remarkably powerful, remarkably quickly: most of us were crying or laughing instantly and frequently as we heard many moving stories about many issues. It’s hard to overstate just how effective many of these stories were when you heard them in person.

When discussing more instrumental concerns – can we use a story to get what we want? – the optimism was more cautious and qualified. Key themes included:

• The balance between effective and ethical storytelling, accepting that if a story is a commodity it can be used by people less sympathetic to our aims, and exploring the ethics of using the lens of sympathetic characters (e.g. white grandparents) to make the case for marginalised groups.
• This ethical dimension was reinforced continuously by stories of vulnerable storytellers and the balance between telling their story and protecting their safety.
• The importance of context: the same story may have more or less impact depending on the nature of the audience; many examples conveyed the sense that a story with huge impact now would have failed 10 or 20 years ago and/ or in a different region.
• The importance of tailoring stories to the biases of audiences and trying to reframe the implications of your audience’s beliefs (one particularly interesting example was of portraying equal marriage in Ireland as the Christian thing to do).
• Many campaigns used humour and positive stories with heroes, based on the assumption that audiences would be put off by depressing stories of problems with no obvious solution but energised by a message of new possibilities.
• Many warned against stories that were too personal, identifying the potential for an audience to want to criticise or fix an individual’s life rather than solve a systemic problem (and this individualistic interpretation was most pronounced among people identifying with right-wing parties).
• Many described the need to experiment/ engage in trial-and-error to identify what works with each audience, including the length of written messages, choice of media, and choice of ‘thin’ stories with clear messages to generate quick attention or ‘thick’ stories which might be more memorable if we have the resources to tell them and people take the time to listen.

Many of these points will seem familiar if you study psychology or the psychology of policymaking. So, the benefit of these experiences is that they tell us how people have applied such insights and how it has helped their cause. Most speakers were confident that they were making an impact.

Why you may not be as impressed: two reasons

The first barrier to getting you enthusiastic is that you weren’t there. If emotional engagement is such a key part of storytelling, and you weren’t there to hear it, why would you care? So, a key barrier to making an ‘impact’ with storytelling is that it is difficult to increase its scale. You might persuade someone if they spent enough time with you, but what if you only had a few seconds in which to impress them or, worse still, you couldn’t impress them because they weren’t interested in the first place? Our worry may be that we can only influence people who are already open to our idea. This isn’t the end of the world, since a key political aim may be to enthuse people who share your beliefs and get them to act (for example, to spread the word to their friends). However, it prompts us to wonder about the varying effect of the same message and the extent to which our message’s power comes from our audience rather than our story.

The second barrier is that, when the question is framed for an academic audience – what is the scientific evidence on the impact of stories? – the answer is not clear.

On the panel devoted to this question (and in a previous session), there were some convincing accounts of the impact of initiatives such as: the Women’s Policy Institute ‘grass roots’ training in California (leading to advocacy prompting 2 dozen bills to be signed over 13 years); Purpose’s branding campaign for the White Helmets (including the miracle baby video which has received tens of millions of views); and, the Frame Works Institute’s ability to change minds with very brief interventions (for example, getting people to think in terms of problem systems more than problem individuals in areas like criminal justice).

However, the academic analysis – with contributions from Francesca Polletta, Jeff Niederdeppe, Douglas Storey, Michael Jones – tended to stress caution or note limited effects:

• Stories work when they create an empathic reaction, but intensely personal experiences are difficult to recreate when mediated (soap operas and ‘edutainment’ come closest).
• Randomised control trials suggest that stories have a measurable effect, but it’s small and compared to no intervention at all rather than a competing approach such as providing evidence in reports (note that most experimental studies do not draw on skilful storytellers)
• Stories work most clearly when they reinforce the beliefs of your allies (and when you refer to heroes, not villains), but the effects are indirect at best with your opponents.

More research required?!

So, you might want more convincing evidence before you take that giant leap to train to become a skilful storyteller: why go for it when its effects are so unclear and difficult to measure?

For me, that response might seem sensible but is also a cop out: unequivocal evidence may never arrive and good science often involves researching as you go. A key insight into policymaking regards a continuous sense of urgency to solve problems: policymakers don’t wait for the evidence to become unequivocal before they act, partly because that sense of clarity may never happen. They feel the need to act on the basis of available evidence. Perhaps scientists should at least think about doing the same when they seek to act on research rather than simply do the research: how long should you postpone potentially valuable action with the old cliché ‘more research required’?

See also: the OSF summary of the workshop