Evidence-informed policymaking: context is everything

I thank James Georgalakis for inviting me to speak at the inaugural event of IDS’ new Evidence into Policy and Practice Series, and the audience for giving extra meaning to my story about the politics of ‘evidence-based based policymaking’. The talk (using powerpoint) and Q&A is here:

 

James invited me to respond to some of the challenges raised to my talk – in his summary of the event – so here it is.

I’m working on a ‘show, don’t tell’ approach, leaving some of the story open to interpretation. As a result, much of the meaning of this story – and, in particular, the focus on limiting participation – depends on the audience.

For example, consider the impact of the same story on audiences primarily focused on (a) scientific evidence and policy, or (b) participation and power.

Normally, when I talk about evidence and policy, my audience is mostly people with scientific or public health backgrounds asking why do policymakers ignore scientific evidence? I am usually invited to ruffle feathers, mostly by challenging a – remarkably prevalent – narrative that goes like this:

• We know what the best evidence is, since we have produced it with the best research methods (the ‘hierarchy of evidence’ argument).
• We have evidence on the nature of the problem and the most effective solutions (the ‘what works’ argument).
• Policymakers seems to be ignoring our evidence or failing to act proportionately (the ‘evidence-policy barriers’ argument).
• Or, they cherry-pick evidence to suit their agenda (the ‘policy based evidence’ argument).

In that context, I suggest that there are many claims to policy-relevant knowledge, policymakers have to ignore most information before making choices, and they are not in control of the policy process for which they are ostensibly in charge.

Limiting participation as a strategic aim

Then, I say to my audience that – if they are truly committed to maximising the use of scientific evidence in policy – they will need to consider how far they will go to get what they want. I use the metaphor of an ethical ladder in which each rung offers more influence in exchange for dirtier hands: tell stories and wait for opportunities, or demonise your opponents, limit participation, and humour politicians when they cherry-pick to reinforce emotional choices.

It’s ‘show don’t tell’ but I hope that the take-home point for most of the audience is that they shouldn’t focus so much on one aim – maximising the use of scientific evidence – to the detriment of other important aims, such as wider participation in politics beyond a reliance on a small number of experts. I say ‘keep your eyes on the prize’ but invite the audience to reflect on which prizes they should seek, and the trade-offs between them.

Limited participation – and ‘windows of opportunity’ – as an empirical finding

I did suggest that most policymaking happens away from the sphere of ‘exciting’ and ‘unruly’ politics. Put simply, people have to ignore almost every issue almost all of the time. Each time they focus their attention on one major issue, they must – by necessity – ignore almost all of the others.

For me, the political science story is largely about the pervasiveness of policy communities and policymaking out of the public spotlight.

The logic is as follows. Elected policymakers can only pay attention to a tiny proportion of their responsibilities. They delegate the rest to bureaucrats at lower levels of government. Bureaucrats lack specialist knowledge, and rely on other actors for information and advice. Those actors trade information for access. In many cases, they develop effective relationships based on trust and a shared understanding of the policy problem.

Trust often comes from a sense that everyone has proven to be reliable. For example, they follow norms or the ‘rules of the game’. One classic rule is to contain disputes within the policy community when actors don’t get what they want: if you complain in public, you draw external attention and internal disapproval; if not, you are more likely to get what you want next time.

For me, this is key context in which to describe common strategic concerns:

• Should you wait for a ‘window of opportunity’ for policy change? Maybe. Or, maybe it will never come because policymaking is largely insulated from view and very few issues reach the top of the policy agenda.
• Should you juggle insider and outsider strategies? Yes, some groups seem to do it well and it is possible for governments and groups to be in a major standoff in one field but close contact in another. However, each group must consider why they would do so, and the trade-offs between each strategy. For example, groups excluded from one venue may engage (perhaps successfully) in ‘venue shopping’ to get attention from another. Or, they become discredited within many venues if seen as too zealous and unwilling to compromise. Insider/outsider may seem like a false dichotomy to experienced and well-resourced groups, who engage continuously, and are able to experiment with many approaches and use trial-and-error learning. It is a more pressing choice for actors who may have only one chance to get it right and do not know what to expect.

Where is the power analysis in all of this?

I rarely use the word power directly, partly because – like ‘politics’ or ‘democracy’ – it is an ambiguous term with many interpretations (see Box 3.1). People often use it without agreeing its meaning and, if it means everything, maybe it means nothing.

However, you can find many aspects of power within our discussion. For example, insider and outsider strategies relate closely to Schattschneider’s classic discussion in which powerful groups try to ‘privatise’ issues and less powerful groups try to ‘socialise’ them. Agenda setting is about using resources to make sure issues do, or do not, reach the top of the policy agenda, and most do not.

These aspects of power sometimes play out in public, when:

• Actors engage in politics to turn their beliefs into policy. They form coalitions with actors who share their beliefs, and often romanticise their own cause and demonise their opponents.
• Actors mobilise their resources to encourage policymakers to prioritise some forms of knowledge or evidence over others (such as by valuing scientific evidence over experiential knowledge).
• They compete to identify the issues most worthy of our attention, telling stories to frame or define policy problems in ways that generate demand for their evidence.

However, they are no less important when they play out routinely:

• Governments have standard operating procedures – or institutions – to prioritise some forms of evidence and some issues routinely.
• Many policy networks operate routinely with few active members.
• Certain ideas, or ways of understanding the world and the nature of policy problems within it, becomes so dominant that they are unspoken and taken for granted as deeply held beliefs. Still, they constrain or facilitate the success of new ‘evidence based’ policy solutions.

In other words, the word ‘power’ is often hidden because the most profound forms of power often seem to be hidden.

In the context of our discussion, power comes from the ability to define some evidence as essential and other evidence as low quality or irrelevant, and therefore define some people as essential or irrelevant. It comes from defining some issues as exciting and worthy of our attention, or humdrum, specialist and only relevant to experts. It is about the subtle, unseen, and sometimes thoughtless ways in which we exercise power to harness people’s existing beliefs and dominate their attention as much as the transparent ways in which we mobilise resources to publicise issues. Therefore, to ‘maximise the use of evidence’ sounds like an innocuous collective endeavour, but it is a highly political and often hidden use of power.

See also:

I discussed these issues at a storytelling workshop organised by the OSF:

See also:

Policy in 500 Words: Power and Knowledge

The politics of evidence-based policymaking

Palgrave Communications: The politics of evidence-based policymaking

Using evidence to influence policy: Oxfam’s experience

The UK government’s imaginative use of evidence to make policy

 

New forms of politics and policymaking in Scotland: participatory and deliberative democracy resurgent? #POLU9SP

When Neil McGarvey and I wrote the first edition of Scottish Politics, we devoted a full chapter (11) to definitions of democracy and post-1999 developments in Scottish democracy. In the second edition, we removed that chapter because there was nothing new to say.

Although the principles behind phrases such as ‘deliberative’ and ‘participative’ democracy are important, I felt that they often represented empty rhetoric in Scotland. In Scotland, we talked a lot about doing things differently, but produced political practices that would not seem out of place in Westminster.

Then came the debate on Scottish independence: the 85% turnout to vote Yes or No, the sense that Scotland was now – or again? – a hub for deliberation and high participation, and a new Scottish Government agenda on translating that participation into something longer term, to produce examples of participatory policymaking.

So, maybe we will have a chapter on participation in the 3^rd edition. I doubt it. I reckon Scottish politics will soon revert to its old self.

I base this expectation on a logical argument about the relationship between representation and other forms of democracy, the previously too-high expectations for new Scottish politics, and a sense of unfulfilled expectations in the Scottish experience since devolution.

What can compete with representative (and pluralist) democracy?

In Scottish Politics (pp220-4), we argue that ‘new politics’ involves reforms to four kinds of democracy:

Representative democracy

This will remain the most important democratic mechanism because ‘direct democracy at a national level would involve too many participants with too little time to devote to politics, and insufficient knowledge to apply to the wide range of responsibilities of the modern state. The main alternative is representative or indirect democracy in which popular sovereignty is expressed through regular elections of representatives acting on their behalf’.

We lose sight of its importance when thinking about other forms of democracy as somehow replacing representative. Instead, we should treat any other activity as a bonus, and consider how it fits in to the bigger picture.

In this category, the most important developments relate to:

(1) the more proportional electoral system – mixed member proportional (MMP) – for the Scottish Parliament (previous lecture)

(2) successful attempts by some parties to make sure that MSPs better represent the social background of the population than their MP counterparts. I say ‘some parties’ but note from Scottish Politics – and these articles by Keating & Cairney, and Cairney, Keating & Wilson – that only Scottish Labour achieved gender parity among their MSPs (these issues are explored in greater depth by scholars such as Kenny and MacKay). In other ways, the experience is more mixed. The Scottish Parliament is often no better at ‘microcosmic’ representation in other key categories – such as ethnicity, disability, and sexuality – and it has its own version of a ‘political class’ divorced from the ‘real world’ outside of politics.

Pluralist Democracy

This category is the most relevant to the course, since it relates to the pervasiveness of ‘policy communities’ or networks in policymaking. Elections produce mass participation every 4 or 5 years, while policymaking and delivery happens all the time. Policymakers and ‘pressure participants’ such as interest groups form relationships, build up trust, and engage regularly with each other. They discuss the principles and details of policies in a way that few actors can do without the specialist knowledge and time to engage.

In this category, the most important developments relate to:

(1) a vague sentiment, expressed by the SCC, to avoid speaking to the ‘usual suspects’ and, instead, to broaden consultation to many participants (an aim explored in the literature on the ‘Scottish policy style’). This sentiment was re-stated by the SNP government in 2007 as an attempt to consult beyond the ‘establishment’. In both cases, it is a soundbite that ignores the logic of consulting with the most interested, expert, and active groups (a point explored well by Jordan and Stevenson, which Neil and I discuss in chapter 11).

(2) A role for Scottish Parliament committees to oversee the quality of Scottish Government consultation, and take action – in relation to draft legislation – if it is not satisfied. I think that 2 committees have used this power in 16 years, but what does this statistic signify: that committees don’t have many powers in practice, or that the Scottish Government consults so well that the power only needs to be used rarely?

Participatory and deliberative democracy

For me, initiatives to increase participatory and deliberative democracy should be seen in this initial context of representative and pluralist democracy. Why start with a focus on initially exciting, but ultimately disappointing, new forms of participation without first stating that most participation is through elections and most deliberation in networks?

The SCC got the ball-of-hype rolling by declaring that Scotland has, ‘consistently declared through the ballot box the wish for an approach to public policy which accords more closely with its collective and community traditions’. We can talk in the lecture about how well substantiated such claims are.

Perhaps more importantly, the SCC did not design the details of devolution. So, while we may wonder what it might have come up with, we can say with greater certainty that the most important developments relate to:

(1) A Scottish Civic Forum to act as a new forum for participation and deliberation among a self-selecting public. It ran from 1999-2006. Well-established interest groups described it as a ‘talking shop’ and did not use it (they had their own access to government), but some community groups and individuals did. Yet, it was difficult to get many people involved (particularly in regional events) and even more difficult to feed its reports into government and parliamentary processes.

(2) A petitions process coordinated by the Scottish Parliament. See Chris Carman’s report and journal article on this. My take is that, although the petitions process gives people a way to engage in politics and, in a very small number of cases, set the parliament’s or government’s agenda on one or two issues, it is peripheral to the policy process.

(3) A wider sense that a powerful Scottish Parliament would become the hub for such initiatives and, through regular debate, become the main arena for public deliberation (see next lecture, but don’t get your hopes up).

Did we revisit those expectations during the debate on Scottish independence?

No, not really. Instead, we saw some interesting attempts to generate interest in participatory and deliberative forms of democracy beyond government and parliament – see ERS Scotland’s Democracy Max, and my take on it – but without meaningful ‘ownership’ of those initiatives by government or parliament. For example, have a look at the Scottish Government’s White Paper on Scottish independence and see what you can spot.

For me, the most telling part of this limited debate was the reference, by advocates of reform, to phrases like ‘politics is broken’ in 2014 when the same phrases were used to push forward political reforms in Scotland from 1999. This sense of déjà vu gives you an indicator of how far the ‘new Scottish politics’ has changed practices in Scotland.

What are the prospects for new forms of participation?

For me, deliberative and participatory politics represents the personal equivalent of things that I would like to do if I had the time. I have a list of colleagues that I’d like to work with more, and I have a vague desire to play a round of golf before I die. At the same time, in most cases, I know that I will never have the time. Indeed, when you care about something enough, you make the time.

To put it another way, when you don’t have the time or enthusiasm for something, it might be better just to stop pretending that a vague aim might pan out one day. It just gets in the way of a more meaningful discussion of what’s actually happening while we pretend that loads of people are participating in new ways of doing politics in Scotland.

Other opinions are available

There are people who know more about these things than me, or are younger and more optimistic about life. For example, Oliver Escobar is doing work on this topic as part of What Works Scotland, and his PhD is here. You might also bump into Peter Matthews or Vikki McCall on the other side of campus and ask them what they are up to.