Monthly Archives: August 2015

The barriers to evidence based policymaking in environmental policy

Kathryn Oliver and colleagues examined 145 studies of the barriers to the use of evidence in health policy. They identified a range of problems, including a gap between ‘supply and demand’, the inability of scientists to provide evidence in a timely and strategic way, and a tendency for policymakers to use short cuts (including personal experience and an ad hoc reliance on experts) rather than conduct systematic searches for policy relevant evidence.

In my work with Adam Wellstead, I am beginning to identify the same findings in environmental policy. Multiple studies suggest that:

  • Current evidence on the nature of environmental problems, or the effectiveness of policy solutions, is incredibly patchy
  • The evidence is not ‘packaged’ well (easy to understand, ‘framed’ in a way that is attractive to policymakers, and/ or accompanied by realistic expectations for policy change)
  • Scientists do not engage well with policymakers, either in networks, academic-practitioner forums, or by using intermediaries or ‘knowledge brokers’
  • Broad differences in academic-policymaking cultures undermine the ability of scientists to engage in politics in a timely manner, or in a way that will maximise the impact of their findings
  • Scientists need to adapt to the vagaries of policymaking, or a tendency for policymakers to: address short term issues rather than plan for the long term; rely on personal experience and limited expert advice; misjudge the risks associated with environmental problems; seek simple, easy to understand, stories rather than the results from sophisticated models; and, use science selectively, often to give a gloss of objectivity to their policy choices.

In other words, most policymakers would make hopeless scientists, and most scientists would be rubbish politicians.

This sort of analysis is interesting enough, and tells us something about the ‘science-policy interface’. Yet, it tends not to be well informed by policy theory or political science. This matters because one might look at the list of barriers and conclude that, if we can overcome them, we can make a big difference to the use of evidence in policy.

That would be a mistake.

Yes, it would be good for policymakers to understand the science a bit more, and for scientists to provide punchy one-page reports, but it is also possible that neither exercise would make much difference. Instead, to understand the profound ‘barriers’ to the use of scientific evidence, we need more scientific knowledge about the policy process itself. This is what most of the environmental literature fails to appreciate, and this lack of knowledge produces a tendency for authors to recommend the same things each time – such as more academic-practitioner workshops to identify barriers – without generating a sense of progress. Such studies should also consider, for example, the huge investment of time and energy that other actors invest, to learn how policymakers think about policy problems, and how to form coalitions with other powerful actors. A glossy report passed on by a ‘knowledge broker’, or a few afternoons in workshops looks paltry in comparison.

In other words, most of this literature does not ask the questions – for example, what is the dominant ‘framing’ of the policy problem; who are the most significant winners and losers with regard to the outcomes of policies; and, what is the effect of multi-level governing arrangements on the use of evidence? – that political scientists would take for granted, and the analysis suffers as a result.

I, and we, are in the process of trying to fill some of those gaps – to use the science of evidence based policy making to inform the scientists who may want to be involved in evidence based policy making. There are more details here – – and the draft chapter on environmental policymaking is here: Cairney Palgrave Pivot CHAPTER 4 environment 25.8.15

Leave a comment

Filed under Evidence Based Policymaking (EBPM), public policy

Barriers/ solutions to the use of evidence in policy (environmental science)

Some examples of the stated ‘barriers’ to the use of evidence in policy, from the environmental science/ policy literature (to match those found in health by Oliver et al, 2014). See here for more details.

Improve the supply of, and/or generate demand for, scientific evidence
(a) the evidence on the problem is not good enough
Until the 1990s, polices to influence climate change behaviour were hindered by the lack of evidence-based indicators of the effect of consumption on the environment. From the 1990s, scientists developed good monitoring tools, but few policymakers became aware of them (Barrett et al, 2005: 38).
Poor policy decisions are often underpinned by poor evidence on ecological history (Hamilton and Penny, 2015)
EU attempts to monitor and advise on the environmental impacts of agricultural practices are hampered by insufficient data (Louwagie et al, 2012: 149-50)
New models should be developed address the paucity of data underpinning policy on climactic vulnerability and adaptation (Malcolm et al, 2014)
The lack of local-area-specific knowledge undermines the effectiveness of otherwise evidence-based land management policies (Molnár, 2014).
Environmental scientists should from evidence based medicine, to producing a database of systematic reviews and policy-relevant synopses (Dicks et al, 2014: 119; Carneiro and da-Silva-Rosa, 2011: 3; Cvitanovic et al, 2013; Cvitanovic et al, 2014; Webb et al, 2012: 203).
(b) the evidence on the solution is not good enough
There is a lack of comprehensive databases of systematic reviews on biodiversity policy. Existing work is presented in a language that is too technical or politically naïve for busy public managers to take on board, and many studies do not provide a clear answer to pressing policy questions (Carneiro and Danton, 2011).
The scientific evidence base on climate change policy interventions is ‘surprisingly weak for such a high profile area’. ‘There is too little systematic climate policy evaluation work in the EU to support systematic evidence- based policy making’ (Haug et al, 2010: 427).
Current performance management practices do not allow us to evaluate the effectiveness of conservation programmes, because organisations measure what is easy to measure (Rissman and Smail, 2015).
(c) the evidence needs to be ‘packaged’ well (easy to understand, framed in a way that is attractive to policymakers, and/ or accompanied by realistic expectations for policy change)
Academics should repackage their work according to the needs of their ‘end user’ – such as by providing pragmatic recommendations or information that helps them predict events and plan ahead – and use ‘knowledge brokers’ (Cvitanovic et al, 2013: 85; Cvitanovic et al, 2014: 35-6).
Policymakers will often not respond to an alleged policy problem if there is not an obvious solution (Lalor and Hickey, 2014: 10-12)
The rise of sophisticated policy assessment tools (such as models) is caused more by technological advance than a demand for information in this form. Simple qualitative stories ‘backed up by illustrative statistics … appeared largely driven by the need to present easily digestible analysis to the decision maker if one wanted the assessment to be instrumentally useful’ (Nilsson et al, 2008: 348).
The carbon capture and storage (CCS) community has a coherent and uniform message for policymakers, which may help explain its major funding successes in the EU (Stephens et al, 2011: 388)
Government reports provide vast amounts of evidence but their links to effective policy are weak, partly because the reports come with unrealistic shopping lists for action (Wellstead and Stedman, 2014: 1000).
Scientists struggle to translate knowledge and concepts about risk to policymakers, stakeholders and the public (Yuen et al, 2013).
(d) engage in networks and academic-practitioner workshops
There is high participant demand to identify best practice in academic-practitioner exchange (or at least to find quick/ easy solutions to gulfs in their relationships), and a belief that regular interaction helps build up trust or ‘social capital’ (Hickey et al, 2013: 539).
To adapt to complex policymaking systems, scientists need to engage in collaborative/ participatory government rather than feed in evidence to the centre (Lalor and Hickey, 2014).
(e) use intermediaries
There is a need for ‘hybrid people’ but an absence of unanimous ‘upper management’ support (in public bodies in Canada and Australia) for ‘knowledge brokers’ (Hickey et al, 2013: 534).
Timing and opportunity
Policymakers value timely and responsive research, but scientists face big time lags in publication (Cvitanovic et al, 2014: 38)
‘Relationships of trust and establishment of expert credibility matter greatly in the acceptance of knowledge claims’ (in international climate change treaty negotiations) (Rowe, 2013: 221)
Despite a new agenda on timely and policy-relevant research (on dryland policies in Africa) the evidence remains ‘sparse’ (Stringer and Dougill, 2013: 328).
Encourage policymaker skills or better government understanding of problems
Governments tend to deal with environmental crises rather than plan for the long term. A lack of government commitment to collecting policy-relevant data produces often undetected policy failures (Clare and Creed, 2014: 243)
Policymakers rely on personal experience and expert advice, not systematic searches of the literature (Carneiro and da-Silva-Rosa, 2011: 1; Cvitanovic, 2013: 85)
Many policy managers do not prioritise scientific evidence and are unaware of advances in adaptation science. Policymakers often have poor knowledge of environmental risks, and their priorities often do not reflect the best evidence (Cvitanovic et al, 2014: 38)
Ministers do not understand the data from the sophisticated policy assessment tools that they, ‘have been so keen to advocate and nurture’ (Nilsson et al, 2008: 350). Rickards et al (2014: 654) provide similar conclusions on scenario planning. As in the nomenclature on evidence-informed policy, they identify ‘scenario methods’ or ‘scenario thinking’
UK government ministers appear unwilling or unable to engage in the systematic review of the evidence on business regulation (Taylor et al, 2013).

1 Comment

Filed under Evidence Based Policymaking (EBPM)

These complaints about ignoring science seem biased and naïve – and too easy to dismiss

This is one of two posts on the use of scientific evidence in environmental policy (a broad term which can include climate change, food, land management, and energy policy). In this one I insult some of the people reacting to the Scottish Government’s decision on GM. In the next one, I present preliminary evidence on a systematic review of the use of evidence in environmental policy.

The Scottish Government recently decided to reinforce a moratorium on the use of GM crops in Scotland. It produced a strong response from many individual scientists, and groups representing scientists, which seems predictable enough. That’s what you would expect some scientists to do if you don’t act solely on their advice.

What I didn’t quite expect was a more general, and often hyperbolic, criticism by non-scientist commentators about the failings of policymakers to listen to scientists.

The general problem for non-scientist commentators is that they don’t really know what ‘the evidence’ is. It’s no great insult to predict that most commentators decrying a lack of reliance on ‘the evidence’ have never read any relevant scientific journal article, systematic review, or peer reviewed report on the science in question. In fact, that’s fine – you probably need some training to understand the technical aspects, particularly on method.

What is not fine is a tendency to accept the word of scientists when they tell you that they know the evidence. In general, they don’t. ‘The evidence’ is often remarkably patchy (a point I’ll reinforce in the next post) and highly-disputed within professions, and canny scientific groups or individuals fill those gaps with an appeal to expert-based authority. This practice should be subject to as much critical analysis and scepticism that we afford to politicians. In fact, if you are a good scientist, you welcome the challenge and the debate (with the exception of cases, such as climate change, in which debate may undermine an urgent need to act).

The general problem for the more naïve scientist commentators is that they often read the scientific reports, take a view on what they mean and what should be done, and simply expect policymakers to do the same. They fail to appreciate how scientific evidence fits into the broader picture of politics and policymaking. They fail to articulate what ‘evidence based policy making’ really means, or should mean in practice, when we have equally or more important ways to make decisions – such as representative government (we elect people to state their beliefs and make decisions on our behalf) and participatory/ deliberative democracy (we seek ways to generate a range of perspectives on a policy problem). In effect, they make an appeal to a process in which scientific knowledge or expert beliefs should be privileged above the beliefs of other people (in this case, it is accompanied by a pretentious reference to the Scottish Enlightenment).

In my view*, both of these approaches are counterproductive. The hyperbolic non-scientist criticisms are too easy to dismiss. As soon as you start equating policy decisions with, for example, witch trials (OK – I’ve only seen two people do it), you can be safely ignored as someone with an axe to grind. The narrow scientist criticisms are just as badly off. As soon as you appeal to ‘the evidence’ without showing any appreciation of the policy process or a wider appreciation of policy decisions, you can be safely dismissed as naïve or self-interested.

Update: maybe more than 2 –

Further reading:

For a pragmatic scientific account see:

Lawton, J. H. (2007) ‘Ecology, politics and policy’, Journal of Applied Ecology, 44,3, 465–74

*These views are my own, they are stated a bit strongly, apologies to Duncan and Euan, etc.


Filed under Evidence Based Policymaking (EBPM), public policy, Scottish politics

Why do people seem so down on e-cigs?

There is a useful summary of e-cigarettes here

Paul Cairney: Politics & Public Policy

It must be very frustrating to give up smoking, with the help of e-cigarettes, only to find that you are no more welcome in public places with a fake cigarette than a real one. UK governments, and many public health advocates, often seem to want to regulate them in the same way, even though the e-cig could be described as a crucial ‘harm reduction’ measure (it’s not exactly healthy, but it’s much better than the other thing you were doing).

Here is a list of historical explanations for this position which won’t make you happy, but can at least distract you while you’re having a sly puff in the toilets:

  1. We’ve been here before with tobacco and harm reduction. So many post-war examples – like the idea of smoking a pipe, putting filter tips on cigarettes, ‘low tar’ cigarettes (which is a bit like ‘less shite in your…

View original post 657 more words

Leave a comment

Filed under Uncategorized

Two first drafts of the SNP 2016 manifesto

People have begun to speculate about how the SNP will incorporate a second referendum into its 2016 Scottish Parliament manifesto. So, I’ve had a go at drafting two different versions.

The first one is easy to write but won’t happen.

The second, more realistic one, is harder to write. In particular, it will be hard to get the wording right when the SNP (in effect) promises a referendum in the next manifesto but not this one.

So, we might expect a lot of coverage of the opposition parties trying to pin down Nicola Sturgeon on the detail. Ordinarily this might be sensible, but not while the SNP remains almost immune to that kind of criticism. Maybe it would be better to focus on what the SNP will do, instead, in the next 5 years. Who knows?

The one that won’t be written:

Nicola Sturgeon stated in 2015 that: “Something material would have to change in terms of the circumstances or public opinion before I think it would be appropriate to have a proposal for a referendum.”

Since we gubbed Labour in the 2015 UK General Election and it looks like we’ll do the same in the Scottish Parliament, we believe that material circumstances have changed and we will have a referendum as soon as possible. Loads of Yes supporters joined the SNP straight after the referendum, and they think we were robbed and that loads of people regret voting No. There was also that poll which suggests that we’re hugely popular even though people don’t think we’re doing a brilliant job, so independence is all we want to talk about.

The more likely one that is longer and trickier to write, and doesn’t quite scan:

Nicola Sturgeon stated in 2015 that: “Something material would have to change in terms of the circumstances or public opinion before I think it would be appropriate to have a proposal for a referendum.”

In the period of the next Scottish Parliament, the biggest potential change of circumstances regards Scotland’s membership of the European Union. Nicola Sturgeon has called on David Cameron to ensure that the UK leaves the EU only if all four nations agree. He has rejected this call. If the UK prepares to leave the EU, despite a majority of voters in Scotland voting to remain, it will cause a constitutional crisis. Under those circumstances, we believe that we should hold a referendum on Scottish independence as soon as practicable (this paragraph seems relatively easy to write).

We are heartened by the surge of support in our membership, to over 100,000 members, and in electoral support for the SNP, which produced 56 of 59 SNP MPs. However, this alone does not represent a change of public opinion towards Scottish independence. It does not guarantee that the majority of voters in Scotland would vote Yes in a second referendum on Scottish independence (this one is tricky to get right, and would be used by opponents).

Instead, the electorate also supports us because we present a strong vision to deal with austerity and support essential public services. They do not support the austerity agenda of the Conservative party, and nor do they find Labour to be a credible alternative (bread and butter stuff).

We believe that, in time, support for our positive vision, and a growing belief that the UK parties’ austerity agenda is damaging Scotland’s economy and widening inequalities, will help produce a clear majority in favour of Scottish independence (also bread and butter).

If that change of public opinion becomes clear over the next five years, we will include a firm commitment to a second independence referendum in our manifesto for the 2021 Scottish Parliament election (no, I can’t get that one to work!).



Filed under Scottish independence, Scottish politics

What can be done about the UK’s ‘glass floor’?

New research on the ‘glass floor’ presents a striking way to understand socioeconomic inequality in the UK. It also highlights ever-present problems in translating such information into policy: we understand the size of the problem well, speculate on its cause badly, and produce vague calls for government action ineffectively. Our initial shock and enthusiasm for policy change translates into disenchantment with yet another ‘too difficult’ problem.

The UK Government’s Social Mobility and Child Poverty Commission has released new research on the life chances of the British population. It identifies a “’glass floor’ in British society” to reject the idea that people get on in life through hard work and merit. Instead, mediocre and lazy children in the right family will do better than bright and hardworking children in the wrong family.

This is horrible paraphrasing of the report, but you get the idea about how most people might notice the report in a hurry, have their beliefs about the lack of a British meritocracy reinforced, then complain that the government is doing enough about it. There wasn’t quite a public outcry (far from it), but you might be forgiven for thinking that the report gives the government plenty of reason to do something. The big question is: will it do anything new with the information?

I wouldn’t rule it out, but would exercise this note of caution: reports like this don’t speak for themselves or give governments a clear impetus to act. Instead, they form part of a larger pattern in this area (of socio-economic inequalities policy), in which we can speak with much more certainty about the size of the problem than (a) its cause, (b) how we should respond, and (c) who exactly should respond.

The size of the problem

The size of the problem is quantified well (it’s not a simple task to measure cognitive ability, class backgrounds and life chances like this) and easy to understand. For example, the commission’s press release states that:

‘Less able, better-off kids are 35% more likely to become high earners than bright poor kids … children from more advantaged social backgrounds who are assessed at age 5 as having low cognitive ability are nonetheless significantly more likely to become high earners than their high ability peers in lower income households. Children from high income backgrounds who show signs of low academic ability at age 5 are 35% more likely to be high earners as adults than children from poorer families who show early signs of high ability’.

The cause of the problem

This is when things get a bit trickier, because although the chair of the commission, Alan Milburn, describes ‘a social scandal that all too often demography is still destiny in Britain’, the commission is not entirely clear on who or what caused it. There is not one simple message about a single villain. Instead, there are at least two, and both stories are not crystal clear.

First, the author of the report, Dr Abigail McKnight, links the outcomes to the behaviour of certain parents:

“The fact that middle class families are successful in hoarding the best opportunities in the education system and in the labour market is a real barrier to the upward social mobility of less advantaged children.”

The keyword there is ‘hoarding’, which suggests inappropriately selfish behaviour. Yet, the chair of the commission, Alan Milburn, is keen not to blame parents: ‘No one should criticise parents for doing their best for their children. That’s what we all want’.

Instead, Milburn sort of blames the government for its current lack of proportionate action: ‘The government should make its core mission the levelling of the playing field so that every child in the country has an equal opportunity to go as far as their abilities can take them’.

The result is a mixed view about the cause of the problem – perhaps it’s the fault of some hoarding parents (the especially rich ones sending their kids to private schools, getting tutors and securing internships for their children) and not so much others (the ones using their own skills to secure a spot for their child in a good state school) – and maybe the solution is to give other parents some of these skills to ‘level the playing field’ a bit.

The realistic solution

This is when things get even trickier, because the report seems to call for the government to do far more than it will, while giving it the ability to say that it is already doing as much as it should.

In the ‘far more than it will’ column is the call to reduce socio-economic inequalities (through wealth and income redistribution?), remove differences in quality between schools, and remove class-based barriers to University admissions.

In the ‘sort of doing it already’ column is the call for the state to intervene early in people’s lives to, in effect, train disadvantaged parents in how to give their children things like ‘soft skills’ related to forming networks and spotting opportunities.

The ultimate complication

The final, and perhaps trickiest, obstacle is about working out who is in charge of taking the next step, to drive this new policy agenda forward. The final paragraph of the main report is instructive:

‘If politicians are serious about their expressed desire to increase social mobility in the UK they will need to address barriers that are preventing less advantaged children from reaching their full potential and remove barriers that block downward mobility’.

It doesn’t say who the politicians are – perhaps for good reason. In areas such as social and economic inequality, it is increasingly difficult to know who is responsible for policy progress. If it’s mainly about economic redistribution, you can call for action from central government – but, let’s be honest, this won’t get you very far. If it’s mainly about training and encouraging ‘soft skills’ like ‘resilience’, central government might produce a broad strategy document, but its localism agenda suggests that it expects local public bodies to take responsibility for social outcomes.

The overall message is that it takes us seconds to understand the problem and call for government action, but a lot longer to decide what we want them to do, and longer still to find the people likely to do it. By that time, our attention will probably have shifted elsewhere, until the next report comes out and we do it all over again. Maybe this time will be different.





Filed under agenda setting, public policy, Social change, UK politics and policy

In UK and Scottish politics, should you assume that people are stupid?

Political commentators often make fun of other political commentators when they complain that the public is stupid. Yet, maybe we all do something similar – assume that most people make quick, emotional and habitual decisions to turn a complex world into a series of simple actions. In that sense, the ‘realistic’ political campaigns (and some policies) favoured by such commentators may be based just as much on the ‘stupidity’ of the target audience.

In Scottish politics, there is a lot of fun to be had (if you like that sort of thing) while taking the piss out of Yes supporters who argue that No supporters were manipulated into their decision. For example, if only No supporters could see through media and partisan manipulation they’d come to a different conclusion. The argument here is that such Yes supporters are implying that temporary No supporters are stupid, since they are more open to manipulation than their critics. Only Yes supporters can see through the lies.

In UK politics, the same fun can be had (if you like that sort of thing) with certain Labour supporters who can’t quite believe that they lost the election, or blame it on people who support Labour values in public but vote Conservative in private. There are also some offshoot debates, often led by John Rentoul, about the idea that the ‘wrong people were voting Labour’ under Blair or that current supporters of Jeremy Corbyn deny the truth about why Labour lost the UK General Election. The debate is not quite the same, but you can detect a similar suggestion that left-wing Labour supporters blame current Conservative supporters for not seeing through the cynicism of Tory campaigns.

In both cases, the proposed solution may be a cold dose of realism about the beliefs of the British public: instead of complaining that too many people are naïve or stupid and hold the wrong beliefs (or are ‘delusional’), try to work with their beliefs to produce a campaign that works for them.

You can see why this approach would present some fairly heated debates, since the counterargument is that you should not accept public beliefs when you consider them repugnant (such as in relation to migration) or dangerously misguided (such as in relation to the causes of the financial crisis or the ways in which parties justify austerity measures, often in relation to the misleading analogy of balancing the household books, which exacerbate socio-economic inequalities).

In time, this pragmatic argument may underpin the second referendum on Scottish independence (who knows?).

In the meantime, it is at the heart of the Labour leadership, in which the heroic/left-wing/ ideologue Jeremy Corbyn is pitted against the three more cynical/ pragmatic/ realistic candidates who want to compete with the Conservatives in part by engaging with their arguments rather than dismissing them as stupid and cynical.

What should we do when people are ‘stupid’

In each case, there are two constants that we should always bear in mind:

  1. Ideas aren’t stupid. Many people focus too much on the stupidity of arguments or ideas when they should really identify the tendency for people to believe them. I don’t want to get into a big philosophical thing about this, but ideas aren’t simply good/ bad and they don’t spread on their own – they need some people to propose them and others to accept them. Your choice is a mix of (a) giving credit to the people who know how to get good ideas going; and/ or (b) assigning blame to the gullible fools that accept them.
  2. People aren’t stupid, but they are open to manipulation. People have a tendency to accept simple stories that chime with their beliefs (and other stuff – Daniel Kahneman’s Thinking Fast and Slow sums up a lot of this kind of argument, but see here and here if you want something more free but less good).

All that remains is to decide what to do about it:

(a) do you start with the assumption that people inevitably have cognitive biases that can be manipulated (particularly when they make snap decisions), to get them to do what you want;


(b) do you assume that people can overcome many cognitive biases, try to educate them, or otherwise help them to think more carefully about issues to make well-considered intelligent decisions?

I don’t know the answer to that question ….

All I’ll point out is that I have (hopefully) reframed the initial premise of this post. Now, it’s the fun-makers and realists who think that people are stupid (the a people). Now, our original villains are really the heroes who have faith in the public to think harder and make the right decision next time (the b people).

except to say that sneering doesn’t help.

A final thought on the strategies of realists is that many of them sneer a lot at the people they think are naïve. This doesn’t just seem rude and annoying – it’s also counterproductive, because politics is often as much about process as outcome. If everyone enters and leaves debates in a respectful way, the losers may be content with a poor outcome. If the winners spend their time sneering at their opponents, and Lording their victories over the losers, it just exposes the very divisions they claim to want to prevent.

1 Comment

Filed under Scottish independence, Scottish politics, UK politics and policy