When are scientists neutral experts or strategic policy makers?

Great post by Professor Karin Ingold on the potential for different roles for scientists in evidence informed policymaking

Integration and Implementation Insights

Community member post by Karin Ingold

karin-ingold Karin Ingold (biography)

What roles can science and scientific experts adopt in policymaking? One way of examining this is through the Advocacy Coalition Framework (Sabatier and Jenkins-Smith 1993). This framework highlights that policymaking and the negotiations regarding a political issue—such as reform of the health system, or the introduction of an energy tax on fossil fuels—is dominated by advocacy coalitions in opposition. Advocacy coalitions are groups of actors sharing the same opinion about how a policy should be designed and implemented. Each coalition has its own beliefs and ideologies and each wants to see its preferences translated into policies.

I build on the work of Weible and colleagues (2010), who distinguish three types of ‘subsystems’ in which policy is made:

  • a collaborative subsystem, where at least two coalitions exist; they have different opinions, but want to overcome them
  • an adversarial subsystem, where…

View original post 526 more words

Leave a comment

Filed under Uncategorized

Evaluation and Governing in the 21st Century

Deirdre Niamh Duffy (2017) Evaluation and Governing in the 21st Century (Palgrave Pivot)

Duffy’s new book engages with the many uses and abuses of evaluation in UK politics. This variation in evaluation practices relates partly to the vagueness of the term. Put simply, evaluation is about measuring the success of public policies. However, such a simple definition is so broad that it can mean everything and therefore nothing.

In that context, Duffy compares the many ways in which people could, do, and should use evaluation to inform policy and policymaking.

In terms of what evaluation could mean, Duffy identifies common definitions of evaluation as:

  • ‘assessment of value/ merit’, involving some sort of combination of the values of the people involved and the methods they use to gather evidence of success
  • “a realistic(ic) ‘science’”, focusing primarily on the allegedly appropriate use of scientific methods to measure success
  • ‘actionable science’, using measures of success to help improve policy.

In terms of what UK governments actually do, Duffy identifies key phases including:

  • A pre-New Labour reluctance to allow outside actors to opine on the success of its policies, combined with a relative lack of sophisticated methodological tools to do so.
  • A New Labour era (from 1997), in which ministers were keen to stress a reliance on evidence based policymaking, to focus on ‘what works’ when identifying promising policies and evaluating their success primarily with reference to technical scientific measures rather than, say, their ideological positions.

One can infer from Duffy’s analysis that there is something to be said for the relative honesty of the pre-Labour position in which governments pretty much told people where to go, and staked their claim as the arbiters of their own success.

In contrast, Labour had a tendency to use the language of evidence to try to depoliticise issues, when there should have been more debate on values and the rationale for policies. It also used this approach to put pressure on delivery organisations – and, by extension, the recipients of policy measures – to do what it wanted. In particular, Duffy focuses on evaluation as the setting of benchmarks, and use of league tables based on proxies of success, to put major pressure on the organisation not doing so well. In part, the government is able to do so by encouraging the fear and shame of the actors leading or working for organisations delivering public services. In this context, ‘evaluation becomes less about EBPM and more about influencing and manipulating behaviour’ (p147).

In terms of what a government should do, Duffy focuses on the potential for evaluation practices to produce positive transformations in policy and practice: ‘evaluation can be reclaimed as part of a transformative project using critical theory’ (p148). Since such transformation would be encouraged via open discussion without a fixed agenda, and without a focus on one best way to use methods to evaluate, it must remain a very general aspiration. Consequently,

For some – particularly those seeking an instrumental guidebook on how to do ‘good evaluation’ – this may seem highly problematic. However, from a critical sociological perspective, it is only through remaining open to potential, as yet unknown emergent transformations that the disciplinary and controlling governing effects of knowledge production processes can be unsettled.

As such, Duffy’s book stands out as a critical theoretical take on the role of evidence in policy making and evaluation. I commend to people who want to broaden their horizons.

duffy

 

 

Leave a comment

Filed under Evidence Based Policymaking (EBPM)

Three habits of successful policy entrepreneurs

Policy and Politics Journal

Paul Cairney Paul Cairney

The ‘multiple streams approach’ (MSA) is one of policy scholarship’s biggest successes. Kingdon’s original is one the highest cited books in policy studies, and there is a thriving programme of empirical application and theoretical refinement.

Yet, I argue that its success is built on shaky foundations because its alleged strength – its flexible metaphor of streams and windows of opportunity – is actually its weakness. Most scholars describe MSA superficially, fail to articulate the meaning of its metaphor, do not engage with state of the art developments, and struggle to apply its concepts systematically to empirical research. These limitations create an acute scientific problem: most scholars apply MSA without connecting it to a coherent research agenda.

In my recent article in Policy & Politics, I seek to solve this problem in three ways.

View original post 390 more words

Leave a comment

Filed under Uncategorized

Introducing our 2018 Policy & Politics special issue on Practical Lessons from Policy Theories

Policy and Politics Journal

Christopher M. Weible and Paul Cairney

Introducing our 2018 Policy & Politics special issue on Practical Lessons from Policy Theories, published in April now available online and in print. (Free to access online until 31 May)

Professors Christopher. M. Weible from the University of Colorado, Denver and Paul Cairney from the University of Stirling talk in the video below about their motivation for producing a special issue on drawing practical lessons from policy theories, and why their subject is so important.

View original post 96 more words

Leave a comment

Filed under Uncategorized

Practical Lessons from Policy Theories

Paul Cairney: Politics & Public Policy

These links to blog posts (the underlined headings) and tweets (with links to their full article) describe a new special issue of Policy and Politics, published in April 2018 and free to access until the end of May.

Weible Cairney abstract

Three habits of successful policy entrepreneurs

View original post 576 more words

Leave a comment

Filed under Uncategorized

Evidence-based policymaking: political strategies for scientists living in the real world

Note: I wrote the following discussion (last year) to be a Nature Comment but it was not to be!

Nature articles on evidence-based policymaking often present what scientists would like to see: rules to minimise bias caused by the cognitive limits of policymakers, and a simple policy process in which we know how and when to present the best evidence.[1]  What if neither requirement is ever met? Scientists will despair of policymaking while their competitors engage pragmatically and more effectively.[2]

Alternatively, if scientists learned from successful interest groups, or by using insights from policy studies, they could develop three ‘take home messages’: understand and engage with policymaking in the real world; learn how and when evidence ‘wins the day’; and, decide how far you should go to maximise the use of scientific evidence. Political science helps explain this process[3], and new systematic and thematic reviews add new insights.[4] [5] [6] [7]

Understand and engage with policymaking in the real world

Scientists are drawn to the ‘policy cycle’, because it offers a simple – but misleading – model for engagement with policymaking.[3] It identifies a core group of policymakers at the ‘centre’ of government, perhaps giving the impression that scientists should identify the correct ‘stages’ in which to engage (such as ‘agenda setting’ and ‘policy formulation’) to ensure the best use of evidence at the point of authoritative choice. This is certainly the image generated most frequently by health and environmental scientists when they seek insights from policy studies.[8]

Yet, this model does not describe reality. Many policymakers, in many levels and types of government, adopt and implement many measures at different times. For simplicity, we call the result ‘policy’ but almost no modern policy theory retains the linear policy cycle concept. In fact, it is more common to describe counterintuitive processes in which, for example, by the time policymaker attention rises to a policy problem at the ‘agenda setting’ stage, it is too late to formulate a solution. Instead, ‘policy entrepreneurs’ develop technically and politically feasible solutions then wait for attention to rise and for policymakers to have the motive and opportunity to act.[9]

Experienced government science advisors recognise this inability of the policy cycle image to describe real world policymaking. For example, Sir Peter Gluckman presents an amended version of this model, in which there are many interacting cycles in a kaleidoscope of activity, defying attempts to produce simple flow charts or decision trees. He describes the ‘art and craft’ of policy engagement, using simple heuristics to deal with a complex and ‘messy’ policy system.[10]

Policy studies help us identify two such heuristics or simple strategies.

First, respond to policymaker psychology by adapting to the short cuts they use to gather enough information quickly: ‘rational’, via trusted sources of oral and written evidence, and ‘irrational’, via their beliefs, emotions, and habits. Policy theories describe many interest group or ‘advocacy coalition’ strategies, including a tendency to combine evidence with emotional appeals, romanticise their own cause and demonise their opponents, or tell simple emotional stories with a hero and moral to exploit the biases of their audience.[11]

Second, adapt to complex ‘policy environments’ including: many policymakers at many levels and types of government, each with their own rules of evidence gathering, network formation, and ways of understanding policy problems and relevant socioeconomic conditions.[2] For example, advocates of international treaties often find that the evidence-based arguments their international audience takes for granted become hotly contested at national or subnational levels (even if the national government is a signatory), while the same interest groups presenting the same evidence of a problem can be key insiders in one government department but ignored in another.[3]

Learn the conditions under which evidence ‘wins the day’ in policymaking

Consequently, the availability and supply of scientific evidence, on the nature of problems and effectiveness of solutions, is a necessary but insufficient condition for evidence-informed policy. Three others must be met: actors use scientific evidence to persuade policymakers to pay attention to, and shift their understanding of, policy problems; the policy environment becomes broadly conducive to policy change; and, actors exploit attention to a problem, the availability of a feasible solution, and the motivation of policymakers, during a ‘window of opportunity’ to adopt specific policy instruments.10

Tobacco control represents a ‘best case’ example (box 1) from which we can draw key lessons for ecological and environmental policies, giving us a sense of perspective by highlighting the long term potential for major evidence-informed policy change. However, unlike their colleagues in public health, environmental scientists have not developed a clear sense of how to produce policy instruments that are technically and politically feasible, so the delivery of comparable policy change is not inevitable.[12]

Box 1: Tobacco policy as a best case and cautionary tale of evidence-based policymaking

Tobacco policy is a key example – and useful comparator for ecological and environmental policies – since it represents a best case scenario and cautionary tale.[13] On the one hand, the scientific evidence on the links between smoking, mortality, and preventable death forms the basis for modern tobacco control policy. Leading countries – and the World Health Organisation, which oversees the Framework Convention on Tobacco Control (FCTC) – frame tobacco use as a public health ‘epidemic’ and allow their health departments to take the policy lead. Health departments foster networks with public health and medical groups at the expense of the tobacco industry, and emphasise the socioeconomic conditions – reductions in (a) smoking prevalence, (b) opposition to tobacco control, and (c) economic benefits to tobacco – most supportive of tobacco control. This framing, and conducive policymaking environment, helps give policymakers the motive and opportunity to choose policy instruments, such as bans on smoking in public places, which would otherwise seem politically infeasible.

On the other hand, even in a small handful of leading countries such as the UK, it took twenty to thirty years to go from the supply of the evidence to a proportionate government response: from the early evidence on smoking in the 1950s prompting major changes from the 1980s, to the evidence on passive smoking in the 1980s prompting public bans from the 2000s onwards. In most countries, the production of a ‘comprehensive’ set of policy measures is not yet complete, even though most signed the FCTC.

Decide how far you’ll go to maximise the use of scientific evidence in policymaking

These insights help challenge the naïve position that, if policymaking can change to become less dysfunctional[1], scientists can be ‘honest brokers’[14] and expect policymakers to use their evidence quickly, routinely, and sincerely. Even in the best case scenario, evidence-informed change takes hard work, persistence, and decades to achieve.

Since policymaking will always appear ‘irrational’ and complex’[3], scientists need to think harder about their role, then choose to engage more effectively or accept their lack of influence.

To deal with ‘irrational’ policymakers, they should combine evidence with persuasion, simple stories, and emotional appeals, and frame their evidence to make the implications consistent with policymakers’ beliefs.

To deal with complex environments, they should engage for the long term to work out how to form alliances with influencers who share their beliefs, understand in which ‘venues’ authoritative decisions are made and carried out, the rules of information processing in those venues, and the ‘currency’ used by policymakers when they describe policy problems and feasible solutions.[2] In other words, develop skills that do not come with scientific training, avoid waiting for others to share your scientific mindset or respect for scientific evidence, and plan for the likely eventuality that policymaking will never become ‘evidence based’.

This approach may be taken for granted in policy studies[15], but it raises uncomfortable dilemmas regarding how far scientists should go, to maximise the use of scientific evidence in policy, using persuasion and coalition-building.

These dilemmas are too frequently overshadowed by claims – more comforting to scientists – that politicians are to blame because they do not understand how to generate, analyse, and use the best evidence. Scientists may only become effective in politics if they apply the same critical analysis to themselves.

[1] Sutherland, W.J. & Burgman, M. Nature 526, 317–318 (2015).

[2] Cairney, P. et al. Public Administration Review 76, 3, 399-402 (2016)

[3] Cairney, P. The Politics of Evidence-Based Policy Making (Palgrave Springer, 2016).

[4] Langer, L. et al. The Science of Using Science (EPPI, 2016)

[5] Breckon, J. & Dodson, J. Using Evidence. What Works? (Alliance for Useful Evidence, 2016)

[6] Palgrave Communications series The politics of evidence-based policymaking (ed. Cairney, P.)

[7] Practical lessons from policy theories (eds. Weible, C & Cairney, P.) Policy and Politics April 2018

[8] Oliver, K. et al. Health Research Policy and Systems, 12, 34 (2016)

[9] Kingdon, J. Agendas, Alternatives and Public Policies (Harper Collins, 1984)

[10] Gluckmann, P. Understanding the challenges and opportunities at the science-policy interface

[11] Cairney, P. & Kwiatkowski, R. Palgrave Communications.

[12] Biesbroek et al. Nature Climate Change, 5, 6, 493–494 (2015)

[13] Cairney, P. & Yamazaki, M. Journal of Comparative Policy Analysis

[14] Pielke Jr, R. originated the specific term The honest broker (Cambridge University Press, 2007) but this role is described more loosely by other commentators.

[15] Cairney, P. & Oliver, K. Health Research Policy and Systems 15:35 (2017)

3 Comments

Filed under Evidence Based Policymaking (EBPM), public policy

How you want your presentation to go, versus how it goes.

At the Los Angeles County Museum of Art

At MOCA

At Broad

At Six Flags MM

At a novelty sweet shop at Citywalk

At the Hollywood Museum

On the walk of fame

At the Getty Museum

At the California Academy of Sciences

At the de Young Museum, San Francisco

… It’s more of a comment than a question …

At the Legion of Honor, San Francisco

At the San Francisco Museum of Modern Art

In retrospect, this constant search for new photos for a joke that became tired by photo 3 could have ruined the trip, but it totally didn’t. No room for The Thinker, though. This look would be welcome during a presentation.

See also:

Leave a comment

Filed under Uncategorized