Category Archives: public policy

Evidence-based policymaking: political strategies for scientists living in the real world

Note: I wrote the following discussion (last year) to be a Nature Comment but it was not to be!

Nature articles on evidence-based policymaking often present what scientists would like to see: rules to minimise bias caused by the cognitive limits of policymakers, and a simple policy process in which we know how and when to present the best evidence.[1]  What if neither requirement is ever met? Scientists will despair of policymaking while their competitors engage pragmatically and more effectively.[2]

Alternatively, if scientists learned from successful interest groups, or by using insights from policy studies, they could develop three ‘take home messages’: understand and engage with policymaking in the real world; learn how and when evidence ‘wins the day’; and, decide how far you should go to maximise the use of scientific evidence. Political science helps explain this process[3], and new systematic and thematic reviews add new insights.[4] [5] [6] [7]

Understand and engage with policymaking in the real world

Scientists are drawn to the ‘policy cycle’, because it offers a simple – but misleading – model for engagement with policymaking.[3] It identifies a core group of policymakers at the ‘centre’ of government, perhaps giving the impression that scientists should identify the correct ‘stages’ in which to engage (such as ‘agenda setting’ and ‘policy formulation’) to ensure the best use of evidence at the point of authoritative choice. This is certainly the image generated most frequently by health and environmental scientists when they seek insights from policy studies.[8]

Yet, this model does not describe reality. Many policymakers, in many levels and types of government, adopt and implement many measures at different times. For simplicity, we call the result ‘policy’ but almost no modern policy theory retains the linear policy cycle concept. In fact, it is more common to describe counterintuitive processes in which, for example, by the time policymaker attention rises to a policy problem at the ‘agenda setting’ stage, it is too late to formulate a solution. Instead, ‘policy entrepreneurs’ develop technically and politically feasible solutions then wait for attention to rise and for policymakers to have the motive and opportunity to act.[9]

Experienced government science advisors recognise this inability of the policy cycle image to describe real world policymaking. For example, Sir Peter Gluckman presents an amended version of this model, in which there are many interacting cycles in a kaleidoscope of activity, defying attempts to produce simple flow charts or decision trees. He describes the ‘art and craft’ of policy engagement, using simple heuristics to deal with a complex and ‘messy’ policy system.[10]

Policy studies help us identify two such heuristics or simple strategies.

First, respond to policymaker psychology by adapting to the short cuts they use to gather enough information quickly: ‘rational’, via trusted sources of oral and written evidence, and ‘irrational’, via their beliefs, emotions, and habits. Policy theories describe many interest group or ‘advocacy coalition’ strategies, including a tendency to combine evidence with emotional appeals, romanticise their own cause and demonise their opponents, or tell simple emotional stories with a hero and moral to exploit the biases of their audience.[11]

Second, adapt to complex ‘policy environments’ including: many policymakers at many levels and types of government, each with their own rules of evidence gathering, network formation, and ways of understanding policy problems and relevant socioeconomic conditions.[2] For example, advocates of international treaties often find that the evidence-based arguments their international audience takes for granted become hotly contested at national or subnational levels (even if the national government is a signatory), while the same interest groups presenting the same evidence of a problem can be key insiders in one government department but ignored in another.[3]

Learn the conditions under which evidence ‘wins the day’ in policymaking

Consequently, the availability and supply of scientific evidence, on the nature of problems and effectiveness of solutions, is a necessary but insufficient condition for evidence-informed policy. Three others must be met: actors use scientific evidence to persuade policymakers to pay attention to, and shift their understanding of, policy problems; the policy environment becomes broadly conducive to policy change; and, actors exploit attention to a problem, the availability of a feasible solution, and the motivation of policymakers, during a ‘window of opportunity’ to adopt specific policy instruments.10

Tobacco control represents a ‘best case’ example (box 1) from which we can draw key lessons for ecological and environmental policies, giving us a sense of perspective by highlighting the long term potential for major evidence-informed policy change. However, unlike their colleagues in public health, environmental scientists have not developed a clear sense of how to produce policy instruments that are technically and politically feasible, so the delivery of comparable policy change is not inevitable.[12]

Box 1: Tobacco policy as a best case and cautionary tale of evidence-based policymaking

Tobacco policy is a key example – and useful comparator for ecological and environmental policies – since it represents a best case scenario and cautionary tale.[13] On the one hand, the scientific evidence on the links between smoking, mortality, and preventable death forms the basis for modern tobacco control policy. Leading countries – and the World Health Organisation, which oversees the Framework Convention on Tobacco Control (FCTC) – frame tobacco use as a public health ‘epidemic’ and allow their health departments to take the policy lead. Health departments foster networks with public health and medical groups at the expense of the tobacco industry, and emphasise the socioeconomic conditions – reductions in (a) smoking prevalence, (b) opposition to tobacco control, and (c) economic benefits to tobacco – most supportive of tobacco control. This framing, and conducive policymaking environment, helps give policymakers the motive and opportunity to choose policy instruments, such as bans on smoking in public places, which would otherwise seem politically infeasible.

On the other hand, even in a small handful of leading countries such as the UK, it took twenty to thirty years to go from the supply of the evidence to a proportionate government response: from the early evidence on smoking in the 1950s prompting major changes from the 1980s, to the evidence on passive smoking in the 1980s prompting public bans from the 2000s onwards. In most countries, the production of a ‘comprehensive’ set of policy measures is not yet complete, even though most signed the FCTC.

Decide how far you’ll go to maximise the use of scientific evidence in policymaking

These insights help challenge the naïve position that, if policymaking can change to become less dysfunctional[1], scientists can be ‘honest brokers’[14] and expect policymakers to use their evidence quickly, routinely, and sincerely. Even in the best case scenario, evidence-informed change takes hard work, persistence, and decades to achieve.

Since policymaking will always appear ‘irrational’ and complex’[3], scientists need to think harder about their role, then choose to engage more effectively or accept their lack of influence.

To deal with ‘irrational’ policymakers, they should combine evidence with persuasion, simple stories, and emotional appeals, and frame their evidence to make the implications consistent with policymakers’ beliefs.

To deal with complex environments, they should engage for the long term to work out how to form alliances with influencers who share their beliefs, understand in which ‘venues’ authoritative decisions are made and carried out, the rules of information processing in those venues, and the ‘currency’ used by policymakers when they describe policy problems and feasible solutions.[2] In other words, develop skills that do not come with scientific training, avoid waiting for others to share your scientific mindset or respect for scientific evidence, and plan for the likely eventuality that policymaking will never become ‘evidence based’.

This approach may be taken for granted in policy studies[15], but it raises uncomfortable dilemmas regarding how far scientists should go, to maximise the use of scientific evidence in policy, using persuasion and coalition-building.

These dilemmas are too frequently overshadowed by claims – more comforting to scientists – that politicians are to blame because they do not understand how to generate, analyse, and use the best evidence. Scientists may only become effective in politics if they apply the same critical analysis to themselves.

[1] Sutherland, W.J. & Burgman, M. Nature 526, 317–318 (2015).

[2] Cairney, P. et al. Public Administration Review 76, 3, 399-402 (2016)

[3] Cairney, P. The Politics of Evidence-Based Policy Making (Palgrave Springer, 2016).

[4] Langer, L. et al. The Science of Using Science (EPPI, 2016)

[5] Breckon, J. & Dodson, J. Using Evidence. What Works? (Alliance for Useful Evidence, 2016)

[6] Palgrave Communications series The politics of evidence-based policymaking (ed. Cairney, P.)

[7] Practical lessons from policy theories (eds. Weible, C & Cairney, P.) Policy and Politics April 2018

[8] Oliver, K. et al. Health Research Policy and Systems, 12, 34 (2016)

[9] Kingdon, J. Agendas, Alternatives and Public Policies (Harper Collins, 1984)

[10] Gluckmann, P. Understanding the challenges and opportunities at the science-policy interface

[11] Cairney, P. & Kwiatkowski, R. Palgrave Communications.

[12] Biesbroek et al. Nature Climate Change, 5, 6, 493–494 (2015)

[13] Cairney, P. & Yamazaki, M. Journal of Comparative Policy Analysis

[14] Pielke Jr, R. originated the specific term The honest broker (Cambridge University Press, 2007) but this role is described more loosely by other commentators.

[15] Cairney, P. & Oliver, K. Health Research Policy and Systems 15:35 (2017)

3 Comments

Filed under Evidence Based Policymaking (EBPM), public policy

Why don’t policymakers listen to your evidence?

Since 2016, my most common academic presentation to interdisciplinary scientist/ researcher audiences is a variant of the question, ‘why don’t policymakers listen to your evidence?’

I tend to provide three main answers.

1. Many policymakers have many different ideas about what counts as good evidence

Few policymakers know or care about the criteria developed by some scientists to describe a hierarchy of scientific evidence. For some scientists, at the top of this hierarchy is the randomised control trial (RCT) and the systematic review of RCTs, with expertise much further down the list, followed by practitioner experience and service user feedback near the bottom.

Yet, most policymakers – and many academics – prefer a wider range of sources of information, combining their own experience with information ranging from peer reviewed scientific evidence and the ‘grey’ literature, to public opinion and feedback from consultation.

While it may be possible to persuade some central government departments or agencies to privilege scientific evidence, they also pursue other key principles, such as to foster consensus driven policymaking or a shift from centralist to localist practices.

Consequently, they often only recommend interventions rather than impose one uniform evidence-based position. If local actors favour a different policy solution, we may find that the same type of evidence may have more or less effect in different parts of government.

2. Policymakers have to ignore almost all evidence and almost every decision taken in their name

Many scientists articulate the idea that policymakers and scientists should cooperate to use the best evidence to determine ‘what works’ in policy (in forums such as INGSA, European Commission, OECD). Their language is often reminiscent of 1950s discussions of the pursuit of ‘comprehensive rationality’ in policymaking.

The key difference is that EBPM is often described as an ideal by scientists, to be compared with the more disappointing processes they find when they engage in politics. In contrast, ‘comprehensive rationality’ is an ideal-type, used to describe what cannot happen, and the practical implications of that impossibility.

The ideal-type involves a core group of elected policymakers at the ‘top’, identifying their values or the problems they seek to solve, and translating their policies into action to maximise benefits to society, aided by neutral organisations gathering all the facts necessary to produce policy solutions. Yet, in practice, they are unable to: separate values from facts in any meaningful way; rank policy aims in a logical and consistent manner; gather information comprehensively, or possess the cognitive ability to process it.

Instead, Simon famously described policymakers addressing ‘bounded rationality’ by using ‘rules of thumb’ to limit their analysis and produce ‘good enough’ decisions. More recently, punctuated equilibrium theory uses bounded rationality to show that policymakers can only pay attention to a tiny proportion of their responsibilities, which limits their control of the many decisions made in their name.

More recent discussions focus on the ‘rational’ short cuts that policymakers use to identify good enough sources of information, combined with the ‘irrational’ ways in which they use their beliefs, emotions, habits, and familiarity with issues to identify policy problems and solutions. Or, they explore how individuals communicate their narrow expertise within a system of which they have almost no knowledge. In each case, ‘most members of the system are not paying attention to most issues most of the time’.

This scarcity of attention helps explain, for example, why policymakers ignore most issues in the absence of a focusing event, policymaking organisations make searches for information which miss key elements routinely, and organisations fail to respond to events or changing circumstances proportionately.

In that context, attempts to describe a policy agenda focusing merely on ‘what works’ are based on misleading expectations. Rather, we can describe key parts of the policymaking environment – such as institutions, policy communities/ networks, or paradigms – as a reflection of the ways in which policymakers deal with their bounded rationality and lack of control of the policy process.

3. Policymakers do not control the policy process (in the way that a policy cycle suggests)

Scientists often appear to be drawn to the idea of a linear and orderly policy cycle with discrete stages – such as agenda setting, policy formulation, legitimation, implementation, evaluation, policy maintenance/ succession/ termination – because it offers a simple and appealing model which gives clear advice on how to engage.

Indeed, the stages approach began partly as a proposal to make the policy process more scientific and based on systematic policy analysis. It offers an idea of how policy should be made: elected policymakers in central government, aided by expert policy analysts, make and legitimise choices; skilful public servants carry them out; and, policy analysts assess the results with the aid of scientific evidence.

Yet, few policy theories describe this cycle as useful, while most – including the advocacy coalition framework , and the multiple streams approach – are based on a rejection of the explanatory value of orderly stages.

Policy theories also suggest that the cycle provides misleading practical advice: you will generally not find an orderly process with a clearly defined debate on problem definition, a single moment of authoritative choice, and a clear chance to use scientific evidence to evaluate policy before deciding whether or not to continue. Instead, the cycle exists as a story for policymakers to tell about their work, partly because it is consistent with the idea of elected policymakers being in charge and accountable.

Some scholars also question the appropriateness of a stages ideal, since it suggests that there should be a core group of policymakers making policy from the ‘top down’ and obliging others to carry out their aims, which does not leave room for, for example, the diffusion of power in multi-level systems, or the use of ‘localism’ to tailor policy to local needs and desires.

Further Reading

The politics of evidence-based policymaking

The politics of evidence-based policymaking: maximising the use of evidence in policy

Images of the policy process

How to communicate effectively with policymakers

Forthcoming special issue in Policy and Politics called ‘Practical lessons from policy theories’, which includes my discussion of how to be a ‘policy entrepreneur’.

1 Comment

Filed under Evidence Based Policymaking (EBPM), Psychology Based Policy Studies, Public health, public policy

Policy concepts in 1000 words: Institutional memory

Guest post by Jack Corbett, Dennis Grube, Heather Lovell and Rodney Scott

Democratic governance is defined by the regular rotation of elected leaders. Amidst the churn, the civil service is expected to act as the repository of received wisdom about past policies, including assessments of what works and what doesn’t. The claim is that to avoid repeating the same mistakes we need to know what happened last time and what were the effects. Institutional memory is thus central to the pragmatic task of governing.

What is institutional memory? And, how is it different to policy learning?

Despite increasing recognition of the role that memory can or should play in the policy process, the concept has defied easy scholarly definition.

In the classic account, institutional memory is the sum total of files, procedures and knowledge held by an organisation. Christopher Pollitt, who has pioneered the study of institutional memory, refers to the accumulated knowledge and experience of staff, technical systems, including electronic databases and various kinds of paper records, the management system, and the norms and values of the organizational culture, when talking about institutional memory. In this view, which is based on the key principles of the new institutionalism, memory is essentially an archive.

The problem with this definition is that it is hard to distinguish the concept from policy learning (see also here). If policy learning is in part about increasing knowledge about policy, including correcting for past mistakes, then we could perhaps conceive of a continuum from learning to memory with an inflection point where one starts and the other stops. But, this is easier to imagine than it is to measure empirically. It also doesn’t acknowledge the forms memories take and the ways memories are contested, suppressed and actively forgotten.

In our recent contribution to this debate (see here and here) we define memories as ‘representations of the past’ that actors draw on to narrate what has been learned when developing and implementing policy. When these narratives are embedded in processes they become ‘institutionalised’. It is this emphasis on embedded narratives that distinguishes institutional memory from policy learning. Institutional memory may facilitate policy learning but equally some memories may prohibit genuine adaptation and innovation. As a result, while there is an obvious affinity between the two concepts it is imperative that they remain distinct avenues of inquiry. Policy learning has unequivocally positive connotations that are echoed in some conceptualisations of institutional memory (i.e. Pollitt). But, equally, memory (at least in a ‘static’ form) can be said to provide administrative agents with an advantage over political principals (think of the satirical Sir Humphrey of Yes Minister fame). The below table seeks to distinguish between these two conceptualisations of institutional memory:

Key debates: Is institutional memory declining?

The scholar who has done the most to advance our understanding of institutional memory in government is Christopher Pollitt. His main contention is that institutional memory has declined over recent decades due to: the high rotation of staff in the civil service, changes in IT systems which prevent proper archiving, regular organisational restructuring, rewarding management skills above all others, and adopting new management ‘fads’ that favour constant change as they become popular. This combination of factors has proven to be a perfect recipe for the loss of institutional memory within organisations.  The result is a contempt for the past that leads to repeated policy failure.

We came to a different view. Our argument is that one of the key reasons why institutional memory is said to have declined is that it has been conceptualised in a ‘static’ manner more in keeping with an older way of doing government. This practice has assumed that knowledge on a given topic is held centrally (by government departments) and can be made explicit for the purpose of archiving. But, if government doesn’t actually work this way (see relevant posts on networks here) then we shouldn’t expect it to remember this way either. Instead of static repositories of summative documents holding a singular ‘objective’ memory, we propose a more ‘dynamic’ people-centred conceptualisation that sees institutional memory as a composite of intersubjective memories open to change. This draws to the fore the role of actors as crucial interpreters of memory, combining the documentary record with their own perspectives to create a story about the past. In this view, institutional memory has not declined, it is simply being captured in a fundamentally different way.

Corbett et al memory

Key debates: How can an institution improve how it remembers?

How an institution might improve its memory is intrinsically linked to how memory is defined and whether or not it is actually in decline. If we follow Pollitt’s view that memory is about the archive of accumulated knowledge that is being ignored or deliberately dismantled by managerialism then the answer involves returning to an older way of doing government that placed a higher value on experience. By putting a higher value on the past as a resource institutions would reduce staff turnover, stop regular restructures and changes in IT systems, etc. For those of us who work in an institution where restructuring and IT changes are the norm, this solution has obvious attractions. But, would it actually improve memory? Or would it simply make it easier to preserve the status quo (a process that involves actively forgetting disruptive but generative innovations)?

Our definition, relying as it does on a more dynamic conceptualisation of memory, is sceptical about the need to improve practices of remembering. But, if an institution did want to remember better we would favour increasing the opportunity for actors within an institution to reflect on and narrate the past. One example of this might be a ‘Wikipedia’ model of memory in which the story of a policy, it success and failure, is constructed by those involved, highlighting points of consensus and conjecture.

Additional reading:

 Corbett J, Grube D, Lovell H, Scott R. “Singular memory or institutional memories? Toward a dynamic approach”. Governance. 2018;00:1–19. https://doi.org/10.1111/gove.12340

 Pollitt, C. 2009. “Bureaucracies Remember, Post‐Bureaucratic Organizations Forget?” Public Administration 87 (2): 198-218.

Pollitt, C. 2000. “Institutional Amnesia: A Paradox of the ‘Information Age’?” Prometheus 18 (1): 5-16.

 

Leave a comment

Filed under 1000 words, public policy, Uncategorized

Policy in 500 words: uncertainty versus ambiguity

In policy studies, there is a profound difference between uncertainty and ambiguity:

  • Uncertainty describes a lack of knowledge or a worrying lack of confidence in one’s knowledge.
  • Ambiguity describes the ability to entertain more than one interpretation of a policy problem.

Both concepts relate to ‘bounded rationality’: policymakers do not have the ability to process all information relevant to policy problems. Instead, they employ two kinds of shortcut:

  • ‘Rational’. Pursuing clear goals and prioritizing certain sources of information.
  • ‘Irrational’. Drawing on emotions, gut feelings, deeply held beliefs, and habits.

I make an artificially binary distinction, uncertain versus ambiguous, and relate it to another binary, rational versus irrational, to point out the pitfalls of focusing too much on one aspect of the policy process:

  1. Policy actors seek to resolve uncertainty by generating more information or drawing greater attention to the available information.

Actors can try to solve uncertainty by: (a) improving the quality of evidence, and (b) making sure that there are no major gaps between the supply of and demand for evidence. Relevant debates include: what counts as good evidence?, focusing on the criteria to define scientific evidence and their relationship with other forms of knowledge (such as practitioner experience and service user feedback), and what are the barriers between supply and demand?, focusing on the need for better ways to communicate.

  1. Policy actors seek to resolve ambiguity by focusing on one interpretation of a policy problem at the expense of another.

Actors try to solve ambiguity by exercising power to increase attention to, and support for, their favoured interpretation of a policy problem. You will find many examples of such activity spread across the 500 and 1000 words series:

A focus on reducing uncertainty gives the impression that policymaking is a technical process in which people need to produce the best evidence and deliver it to the right people at the right time.

In contrast, a focus on reducing ambiguity gives the impression of a more complicated and political process in which actors are exercising power to compete for attention and dominance of the policy agenda. Uncertainty matters, but primarily to describe the role of a complex policymaking system in which no actor truly understands where they are or how they should exercise power to maximise their success.

Further reading:

Framing

The politics of evidence-based policymaking

To Bridge the Divide between Evidence and Policy: Reduce Ambiguity as Much as Uncertainty

How to communicate effectively with policymakers: combine insights from psychology and policy studies

Here is the relevant opening section in UPP:

p234 UPP ambiguity

Leave a comment

Filed under 500 words, agenda setting, Evidence Based Policymaking (EBPM), public policy, Storytelling

The global study of governance and public policy

Cairney GPP Iran

I was invited by Dr Emamian from the Governance and Policy Think Tank to deliver this short lecture at the first ‘governance and public policy conference’ in Iran. I was unable to attend, so recorded a set of short video presentations supplemented by blog discussion. The topics to be covered include the importance of a scholarly network for policy studies, the need for a set of core policy concepts to act as a technical language for that network, and the need to apply that language to explain shifts in government and regulation towards ‘regulatory governance’.

Please note that my choice to record the videos in my garden (while I look up) seemed good at the time, for some very good reasons that I won’t get into. However, you will see that I become increasingly cold and annoyed at being cold. I can only apologize for my face and the fact that I was too cold to remember to put on my professional voice.

Using shared concepts in a scholarly network of policy researchers

Our aim may be to produce a global network of policy scholars, in two main ways:

  1. To foster meetings and discussion, such as via this conference and others such as the ICPP and ECPR
  2. To make sure that we are talking about the same thing. Most of the theories to which I refer are based on studies of countries like the US and UK. Their prominence contributes to a ‘global north’ perspective which can be useful in the abstract but with uncertain applicability across the globe.

For example, when considering the applicability of US-inspired theories, think about their taken-for-granted assumptions about the nature of a political system, in which leaders in many levels and types of government are elected regularly, there is a constitution guaranteeing a division of powers across legislative-executive-judicial branches and between federal/subnational levels, and people describe a ‘pluralist’ system in which many groups mobilise and counter-mobilise to influence policy.

What happens when we stop taking this political context for granted? Do these theories remain as relevant?

Which concepts do we use?

I describe two main abstract concepts then invite you to think about how to apply them in more concrete circumstances.

  1. Bounded rationality, not comprehensive rationality.

No-one can understand fully the world in which we live. Individuals can only understand and pay attention to a tiny part of key aspects of the world such as political systems.

Indeed, a handy phrase to remember is that almost all people must ignore almost everything almost all of the time.

Yet, they must make choices despite uncertainty, perhaps by adopting ‘fast and frugal’ heuristics. In other words, we may see all human choices as flawed when compared with an ideal of perfect decision-making. On the other hand, we may marvel at the ways in which humans make often-good choices despite their limitations.

Individual policymakers use two short-cuts to gather enough information to make choices:

  • ‘Rational’, in which they adopt measures to ensure that they have good enough information to inform decisions. For example, they prioritise certain written sources of information and draw on people they consider to be experts.
  • ‘Irrational’, in which they rely on things like gut instinct, habit, and emotion to make snap decisions.

In that context, policy scholarship involves the study of how people make and influence those choices. One part is about the role of evidence, in which people produce information to reduce uncertainty about the nature of the world. However, the more important study is of how people understand the world in the first place. As policy scholars, we focus on ambiguity, to describe the many ways in which people choose to understand the same problems, and the exercise of power to influence those choices.

  1. A complex policymaking environment, not a policy cycle.

Things get more complicated when we move from the analysis of (a) key individuals to (b) the interaction between many individuals and organisations in a complex policymaking ‘system’ or ‘environment’. Policy scholars describe this environment in many different ways, using different concepts, but we can identify a core set of terms on which to focus:

  • Actors. There are many policy influencers and policymakers in many authoritative venues spread across many levels and types of government.
  • Institutions. Each venue has its own rules, including the formal, written-down, and easy to understand rules, versus the informal norms, cultures, and practices which are difficult to identify and describe.
  • Networks. Policymakers and influencers form relationships based on factors such as trust, authority, and the exchange of resources such as information and support.
  • Ideas. People communicate their beliefs, about policy problems and potential solutions, within a wider understanding of the world (often described as a paradigm or hegemony). Some of that understanding is taken-for-granted and not described, and people limit their analysis and argument according to the ways in which they think other people see the world.
  • Socioeconomic context and events. Policymakers often have to respond to policy conditions and events over which they have limited control, such geographic, demographic, and economic factors. These factors help produce non-routine ‘events’ alongside more predictable events such as elections (or other means to ensure a change of government).

In that context, policy scholarship focuses on producing theories to explain what happens when policymakers have limited control over their political systems and policymaking environments.

How far do these concepts travel?

As you can see, these concepts are widely applicable because they are abstract. What happens when we try to apply them to specific countries or case studies? For example:

  1. We talk about policymakers using cognitive, moral, and emotional shortcuts, but those shortcuts can vary profoundly across the globe.
  2. Each political system has a different collection of authoritative venues, formal and informal rules of politics, networks of power, ways to describe how the world works and should work, and socioeconomic context.

This is where our global network becomes valuable, to help us describe how we make sense of the same concepts in very different ways, and consider the extent to which such discussions are comparable.

Example: how do governments address an ‘era of governance’?

One way to foster such discussion is to consider how governments address the limits to their powers. These limits are described in many different ways, from a focus on ‘complexity’ and policy outcomes which ‘emerge’ from local activity (despite attempts by central governments to control outcomes), to a focus on the shift from ‘government’ to ‘governance’.

As policy scholars, we can make several useful distinctions to describe these dynamics, such as to separate an actual shift in policymaking from government to governance, versus a shift in the way we now describe government.

Or, we can separate how governments can, do, and should address the limits to their powers.

I’d say that most policy scholarship focuses on how governments operate: how they actually address problems and what are the – intended and unintended – consequences.

However, these studies are trying to describe the tensions between what governments can do, given the limits I describe, and what they think they should do, given their position of authority and their need to describe their success.

For example, some systems may be more conducive to the support for ‘polycentric governance’, in which many authoritative venues cooperate to address problems, while others are built on the idea of central control and the concentration of authority in a small group of actors.

Therefore, the study of actual policymaking and outcomes will vary markedly according to the ways in which government actors feel they need to assert an image of control over a policy environment which is almost immune to control.

Perhaps an ‘era of governance’ describes some recognition by many governments that they need to find new ways to address their limited control over policy outcomes, both domestically and globally. However, an enduring theme in political science and policy studies is that we do not explain policymaking well if we restrict our attention to the ‘rational’ decisions of a small number of actors. Let’s not make too many assumptions about their power and motive.

Further reading:

The New Policy Sciences

How to Communicate Effectively with Policymakers

The Politics of Evidence Based Policymaking

Comparison of Theories of the Policy Process

Key policy theories and concepts in 1000 words.

Leave a comment

Filed under public policy

Debating the politics of evidence-based policy

Joshua Newman has provided an interesting review of three recent books on evidence/ policy (click here). One of those books is mine: The Politics of Evidence-Based Policy Making (which you can access here).

His review is very polite, for which I thank him. I hope my brief response can be seen in a similarly positive light (well, I had hoped to make it brief). Maybe we disagree on one or two things, but often these discussions are about the things we emphasize and the way we describe similar points.

There are 5 points to which I respond because I have 5 digits on my right hand. I’d like you to think of me counting them out on my fingers. In doing so, I’ll use ‘Newman’ throughout, because that’s the academic convention, but I’d also like to you imagine me reading my points aloud and whispering ‘Joshua’ before each ‘Newman’.

  1. Do we really need to ‘take the debate forward’ so often?

I use this phrase myself, knowingly, to keep a discussion catchy, but I think it’s often misleading. I suggest not to get your hopes up too high when Newman raises the possibility of taking the debate forward with his concluding questions. We won’t resolve the relationship between evidence, politics & policy by pretending to reframe the same collection of questions about the prospect of political reform that people have been asking for centuries. It is useful to envisage better political systems (the subject of Newman’s concluding remarks) but I don’t think we should pretend that this is a new concern or that it will get us very far.

Indeed, my usual argument is that researchers need to do something (such as improve how we engage in the policy process) while we wait for political system reforms to happen (while doubting if they will ever happen).

Further, Newman does not produce any political reforms to address the problems he raises. Rather, for example, he draws attention to Trump to describe modern democracies as ‘not pluralist utopias’ and to identify examples in which policymakers draw primarily on beliefs, not evidence. By restating these problems, he does not solve them. So, what are researchers supposed to do after they grow tired of complaining that the world does not meet their hopes or expectations?

In other words, for me, (a) promoting political change and (b) acting during its absence are two sides of the same coin. We go round and round more often than we take things forward.

  1. What debate are we renaming?

Newman’s ‘we’ve heard it before’ argument seems more useful, but there is a lot to hear and relatively few people have heard it. I’d warn against the assumption that ‘I’ve heard this before’ can ever equal ‘we’ve heard it before’ (unless ‘we’ refers to a tiny group of specialists talking only to each other).

Rather, one of the most important things we can do as academics is to tell the same story to each other (to check if we understand the same story, in the same way, and if it remains useful) and to wider audiences (in a way that they can pick up and use without dedicating their career to our discipline).

Some of our most important insights endure for decades and they sometimes improve in the retelling. We apply them to new eras, and often come to the same basic conclusions, but it seems unhelpful to criticise a lack of complete novelty in individual texts (particularly when they are often designed to be syntheses). Why not use them to occasionally take a step back to discuss and clarify what we know?

Perhaps more importantly, I don’t think Newman is correct when he says that each book retells the story of the ‘research utilization’ literature. I’m retelling the story of policy theory, which describes how policymakers deal with bounded rationality in a complex policymaking environment. Policy theory’s intellectual histories often provide very different perspectives – of the policymaker trying to make good enough decisions, rather than the researcher trying to improve the uptake of their research – than the agenda inspired by Weiss et al (see for example The New Policy Sciences).

  1. Don’t just ‘get political’; understand the policy process

I draw on policy theory because it helps people understand policymaking. It would be a mistake to conclude from my book that I simply want researchers to ‘get political’. Rather, I want them to develop useful knowledge of the policy process in which they might want to engage. This knowledge is not freely available; it takes time to understand the discipline and reflect on policy dynamics.

Yet, the payoff can be profound, if only because it helps people think about the difference between two analytically separate causes of a notional ‘evidence policy gap’: (a) individuals making choices based on their beliefs and limited information (which is relatively easy to understand but also to caricature), and (b) systemic or ‘environmental’ causes (which are far more difficult to conceptualise and explain, but often more useful to understand).

  1. Don’t throw out the ‘two communities’ phrase without explaining why

Newman criticises the phrase ‘two communities’ as a description of silos in policymaking versus research, partly because (a) many policymakers use research frequently, and (b) the real divide is often between users/ non-users of research within policymaking organisations. In short, there are more than two communities.

I’d back up his published research with my anecdotal experience of giving talks to government audiences: researchers and analysts within government are often very similar in outlook to academics and they often talk in the same way as academics about the disconnect between their (original or synthetic) research and its use by their ‘operational’ colleagues.

Still, I’m not sure why Newman concludes that the ‘two communities’ phrase is ‘deeply flawed and probably counter-productive’. Yes, the world is more nuanced and less binary than ‘two communities’ suggests. Yes, the real divide may be harder to spot. Still, as Newman et al suggest: ‘Policy makers and academics should focus on bridging instruments that can bring their worlds closer together’. This bullet point from their article seems, to me, to be the point of using the phrase ‘two communities’. Maybe Caplan used the phrase differently in 1979, but to assert its historic meaning then reject the phrase’s use in modern discussion seems less useful than simply clarifying the argument in ways such as:

  • There is no simple policymaker/ academic divide but, … note the major difference in requirements between (a) people who produce or distribute research without taking action, which allows them (for example) to be more comfortable with uncertainty, and (b) people who need to make choices despite having incomplete information to hand.
  • You might find a more receptive audience in one part of government (e.g. research/ analytical) than another (e.g. operational), so be careful about generalising from singular experiences.
  1. Should researchers engage in the policy process?

Newman says that each book, ‘unfairly places the burden of resolving the problem in the hands of an ill-equipped group of academics, operating outside the political system’.

I agree with Newman when he says that many researchers do not possess the skills to engage effectively in the policy process. Scientific training does not equip us with political skills. Indeed, I think you could read a few of my blog posts and conclude, reasonably, that you would like nothing more to do with the policy process because you’d be more effective by focusing on research.

The reason I put the onus back on researchers is because I am engaging with arguments like the one expressed by Newman (in other words, part of the meaning comes from the audience). Many people conclude their evidence policy discussions by identifying (or ‘reframing’) the problem primarily as the need for political reform. For me, the focus on other people changing to suit your preferences seems unrealistic and misplaced. In that context, I present the counter-argument that it may be better to adapt effectively to the policy process that exists, not the one you’d like to see. Sometimes it’s more useful to wear a coat than complain about the weather.

See also:  The Politics of Evidence 

The Politics of Evidence revisited

 

Pivot cover

1 Comment

Filed under Evidence Based Policymaking (EBPM), public policy

What do we need to know about the politics of evidence-based policymaking?

Today, I’m helping to deliver a new course – Engaging Policymakers Training Programme – piloted by the Alliance for Useful Evidence and the UCL. Right now, it’s for UCL staff (and mostly early career researchers). My bit is about how we can better understand the policy process so that we can engage in it more effectively.  I have reproduced the brief guide below (for my two 2-hour sessions as part of a wider block). If anyone else is delivering something similar, please let me know. We could compare notes. 

This module will be delivered in two parts to combine theory and practice

Part 1: What do we need to know about the politics of evidence-based policymaking?

Policy theories provide a wealth of knowledge about the role of evidence in policymaking systems. They prompt us to understand and respond to two key dynamics:

  1. Policymaker psychology. Policymakers combine rational and irrational shortcuts to gather information and make good enough decisions quickly. To appeal to rational shortcuts and minimise cognitive load, we reduce uncertainty by providing syntheses of the available evidence. To appeal to irrational shortcuts and engage emotional interest, we reduce ambiguity by telling stories or framing problems in specific ways.
  2. Complex policymaking environments. These processes take place in the context of a policy environment out of the control of individual policymakers. Environments consist of: many actors in many levels and types of government; engaging with institutions and networks, each with their own informal and formal rules; responding to socioeconomic conditions and events; and, learning how to engage with dominant ideas or beliefs about the nature of the policy problem. In other words, there is no policy cycle or obvious stage in which to get involved.

In this seminar, we discuss how to respond effectively to these dynamics. We focus on unresolved issues:

  1. Effective engagement with policymakers requires storytelling skills, but do we possess them?
  2. It requires a combination of evidence and emotional appeals, but is it ethical to do more than describe the evidence?
  3. The absence of a policy cycle, and presence of an ever-shifting context, requires us to engage for the long term, to form alliances, learn the rules, and build up trust in the messenger. However, do we have and how should we invest the time?

The format will be relatively informal. Cairney will begin by making some introductory points (not a powerpoint driven lecture) and encourage participants to relate the three questions to their research and engagement experience.

Gateway to further reading:

  • Paul Cairney and Richard Kwiatkowski (2017) ‘How to communicate effectively with policymakers: combine insights from psychology and policy studies’, Palgrave Communications
  • Paul Cairney and Kathryn Oliver (2017) ‘Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy?’ Health Research Policy and Systems (HARPS), DOI: 10.1186/s12961-017-0192-x
  • Paul Cairney, Kathryn Oliver, and Adam Wellstead (2016) ‘To Bridge the Divide between Evidence and Policy: Reduce Ambiguity as Much as Uncertainty’, Public Administration Review, Early View (forthcoming) DOI:10.1111/puar.12555 PDF

Part 2: How can we respond pragmatically and effectively to the politics of EBPM?

In this seminar, we move from abstract theory and general advice to concrete examples and specific strategies. Each participant should come prepared to speak about their research and present a theoretically informed policy analysis in 3 minutes (without the aid of powerpoint). Their analysis should address:

  1. What policy problem does my research highlight?
  2. What are the most technically and politically feasible solutions?
  3. How should I engage in the policy process to highlight these problems and solutions?

After each presentation, each participant should be prepared to ask questions about the problem raised and the strategy to engage. Finally, to encourage learning, we will reflect on the memorability and impact of presentations.

Powerpoint: Paul Cairney A4UE UCL 2017

1 Comment

Filed under Evidence Based Policymaking (EBPM), Psychology Based Policy Studies, public policy