Category Archives: public policy

Beware the well-intentioned advice of unusually successful academics

This post – by Dr Kathryn Oliver and me – originally appeared on the LSE Impact Blog. I have replaced the picture of a thumb up with a cat hanging in there. 

Many academics want to see their research have an impact on policy and practice, and there is a lot of advice on how to seek it. It can be helpful to take advice from experienced and successful people. However, is this always the best advice? Guidance based on best practice and success stories in particular, often reflect unequal access to policymakers, institutional support, and credibility attached to certain personal characteristics.

To take stock of the vast amount of advice being offered to academics, we decided to compare it with the more systematic analyses available in the peer-reviewed literature, on the ‘barriers’ between evidence and policy, and policy studies. This allowed us to situate this advice in a wider context, see whether it was generalisable across settings and career stages, and to think through the inconsistencies and dilemmas which underlie these suggestions.

The advice: Top tips on influencing policy

The key themes and individual recommendations we identified from the 86 most-relevant publications are:

  1. Do high quality research: Use well-established research designs, methods, or metrics.
  2. Make your research relevant and readable: Provide easily-understandable, clear, relevant and high-quality research. Aim for the general reader. Produce good stories based on emotional appeals or humour.
  3. Understand the policymaking context. Note the busy and constrained lives of policy actors. Maximise established ways to engage, such as in advisory committees. Be pragmatic, accepting that research rarely translates directly into policy.
  4. Be ‘accessible’ to policymakers. This may involve discussing topics beyond your narrow expertise. Be humble, courteous, professional, and recognise the limits to your skills.
  5. Decide if you want to be an ‘issue advocate’. Decide whether to simply explain the evidence, remain an ‘honest broker, or recommend specific policy options. Negative consequences may include peer criticism, being seen as an academic lightweight, being used to add legitimacy to a policy position, and burnout.
  6. Build relationships (and ground rules) with policymakers: Relationship-building requires investment and skills, but working collaboratively is often necessary. Academics could identify policy actors to provide insights into policy problems, act as champions for their research, and identify the most helpful policy actors.
  7. Be ‘entrepreneurial’ or find someone who is. Be a daring, persuasive scientist, comfortable in policy environments and available when needed. Or, seek brokers to act on your behalf.
  8. Reflect continuously: should you engage, do you want to, and is it working? Academics may enjoy the work or are passionate about the issue. Even so, keep track of when and how you have had impact, and revise your practices continuously.

hang-in-there-baby

Inconsistencies and dilemmas

This advice tends not to address wider issues. For example, there is no consensus over what counts as good evidence for policy, or therefore how best to communicate good evidence. We know little about how to gain the wide range of skills that researchers and policymakers need to act collectively, including to: produce evidence syntheses, manage expert communities, ‘co-produce’ research and policy with a wide range of stakeholders, and be prepared to offer policy recommendations as well as scientific advice. Further, a one-size fits-all model won’t help researchers navigate a policymaking environment where different venues have different cultures and networks. Researchers therefore need to decide what policy engagement is for—to frame problems or simply measure them according to an existing frame—and how far researchers should go to be useful and influential. If academics need to go ‘all in’ to secure meaningful impact, we need to reflect on the extent to which they have the resources and support to do so. This means navigating profound dilemmas:

Source: The dos and don’ts of influencing policy: a systematic review of advice to academics

 

Can academics try to influence policy? The financial costs of seeking impact are prohibitive for junior or untenured researchers, while women and people of colour may be more subject to personal abuse. Such factors undermine the diversity of voices available.

How should academics influence policy? Many of these new required skills – such as storytelling – are not a routine part of academic training, and may be looked down on by our colleagues.  

What is the purpose of academics engagement in policymaking? To go beyond tokenistic and instrumental engagement is to build genuine rapport with policymakers, which may require us to co-produce knowledge and cede some control over the research process. It involves a fundamentally different way of doing public engagement: one with no clear aim in mind other than to listen and learn, with the potential to transform research practices and outputs.

Where is the evidence that this advice helps us improve impact?

The existing advice offered to academics on how to create impact is – although often well-meaning – not based on systematic research or comprehensive analysis of empirical evidence. Few advice-givers draw clearly on key literatures on policymaking or evidence use. This leads to significant misunderstandings, which can have potentially costly repercussions for research, researchers and policy.  These limitations matter, as they lead to advice which fails to address core dilemmas for academics—whether to engage, how to engage, and why—which have profound implications for how scientists and universities should respond to the calls for increased impact.

Most tips focus on individual experience, whereas engagement between research and policy is driven by systemic factors. Many of the tips may be sensible and effective, but often only within particular settings. The advice is likely to be useful mostly to a relatively similar group of people who are confident and comfortable in policy environments, and have access and credibility within policy arenas. Thus, the current advice and structures may help reproduce and reinforce existing power dynamics and an underrepresentation of people who do not fit a very narrow mould.

The overall result may be that each generation of scientists has to fight the same battles, and learn the same lessons over again. Our best response as a profession is to interrogate current advice, shape and frame it, and to help us all to find ways to navigate the complex practical, political, moral and ethical challenges associated with being researchers today. The ‘how to’ literature can help, but only if authors are cognisant of their wider role in society and complex policymaking systems.

This blog post is based on the authors’ co-written articles, The dos and don’ts of influencing policy: a systematic review of advice to academics, published in Palgrave Communications, and ‘How should academics engage in policymaking to achieve impact?’  published in Political Studies Review 

About the authors

Kathryn Oliver is Associate Professor of Sociology and Public Health, London School of Hygiene and Tropical Medicine (@oliver_kathryn ). Her interest is in how knowledge is produced, mobilized and used in policy and practice, and how this affects the practice of research. She co-runs the research collaborative Transforming Evidence with Annette Boaz. https://transformure.wordpress.com and her writings can be found here: https://kathrynoliver.wordpress.com

Paul Cairney is Professor of Politics and Public Policy, University of Stirling, UK (@Cairneypaul).  His research interests are in comparative public policy and policy theories, which he uses to explain the use of evidence in policy and policymaking, in one book (The Politics of Evidence-Based Policy Making, 2016), several articles, and many, many blog posts: https://paulcairney.wordpress.com/ebpm/

See also:

  1. Adam Wellstead, Paul Cairney, and Kathryn Oliver (2018) ‘Reducing ambiguity to close the science-policy gap’, Policy Design and Practice, 1, 2, 115-25 PDF
  2. Paul Cairney and Kathryn Oliver (2017) ‘Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy?’ Health Research Policy and Systems (HARPS), DOI: 10.1186/s12961-017-0192-x PDF AM
  3. Paul Cairney, Kathryn Oliver, and Adam Wellstead (2016) ‘To Bridge the Divide between Evidence and Policy: Reduce Ambiguity as Much as Uncertainty’, Public Administration Review, 76, 3, 399–402 DOI:10.1111/puar.12555 PDF

1 Comment

Filed under Evidence Based Policymaking (EBPM), Folksy wisdom, public policy, Storytelling, UK politics and policy

Can Westminster take back control after Brexit?

All going well, this discussion will be in a box in Chapter 8 of Understanding Public Policy 2nd ed.

“The ‘Brexit’ referendum was dominated by a narrative of taking back control of policy and policy making. Control of policy would allow the UK government to make profound changes to immigration and spending. Control of policymaking would allow Parliament and the public to hold the UK government directly to account, in contrast to a more complex and distant EU policy process less subject to direct British scrutiny.

Such high level political debate is built on the false image of a small number of elected policymakers – and the Prime Minister in particular – responsible for the outcomes of the policy process.

There is a strange disconnect between the ways in which elected politicians and elected policymakers describe UK policymaking. Ministers have mostly given up the language of control; modern manifestos no longer make claims – such as to secure ‘full employment’ or eradicate health inequalities – that suggest they control the economy or can solve problems by providing public services. Yet, much Brexit rhetoric suggests that a vote to leave the EU will put control back in the hands of ministers to solve major problems.

The main problem with the latter way of thinking is that it is rejected continuously in the modern literature on policymaking. Policymaking is multi-centric: responsibility for outcomes is spread across many levels and types of government, to the extent that it is not possible to simply know who is in charge and to blame.

Some multi-level governance (MLG) relates to the choice to share power with EU, devolved, and local policymaking organisations.

However, most MLG is necessary because ministers do not have the cognitive or coordinative capacity to control policy outcomes.

They can only pay attention to a tiny proportion of their responsibilities, and have to delegate the rest. Most decisions are taken in their name but without their intervention. They occur within a policymaking environment over which ministers have limited knowledge and control.

The problem with using Brexit as a lens through which to understand British politics is that it emphasises the choice to no longer spread power across a political system, without acknowledging the necessity of doing so.

Our understanding of the future of UK policy and policymaking is incomplete without a focus on the concepts and evidence that help us understand why UK ministers must accept their limitations and act accordingly.

Yet, clearly the Westminster model archetype remains important even if it does not exist (Duggett, 2009). Policy studies have challenged successfully its image of central control, but, the model’s importance resides in its rhetorical power in wider politics when people maintain a simple argument during general election and referendum debates: we know who is – or should be – in charge. This perspective has a profound effect on the ways in which policymakers defend their actions, and political actors compete for votes, even when it is ridiculously misleading (Rhodes, 2013; Bevir, 2013)”

See also Policy Concepts in 1000 Words: the Westminster Model and Multi-level Governance

Leave a comment

Filed under POLU9UK, public policy, UK politics and policy

Understanding Public Policy 2nd edition?

Almost. I have done a full draft that I will redraft one more time following external feedback and review (then during copy-editing). I am hoping that you might also read some of it and give me feedback, if only to point out big mistakes before it is too late. To be honest, by this stage, I won’t be adding major new sections or chapters (and I no longer want to read this thing), but please let me know if there are big gaps that I should fill in the third edition.

I have included below the introduction and conclusion (and each chapter should also have its own entry (or multiple entries) in the 1000 Words and 500 Words series) and invite you to get in touch – via email or Twitter DM – if you would like a copy of the whole thing.

Preface

Chapter 1 Introduction to policy and policymaking

Chapter 13 Conclusion

New references

Old references

If you would like to see the likely cover:

2nd ed cover

 

 

 

1 Comment

Filed under 1000 words, 500 words, agenda setting, Evidence Based Policymaking (EBPM), Policy learning and transfer, public policy

Policy in 500 Words: The advocacy coalition framework

Here is the ACF story.

People engage in politics to turn their beliefs into policy. They form advocacy coalitions with people who share their beliefs, and compete with other coalitions. The action takes place within a subsystem devoted to a policy issue, and a wider policymaking process that provides constraints and opportunities to coalitions.

The policy process contains multiple actors and levels of government. It displays a mixture of intensely politicized disputes and routine activity. There is much uncertainty about the nature and severity of policy problems. The full effects of policy may be unclear for over a decade. The ACF sums it up in the following diagram:

acf

Policy actors use their beliefs to understand, and seek influence in, this world. Beliefs about how to interpret the cause of and solution to policy problems, and the role of government in solving them, act as a glue to bind actors together within coalitions.

If the policy issue is technical and humdrum, there may be room for routine cooperation. If the issue is highly charged, then people romanticise their own cause and demonise their opponents.

The outcome is often long-term policymaking stability and policy continuity because the ‘core’ beliefs of coalitions are unlikely to shift and one coalition may dominate the subsystem for long periods.

There are two main sources of change.

  1. Coalitions engage in policy learning to remain competitive and adapt to new information about policy. This process often produces minor change because coalitions learn on their own terms. They learn how to retain their coalition’s strategic advantage and use the information they deem most relevant.
  2. ‘Shocks’ affect the positions of coalitions within subsystems. Shocks are the combination of external events – such as the election of a new government with different ideas, or the effect of socio-economic change – and the reaction by coalitions. Events may prompt major change as members of a dominant coalition question their beliefs in the light of new evidence (internal shock). Or, another coalition may adapt more readily to its new policy environment and exploit events to gain competitive advantage (external shock).

The ACF began as the study of US policymaking, focusing largely on environmental issues. It has changed markedly to reflect the widening of ACF scholarship to new policy areas, political systems, and methods.

For example, the flow diagram’s reference to the political system’s long term coalition opportunity structures is largely the response to insights from comparative international studies:

  • A focus on the ‘degree of consensus needed for major policy change’ reflects applications in Europe that highlighted the important of proportional electoral systems
  • A focus on the ‘openness of the political system’ partly reflects applications to countries without free and fair elections, and/ or systems that do not allow people to come together easily as coalitions to promote policy change.

As such, like all theories in this series, the ACF discusses elements that it would treat as (a) universally applicable, such as the use of beliefs to address bounded rationality, and (b) context-specific, such as the motive and opportunity of specific people to organize collectively to translate their beliefs into policy.

See also:

The 500 and 1000 Words series

Why Advocacy Coalitions Matter and How to Think about Them

Three lessons from a comparison of fracking policy in the UK and Switzerland

Bonus material

Scottish Independence and the Devil Shift

Image source: Weible, Heikkila, Ingold, and Fischer (2016: 6)

 

 

 

1 Comment

Filed under 500 words, public policy

Policy in 500 Words: Ecology of Games

The ‘Ecology of Games Framework’ (EG) combines insights from many approaches to analyze ‘institutional complexity’ and ‘complex institutional systems’.

The focus is on actors learning how to secure ‘mutually beneficial outcomes’, cooperating to produce and deliver agreed solutions, and bargaining within a system over which no actor has control. Therefore, it is worth reading the posts on game theory, the IAD, and SES first (especially if, like me, you associated ‘game’ with tig, then Monopoly, then The Wire).

Dz3Hmy2VsAAoO72.png

Insights from three key approaches

EG connects Norton’s ‘ecology of games’, the IAD, and insights from complexity theory to reinforce the idea that institutional arrangements are not simple and orderly.

In simple games, we need only analyse the interaction between a small number of actors with reference to one set of self-contained rules providing clear sanctions or payoffs. In real world policymaking, many different games take place at the same time in different venues.

Some policy games may be contained within a geographical area – such as California – but there are no self-contained collective action problems:

  • Examples such as ‘biodiversity’, ‘ecology’ or ‘environmental’ policies command a collection of interdependent policies relating to issues like local planning, protected species, water management, air pollution, transport, energy use, and contributors to such policies or policy problems in other areas of government (such as public services).
  • Each contributor to policy may come from different institutions associated with many policymaking venues spread across many levels and types of government.

Consequently, many games interact with each other. The same actor might participate in multiple games subject to different rules. Further, each game produces ‘externalities’ for the others; the ‘payoffs’ to each game are connected and complicated.

A focus on ‘complex adaptive systems’ suggests that central governments do not have the resources to control – or understand fully – interaction at this frequency and scale. Rather, policymaking influences are:

  • Internal to the game, when actors (a) follow and shape the rules of each institution, and (b) learn through trial and error.
  • External to the game, when physical resources change, or central levels of government change the resources of local actors.

Insights from the wider literature

The EG brings in wider insights – from theories in the 500 and 1000 Words series – to analyse this process. Examples include:

Consequently, we have come a long way from simple assumptions about human behaviour outlined in our first post in this series.

As with the IAD, the EG emphasis is on (a) finding solutions to complex (largely environmental) policy problems, with reference to (b) initiatives consistent with self-organising systems such as ‘collaborative governance’. Like most posts in this series, it rejects a naïve attachment to a single powerful central government. Policymaking is multi-centric, and solutions to complex problems will emerge in that context.

See also:

Policy Concepts in 1000 Words: the Institutional Analysis and Development Framework (IAD) and Governing the Commons

Policy Concepts in 1000 Words: it’s time for some game theory

Policy in 500 Words: the Social-Ecological Systems Framework

Policy Concepts in 1000 Words: Rational Choice and the IAD (the older post for the 1st edition)

Policy Concepts in 1000 Words: Multi-centric Policymaking

How to Navigate Complex Policy Designs

How can governments better collaborate to address complex problems?

 

3 Comments

Filed under 500 words, public policy

Policy in 500 Words: the Social-Ecological Systems Framework

Think of the SES framework as the combination of:

  • the IAD approach to analyzing ‘the commons’, and
  • ecological sciences approaches to ‘complex social-ecological systems’

The result is a framework that resembles CPR studies in key respects. Ostrom’s 2009 article in Science provides a visual emphasis on the interactions between ‘first-level’ concepts including users, their governance system, resource system (such as a protected park) and resource units (such as its trees):

SES Science Ostrom

It also raises similar questions, such as ‘When will the users of a resource invest time and energy to avert “a tragedy of the commons”’?

It answers them with reference to ‘second level’ concepts describing factors that encourage users to (a) value long term sustainability and (b) self-organize to secure this outcome. This table summarizes many of them:

SES table 7.2

Note that Ostrom describes their effect as indicative because, ‘As in most complex systems, the variables interact in a nonlinear fashion …Simple blueprint policies do not work’.

As a result, we have a super-complicated framework to help us understand an even more super-complicated world. For some, the SES framework serves to ‘diagnose’ the sustainability of social-ecological systems and explore the prospect of more effective self-organisation to manage resources. However, as with the IAD, effective use of the framework itself requires a fair amount of immersion in the language of analysis.

See also:

Policy Concepts in 1000 Words: the Institutional Analysis and Development Framework (IAD) and Governing the Commons

Policy in 500 Words: Ecology of Games

Policy Concepts in 1000 Words: it’s time for some game theory

Policy Concepts in 1000 Words: Rational Choice and the IAD (the older post for the 1st edition)

Policy Concepts in 100 Words: Multi-centric Policymaking

How to Navigate Complex Policy Designs

How can governments better collaborate to address complex problems?

 

3 Comments

Filed under 500 words, public policy

Policy Concepts in 1000 Words: the Institutional Analysis and Development Framework (IAD) and Governing the Commons

The IAD provides a language, and way of thinking, about the ways in which different institutions foster collective action. The language is so complicated that I have cheated by summarising key terms in this box (and describing polycentric governance in a different post) to stay within the 1000 words limit:

IAD box 2.2.19

Governing the Commons

For me, the best way to understand the IAD is through the lens of Governing the Commons (and the research agenda it inspired), which explains how to rethink ‘tragedies of the commons’ and encourage better management of common pool resources (CPRs).

Ostrom rejects the uncritical use of rational choice games to conclude – too quickly – that disastrous collective action problems are inevitable unless we ‘privatize’ commons or secure major government intervention (which is tricky anyway when global problems require international cooperation). The tragedy of the commons presents a too-bleak view of humanity, in which it would be surprising to find cooperation even when the fate of the world is in human hands.

Alternatively, what if there is evidence that people often work collectively and effectively without major coercion? People are social beings who share information, build trust by becoming known as reliable and predictable, and come together to produce, monitor and enforce rules for the group’s benefit. They produce agreements with each other that could be enforced if necessary.

The IAD helps us analyse these cooperative arrangements. Ostrom describes 8 ‘design principles’ of enduring and effective CPR management shared by many real world examples:

  1. CPRs have clear boundaries. Users know what they are managing, and can identify legitimate users.
  2. The rules suit local conditions. Users know what they (a) are expected to contribute to management and (b) receive from CPRs.
  3. The actors affected by the rules help shape them (at low cost).
  4. CPR monitors are users or accountable to users. They monitor (a) the conduct of users and (b) the state of the CPR. The costs of mutual monitoring are low, and their consequences felt quickly.
  5. The penalties for rule-breaking are low if the choice is a one-off and understandable under the circumstances (to avoid alienating the user). The penalties are high if the choice is part of a pattern which makes other users feel like ‘suckers’, or if rule-breaking would be catastrophic.
  6. Conflict resolution is frequent, rapid and low cost.
  7. Users have the right to self-organise without too much outside interference.
  8. Many projects are connected geographically and at different scales – local, regional, national – in ways that do not undermine individual projects.

These design principles help explain why some communities manage CPRs successfully. They allow users to share the same commitment and expect the long-term benefits to be worthwhile.

However, Ostrom stressed that there is no blueprint – no hard and fast rules – to CPR management. There are three particular complications:

  1. Trust

Good management requires high trust to encourage norms of reciprocity. Trust is crucial to minimizing the costs of compliance monitoring and enforcement. Trust may develop when participants communicate regularly, share an understanding of their common interests, reciprocate each other’s cooperation, and have proven reliable in the past.

Design principles are important to developing trust and solidarity, but so are evolutionary’ changes to behaviour. Actors have often learned about rule efficacy – to encourage cooperation and punish opportunism – through trial-and-error over a long period, beginning with simple, low-cost operational rules producing quick wins.

  1. Rules, rules on rules, more rules, then even more rules

Institutions contain a large, complicated set of rules that serve many different purposes, and need to be understood and analysed in different ways.

Different purposes include:

  • how many actors are part of an action situation, and the role they play
  • what they must/ must not do
  • who is eligible to participate
  • who can move from one role to another
  • who controls membership, and how
  • how many participants are involved in a choice
  • what will happen if there is no agreement
  • how to manage and communicate information
  • the rewards or sanctions
  • the range of acceptable actions or outcomes from action.

 

We also need to analyse the relative costs and simplicity of different rules, and the rules about the other rules, including

  • ‘operational’ rules on day-to-day issues (such as specific payoffs/ sanctions for behaviour)
  • ‘collective choice’ rules about how to make those rules
  • ‘constitutional’ rules on who can decide those rules and who can monitor and enforce, and
  • ‘metaconstitutional’ analysis of how to design these constitutions with reference to the wider political and social context.

 

  1. The world is too complex to break down into simple pieces

By now, you may be thinking that the IAD – and analysis of resource management – is complicated. This is true, partly because each case study – of the physical conditions and social practices regarding resource management – is different in some way. We can use the IAD to compare experiences, but accept that a profoundly successful scheme in one context may fail miserably in another.

Simplicity versus complexity: the world is complex, but should our analysis follow suit?

Indeed, this is why we need to think about rational choice games and the IAD simultaneously, to understand the analytical trade-offs.

Game theory laboratory experiments – built on simple rules and relatively small numbers of parameters – produce parsimonious analysis and results that we can understand relatively easily.

We may reject simple games as unrealistic, but what if we take this criticism to its extreme?

IAD in-depth field studies embrace complexity to try to understand the key dimensions of each study’s context. When we put them all together, there are too many concepts, variables, global applications, and variations-by-context, to contain in a simple theory.

The IAD addresses this trade off by offering a language to help organize research, encouraging people to learn it then use it to apply many different theories to explain different parts of the whole picture.

In other words, it is OK to reject simple models as unrealistic, but to embrace real-world complexity may require a rather complicated language.

See also:

Policy Concepts in 1000 Words: it’s time for some game theory

Policy Concepts in 1000 Words: Rational Choice and the IAD (the older post for the 1st edition)

Policy Concepts in 100 Words: Multi-centric Policymaking

Policy in 500 Words: the Social-Ecological Systems Framework

Policy in 500 Words: Ecology of Games

How to Navigate Complex Policy Designs

How can governments better collaborate to address complex problems?

 

 

3 Comments

Filed under 1000 words, public policy