Policy Analysis in 750 Words: How to deal with ambiguity

This post forms one part of the Policy Analysis in 750 words series. It draws on this 500 Words post, then my interpretation of co-authored work with Drs Emily St Denny and John Boswell (which I would be delighted to share if it gets published). It trails off at the end.

In policy studies, ambiguity describes the ability to entertain more than one interpretation of a policy problem. There are many ways to frame issues as problems. However, only some frames receive high policymaker attention, and policy change relates strongly to that attention. Resolving ambiguity in your favour is the prize.

Policy studies focus on different aspects of this dynamic, including:

1. The exercise of power, such as of the narrator to tell stories and the audience to engage with or ignore them.
2. Policy learning, in which people collaborate (and compete) to assign concrete meaning to abstract aims.
3. A complex process in which many policymakers and influencers are cooperating/ competing to define problems in many policymaking centres.

They suggest that resolving ambiguity affects policy in different ways, to influence the:

• Single policy instrument produced by one centre,
• Accumulation of separate instruments emerging from many centres, or
• Outcome of collaboration across those centres.

The latter descriptions, reflecting multi-centric policymaking, seem particularly relevant to major contemporary policy problems – such as global public health and climate crises – in which cooperation across (and outside of) many levels and types of government is essential.

Resolving ambiguity in policy analysis texts

This context helps us to interpret common (Step 1) advice in policy analysis textbooks: define a policy problem for your client, using your skills of research and persuasion but tailoring your advice to your client’s interests and beliefs. Yet, gone are the mythical days of elite analysts communicating to a single core executive in charge of formulating and implementing all policy instruments. Many analysts engage with many centres producing (or co-producing) many instruments. Resolving ambiguity in one centre does not guarantee the delivery of your aims across many.

Two ways to resolve ambiguity in policy analysis

Classic debates would highlight two different responses:

• ‘Top down’ accounts see this issue through the lens of a single central government, examining how to reassert central control by minimising implementation gaps.

Policy analysis may focus on (a) defining the policy problem, and (b) ensuring the implementation of its solution.

• ‘Bottom up’ accounts identify the inevitability (and legitimacy) of policy influence in multiple centres. Policy analysis may focus on how to define the problem in cooperation with other centres, or to set a strategic direction and encourage other centres to make sense of it in their context.

This terminology went out of fashion, but note the existence of each tendency in two ideal-type approaches to contemporary policy problems:

1. Centralised and formalised approaches.

Seek clarity and order to address urgent policy problems. Define the policy problem clearly, translate that definition into strategies for each centre, and develop a common set of effective ‘tools’ to ensure cooperation and delivery.

Policy analysis may focus on technical aspects, such as how to create a fine-detail blueprint for action, backed by performance management and accountability measures that tie actors to specific commitments.

The tagline may be: ambiguity is a problem to be solved, to direct policy actors towards a common goal.

2. Decentralised, informal, collaborative approaches.

Seek collaboration to make sense of, and address, problems. Reject a single definition of the problem, encourage actors in each centre (or in concert) to deliberate to make sense of problems together, and co-create the rules to guide a continuous process of collective behaviour.

Policy analysis may focus on how to contribute to a collaborative process of sense-making and rule-making.

The tagline may be: ambiguity presents an opportunity to energise policy actors, to harness the potential for innovation arising from deliberation.

Pick one approach and stick with it?

Describing these approaches in such binary terms makes the situation – and choice between approaches – look relatively straightforward. However, note the following issues:

• Many policy sectors (and intersectoral agendas) are characterised by intense disagreement on which choice to make. These disagreements intersect with others (such as when people seek not only transformative policy change to solve global problems, but also equitable process and outcomes).
• Some sectors seem to involve actors seeking the best of both worlds (centralise and localise, formalise and deliberate) without recognising the trade-offs and dilemmas that arise.
• I have described these options as choices, but did not establish if anyone is in the position to make or contribute to that choice.

In that context, resolving ambiguity in your favour may still be the prize, but where would you even begin?

Further reading

Well, that was an unsatisfying end to the post, eh? Maybe I’ll write a better one when some things are published. In the meantime, some of these papers and posts explore some of these issues:

• Policy Concepts in 1000 Words: the Institutional Analysis and Development Framework (IAD) and Governing the Commons – the IAD is your go-to framework to study collaborative governance
• Improving policy implementation through collaborative policymaking
• The future of public health policymaking after COVID-19: lessons from Health in All Policies – identifies these dilemmas in relation to health equity strategies
• The future of education equity policy: ‘neoliberal’ versus ‘social justice’ approaches – identifies equivalent debates in education
• A systematic review of energy systems – tries to discourage people from thinking that we can solve these issues with a vague reference to ‘systems thinking’
• The contested relationship between governance and evidence – relates ‘evidence based’ to ‘collaborative’ approaches to governance (as does Who should be involved in the process of policy analysis?)
• This page  describes a book and many posts on ‘prevention’ policy. We complain that governments use the phrase ‘prevention is better than cure’ without defining prevention, and that they want centralised and decentralised approaches to ‘preventive policymaking’.
• The myth of ‘evidence based policymaking’ in a decentred state argues that policymakers don’t really know what they want, and couldn’t get it if they tried.
• The politics of policy design – tries to make you feel good about never really resolving these dilemmas.

Policy Analysis in 750 Words: complex systems and systems thinking

This post forms one part of the Policy Analysis in 750 words series overview and connects to previous posts on complexity. The first 750 words tick along nicely, then there is a picture of a cat hanging in there baby to signal where it can all go wrong. I updated it (22.6.20) to add category 11 then again (30.9.20) when I realised that the former category 11 was a lot like 6.

There are a million-and-one ways to describe systems and systems thinking. These terms are incredibly useful, but also at risk of meaning everything and therefore nothing (compare with planning and consultation).

Let’s explore how the distinction between policy studies and policy analysis can help us clarify the meaning of ‘complex systems’ and ‘systems thinking’ in policymaking.

For example, how might we close a potentially large gap between these two stories?

1. Systems thinking in policy analysis.

• Avoid the unintended consequences of too-narrow definitions of problems and processes (systems thinking, not simplistic thinking).
• If we engage in systems thinking effectively, we can understand systems well enough to control, manage, or influence them.

2. The study of complex policymaking systems.

• Policy emerges from complex systems in the absence of: (a) central government control and often (b) policymaker awareness.
• We need to acknowledge these limitations properly, to accept our limitations, and avoid the mechanistic language of ‘policy levers’ which exaggerate human or government control.

Complex systems thinking could be the future of policymaking.

But what does it even mean?https://t.co/I5DwY884Vf

— Apolitical (@apoliticalco) March 19, 2019

See also: Systems science and systems thinking for public health: a systematic review of the field

Six meanings of complex systems in policy and policymaking

Let’s begin by trying to clarify many meanings of complex system and relate them to systems thinking storylines.

For example, you will encounter three different meanings of complex system in this series alone, and each meaning presents different implications for systems thinking:

1. A complex policymaking system

Policy outcomes seem to ‘emerge’ from policymaking systems in the absence of central government control. As such, we should rely less on central government driven targets (in favour of local discretion to adapt to environments), encourage trial-and-error learning, and rethink the ways in which we think about government ‘failure’ (see, for example, Hallsworth on ‘system stewardship’, the OECD on ‘Systemic Thinking for Policy Making‘, and this thread)

• Systems thinking is about learning and adapting to the limits to policymaker control.

“Outcomes are emergent properties of complex systems that we simply cannot force through more rigorous measuring” – @JohnBurgoyne07 #CFGCommAcc20 https://t.co/JCCwMxN0nY

— CPI (@CPI_foundation) February 11, 2020

2. Complex policy problems

Dunn (2017:  73) describes the interdependent nature of problems:

“Subjectively experienced problems – crime, poverty, unemployment, inflation, energy, pollution, health, security – cannot be decomposed into independent subsets without running the risk of producing an approximately right solution to the wrong problem. A key characteristic of systems of problems is that the whole is greater – that is, qualitatively different – than the simple sum of its parts” (contrast with Meltzer and Schwartz on creating a ‘boundary’ to make problems seem solveable).

• Systems thinking is about addressing policy problems holistically.

3. Complex policy mixes

What we call ‘policy’ is actually a collection of policy instruments. Their overall effect is ‘non-linear’, difficult to predict, and subject to emergent outcomes, rather than cumulative (compare with Lindblom’s hopes for incrementalist change).

This point is crucial to policy analysis: does it involve a rethink of all instruments, or merely add a new instrument to the pile?

• Systems thinking is about anticipating the disproportionate effect of a new policy instrument.

These three meanings are joined by at least three more (from Munro and Cairney on energy systems):

4. Socio-technical systems (Geels)

Used to explain the transition from unsustainable to sustainable energy systems.

• Systems thinking is about identifying the role of new technologies, protected initially in a ‘niche’, and fostered by a supportive ‘social and political environment’.

5. Socio-ecological systems (Ostrom)

Used to explain how and why policy actors might cooperate to manage finite resources.

• Systems thinking is about identifying the conditions under which actors develop layers of rules to foster trust and cooperation.

6. Performing the metaphor of systems

Governments often use the language of complex systems – rather loosely – to indicate an awareness of the interconnectedness of things. They often perform systems thinking to give the impression that they are thinking and acting differently, but without backing up their words with tangible changes to policy instruments.

• Systems thinking is about projecting the sense that (a) policy and policymaking is complicated, but (b) governments can still look like they are in control.

Four more meanings of systems thinking

Now, let’s compare these storylines with a small sample of wider conceptions of systems thinking:

7. The old way of establishing order from chaos

Based on the (now-diminished) faith in science and rational management techniques to control the natural world for human benefit (compare Hughes and Hughes on energy with Checkland on ‘hard’ v ‘soft’ systems approaches, then see What you need as an analyst versus policymaking reality and Radin on the old faith in rationalist governing systems).

• Systems thinking was about the human ability to turn potential chaos into well-managed systems (such as ‘large technical systems’ to distribute energy)

8. The new way of accepting complexity but seeking to make an impact

Based on the idea that we can identify ‘leverage points’, or the places that help us ‘intervene in a system’ (see Meadows then compare with Arnold and Wade).

• Systems thinking is about the human ability to use a small shift in a system to produce profound changes in that system.

9. A way to rethink cause-and-effect

Based on the idea that current research methods are too narrowly focused on linearity rather than the emergent properties of systems of behaviour (for example, Rutter et al on how to analyse the cumulative effect of public health interventions, and Greenhalgh on responding more effectively to pandemics).

• Systems thinking is about rethinking the ways in which governments, funders, or professions conduct policy-relevant research on social behaviour.

This is a brilliant paper https://t.co/dRnN651jfQ https://t.co/CyqHZs0453

— Professor Paul Cairney (@CairneyPaul) July 1, 2020

10. A way of thinking about ourselves

Embrace the limits to human cognition, and accept that all understandings of complex systems are limited.

• Systems thinking is about developing the ‘wisdom’ and ‘humility’ to accept our limited knowledge of the world.

HT @suzannewTHF … a neat graphic explanation of complexity theory

"we need to let go of our cultural obsession with prediction and control and and develop the wisdom and humility to sit with multiplicity"https://t.co/0glbccWm2j?@harryrutter @KarabiGlobal @tim_hobbs_lab

— Jo Bibby (@JoBibbyTHF) December 19, 2019

How can we clarify systems thinking and use it effectively in policy analysis?

Now, imagine you are in a room of self-styled systems thinkers, and that no-one has yet suggested a brief conversation to establish what you all mean by systems thinking. I reckon you can make a quick visual distinction by seeing who looks optimistic.

I’ll be the morose-looking guy sitting in the corner, waiting to complain about ambiguity, so you would probably be better off sitting next to Luke Craven who still ‘believes in the power of systems thinking’.

If you can imagine some amalgam of these pessimistic/ optimistic positions, perhaps the conversation would go like this:

1. Reasons to expect some useful collaboration.

Some of these 10 discussions seem to complement each other. For example:

• We can use 3 and 9 to reject one narrow idea of ‘evidence-based policymaking’, in which the focus is on (a) using experimental methods to establish cause and effect in relation to one policy instrument, without showing (b) the overall impact on policy and outcomes (e.g. compare FNP with more general ‘families’ policy).
• 1-3 and 10 might be about the need for policy analysts to show humility when seeking to understand and influence complex policy problems, solutions, and policymaking systems.

In other words, you could define systems thinking in relation to the need to rethink the ways in which we understand – and try to address – policy problems. If so, you can stop here and move on to the next post. There is no benefit to completing this post.

2. Reasons to expect the same old frustrating discussions based on no-one defining terms well enough (collectively) to collaborate effectively (beyond using the same buzzwords).

Although all of these approaches use the language of complex systems and systems thinking, note some profound differences:

Holding on versus letting go.

• Some are about intervening to take control of systems or, at least, make a disproportionate difference from a small change.
• Some are about accepting our inability to understand, far less manage, these systems.

Talking about different systems.

• Some are about managing policymaking systems, and others about social systems (or systems of policy problems), without making a clear connection between both endeavours.

For example, if you use approach 9 to rethink societal cause-and-effect, are you then going to pretend that you can use approach 7 to do something about it? Or, will our group have a difficult discussion about the greater likelihood of 6 (metaphorical policymaking) in the context of 1 (the inability of governments to control the policymaking systems we need to solve the problems raised by 9).

In that context, the reason that I am sitting in the corner, looking so morose, is that too much collective effort goes into (a) restating, over and over and over again, the potential benefits of systems thinking, leaving almost no time for (b) clarifying systems thinking well enough to move on to these profound differences in thinking. Systems thinking has not even helped us solve these problems with systems thinking.

See also:

Why systems thinkers and data scientists should work together to solve social challenges

Understanding Public Policy 2nd edition

All going well, it will be out in November 2019. We are now at the proofing stage.

I have included below the summaries of the chapters (and each chapter should also have its own entry (or multiple entries) in the 1000 Words and 500 Words series).

 

Policy in 500 Words: Ecology of Games

The ‘Ecology of Games Framework’ (EG) combines insights from many approaches to analyze ‘institutional complexity’ and ‘complex institutional systems’.

The focus is on actors learning how to secure ‘mutually beneficial outcomes’, cooperating to produce and deliver agreed solutions, and bargaining within a system over which no actor has control. Therefore, it is worth reading the posts on game theory, the IAD, and SES first (especially if, like me, you associated ‘game’ with tig, then Monopoly, then The Wire).

Insights from three key approaches

EG connects Norton’s ‘ecology of games’, the IAD, and insights from complexity theory to reinforce the idea that institutional arrangements are not simple and orderly.

In simple games, we need only analyse the interaction between a small number of actors with reference to one set of self-contained rules providing clear sanctions or payoffs. In real world policymaking, many different games take place at the same time in different venues.

Some policy games may be contained within a geographical area – such as California – but there are no self-contained collective action problems:

• Examples such as ‘biodiversity’, ‘ecology’ or ‘environmental’ policies command a collection of interdependent policies relating to issues like local planning, protected species, water management, air pollution, transport, energy use, and contributors to such policies or policy problems in other areas of government (such as public services).
• Each contributor to policy may come from different institutions associated with many policymaking venues spread across many levels and types of government.

Consequently, many games interact with each other. The same actor might participate in multiple games subject to different rules. Further, each game produces ‘externalities’ for the others; the ‘payoffs’ to each game are connected and complicated.

A focus on ‘complex adaptive systems’ suggests that central governments do not have the resources to control – or understand fully – interaction at this frequency and scale. Rather, policymaking influences are:

• Internal to the game, when actors (a) follow and shape the rules of each institution, and (b) learn through trial and error.
• External to the game, when physical resources change, or central levels of government change the resources of local actors.

Insights from the wider literature

The EG brings in wider insights – from theories in the 500 and 1000 Words series – to analyse this process. Examples include:

• Actors deal with bounded rationality primarily with reference to their ‘social tribal instincts’.
• They form coalitions with their allies to get what they want from different games
• They engage in ‘policy learning’ to update their policy knowledge and enhance their policymaking position.

Consequently, we have come a long way from simple assumptions about human behaviour outlined in our first post in this series.

As with the IAD, the EG emphasis is on (a) finding solutions to complex (largely environmental) policy problems, with reference to (b) initiatives consistent with self-organising systems such as ‘collaborative governance’. Like most posts in this series, it rejects a naïve attachment to a single powerful central government. Policymaking is multi-centric, and solutions to complex problems will emerge in that context.

See also:

Policy Concepts in 1000 Words: the Institutional Analysis and Development Framework (IAD) and Governing the Commons

Policy Concepts in 1000 Words: it’s time for some game theory

Policy in 500 Words: the Social-Ecological Systems Framework

Policy Concepts in 1000 Words: Rational Choice and the IAD (the older post for the 1^st edition)

Policy Concepts in 1000 Words: Multi-centric Policymaking

How to Navigate Complex Policy Designs

How can governments better collaborate to address complex problems?

Yay! I didn't realize that @CairneyPaul has summarized our work on the "ecology of games" in 500 words! Who needs another 7000 word article at this point? All you need is right here! https://t.co/Y5f9O4Gaub pic.twitter.com/CgELtL8CG7

— Mark Lubell (@EnvPolicyCenter) February 20, 2019

 

Policy Concept in 1000 Words: Multi-centric Policymaking

Many theories in this 1000 words series describe multiple policymaking venues. They encourage us to give up on the idea of an all-knowing, all-powerful national central government. Instead, there are many venues in which to make authoritative choices, each contributing to what we call policy.

The word ‘multi-centric’ (coined by Professor Tanya Heikkila, with me and Dr Matt Wood) does not suggest that every venue is of equal importance or power. Rather, it prompts us not to miss something important by focusing too narrowly on one single (alleged) centre of authority.

To some extent, multi-centric policymaking results from choice. Many federal political systems have constitutions that divide power between executive, legislative, and judicial branches, or give some protection to subnational governments. Many others have become ‘quasi-federal’ more organically, by sharing responsibilities with supranational and subnational governments. In such cases, there is explicit choice to distribute power and share responsibility for making policy (albeit with some competition to assert power or shuffle-off responsibility).

However, for the most part, this series helps explain the necessity of multi-centric policymaking with reference to two concepts:

1. Bounded rationality. Policymakers are only able to pay attention to – and therefore understand and seek to control – a tiny proportion of their responsibilities.
2. Complex policymaking environments. Policymakers operate in an environment over which they have limited understanding and even less control. It contains many policymakers and influencers spread across many venues, each with their own institutions, networks, ideas (and ways to frame policy), and responses to socio-economic context and events.

Both factors combine to provide major limits to single central government control. Elected policymakers deal with bounded rationality by prioritising some issues and, necessarily, delegating responsibility for the rest. Delegation may be inside or outside of central government.

1000 Words theories describing multi-centric government directly

Multi-level governance describes the sharing of power vertically, between many levels of government, and horizontally, between many governmental, quasi-non-governmental and non-governmental organisations. Many studies focus on the diffusion of power within specific areas like the European Union – highlighting choice – but the term ‘governance’ has a wider connection to the necessity of MLG.

For example, part of MLG’s origin story is previous work to help explain the pervasiveness of policy networks:

• Policymakers at the ‘top’ ask bureaucrats to research and process policy on their behalf
• Civil servants seek information and advice from actors outside of government
• They often form enduring relationships built on factors such as trust.
• Such policymaking takes place away from a notional centre – or at least a small core executive – and with limited central attention.

Polycentricity describes (a) ‘many decision centers’ with their own separate authority, (b) ‘operating under an overarching set of rules’, but with (c) a sense of ‘spontaneous order’ in which no single centre controls the rules or outcomes. Polycentric governance describes ‘policymaking centres with overlapping authority; they often work together to make decisions, but may also engage in competition or conflict’.

This work on polycentric governance comes primarily from the Institutional Analysis and Development (IAD) framework that helps compare the effectiveness of institutions designed to foster collective action. For example, Ostrom identifies the conditions under which non-governmental institutions can help manage ‘common pool resources’ effectively, while IAD-inspired studies of municipal governance examine how many ‘centres’ can cooperate as or more effectively than a single central government.

Complexity theory has a less clear origin story, but we can identify key elements of complex systems:

• They are greater than the sum of their parts
• They amplify or dampen policymaking activity, so the same action can have a maximal or no impact
• Small initial choices can produce major long term momentum
• There are regularities of behaviour despite the ever-present potential for instability
• They exhibit ‘emergence’. Local outcomes seem to defy central direction.

Systems contain many actors interacting with many other actors. They follow and reproduce rules, which help explain long periods of regular behaviour. Or, many actors and rules collide when they interact, producing the potential for many bursts of instability. In each case, the system is too large and unpredictable to be subject to central control.

1000 Words theories describing multi-centric government indirectly

Many other theories in this series describe multi-centric policymaking – or aspects of it – without using this term directly. Examples include:

Punctuated equilibrium theory suggests that (a) policymakers at the ‘centre’ of government could pay attention to, and influence, most issues, but (b) they can only focus on a small number and must ignore the rest. Very few issues reach the ‘macropolitical’ agenda. Multiple policymaking organisations process the rest out of the public spotlight.

Multiple streams analysis turns the notion of a policy cycle on its head, and emphasises serendipity over control. Policy does not change until three things come together at the right ‘window of opportunity’: attention to a problem rises, a feasible solution exists, and policymakers have the motive and opportunity to act. Modern MSA studies show that such windows exist at multiple levels of government.

The advocacy coalition framework describes the interaction between many policymakers and influencers. Coalitions contain actors from many levels and types of government, cooperating and competing within subsystems (see networks). They are surrounded by a wider context – over which no single actor has direct control – that provides the impetus for ‘shocks’ to each coalition.

In such accounts, the emphasis is on high levels of complexity, the potential for instability, and the lack of central control over policymaking and policy outcomes. The policy process is not well described with reference to a small group of policymakers at the heart of government.

The implications for strategy and accountability

Making Policy in a Complex World explores the implications of multi-centric policymaking for wider issues including:

1. Accountability. How do we hold elected policymakers to account if we no longer accept that there is a single government to elect and scrutinise? See MLG for one such discussion.
2. Strategy. How can people act effectively in a policy process that seems too complex to understand fully? See this page on ‘evidence based policymaking’

Further Reading:

Key policy theories and concepts in 1000 words

Policy in 500 words

5 images of the policy process

[right click for the audio]

Making Policy in a Complex World (preview PDF ) also provides a short explainer of key terms as follows: