Tag Archives: Complex systems

How can political actors take into account the limitations of evidence-based policy-making? 5 key points

These notes are for my brief panel talk at the European Parliament-European University Institute ‘Policy Roundtable’: Evidence and Analysis in EU Policy-Making: Concepts, Practice and Governance. As you can see from the programme description, the broader theme is about how EU institutions demonstrate their legitimacy through initiatives such as stakeholder participation and evidence-based policymaking (EBPM). So, part of my talk is about what happens when EBPM does not exist.

The post is a slightly modified version of my (recorded) talk for Open Society Foundations (New York) but different audiences make sense of these same basic points in very different ways.

  1. Recognise that the phrase ‘evidence-based policy-making’ means everything and nothing

The main limitation to ‘evidence-based policy-making’ is that no-one really knows what it is or what the phrase means. So, each actor makes sense of EBPM in different ways and you can tell a lot about each actor by the way in which they answer these questions:

  • Should you use restrictive criteria to determine what counts as ‘evidence’? Some actors equate evidence with scientific evidence and adhere to specific criteria – such as evidence-based medicine’s hierarchy of evidence – to determine what is scientific. Others have more respect for expertise, professional experience, and stakeholder and service user feedback as sources of evidence.
  • Which metaphor, evidence based or informed is best? ‘Evidence based’ is often rejected by experienced policy participants as unrealistic, preferring ‘informed’ to reflect pragmatism about mixing evidence and political calculations.
  • How far do you go to pursue EBPM? It is unrealistic to treat ‘policy’ as a one-off statement of intent by a single authoritative actor. Instead, it is made and delivered by many actors in a continuous policymaking process within a complicated policy environment (outlined in point 3). This is relevant to EU institutions with limited resources: the Commission often makes key decisions but relies on Member States to make and deliver, and the Parliament may only have the ability to monitor ‘key decisions’. It is also relevant to stakeholders trying to ensure the use of evidence throughout the process, from supranational to local action.
  • Which actors count as policymakers? Policymaking is done by ‘policymakers’, but many are unelected and the division between policymaker/ influencer is often unclear. The study of policymaking involves identifying networks of decision-making by elected and unelected policymakers and their stakeholders, while the actual practice is about deciding where to draw the line between influence and action.
  1. Respond to ‘rational’ and ‘irrational’ thought.

Comprehensive rationality’ describes the absence of ambiguity and uncertainty when policymakers know what problem they want to solve and how to solve it, partly because they can gather and understand all information required to measure the problem and determine the effectiveness of solutions.

Instead, we talk of ‘bounded rationality’ and how policymakers deal with it. They employ two kinds of shortcut: ‘rational’, by pursuing clear goals and prioritizing certain kinds and sources of information, and ‘irrational’, by drawing on emotions, gut feelings, deeply held beliefs, habits, and familiarity, make decisions quickly.

I say ‘irrational’ provocatively, to raise a key strategic question: do you criticise emotional policymaking (describing it as ‘policy based evidence’) and try somehow to minimise it, adapt pragmatically to it, or see ‘fast thinking’ more positively in terms of ‘fast and frugal heuristics’? Regardless, policymakers will think that their heuristics make sense to them, and it can be counterproductive to simply criticise their alleged irrationality.

  1. Think about how to engage in complex systems or policy environments.

Policy cycle’ describes the idea that there is a core group of policymakers at the ‘centre’, making policy from the ‘top down’, and pursuing their goals in a series of clearly defined and well-ordered stages, such as: agenda setting, policy formulation, legitimation, implementation, and evaluation. In this context, one might identify how to influence a singular point of central government decision.

However, a cycle model does not describe policymaking well. Instead, we tend to identify the role of less ordered and more unpredictable complex systems, or policy environments containing:

  • A wide range of actors (individuals and organisations) influencing policy at many levels of government. Scientists and practitioners are competing with many actors to present evidence in a particular way to secure a policymaker audience.
  • A proliferation of rules and norms maintained by different levels or types of government. Support for particular ‘evidence based’ solutions varies according to which organisation takes the lead and how it understands the problem.
  • Important relationships (‘networks’) between policymakers and powerful actors. Some networks are close-knit and difficult to access because bureaucracies have operating procedures that favour particular sources of evidence and some participants over others, and there is a language – indicating what ways of thinking are in good ‘currency’ – that takes time to learn.
  • A tendency for certain ‘core beliefs’ or ‘paradigms’ to dominate discussion. Well-established beliefs provide the context for policymaking: new evidence on the effectiveness of a policy solution has to be accompanied by a shift of attention and successful persuasion.
  • Policy conditions and events that can reinforce stability or prompt policymaker attention to lurch at short notice. In some cases, social or economic ‘crises’ can prompt lurches of attention from one issue to another, and some forms of evidence can be used to encourage that shift, but major policy change is rare.

For stakeholders, an effective engagement strategy is not straightforward: it takes time to know ‘where the action is’, how and where to engage with policymakers, and with whom to form coalitions. For the Commission, it is difficult to know what will happen to policy after it is made (although we know the end point will not resemble the starting point). For the Parliament, it is difficult even to know where to look.

  1. Recognise that EBPM is only one of many legitimate ‘good governance’ principles.

There are several principles of ‘good’ policymaking and only one is EBPM. Others relate to the value of pragmatism and consensus building, combining science advice with public values, improving policy delivery by generating ‘ownership’ of policy among key stakeholders, and sharing responsibility with elected national and local policymakers.

Our choice of which principle and forms of evidence to privilege are inextricably linked. For example, some forms of evidence gathering seem to require uniform models and limited local or stakeholder discretion to modify policy delivery. The classic example is a programme whose value is established using randomised control trials (RCTs). Others begin with local discretion, seeking evidence from stakeholders, professional groups, service user and local practitioner experience. This principle seems to rule out the use of RCTs, at least as a source of a uniform model to be rolled out and evaluated. Of course, one can try to pursue both approaches and a compromise between them, but the outcome may not satisfy advocates of either approach to EBPM or help produce the evidence that they favour.

  1. Decide how far you’ll go to achieve EBPM.

These insights should prompt us to see how far we are willing, and should, go to promote the use of certain forms of evidence in policymaking

  • If policymakers and the public are emotional decision-makers, should we seek to manipulate their thought processes by using simple stories with heroes, villains, and clear but rather simplistic morals?
  • If policymaking systems are so complex, should stakeholders devote huge amounts of resources to make sure they’re effective at each stage?
  • Should proponents of scientific evidence go to great lengths to make sure that EBPM is based on a hierarch of evidence? There is a live debate on science advice to government on the extent to which scientists should be more than ‘honest brokers’.
  • Should policymakers try to direct the use of evidence in policy as well as policy itself?

Where we go from there is up to you

The value of policy theory to this topic is to help us reject simplistic models of EBPM and think through the implications of more sophisticated and complicated processes. It does not provide a blueprint for action (how could it?), but instead a series of questions that you should answer when you seek to use evidence to get what you want. They are political choices based on value judgements, not issues that can be resolved by producing more evidence.



Filed under Evidence Based Policymaking (EBPM), public policy

Policymaking in the UK: do you really know who is in charge and who to blame? #POLU9UK

This week, we continue with the idea of two stories of British politics. In one, the Westminster model-style story, the moral is that the centralisation of power produces clear lines of accountability: you know who is in charge and, therefore, the heroes or villains. In another, the complex government story, the world seems too messy and power too diffuse to know all the main characters.

Although some aspects of these stories are specific to the UK, they relate to some ‘universal’ questions and concepts that we can use to identify the limits to centralised power. Put simply, some rather unrealistic requirements for the Westminster story include:

  1. You know what policy is, and that it is made by a small number of actors at the heart of government.
  2. Those actors possess comprehensive knowledge about the problems and solutions they describe.
  3. They can turn policy intent into policy outcomes in a straightforward way.

If life were that simple, I wouldn’t be asking you to read the following blog posts (underlined) which complicate the hell out of our neat story:

You don’t know what policy is, and it is not only made by a small number of actors at the heart of government.

We don’t really know what government policy is. In fact, we don’t even know how to define ‘public policy’ that well. Instead, a definition like ‘the sum total of government action, from signals of intent to the final outcomes’ raises more issues than it settles: policy is remarkably difficult to identify and measure; it is made by many actors inside, outside, and sort of inside/outside government; the boundary between the people influencing and making policy is unclear; and, the study of policy is often about the things governments don’t do.

Actors don’t possess comprehensive knowledge about the problems and solutions they describe

It’s fairly obvious than no-one possesses all possible information about policy problems and the likely effects of proposed solutions. It’s not obvious what happens next. Classic discussions identified a tendency to produce ‘good enough’ decisions based on limited knowledge and cognitive ability, or to seek other measures of ‘good’ policy such as their ability to command widespread consensus (and no radical movement away from such policy settlements). Modern discussions offer us a wealth of discussions of the implications of ‘bounded rationality’, but three insights stand out:

  1. Policymakers pay disproportionate attention to a tiny proportion of the issues for which they are responsible. There is great potential for punctuations in policy/ policymaking when their attention lurches, but most policy is made in networks in the absence of such attention.
  2. Policymakers combine ‘rational’ and ‘irrational’ ways to make decisions with limited information. The way they frame problems limits their attention to a small number of possible solutions, and that framing can be driven by emotional/ moral choices backed up with a selective use of evidence.
  3. It is always difficult to describe this process as ‘evidence-based policymaking’ even when policymakers have sincere intentions.

Policymakers cannot turn policy intent into policy outcomes in a straightforward way

The classic way to describe straightforward policymaking is with reference to a policy cycle and its stages. This image of a cycle was cooked up by marketing companies trying to sell hula hoops to policymakers and interest groups in the 1960s. It is not an accurate description of policymaking (but spirographs are harder to sell).

Instead, for decades we have tried to explain the ‘gap’ between the high expectations of policymakers and the actual – often dispiriting- outcomes, or wonder if policymakers really have such high expectations for success in the first place (or if they prefer to focus on how to present any of their actions as successful). This was a key topic before the rise of ‘multi-level governance’ and the often-deliberate separation of central government action and expected outcomes.

The upshot: in Westminster systems do you really know who is in charge and who to blame?

These factors combine to generate a sense of complex government in which it is difficult to identify policy, link it to the ‘rational’ processes associated with a small number of powerful actors at the heart of government, and trace a direct line from their choices to outcomes.

Of course, we should not go too far to argue that governments don’t make a difference. Indeed, many ministers have demonstrated the amount of damage (or good) you can do in government. Still, let’s not assume that the policy process in the UK is anything like the story we tell about Westminster.

Seminar questions

In the seminar, I’ll ask you reflect on these limits and what they tell us about the ‘Westminster model’. We’ll start by me asking you to summarise the main points of this post. Then, we’ll get into some examples in British politics.

Try to think of some relevant examples of what happens when, for example, minsters seem to make quick and emotional (rather than ‘evidence based’) decisions: what happens next? Some obvious examples – based on our discussions so far – include the Iraq War and the ‘troubled families’ agenda, but please bring some examples that interest you.

In group work, I’ll invite you to answer these questions:

  1. What is UK government policy on X? Pick a topic and tell me what government policy is.
  2. How did the government choose policy? When you decide what government policy is, describe how it made its choices.
  3. What were the outcomes? When you identify government policy choices, describe their impact on policy outcomes.

I’ll also ask you to identify at least one blatant lie in this blog post.

1 Comment

Filed under POLU9UK, Uncategorized

Week 2. Two stories of British politics: the Westminster model versus Complex Government #POLU9UK

I want you to think about the simple presentation of complex thought.

  • How do we turn a world which seems infinitely complex into an explanation which describes that world in a few minutes or seconds?
  • How do we choose the information on which to focus, at the expense of all other information, and generate support for that choice?
  • How do we persuade other people to act on that information?

To that end, this week we focus on two stories of politics, and next month you can use these questions to underpin your coursework.

Imagine the study of British politics as the telling of policymaking stories.

We can’t understand or explain everything about politics. Instead, we turn a complex world into a set of simple stories in which we identify, for example, the key actors, events and outcomes. Maybe we’ll stick to dry description, or maybe we’ll identify excitement, heroes, villains, and a moral. Then, we can compare these tales, to see if they add up to a comprehensive account of politics, or if they give us contradictory stories and force us to choose between them.

As scholars, we tell these stories to help explain what is happening, and do research to help us decide which story seems most convincing. However, we also study policymakers who use such stories to justify their action, or the commentators using them to criticise the ineffectiveness of those policymakers. So, one intriguing and potentially confusing prospect is that we can tell stories about policymakers (or their critics) who tell misleading stories!

Remember King Canute (Cnut)

King Canute


If you’re still with me, have a quick look at Hay’s King Canute article (or my summary of it). Yes, that’s right: he got a whole article out of King Canute. I couldn’t believe it either. I was gobsmacked when I realised how good it was too. For our purposes, it highlights three things:

  1. We’ll use the same shorthand terms – ‘Westminster model’, ‘complex government’ – but let’s check if we tell the same stories in the same way.
  2. Let’s check if we pick the same moral. For example, if ministers don’t get what they want, is it because of bad policymaking or factors outside of their control? Further, are we making empirical evaluations and/or moral judgements?
  3. Let’s identify how policymakers tell that story, and what impact the telling has on the outcome. For example, does it help get them re-elected? Does the need or desire to present policymaking help or hinder actual policymaking? Is ‘heresthetic’ a real word?

The two stories

This week, we’ll initially compare two stories about British politics: the Westminster Model and Complex Government. I present them largely as contrasting accounts of politics and policymaking, but only to keep things simple at first.

One is about central control in the hands of a small number of ministers. It contains some or all of these elements, depending on who is doing the telling:

  1. Key parts of the Westminster political system help concentrate power in the executive. Representative democracy is the basis for most participation and accountability. The UK is a unitary state built on parliamentary sovereignty and a fusion of executive and legislature, not a delegation or division of powers. The plurality electoral system exaggerates single party majorities, the whip helps maintain party control of Parliament, the government holds the whip, and the Prime Minister controls membership of the government.
  2. So, you get centralised government and you know who is in charge and therefore to blame.

Another is about the profound limits to the WM:

  1. No-one seems to be in control. The huge size and reach of government, the potential for ministerial ‘overload’ and need to simplify decision-making, the blurry boundaries between the actors who make and influence policy, the multi-level nature of policymaking, and, the proliferation of rules and regulations, many of which may undermine each other, all contribute to this perception.
  2. If elected policymakers can’t govern from the centre, you don’t get top-down government.

What is the moral of these stories?

For us, a moral relates to (a) how the world works or should work, (b) what happens when it doesn’t work in the way we expect, (c) who is to blame for that, and/ or (d) what we should do about it.

For example, what if we start with the WM as a good thing: you get strong, decisive, and responsible government and you know who is in charge and therefore to blame. If it doesn’t quite work out like that, we might jump straight to pragmatism: if elected policymakers can’t govern from the centre, you don’t get strong and decisive government, it makes little sense to blame elected policymakers for things outside of their control, and so we need more realistic forms of accountability (including institutional, local, and service-user).

Who would buy that story though? We need someone to blame!

Yet, things get complicated when you try to identify a moral built on who to blame for it:

There is a ‘universal’ part of the story, and it is difficult to hold a grudge against the universe. In other words, think of the aspects of policymaking that seem to relate to limitations such as ‘bounded rationality’. Ministers can only pay attention to a fraction of the things for which they are formally in charge. So, they pay disproportionate attention to a small number of issues and ignore the rest. They delegate responsibility for those tasks to civil servants, who consult with stakeholders to produce policy. Consequently, there is a blurry boundary between formal responsibility and informal influence, often summed up by the term governance rather than government. A huge number of actors are involved in the policy process and it is difficult to separate their effects. Instead, think of policy outcomes as the product of collective action, only some of which is coordinated by central government. Or, policy outcomes seem to ‘emerge’ from local practices and rules, often despite central government attempts to control them.

There is UKspecific part of the story, but it’s difficult to blame policymakers that are no longer in government. UK Governments have exacerbated the ‘governance problem’, or the gap between an appearance of central control and what central governments can actually do. A collection of administrative reforms from the 1980s, many of which were perhaps designed to reassert central government power, has reinforced a fragmented public landscape and a periodic sense that no one is in control. Examples include privatisation, civil service reforms, and the use of quangos and non-governmental organisations to deliver policies. Further, a collection of constitutional reforms has shifted power up to the EU and down to devolved and regional or local authorities.

How do policymakers (and their critics) tell these stories, how should they tell them, and what is the effect in each case?

Let’s see how many different stories we can come up with, perhaps with reference to specific examples. Their basic characteristics might include:

  • Referring primarily to the WM, to blame elected governments for not fulfilling their promises or for being ineffectual. If they are in charge, and they don’t follow through, it’s their fault linked to poor judgement.
  • Referring to elements of both stories, but still blaming ministers. Yes, there are limits to central control but it’s up to ministers to overcome them.
  • Referring to elements of both stories, and blaming other people. Ministers gave you this task, so why didn’t you deliver?
  • Referring to CG, and blaming more people. Yes, there are many actors, but why the hell can’t they get together to fix this?
  • Referring to CG and wondering if it makes sense to blame anyone in particular. It’s the whole damn system! Government is a mystery wrapped in a riddle inside an enigma.

Joe Pesci JFK the system

In broader terms, let’s discuss what happens when our two initial stories collide: when policymakers need to find a way to balance a pragmatic approach to complexity and the need to describe their activities in a way that the public can understand and support.

For example, do they try to take less responsibility for policy outcomes, to reflect their limited role in complex government, and/ or try to reassert central control, on the assumption that they may as well be more influential if they will be held responsible?

The answer, I think, is that they try out lots of solutions at the same time:

  • They try to deliver as many manifesto promises as possible, and the manifesto remains a key reference point for ministers and civil servants.
  • They often deal with ‘bounded rationality’ by making quick emotional and moral choices about ‘target populations’ before thinking through the consequences
  • In cases of ‘low politics’ they might rely on policy communities and/ or seek to delegate responsibility to other public bodies
  • In cases of ‘high politics’, they need to present an image of governing competence based on central control, so they intervene regularly
  • Sometimes low politics becomes high politics, and vice versa, so they intervene on an ad hoc basis before ignoring important issues for long periods.
  • They try to delegate and centralise simultaneously, for example via performance management based on metrics and targets.

We might also talk, yet again, about Brexit. If Brexit is in part a response to these problems of diminished control, what stories can we identify about how ministers plan to take it back? What, for example, are the Three Musketeers saying these days? And how much control can they take back, given that the EU is one small part of our discussion?

Illustrative example: (1) troubled families

I can tell you a quick story about ‘troubled families’ policy, because I think it sums up neatly the UK Government’s attempt to look in control of a process over which it has limited influence:

  • It provides a simple story with a moral about who was to blame for the riots in England in 2011: bad parents and their unruly children (and perhaps the public sector professionals being too soft on them).
  • It sets out an immediate response from the centre: identify the families, pump in the money, turn their lives around.
  • But, if you look below the surface, you see the lack of control: it’s not that easy to identify ‘troubled families’, the government relies on many local public bodies to get anywhere, and few lives are actually being ‘turned around’.
  • We can see a double whammy of ‘wicked problems’: the policy problem often seems impervious to government action, and there is a lack of central control of that action.
  • So, governments focus on how they present their action, to look in control even when they recognise their limits.

Illustrative example: (2) prevention and early intervention

If you are still interested by this stage, look at this issue in its broader context, of the desire of governments to intervene early in the lives of (say) families to prevent bad things happening. With Emily St Denny, I ask why governments seem to make a sincere commitment to this task but fall far shorter than they expected. The key passage is here:

“Our simple answer is that, when they make a sincere commitment to prevention, they do not know what it means or appreciate scale of their task. They soon find a set of policymaking constraints that will always be present. When they ‘operationalise’ prevention, they face several fundamental problems, including: the identification of ‘wicked’ problems (Rittell and Webber, 1973) which are difficult to define and seem impossible to solve; inescapable choices on how far they should go to redistribute income, distribute public resources, and intervene in people’s lives; major competition from more salient policy aims which prompt them to maintain existing public services; and, a democratic system which limits their ability to reform the ways in which they make policy. These problems may never be overcome. More importantly, policymakers soon think that their task is impossible. Therefore, there is high potential for an initial period of enthusiasm and activity to be replaced by disenchantment and inactivity, and for this cycle to be repeated without resolution”.

Group exercise.

Here is what I’ll ask you to do this week:

  • Describe the WM and CG stories in some depth in your groups, then we’ll compare your accounts.
  • Think of historical and contemporary examples of decision-making which seem to reinforce one story or the other, to help us decide which story seems most convincing in each case.
  • Try to describe the heroes/ villains in these stories, or their moral. For example, if the WM doesn’t explain the examples you describe, what should policymakers do about it? Will we only respect them if they refuse to give up, like Forest Gump or the ‘never give up, never surrender’ guy in Galaxy Quest? Or, if we would like to see pragmatic politicians, how would we sell their behaviour as equally heroic?


Filed under POLU9UK, public policy, UK politics and policy

Is there any hope for evidence in emotional debates and chaotic government?

Two recent news features sum up the role of emotion and ‘chaos’ in policymaking.

The first is ‘Irritation and anger’ may lead to Brexit, says influential psychologist in The Telegraph. The headline suggests that people may vote Leave for emotional reasons, rather than with reference to a more ‘rational’ process in which people identify the best evidence and use it to weight up the short and long term consequences of action.

Yet, the article confirms that we’re all at it! To be human is to use emotional, gut-level, and habitual thinking to turn a complex world, with too much information, into a good enough decision in a necessary short amount of time.

In debate, evidence is mentioned a lot, but only to praise the evidence backing my decision and rejecting yours. Or, you only trust the evidence from people you trust. If you trust the evidence from certain scientists, you stress their scientific credentials. If not, you find some from other experts. Or, if all else is lost, you reject experts as condescending elites with a hidden agenda. Or, you say simply that they can’t be that clever if they agree with smarmy Cameron/ Johnson.

Lesson 1: you can see these emotional and manipulative approaches to policymaking play out in the EU referendum. Don’t assume that policymaking behind closed doors, on other issues, is any different.

The second feature is Lost in Transit: chaos in government research in The Economist.

It describes a Sense about Science report which (a) was commissioned ‘following a spate of media stories about government research being suppressed or delayed’, and (b) finds that ‘The UK government spends around £2.5 billion a year on research for policy, but does not know how many studies it has commissioned or which of them have been published’.

The Economist reports the perhaps-unexpected result of the inquiry:

But the main gripe is the sheer disorganisation of it all. The report’s afterword states that “Sir Stephen looked for suppression and found chaos”.

Such accounts reflect the two contradictory stories that we often tell about government. The first relates to the Westminster model of democratic accountability which helps concentrate power at the centre of government: if you know who is in charge, you know who to blame.

The second, regarding complex government, describes a complicated world of public policy in which no-one seems to be in control. For example, we make reference to: the huge size and reach of government; the potential for ministerial ‘overload’ and need to simplify decision-making; the blurry boundaries between the actors who make and influence policy; the multi-level nature of policymaking; and, the proliferation of rules and regulations, many of which may undermine each other.

The problem with the first story is that (a) although it is easy to tell during elections and inquiries, (b) you always struggle to find it when you actually study government.

The problem with the second is that, (a) although it seems realistic when you study government, (b) few people will buy it when they are seeking to hold ministers and governments to account. This problem may be exacerbated by the terms of reference of reports: few will accept a pragmatic response, based on the second story of complexity, if you start out by using the first story of central control to say that you will track down and solve the problem!

Lesson 2: if you assume central control you will find chaos (and struggle to produce feasible recommendations to deal with it). The manipulation of evidence takes place in a complex policymaking system over which no individual or ‘core executive’ has control. Indeed, no single person or organisation could even pay attention to all that goes on within government. This insight requires pragmatic inquiries and solutions, not the continuous reassertion of central control and discovery of ‘chaos’.

It might be possible to develop a third lesson if we put these two together. One part of the EU debate reflects our inability to understand EU policymaking and relate it to the relatively clear processes in the UK, in which you know who is in charge and therefore who to blame. The EU seems less democratic because it is so complex and remote. Yet, if we follow this other story about complexity in the UK, we often find that UK politics is also difficult to follow. Its image does not describe reality.

Lesson 3: when you find policymaking complexity in the EU, don’t assume it is any better in the UK! Instead, try to compare like with like.

See also

I expand on both lessons in The Politics of Evidence-Based Policymaking

Cock-up, not conspiracy, conceals evidence for policy

Government buries its own research – and that’s bad for democracy

The rationality paradox of Nudge: rational tools of government in a world of bounded rationality


Filed under Evidence Based Policymaking (EBPM), public policy, UK politics and policy

Key theories in policymaking: how to explain what is going on in Scotland #POLU9SP

This is quite a long read and I will likely spread the discussion over more than one lecture. In this post I provide the ‘universal’ concepts to consider, and we can discuss in the lecture how they apply specifically to Scottish policymaking.

In week 1, we discussed the concept of ‘bounded rationality’ and theories which explore its implications. What happens when policymakers do not have the ability to gather all relevant information and make policy in an orderly way? They focus on a small number of issues and ignore or delegate the rest, their attention lurches from issue to issue, they make quick and often emotional decisions, and they are susceptible to persuasion and simple stories which exploit their emotions and biases and reinforce their beliefs.

We then discussed the ‘universal’ and territorial aspects of such theories: all policymakers face these limitations, but a Scottish frame of reference influences the kinds of issues which receive most attention and the stories that gain most traction.

So far, our conceptual focus has been on the role of key individuals: the policymakers in charge, and the actors who seek influence. This week, we extend the analysis to the ‘environments’ or ‘systems’ in which they operate. Most theories combine this dual focus on:

  1. the cognitive limits of policymakers and the ways in which they think, and
  2. the conditions under which they make choices, including: the rules they follow, the networks in which they participate, the socioeconomic context in which they operate, and their deeply held beliefs.

There are too many relevant concepts and theories to cover in one lecture, and I wouldn’t expect you to become familiar with all of them by the end of the year (in fact, it would be better to have an inside-out knowledge of one).

Instead, as an introduction to the study of policymaking, we can discuss the concepts that most theories have in common, and look at one example of a theory – complexity theory – which can be used to consolidate several discussions.

Key questions in policy theory

These concepts can be turned into the questions we should ask when we try to understand and explain the environment in which ‘boundedly rational’ policymakers make choices:

  • Actors. Which actors are involved in policymaking, and at what level of government do they operate?

This is not an easy question to answer if we accept the need to focus on more than just the elected people in ministerial posts. There may be thousands of actors involved in policymaking, and ‘actors’ is a very broad term which includes individuals or organisations, including private companies, interest groups and governments bodies.

A common argument in policy theory is that we have witnessed a shift since the early post-war period, characterised by centralized and exclusive policymaking, towards a fragmented multi-level system involving a much larger number of actors.

  • Institutions. What rules or ‘standard operating procedures’ have developed within policymaking organisations and how do they influence the development of policy?

The term ‘institution’ is often used loosely to describe important organisations such as governments or legislatures. Really, it refers to the rules, ‘norms’, and other practices that influence policymaking behaviour.  Some rules are visible or widely understood, such as constitutions or the ‘standing orders’ of parliaments. Others are less visible – the ‘rules of the game’ in politics, or organisational ‘cultures’ – and may only be understood following in-depth study of particular organisations.

These rules develop in different ways in many parts of government, prompting us to consider what happens when many different actors develop different expectations of politics and policymaking.  For example, it might help explain a gap between policies made in one organisation and implemented by another. It might cause government policy to be contradictory, when many different organisations produce their own policies without coordinating with others. Or, governments may contribute to a convoluted statute book by adding to laws and regulations without thinking how they all fit together. Such problems may be magnified when policymaking is multi-level.

  • Networks or ‘subsystems’. What is the balance of power between ‘pressure participants’ such as interest groups?

In week 1, we linked the formation of policy networks to bounded rationality: ministers and senior civil servants delegate, and more junior civil servants form relationships with the actors giving them information and advice. In some cases, we can identify close relationships based, for example, on a shared understanding of the policy problem and an adherence to unwritten ‘rules of the game’. In others, these networks are large, there are many actors involved, there is less incentive to follow the same rules (such as to keep discussions relatively quiet), and there is more competition to ‘frame’ the problem to be solved.

  • Context and events. How does the socio-economic and political context influence policy? Which events have prompted or undermined policy development?

‘Context’ describes the policy conditions that policymakers take into account when identifying problems, such as a country’s geography, demographic profile, economy, and social attitudes. This wider context is in addition to the ‘institutional’ context, when governments inherit the laws and organisations of their predecessors.

Important ‘game changing’ events can be routine, such as when elections produce new governments with new ideas, or unanticipated, such as when crises or major technological changes prompt policymakers to reconsider existing policies.

In each case, we should consider the extent to which policymaking is in the control of policymakers. In some cases, the role of context seems irresistible – examples include major demographic change (such as an ageing population), the role of technology in driving healthcare demand, climate change, extreme events, and ‘globalisation’. Yet, governments often show that they can ignore such issues for long periods of time or, at least, decide how and why they are important.

  • Ideas.  What is the role of beliefs, knowledge, evidence and learning in shaping the way that policymakers understand and seek to solve problems?

‘Ideas’ is a quite-vague term to describe beliefs, or ways of thinking, and the extent to which they are shared within groups, organisations, networks and political systems. There are three kinds of inter-related ideas to discuss. First, an idea can be a policy solution: ‘I have an idea’ to solve a policy problem. Second, ideas relate to persuasion as a resource in the policy process. Actors can use money and other sources of power to influence policy, but also argument and manipulation. Third, ideas refer to shared beliefs or a shared language used by policy participants: the ‘core beliefs’, ‘paradigms’, ‘hegemonic’ ideas, or ‘monopolies’ of ideas that are often so important because people take them for granted.

The latter provides the context in which people use arguments and persuasion and within which certain policy solutions are conceivable. So, not everyone has the same opportunity to raise attention to problems and propose their favoured solutions. Some can exploit a dominant understanding of the policy problem, while others have to work harder to challenge existing beliefs. A focus on ideas is a focus on power: to persuade the public, media and/ or government that there is a reason to make policy; and, to keep some issues on the agenda at the expense of others.

Using theory to bring these concepts together

Any one of these elements could be used to explain why the policy process is ‘complex’ and so difficult to understand and predict. Or, many theories try to explain how the policy process works by describing the interaction between all of those elements: https://paulcairney.wordpress.com/1000-words/

One example is complexity theory, which discusses the properties of complex policymaking systems:

  1. The same amount of activity can have no effect or a huge effect. For example, someone presenting the same information may be ignored by policymakers or receive disproportionate attention.
  2. Decisions made in the past can produce a long term momentum or ‘path dependence’. For example, the current ‘welfare state’ or national health service can be traced at least as far back as the 1940s.
  3. Policymaking involves long periods of organizational inactivity followed by bursts of activity in some areas. Policy can remain the same for decades, only to change dramatically and quickly.
  4. ‘Systemic’ behaviour results from the interaction between actors who share information and follow particular rules. Policymaking behaviour often seems to ‘emerge’ from local interaction and despite the efforts of central governments to control it.

Complexity theory is a good example to consider because it helps us think about the ways in which you might present policy advice when you have to take into account the vagaries of policymaking. We can discuss this link between theory and advice in the next lecture.


Filed under POLU9SP, Scottish politics

Now that the SNP dominates Scottish politics, it should it have a vision for Scottish Government

This is the claim made in different ways by Scotland’s dream team of Andrew Tickell and Chris Deerin. For Tickell, it’s time to stop gloating about wiping out Scottish Labour and start talking about what the SNP stands for in policy terms. For Deerin, it’s time for Nicola Sturgeon to demonstrate what she has achieved in government rather than in party politics; to show that the SNP has any ideas worthy of its popularity.

deerin policy

In this post, I’ll show you that the Scottish Government has a vision. It might seem bland and motherhood/ apple pie, and maybe almost no one has heard of it, but if you accept it then the questions raised by Tickell and Deerin will seem off-target and unambitious. Instead, there is another, far less discussed, problem that is so big that it goes to the heart of our political system, but so abstract that only the die-hards will want to read further. I’ll put in a picture of Joe Pesci (in JFK) here to suggest that talking about the whole system of government might be entertaining.

Joe Pesci JFK the system

The Scottish Government’s vision

Since 2007, the SNP Government has presented a ‘ten year vision’ which includes:

  • a ‘core purpose – to create a more successful country, with opportunities for all of Scotland to flourish, through increasing sustainable economic growth’; linked to
  • a ‘purpose framework’, with targets gauging its economic growth, productivity, labour market participation, population, income inequality, regional inequality and (emissions based) sustainability;
  • five ‘strategic objectives’: Enabling businesses and people to increase their wealth and more people to share fairly in that wealth, Helping people to sustain and improve their health, especially in disadvantaged communities, ensuring better, local and faster access to health care, Helping communities to flourish, becoming stronger, safer places to live, offering improved opportunities and a better quality of life, Expanding opportunities to succeed from nurture through to lifelong learning ensuring higher and more widely shared achievements, and Improving Scotland’s natural and built environment and the sustainable use and enjoyment of it.
  • sixteen ‘National Outcomes’ and fifty ‘National Indicators’ of progress towards the fulfilment of its vision.

It also has a clear narrative of how it wants to make policy. Its ‘governance’ strategy is based on a desire to delegate or share policymaking responsibility with key stakeholders in the public, private and third sectors. So, it has set up or reinforced local partnerships, charged with turning a broad vision into something that can be delivered according to the politics and circumstances of local areas. It talks about making sure that service users are at the heart of designing public services. It has also developed a desire to harness high levels of pre-referendum participation by stating its broad aims and inviting the public to help make sense of them. Or, it talks about a new era in which governments no longer do policy to people but with them – which translates into, for example, going into a consultation or meeting without having a fixed Scottish Government position or knowing what the outcome will be. Maybe it will also experiment with other ways to foster participation, such as citizen assemblies or juries or civic forums.

Why this vision is so important

This is a bland vision which includes aims that most people would struggle to criticise. That’s what makes it so important. It’s the kind of vision that political parties try to come up with, using a new mix of words, at every Scottish election. It generates so much cross-party support that it can relied upon as a reference point for the long term. So, for example, when we identify current major inequalities in health or education attainment, we can make reference to the long term vision and ask if there has been stagnation or progress. Then, when we think about that progress, we can point to meaningful outcomes (e.g. increases in community wellbeing) rather than just adherence to arbitrary inputs (e.g. teacher numbers) or outputs (e.g. meeting waiting times). It allows us to come up with more realistic measure of progress than 8-month or annual anniversaries in power – and, as such provides an antidote (albeit only for the few people seeking one) to the usual party political evaluations provided by politicians.

It also allows us to measure progress or decline in one area in relation to another, so that we don’t just extrapolate from one good or bad experience to produce a misleading overall picture. Maybe it even gives you a reference point for debates on, for example, the idea that policymaking remains excessively centralist (or the opposite). Certainly, it presents a realistic response to complex government: it helps elected governments recognise their limited abilities to centralise power to deliver their promises. Within this vision is the sense that elected policymakers are one of many groups of actors with the ability to influence outcomes. To assert otherwise is to mislead the electorate.

In short, even if you put the finest minds on the case – perhaps a collection of party worthies, civil servants, scientists, economists, philosophers, novelists, and poets – I doubt they could come up with a markedly better vision for Scottish policy.

Why this vision is incompatible with the way we understand politics

The bigger problem with this sort of vision is that it is, in important ways, the enemy of the kind of politics that we hold most dear. This can be a positive thing, if a technocratic vision makes up for some of the worst excesses of representative government and party politics. For example, parties don’t really have long term visions – they pretend to have them to win elections. They try to come up with new words to describe what is pretty much the Scottish Government position anyway, but without the sense that any party can promise to be in government long enough to see it through.

Parties also produce manifestos with much more specific aims which scupper long term visions – such as when they make silly short term promises on police officer or teacher or nurse numbers or hospital waiting times, which distort long term justice, education and health policy planning by limiting choice and putting short term aims on inputs/ outputs further up the agenda than long term outcomes. Or, parties make very firm commitments to policy decisions – which makes a nonsense of the idea that politicians and civil servants go into a room without knowing what the outcome will be. Or, more generally, they give the sense, during electoral competition, that central government makes the decisions and is accountable for everything that happens – which goes against the idea that the Scottish Government can delegate and share responsibility with local bodies.

But the adherence to a non-party-specific vision can also take us too far in the other direction. The worrying thing may not be that representative government distorts politics – since it’s the best form of politics we have – but that the alternative distorts representative government. Consider, for example, the implication of the Scottish Government sharing responsibility with everyone – local public bodies, stakeholders, service users – for long term outcomes. With that system, how can you decide how well things are going or who is to blame when things go badly? Sharing responsibility with so many people means not taking the blame (unless we are scapegoating or holding to account some individuals when they appear to be incompetent). It means that the call for an 8-month progress report is too easy to bat away. It means that the Scottish Parliament will struggle to hold the Scottish Government to account for policy progress, and that the electorate will struggle to reward or punish elected governments for what goes on in their name.

If you want the SNP to have a vision, and win elections, keep it bland

Beyond Scottish independence, we know roughly what the SNP wants and that it will take far longer to achieve than in the space of a few elections. We might want some milestones along the way, but also recognise that ‘quick wins’ are not the same as important wins.

In that sense, if you value representative government and party competition, it seems OK to accept that the presentation of a vision is a means to an end to win a game. You don’t bare your soul in a manifesto. You tell people that everything will be OK: the long term vision is already there and we are the most competent group of people to ensure its progress. The SNP is very good at this. It won the 2011 election on the back of it. It is one unheralded part of the SNP’s strategy to remain popular and in government: maintaining an image of governing competence by presenting a bland vision and avoiding major mistakes. It shouldn’t be underrated, particularly since it has allowed the SNP to remain Teflon in the face of a growing number of specific criticisms of its record.

The worrying bit

In short, if any political party tried to sell me another vision in its manifesto, or a new leader tried to claim the credit for policy progress in 8 months, I would take an instant distrust to them (in contrast to the politicians that live to stick it to other politicians – at least you know where you stand).

The only problem is that I don’t really know what I want from them instead, beyond a bland commitment to make society fairer. I know that they can only do so much in government, that they are part of a larger complex system, that their influence on policy outcomes is limited, and that they shouldn’t really take the bulk of the credit or blame for what happens. I know that their manifestos shouldn’t be too specific because some promises have a bewitching effect on debates and distorting effect on policy. Beyond hoping for competent and trustworthy people who seem to share my outlook on life, I’m not sure what else to root for. That seems much more of a problem to me than the lack of a political party’s soul or a leader’s 8-month narrative.

See also:

The language of complexity does not mix well with the language of Westminster-style accountability

What is ‘Complex Government’ and what can we do about it?

Scotland’s Future Political System

1 Comment

Filed under Scottish politics

Policy Concepts in 1000 Words: Complex Systems

complexity za

(podcast download)

There is an unnecessary tendency for proponents of complexity theory to say that it is radically new; a scientific revolution; that it will change the way we think about, and study, the natural and social world. It suggests that we shift our analysis from individual parts of a system to the system as a whole; as a network of elements that interact and combine to produce systemic behaviour that cannot be broken down merely into the actions of its constituent parts. The metaphor of a microscope or telescope, in which we zoom in to analyse individual components or zoom out to see the system as a whole, sums up this alleged shift of approach.

Complexity theory has been applied to a wide range of activity, from the swarming behaviour of bees, the weather and the function of the brain, to social and political systems.  The argument is that all such systems have common properties, including:

  1. A complex system is greater than the sum of its parts; those parts are interdependent – elements interact with each other, share information and combine to produce systemic behaviour.
  2. Some attempts to influence complex systems are dampened (negative feedback) while others are amplified (positive feedback). Small actions can have large effects and large actions can have small effects.
  3. Complex systems are particularly sensitive to initial conditions that produce a long-term momentum or ‘path dependence’.
  4. They exhibit ‘emergence’, or behaviour that results from the interaction between elements at a local level rather than central direction.
  5. They may contain ‘strange attractors’ or demonstrate extended regularities of behaviour which may be interrupted by short bursts of change.

As you might expect from a theory of many things, the language is vague and needs some interpretation in each field. In the policymaking field, the identification of a complex system is often used to make the following suggestions:

  • Law-like behaviour is difficult to identify – so a policy that was successful in one context may not have the same effect in another.
  • Policymaking systems are difficult to control; policy makers should not be surprised when their policy interventions do not have the desired effect.
  • Policy makers in the UK have been too driven by the idea of order, maintaining rigid hierarchies and producing top-down, centrally driven policy strategies.  An attachment to performance indicators, to monitor and control local actors, may simply result in policy failure and demoralised policymakers.
  • Policymaking systems or their environments change quickly. Therefore, organisations must adapt quickly and not rely on a single policy strategy.

On this basis, there is a tendency in the literature to encourage the delegation of decision-making to local actors:

  1. Rely less on central government driven targets, in favour of giving local organisations more freedom to learn from their experience and adapt to their rapidly-changing environment.
  2. To deal with uncertainty and change, encourage trial-and-error projects, or pilots, that can provide lessons, or be adopted or rejected, relatively quickly.
  3. Encourage better ways to deal with alleged failure by treating ‘errors’ as sources of learning (rather than a means to punish organisations) or setting more realistic parameters for success/ failure.
  4. Encourage a greater understanding, within the public sector, of the implications of complex systems and terms such as ‘emergence’ or ‘feedback loops’.

In other words, this literature, when applied to policymaking, tends to encourage a movement from centrally driven targets and performance indicators towards a more flexible understanding of rules and targets by local actors who are more able to understand and adapt to rapidly-changing local circumstances.

Although complexity theory is described as new, these are familiar arguments in policy studies. Relevant texts in the policymaking literature include punctuated equilibrium theory, historical institutionalism and the following:

Lipsky’s idea of ‘street level bureaucracy’. He suggests that there are so many targets, rules and laws that no public agency or official can be reasonably expected to fulfil them all.  In fact, many may be too vague or even contradictory, requiring ‘street level bureaucrats’ to choose some over others.  The potential irony is that the cumulative pressure from more central government rules and targets effectively provides implementers with a greater degree of freedom to manage their budgets and day-to-day activities. Alternatively, central governments must effectively reduce their expectations by introducing performance measures which relate to a small part of government business (see this discussion of Street Level Organizations https://paulcairney.wordpress.com/2013/09/09/street-level-bureacrats/).

Hjern’s focus on intra-departmental conflict, when central government departments pursue programmes with competing aims, and interdependence, when policies are implemented by multiple organizations. Programmes are implemented through ‘implementation structures’ where ‘parts of many public and private organizations cooperate in the implementation of a programme’. Although national governments create the overall framework of regulations and resources, and there are ‘administrative imperatives’ behind the legislation authorizing a programme, the main shaping of policy takes place at local levels.

Governance.  A lack of central control has prompted governments in the past to embrace New Public Management (NPM) and seek to impose order through hierarchy and targeting. However, local implementation networks (with members from the public, third and private sectors) have often proved not be amenable to such direct control.

Lindblom’s discussion of incrementalism in 1959: ‘Making policy is at best a very rough process. Neither social scientists, nor politicians, nor public administrators yet know enough about the social world to avoid repeated error in predicting the consequences of policy moves. A wise policy-maker consequently expects that his policies will achieve only part of what he hopes and at the same time will produce unanticipated consequences he would have preferred to avoid. If he proceeds through a succession of incremental changes, he avoids serious lasting mistakes’ (or she/ her).

Consequently, we should reject the idea of theoretical novelty for novelty’s sake. The value of complexity theory is not that it is trendy – it is that it allows us to use our knowledge of the natural and social world to understand and influence real world problems.

Where do we go from here?

Empirical issues: How does ‘complexity thinking’ improve our understanding of politics and policymaking?

Normative issues: The language of complexity does not mix well with the language of Westminster-style accountability

See ‘Complexity Theory in Political Science and Public Policy’ PDF Paywall Green and the introduction and conclusion to our Handbook on Complexity and Public Policy.

See also Professor Robert Geyer on Complexity and the Stacey Diagram – http://vimeo.com/25979052

Click on the picture at the top of this post for a nice surprise.

Series: Policy Concepts in 1000 words


Filed under 1000 words, public policy, UK politics and policy