Tag Archives: Bounded rationality

How to present policy analysis to many audiences #POLU9SP

This is the only week in which I will encourage you to find advice on policy analysis by using a search engine like Google. However, you need to do it with a critical eye. You will find two kinds of advice, only one of which is useful in a straightforward way:

  1. Most of the advice will tell you to be concise, to keep it short for a busy audience, readable, to minimise jargon, and up-to-date, to give the impression of comprehensive research. This advice is sound.
  2. Some will get you to organise the advice as if the policy process were ordered into a series of discrete stages, akin to a policy cycle. This advice can be problematic.

A key message of this course is that policy processes are more like complex systems than ordered cycles. Therefore, good policy analysis should reflect on policymaking as much as policy. There is no point in coming up with a well-researched analysis of a problem and a brilliant solution if, for example, you recommend it to someone without the willingness or ability to pay attention to it or use it.

We can reinforce this point with reference to the previous lecture’s focus on 5 key concepts common to policy theories:

  1. There are many actors involved in policymaking, at multiple levels. Therefore, you shouldn’t assume that you can make a recommendation effectively by only speaking to one policymaker or organisation, or only seeing the problem through their eyes.
  2. Many ‘institutions’ have their own ‘standard operating procedures’ or rules which take time to understand and influence.
  3. Many policymakers or departments have a well-established or powerful clientele which you should take into account.
  4. Your analysis might be a quick response to major social change or a key event, not a process in which you have years to plan.
  5. Your analysis may be more persuasive if couched in a language that tends to dominate discussion or with reference to the entrenched beliefs of policymakers or key participants.

Or, we can make the point with reference to the implications of complexity theory, discussed in the last lecture:

  • In complex systems, it is unwise to expect ‘law-like behaviour’; a policy that was successful in one context may not have the same effect in another.
  • Policymaking systems are difficult to control and analysts should be prepared for the possibility that their policy interventions do not have the desired effect. Governments may be learning that they cannot simply make a policy and expect successful implementation by using a top-down, centrally driven policy strategy.
  • Policymaking systems or their environments change quickly. Therefore, organisations must learn and adapt quickly.

Complexity, the Scottish approach, and policy analysis

The advice that tends to come from complexity studies is generally consistent with the ‘Scottish approach’ that we have begun to discuss: rely less on short term and rigid targets, in favour of giving local organisations more freedom to learn from their experience and adapt to their rapidly-changing environment; encourage trial-and-error projects that can provide lessons, or be adopted or rejected, relatively quickly; and, set broad or realistic parameters for success and failure. However, policymakers are also under pressure to act to solve emerging problems and take responsibility for national strategies.

This context makes policy analysis tricky: you want to recognise the ways in which governments understand their policymaking task, and seek to operate for the long term, but also the pressures they are under to act quickly and take control.

Bardach’s 8 steps

One text which tries to combine policy analysis with a recognition of policymaking context is Eugene Bardach’s A Practical Guide for Policy Analysis. His steps to policy analysis provide one (but not the only) way to structure your coursework in a way that combines empirical analysis and political awareness:

  1. ‘Define the problem’ involves the selective use of information (and eye-catching statistics), ideology, and persuasion, to help your problem compete with others.
  2. ‘Assemble some evidence’ involves ‘hustling’ relevant data in an efficient way.
  3. ‘Construct the alternatives’ involves generating politically feasible policy solutions.
  4. ‘Select the criteria’ involves recognising the political nature of policy evaluation, which is influenced by the measures you choose to determine success.
  5. ‘Project the outcomes’ involves predicting only relevant outcomes; the ones that key actors care about (such as cost savings or value for money).
  6. ‘Confront the trade-offs’ can involve working out how much of a bad service policymakers will accept to cut costs.
  7. ‘Decide’ involves looking at your case to see if it is persuasive.
  8. ‘Tell your story’ involves tailoring your case to the biases of your audience. A policy analysis is not for everyone. It is a case made to a specific audience, or several stories for different audiences.

Examples in Scotland

You should think about possible examples as soon as possible, and we can discuss your ideas in lectures this week and tutorials in week 3. Possible examples include:

  • Why and how would you introduce more class testing in compulsory education? In this case, you could identify the relevance of the Scottish system of education, the moves in the 1990s to reject testing, and the role and influence of education unions or the teaching profession, and weigh these factors against the need to be seen to be responding to things like a perceived decline in pupil attainment and major inequalities in attainment.
  • Can you make a case for private sector delivery in the Scottish NHS? The NHS was a key feature of the independence debate. Would it be feasible for any party to propose more private sector delivery? If only the Conservatives would try, how would you recommend they do it?
  • How would you structure your advice on fracking to the current SNP-led Scottish Government?

The coursework

It will be difficult for you to reflect on all of these factors in each piece of coursework, particularly since the blog needs to be eye-catching and the policy analysis needs to be short. However, you will have more space to explain your reasoning in the oral presentation and explore the politics of policymaking, and explain policy change, in the longer essay. This combination of exercises will, I hope, give you the chance to communicate similar information to different audiences, and reflect on how you do it.

Leave a comment

Filed under POLU9SP, Scottish politics

Can you separate the facts from your beliefs when making policy?

A key argument in policy studies is that it is impossible to separate facts and values when making policy. We often treat our beliefs as facts, or describe certain facts as objective, but perhaps only to simplify our lives or support a political strategy (a ‘self-evident’ fact is very handy for an argument). People make empirical claims infused with their values and often fail to realise just how their values or assumptions underpin their claims.

This is not an easy argument to explain. One strategy is to use extreme examples to make the point. For example, Herbert Simon points to Hitler’s Mein Kampf as the ultimate example of value-based claims masquerading as facts. We can also draw on some embarrassing historic academic research which states that the evidence exists to show that men are more intelligent than women and some races are demonstrably superior to others. In such cases, we would point out, for example, that the design of the research helped produce such conclusions: our values underpin our assumptions about how to measure intelligence or other measures of superiority.

‘Wait a minute, though’ (you might say). “What about simple examples in which you can state facts with relative certainty – such as the statement ‘there are 449 words in this post’”. ‘Fair enough’, I’d say (you will have to speak with a philosopher to get a better debate about the meaning of your 449 words claim). But this statement doesn’t take you far in policy terms. Instead, you’d want to say that there are too many or too few words, before you decided what to do about it.

In that sense, we have the most practical explanation of the unclear fact/ value distinction: the use of facts in policy is to underpin evaluations based on values. For example, we might point to the routine uses of data to argue that a public service is in ‘crisis’ or that there is a public health related epidemic. We might argue that people only talk about ‘policy problems’ they think we have a duty to solve them.

Or, facts and values often seem the hardest to separate when we evaluate the success and failure of policy solutions, since the measures used for evaluation are as political as any other part of the policy process. The gathering and presentation of facts is inherently a political exercise, and our use of facts to encourage a policy response is inseparable from our beliefs about how they world should work.

To think further about the relevance of this discussion, see this post on policy evaluation, this page on the use of evidence in policymaking, this book by Douglas, and this short commentary on ‘honest brokers’ by Jasanoff.

8 Comments

Filed under Evidence Based Policymaking (EBPM), UK politics and policy

The language of complexity does not mix well with the language of Westminster-style accountability

westminster-times

A common argument in British politics is that the UK Government has exacerbated its own ‘governance problem’. A collection of post-war reforms, many of which were perhaps designed to reinforce central control, has produced a fragmented public landscape and a periodic sense that no one is in control. This outcome presents major problems for the ‘Westminster’ narrative of central government and ministerial accountability to the public via Parliament. If ministers are not in control of their departments, how can we hold them to account in a meaningful way?

Yet, in many cases, it is misleading to link these outcomes to specific decisions or points in time, since many aspects of the ‘governance problem’ are universal: policymakers can only pay attention to a small fraction of the issues for which they are responsible; they do not have enough information to make decisions without major uncertainty; policy problems are too multi-faceted and ‘cross-cutting’ to allow policymaking without ambiguity; there is an inescapable logic to delegating decisions to ‘policy communities’ which may not talk to each other or account meaningfully to government; and, delivery bodies will always have discretion in the way they manage competing government demands.

In this context, policymaking systems can be described usefully as complex systems, in which behaviour is always difficult to predict, and outcomes often seem to emerge in the absence of central control. Further, the literature on complexity provides some advice about how governments should operate within complex systems. Unfortunately, much of this literature invites policymakers to give up on the idea that they can control policy processes and outcomes. While this may be a pragmatic response, it does not deal well with the need for elected policymakers to account for their actions in a very particular way. What seems sensible to one audience may be indefensible to another. In particular, the language of complexity does not mix well with the language of Westminster-style accountability.

What we need is a response that sets out a governmental acknowledgement of the limits to its powers, combined with the sense that we can still hold elected policymakers to account in a meaningful way. Ideally, this response should be systematic enough to allow us to predict when ministers will take responsibility for their actions, redirect attention to other accountable public bodies, and/ or identify the limited way in which they can be held responsible for certain outcomes. Beyond this ideal, we may settle for a government strategy based on explicit trade-offs between pragmatism, in which governments acknowledge the effect of administrative devolution (or, in the case of local authorities, political devolution), and meaningful representation, in which they maintain some degree of responsibility for decisions made in their name.

The aim of this paper is to draw lessons from the Scottish experience, which demonstrates an attempt to mix strategic responsibility with an element of flexibility and delegation. While we should not exaggerate the coherence of government strategies, we can meaningfully describe a ‘Scottish policy style’, identified in empirical studies, and a ‘Scottish approach’ as a self-styled description of policymaking by the Scottish Government. Further, the Scottish context is comparable enough to the UK to offer lessons. Although much of the rhetoric of ‘new Scottish politics’ suggests that it is markedly different from ‘old Westminster’, it has inherited a Westminster-style focus on government accountability to the public via Parliament (and an assumption that ‘the government governs’). Although Scotland is smaller, and the Scottish Government is able to design a governance style based on greater personal contact with interest groups and public bodies, this only serves to reinforce the importance of ‘universal’ problems when the problems that arise in Scotland resemble those faced in the UK. Overall, Scottish policymaking demonstrates that many problems related to ‘governance’ cannot be solved. Rather, the Scottish experience prompts us to identify important trade-offs between the delegation of administrative functions and the maintenance of central accountability.

To explain these issues, the paper first summarises the main ways in which UK governments have allegedly exacerbated governance problems. Second, it separates this focus on specific outcomes from the universal constraints on central control common to all complex policymaking systems. Third, it contrasts the practical advice that arises from a focus on complexity theory with the political imperative, in Westminster systems, to present policy outcomes as the responsibility of ministers. Fourth, it identifies the balance struck between accountability and delegation by the Scottish Government since 2007, and the transferable lessons to other systems.

The paper is here: Cairney Governance Complexity Accountability Scotland 20.11.14

Some of the wider issues are discussed here:

Policy Concepts in 1000 Words: the Westminster Model and Multi-level Governance

Policy Concepts in 1000 Words: Complex Systems

What is ‘Complex Government’ and what can we do about it?

Sharing professional and academic knowledge: The role of academic-practitioner workshops (on turning policy and complexity theories into something consistent with Westminster politics)

Life goes on after the Scottish independence referendum (3000 words and a lecture)

The Scottish political system and policy process share the same ‘complex government’ features as any country (LSE 1200 words)

15 Comments

Filed under public policy, Scottish politics, UK politics and policy

Life goes on after the Scottish independence referendum

This is the 3000 word experimental-album post, and there is also a shorter radio-edit single for the LSE blog. I discussed these issues at my inaugural lecture and the audio is available here:

There was a no vote in the Scottish independence referendum. Almost immediately, David Cameron announced that Lord Smith of Kelvin would take charge of the process to turn broad UK party promises on further devolution into a more detailed plan. I discuss the main issues regarding that plan here, but in this post I want to focus more on the bigger picture, to link the discussion of Scottish devolution to academic work on the ‘universal’ challenges that all governments face:

  • Does anyone understand the policy process in Scotland?
  • Can anyone control or influence that process?
  • If not, can we hold them to account?
  • To what extent does the Scottish Government face the same challenges as any other?
  • Do Scottish political institutions have the capacity to address them in a distinctive way?

My aim is not to deny that Scottish politics is distinctive, but to argue that its political system, and policy process, shares the same ‘complex government’ features as any country. This may provide a useful sense of perspective after a long period of excitement about one aspect of British politics – which has produced the idea that (a) people know how the Scottish policy process would work after a yes or no vote, and (b) that major constitutional change produces a major change in policy and policymaking. I don’t think that either of those beliefs is true.

I also use the ‘will life go on?’ question, partly to be sarcastic, and partly to show that government and society have an auto-pilot function: while we have been obsessed with the referendum, 500,000 public employees in Scotland have continued delivering public policies out of the public spotlight, and citizens have continued to interact with public services.

In short, my aim is to show you the links between two separate-looking concerns:

Does anyone understand the policy process in Scotland and the UK?

You might get the impression from the debate on the referendum that one side knows how Scottish policymaking works; that if you vote yes or no, you guarantee a particular outcome or, at least, guard against a bad outcome. Yet, the policy process is too complex to for anyone understand fully – from the citizen, dipping in and out of political debate, to the policymaker trying to make a difference, and the academics, still confused after decades of study.

Instead, politicians and campaigners find ways to simplify the process enough to understand and explain, while academics like me develop a language to show why we couldn’t possibly understand the process. We focus on five elements which, on their own, show the complexity of policymaking and, combined, make us thoroughly confused:

  1. ‘Bounded rationality’ suggests that policymakers do not have the time, resources and cognitive ability to consider all information, possibilities, solutions, or consequences of their actions. Instead, they use informational shortcuts or heuristics to produce good-enough decisions. They may be ‘goal-oriented’, but also use emotional, intuitive and often unreliable ‘heuristics’ to make decisions quickly. Their attention may lurch dramatically from one issue to another, and they may draw on quick, emotional judgements to treat different social groups as deserving of government benefits or sanctions.
  2. Institutions are the rules, norms, and practices that influence political behaviour. Some are visible and widely understood – such as constitutions – and others are informal, often only understood by a small number of people. These are the rules that organisations develop to run a complex world into something understandable and manageable. Yet, different rules develop in many parts of government, or government ‘silos’, often with little reference to each other. This can produce: unpredictable outcomes when people follow often contradictory rules when they interact; a multiplicity of accountability and performance management processes which do not ‘join up’; and, a convoluted statute book, made more complex by the interaction between laws and regulations designed for devolved, UK and EU matters.
  3. Policy networks show us how policymakers deal with their ability to pay attention to only a fraction of the things for which they are responsible. We begin with the huge reach and responsibilities of governments, producing the potential for ministerial ‘overload’. Governments divide responsibilities into broad sectors and specialist subsectors, and senior policymakers delegate responsibility to civil servants. ‘Policy community’ describes the relationships that often develop between the actors responsible for policy decisions and the participants, such as interest groups (and businesses, public sector organisations, and other types of government body), with which they engage. For example, civil servants seek information from groups. Or, they seek legitimacy for their policies through group ‘ownership’. Groups use their resources – based on what they provide (expertise, advice, research) and/ or who they represent (a large membership; an important profession; a high status donor or corporation) – to secure regular access to government. In some cases, the relationships between policymakers and participants endure, they ‘co-produce’ policy, and we use the term ‘governance’ to describe a messy world in which it is difficult to attribute outcomes simply to the decisions of governments. Multi-level governance describes this messy process involving the blurry boundaries between policy produced by elected policymakers and civil servants, and the influence of a wide range of governmental, non-governmental and quasi-non-governmental bodies.
  4. Ideas are beliefs or ways of thinking. Some ways of thinking are accepted to such an extent that they are taken for granted or rarely challenged (we often call them ‘paradigms’). Others regard new ways of thinking, or new solutions to problems, and the persuasion necessary to prompt other actors to rethink their beliefs. The policy process involves actors competing to raise attention to problems and propose their favoured solutions. Not everyone has the same opportunity. Some can exploit a dominant understanding of the policy problem, while others have to work harder to challenge existing beliefs. A focus on ideas is a focus on power: to persuade the public, media and/ or government that there is a reason to make policy; and, to keep some issues on the agenda at the expense of others.
  5. Context describes a policymaker’s environment. It includes the policy conditions that policymakers take into account when identifying problems, such as a political system’s geography, demographic profile, economy, and mass behaviour. It can refer to a sense of policymaker ‘inheritance’ – of laws, rules, and programs – when they enter office. Or, we may identify events, either routine, such as elections, or unanticipated, including social or natural crises or major scientific breakthroughs and technological change. In each case, we consider if a policymaker’s environment is in her control and how it influences her decisions. In some cases, the role of context seems irresistible – examples include major demographic change, the role of technology in driving healthcare demand, climate change, extreme events, and ‘globalisation’. Yet, governments have shown that they can ignore such issues for long periods of time.

Can anyone control or influence that process? If not, can we hold them to account?

Each of these five elements could contribute to a sense of complexity. When combined, they suggest that the world of policymaking is too complex to predict or fully understand. They expose slogans such as ‘joined up’ or ‘holistic’ government as attempts to give the appearance of order to policymaking when we know that policymakers can only pay attention to a small portion of the issues for which they are responsible.

The idea of ‘complex government’ can be used to reject the idea – associated with the ‘Westminster model’ – that power is concentrated in the hands of a small number of people in central government. Instead, governments develop strategies to deal with the fact that their powers are rather limited in practice.

Consequently, there is a profoundly important tension between the reality of complex government and the assertion of government control and accountability. Policymakers have to justify their activities with regard to the idea of accountability to the public via ministers and Parliament. We expect ministers to deliver on their promises, and few are brave enough to admit their limitations.

Complex government also prompts us to consider how we can hold policymakers to account if the vast majority of the population does not understand how the policy process works; if policy outcomes seem to emerge in unpredictable or uncontrollable ways, or the allegation of complexity is used to undermine popular participation or obscure accountability. The aim of political reformers, to go beyond representative government and produce more participatory forms of democracy, may solve a general sense of detachment by the political class, and aid the transparency of some aspects of policymaking, but it will not solve this bigger problem.

To what extent does the Scottish Government face the same challenges as any other?

Right now, the Scottish Government faces the same task as a large number of countries:

  1. In the aftermath of economic crisis, and reduced budgets, it has to consider how to deliver similar levels of public services – including health, education, emergency services, and housing – at lower cost.
  2. It also seeks to reduce inequalities – albeit without the policy levers that could make the biggest difference.
  3. It needs to find a balance, to address an inescapable trade-off between a degree of uniformity of national policies and local discretion. People understand this problem in different ways; some bemoan the ‘fragmentation’ of public services and the potential for a ‘postcode lottery’, while others identify more positive notions of flexible government, the potential for innovation, and the value of ‘community-led’ policies or individualised, ‘co-produced’, services.
  4. It needs to find a way to ‘join up’ its public services – to make, for example, health speak to education, social work and policing.

As in many countries, one potential solution to all four problems is the idea of ‘prevention’ or ‘early intervention’. Preventative spending’ and ‘prevention’ are terms used by many governments, and in many policy studies, to describe a broad aim to reduce public service costs (and ‘demand’) by addressing policy problems at an early stage. The argument is that too much government spending is devoted to services to address severe social problems at a late stage. The aim is for governments to address a wide range of longstanding problems – including crime and anti-social behaviour, ill health and unhealthy behaviour, low educational attainment, and unemployment (and newer problems relating to climate change and anti-environmental behaviour) – by addressing them at source, before they become too severe and relatively expensive.

Yet, as in all countries, it cannot simply make this happen, for three main reasons:

  • Inequalities are often described as ‘wicked’ problems because they seem intractable – because governments do not appear to have the means, or perhaps the ability and willingness, to solve them. For example, health inequalities could be caused largely by income inequality, which the Scottish Government would struggle to address, and the UK Government may be unwilling to address radically. Or, we have a mix of solutions, from the often-innocuous (more spending on pre-school education), to the sensitive (restricting the use of alcohol and tobacco) and the downright controversial (preventing crime before it happens).
  • We are back to the idea of complex government – to address social and economic problems at this scale requires something akin to complete central government control over policies and outcomes. Instead, governments try to find ways to cooperate with a wide range of actors to secure some of their aims while dealing with the unintended consequences of their policies.
  • No one is quite sure what ‘early intervention’ or ‘prevention’ is. It sounds intuitive but, when you get into the details and need to produce a detailed plan, ambiguity and uncertainty replace intuition and a shared understanding of what to do.

So, policymakers have a limited amount of control over this process and they face the same problems as any government: the ability to pay attention to only a small proportion of issues, or to a small proportion of public service activity; the tendency for problems to be processed in government ‘silos’ (by one part of government, not communicating well with others); the potential for policymakers, in different departments or levels of government, to understand and address the policy problem in very different ways; and, ‘complexity’, which suggests that policy outcomes often ‘emerge’ from local action in the absence of central control.

These problems can only be addressed in a limited way by government strategies based on: the use of accountability and performance measures; the encouragement of learning and cooperation between public bodies; and, the development of a professional culture in which many people are committed to the same policy approach.

Do Scottish political institutions have the capacity to address them in a distinctive way?

The Scottish Government addresses this problem in two potentially-distinctive ways:

  • Policymaking culture. Many studies explore the idea of a ‘Scottish policy style’, which refers to the ways in which the Scottish Government makes policy following consultation and negotiation with pressure participants such as interest groups, local government organisations and unions
  • Administrative organisation. Many studies explore a distinctive ‘governance’ style, or a relative ability or willingness to devolve the delivery of policy to other organisations in a meaningful way. It sets a broad national strategy, the National Performance Framework, invites local bodies to produce policies consistent with it, and measures performance using broad, long term outcomes. For example, it now encourages local authorities to cooperate with a range of other bodies in the public sector (including health, enterprise, police, fire and transport), private and ‘third’ sector (mostly voluntary or charitable organisations) via established ‘Community Planning Partnerships’ (CPPs), to produce a ‘strategic vision’ for each local area.

In both cases, we usually find that the comparator is ‘Westminster’. Scotland can do things differently (at least when funding is not a problem) because it is smaller, which allows its government to develop closer relationships with key actors, and develop relatively high levels of trust in other bodies to deliver public services.

So, yes, in the context of all that I have said about governments facing the same challenges, and addressing them in similar ways, the Scottish Government has some distinctive policymaking elements.

What about the Scottish Parliament and other bodies?

Yet, consider the effect of this distinctiveness on the rest of the political system. My description of the policy process should already give the sense that it is driven primarily by government, and that parliament and ‘the people’ don’t play much of a role. What if policymaking follows its current trajectory, with more powers devolved to local authorities and a range of bodies involved in CPPs?

  1. This development has great potential to undermine traditional forms of parliamentary scrutiny. The Scottish Parliament already lacks the ability to gather information independently – it tends to rely on bodies such as the Scottish Government to provide that information. It does not get enough information from the Scottish Government about what is going on locally. Scotland lacks the top-down performance management system that we associate with the UK Government, and a greater focus on long term outcomes removes an important and regular source of information on public sector performance. Local and health authorities also push back against calls for detailed information. More devolution to local authorities would exacerbate this tension between local and national accountability.
  2. A second consequence of devolving more power locally is interest groups must reorganise, to shift from lobbying one national government to 32 local governments. Such a shift would produce new winners and losers. The well-resourced professional groups can adapt their multi-level lobbying strategies, while the groups working on a small budget, only able to lobby the Scottish Government, will struggle.
  3. These trends may prompt a new agenda on local participatory capacity, to take on the functions performed less by these national organisations. For example, the ERS Scotland’s suggestion is that more local devolution could produce a more active local population. Even so, we still need to know more about how and why people organise. For example, local communities may organise in an ad hoc way to address major issues in their area as they arise; to engage in a small part of the policy process at a particular time. They do not have the resources to engage in a more meaningful way, compared to a Parliament and collection of established groups which maintain a constant presence and develop knowledge of the details of policies over time.

Conclusion

The conclusion is that, if we focus on the wider policymaking and political process, we should get a stronger sense that a Yes vote or major further devolution would not produce radical change. The idea of giving a Scottish Government the powers to make radical changes to inequalities, public services, and outcomes, should take second stage to the idea that all governments are constrained by a lack of resources to make a quick and fundamental difference to the economy and society. No-one really understands the policy process, and no-one is in the position to control it. Rather, people pay attention to a small number of issues, and work with a large number of other people to negotiate some changes in some areas. This process involves major trade-offs, and the knowledge that attention to a small number of priorities means ignoring the rest.

So, too, should we be sceptical about the idea of a new era of popular participation, sweeping the nation and changing the way we do politics in Scotland. Even the Scottish Parliament struggles to know what happens in the Scottish Government and beyond. Even well-resourced interest groups struggle to keep track of an increasingly devolved system. So, what chance would citizens have if they did not devote their whole lives to politics? We should encourage popular participation, as the right thing to do, but without creating false expectations about the results.

Final note: in the Q&A I mention the Gilmore Girls. To follow up the reference,  see

https://paulcairney.wordpress.com/2013/09/05/democracy-max/

4 Comments

Filed under agenda setting, ESRC Scottish Centre for Constitutional Change, public policy, Scottish independence, Scottish politics, UK politics and policy

Scottish Independence and the Devil Shift

Intuitively speaking, the Scottish independence debate reinforces the idea of the ‘devil shift’ in politics. Sabatier, Hunter and McLaughlin used this phrase in 1987 to describe how advocacy coalitions (people who share the same beliefs and, to some extent, coordinate their behaviour) assess the behaviour of their opponents in ‘high conflict situations’: ‘anyone who disagrees with them must be mistaken about the facts, operating from the wrong value premises, or acting from evil motive’.

They argue that, if each coalition acted simply in a ‘rational’ way – basing their decisions on a combination of information gathering and reason – they would develop a good sense of perspective when engaging their opponents. Instead, people also use short cuts to gather information and make decisions, and this has an effect on the way that they see their opponents.

For example, people may regret losses more than they value gains, so they feel that their opponents ‘win’ more disputes, or that their own wins over their opponents are less substantial than their losses. Further, the emotional stakes are high, and people within coalitions may be more likely to feel that their opponents are more malicious or ‘evil’ than we, as outside observers, would think. This may be exacerbated by any defeat, which people may attribute to the power of their opponents rather than the power of their opponent’s arguments. They pursue 4 main hypotheses:

  1. ‘Actors will impugn the motives and/or reasonableness of their opponents while perceiving themselves to be reasonable people acting out of concern for the public welfare’.
  2. ‘Actors will evaluate their opponents’ behavior in harsher terms than will most members of their policy community, while evaluating their own behavior in more favorable terms’.
  3. ‘Actors will perceive their opponents to be more influential, and themselves to be less influential, than will most members of their policy community’.
  4. ‘The amount of distortion (or “devil shift”) is correlated with the distance between one’s beliefs and those of one’s opponents’.

I said ‘intuitively speaking’, because this just ‘chimes’ with my impression of a lot of the debate so far. We are used to political parties and campaign managers demonising their opponents as a strategy, knowing that their actual views are more sensible when they remove the public mask. What I’m not used to is ordinarily sensible people – some of them, shock horror, are academics – leaving reason at the door when describing the ideas or characteristics of their opponents (no, not you – I didn’t mean you). Too many people seem willing to demonise and overestimate the power of the people representing each campaign; to eulogise their own beliefs and predict the apocalypse if the vote goes the wrong way.

But maybe that’s because I’ve gotten in with a bad crowd and/ or I spend too much time on social media (which exacerbates a tendency to speak before thinking things through).

If we are being a bit more scientific about it, how would we demonstrate this ‘devil shift’. It’s not easy to go beyond intuition to produce something worth pursuing as a student dissertation or publishing in an academic journal. For example, Sabatier et al suggest that it is difficult to measure the strength and consistency of beliefs in coalitions without a large survey – and they go to great lengths to produce data to inform their hypotheses. I wonder if something similar is now possible to pursue using a combination of deduction and the gathering of data from places such as social media. This is yet another example of a study that I would like to see rather than one I would like to do.

See also: a discussion of a different kind of link between the church and the debate http://www.bbc.co.uk/news/uk-scotland-29036613

7 Comments

Filed under public policy, Scottish independence, Scottish politics

Scottish Independence: Will Anything Really Change?

Some issues are territorial. Others are universal.

I like to tell people that very little will change after an event such as an election or referendum. While we focus on the exciting world of party politics, elections, campaigns and a referendum, the humdrum world of policymaking goes on, often unnoticed. Indeed, one partly causes the other: our attention to a small number of high profile people allows a large number of people to carry on, delivering public policy, out of the spotlight. Some issues receive huge amounts of attention, and policy might change if there is a problem that can be solved and the solution is not too expensive or controversial. Others receive almost no attention – yet, life goes on, and policy is often delivered in much the same way as in the past. When we elect governments, or choose a completely new kind of government, we expect ministers to solve problems for us. Yet, they can only pay attention to a tiny proportion of the things for which they are responsible. No individual, or small group of people at the heart of government, has the ability to understand or control the complex government of which it is in charge. In Scotland, regardless of a Yes or No vote, one of those key issues is about how to organise, deliver and reform public services. It demonstrates two main problems that you find in any study of government and policymaking. First, there is an inescapable trade-off between a desire to harmonise national policies and to encourage local discretion. Policymakers and policy participants understand this problem in different ways; some bemoan the ‘fragmentation’ of public services and the potential for a ‘postcode lottery’, while others identify more positive notions of flexible government, the potential for innovation, and the value of ‘community-led’ policies or individualised, ‘co-produced’, services. Second, policymakers have a limited amount of control over this trade-off. They do not simply choose a level of fragmentation. Instead, they face the same problems as any government: the ability to pay attention to only a small proportion of issues, or to a small proportion of public service activity; the tendency for problems to be processed in government ‘silos’ (by one part of government, not communicating well with others); the potential for policymakers, in different departments or levels of government, to understand and address the policy problem in very different ways; and, ‘complexity’, which suggests that policy outcomes often ‘emerge’ from local action in the absence of central control. These problems can only be addressed in a limited way by government strategies based on: the use of accountability and performance measures; the encouragement of learning and cooperation between public bodies; and, the development of a professional culture in which many people are committed to the same policy approach. In our new paper, my colleagues Emily St Denny, Siabhainn Russell and I look at how the Scottish Government addresses these ‘universal’ problems. We describe its reputation for making policy in a distinctive way; its reputation for pursuing a consultative and cooperative style when it makes and implements policy. It works with a wide range of bodies to build support for its policy aims. Its policy delivery involves a broad national strategy combined with a commitment to trust bodies such as local authorities to meet its aims. In turn, local authorities work with a wide range of bodies in the public, voluntary and private sector to produce shared aims relevant to their local areas. This approach could help address problems associated with ‘silos’, ‘ambiguity’ and local discretion, if policy is ‘co-produced’ and ‘owned’ by national and local bodies. On the other hand, the ‘Scottish Approach’ implies a decision to encourage discretion, the production of a meaningful degree of local policymaking, and perhaps even the acceptance that some policies may ‘emerge’ in the absence of central direction and traditional accountability measures. To show how complicated government is, we select problems and strategies that seem more likely to exacerbate these ‘universal’ problems more than others. We outline two policy areas, on prevention and transition, which cut across many government departments, involve many levels of government (local, Scottish, UK, and perhaps EU) and types of government (including education, social work, health and police authorities), and seem particularly difficult to define and manage. In both cases, the problem is not one of disagreement. In fact, there is a widespread commitment to both issues, and to achieve a ‘decisive shift to prevention’ in particular. Rather, the problem is often one of ambiguity – for example, people are not quite sure what prevention means in practice, when applied to different kinds of policy problem – or ‘fragmentation’, when a range of public bodies have to work together to produce more specific aims and objectives. These ‘universal’ points are important when we consider Scottish policymaking in the context of constitutional change: a shift of policymaking responsibility from the UK to Scotland may reduce one aspect of complex government – such as the link between the social security system (currently reserved to the UK) and public services – but many would still remain. In cases such as prevention, Scottish independence could have an impact on budget and policy priorities. It would not, however, solve the problem of how to define and address a cross-cutting and ambiguous problem. That problem would remain, and life would go on regardless of a Yes and No vote. The paper is here: Cairney Russell St Denny ECPR 19.8.14

4 Comments

Filed under public policy, Scottish independence, Scottish politics

Four obstacles to evidence based policymaking (EBPM)

  1. Even if ‘the evidence’ exists, it doesn’t tell you what to do.
  • Sometimes there is clear evidence of a problem but not its solution.
  • The evidence may tell us that something is effective, but not if it is appropriate.
  • Scientists may exaggerate scientific consensus when they become advocates.
  • Scientists often disagree about what they are doing, how they should do it, and how science should contribute to policy.
  • These problems are exacerbated when: problems cross-cut traditional policy areas and disciplinary boundaries, the evidence base is patchy, and, the evidence comes from abroad, in an unfamiliar or unsystematic way.
  1. The demand for evidence does not match the supply.
  • Governments may fund research to seek a ‘magic bullet’ or killer piece of information to remove the need for political choice.
  • Research studies often focus on the narrow, measurable aspects of interventions but policymakers consider complex problems.
  • Policymakers pay attention to, or understand, the evidence in different ways than specialist scientists.
  • Their demand for information may be unpredictable.
  • They seek many sources of information – scientific, practical, opinion.
  • They often have to make decisions quickly and despite uncertainty.
  • They use research selectively: to bolster their case, legitimise their actions, and show that they are acting.
  • People providing evidence want an instant impact, but the effect may be more subtle, taking years or decades to filter through.
  1. People make choices in a complex policymaking system in which the role of evidence is often unclear.
  • The policy process contains many policymakers and it takes time to understand how the system works.
  • Scientists are competing with a wide range of actors (more knowledgeable of the policy process) to secure a policymaker audience and present evidence in a particular way.
  • Support for evidence-based solutions varies according to which department or unit takes the lead and how it understand the problem.
  • Bureaucracies and public bodies have operating procedures that favour particular sources of evidence and some participants over others.
  • Well-established beliefs provide the context for a consideration of new evidence.
  • Attention to evidence may lurch unpredictably following shifts in the policy environment.
  1. Evidence-based policymaking is not the same as good policymaking.
  • Minimising the evidence-policy gap means centralising power in the hands of a small number of policymakers and ensuring that scientific evidence is the sole source of knowledge for policymakers.
  • Governments may legitimately pursue alternative forms of ‘good’ policymaking based on consulting widely and generating a degree of societal, practitioner and user consensus.

Armed with this knowledge, as scientists we can choose how to adapt to those circumstances by, for example: identifying where the action takes place; learning about the properties of policymaking systems, the rules of the game, and how to frame evidence to fit policy agendas; forming coalitions with other influential actors; and, engaging in the policy process long enough to exploit windows of opportunity.

See also:

This post is one of many on EBPM. The full list is here: https://paulcairney.wordpress.com/ebpm/

How Can Policy Theory Have an Impact on Policy Making?

A ‘decisive shift to prevention’: how do we turn an idea into evidence based policy?

For a whole bunch of posts on the policy theory discussed in the originally more detailed argument see https://paulcairney.wordpress.com/1000-words/

6 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), public policy, UK politics and policy

What is ‘Complex Government’ and what can we do about it?

‘Complex government’ relates to many factors:

  • the size and multi-level nature of government
  • the proliferation of rules, regulations and public bodies
  • a crowded arena with blurry boundaries between policymakers and the actors who influence them; and
  • general uncertainty when people interact in unpredictable ways within a changeable policy environment.

Complex government is difficult to understand, control, influence and hold to account.

This brief article considers it from various perspectives: scholars trying to conceptualise it; policymakers trying to control or adapt to it; and, scientists, interest groups and individuals trying to influence it.

Cairney Complex Government 14.5.14 (submitted to a special issue – ‘Complex Government’ – in Public Money and Management)

See also: Key policy theories and concepts in 1000 words.

7 Comments

Filed under Evidence Based Policymaking (EBPM), public policy, UK politics and policy

The Psychology of Evidence Based Policymaking: Who Will Speak For the Evidence if it Doesn’t Speak for Itself?

Let’s begin with a simple – and deliberately naïve – prescription for evidence based policymaking (EBPM): there should be a much closer link between (a) the process in which scientists and knowledge brokers identify major policy problems, and (b) the process in which politicians make policy decisions. We should seek to close the ‘evidence-policy gap’. The evidence should come first and we should bemoan the inability of policymakers to act accordingly. I discuss why that argument is naïve here and here, but in terms of the complexity of policy processes and the competing claims for knowledge-based policy. This post is about the link between EBPM and psychology.

Let’s consider the role of two types of thought process common to all people, policymakers included: (a) the intuitive, gut, emotional or other heuristics we use to process and act on information quickly; and (b) goal-oriented and reasoned, thoughtful behaviour. As Daniel Kahneman’s Thinking, Fast and Slow (p 20) puts it: ‘System 1 operates automatically and quickly, with little or no effort and no sense of voluntary control. System 2 allocates attention to the effortful mental activities that demand it, including complex computations … often associated with the subjective experience of agency, choice and concentration’

The naïve description of EBPM requires System 2 (‘slow’) thinking, but what happens if most policymaking is characterised by System 1 (‘fast’)? The answer is ‘a whole bunch of cognitive shortcuts’, including:

  • the ‘availability heuristic’, when people relate the size, frequency or probability of a problem to how easy it is to remember or imagine
  • the ‘representativeness heuristic’, when people overestimate the probability of vivid events
  • ‘prospect theory’, when people value losses more than equivalent gains
  • ‘framing effects’, based on emotional and moral judgements
  • confirmation bias
  • optimism bias, or unrealistic expectations about our aims working out well when we commit to them
  • status quo bias
  • a tendency to use exemplars of social groups to represent general experience; and
  • a ‘need for coherence’ and to establish patterns and causal relationships when they may not exist (see Paul Lewis, p 7).

The ‘availability heuristic’ may also be linked to more recent studies of ‘processing fluency’ – which suggests that people’s decisions are influenced by their familiarity with things; with the ease in which they process information (see Alter and Oppenheimer, 2009). Fluency can take several forms, including conceptual, perceptual, and linguistic. For example, people may pay more attention to an issue or statement if they already possess some knowledge of it and find it easy to understand or recall. They may pay attention to people when their faces seem familiar and find fewer faults with systems they comprehend. They may place more value on things they find familiar, such as their domestic currency, items that they own compared to items they would have to buy, or the stocks of companies with more pronounceable names – even if they are otherwise identical. Or, their ability to imagine things in an abstract or concrete form may relate to their psychological ‘distance’ from it.

Is fast thinking bad thinking? Views from psychology

Alter and Oppenheimer use these insights to warn policymakers against taking the wrong attitude to regulation or spending based on flawed assessments of risk – for example, they might spend disproportionate amounts of money on projects designed to address risks with which they are most familiar (Slovic suggests that feelings towards risk may even be influenced by the way in which it is described, for example as a percentage versus a 1 in X probability). Alter and Oppenheimer also worry about medical and legal judgements swayed by fluid diagnoses and stories. Haidt argues that the identification of the ‘intuitive basis of moral judgment’ can be used to help policymakers ‘avoid mistakes’ or allow people to develop ‘programs’ or an ‘environment’ to ‘improve the quality of moral judgment and behavior’. These studies compare with arguments focusing on the positive role of emotions of decision-making, either individually (Frank) or as part of social groups, with emotional responses providing useful information in the form of social cues (Van Kleef et al).

Is fast thinking bad thinking? Views from the political and policy sciences

Social Construction Theory suggests that policymakers make quick, biased, emotional judgements, then back up their actions with selective facts to ‘institutionalize’ their understanding of a policy problem and its solution. They ‘socially construct’ their target populations to argue that they are deserving either of governmental benefits or punishments. Schneider and Ingram (forthcoming) argue that the outcomes of social construction are often dysfunctional and not based on a well-reasoned, goal-oriented strategy: ‘Studies have shown that rules, tools, rationales and implementation structures inspired by social constructions send dysfunctional messages and poor choices may hamper the effectiveness of policy’.

However, not all policy scholars make such normative pronouncements. Indeed, the value of policy theory is often to show that policy results from the interaction between large numbers of people and institutions. So, the actions of a small number of policymakers would not be the issue; we need to know more about the cumulative effect of individual emotional decision making in a collective decision-making environment – in organisations, networks and systems. For example:

  • The Advocacy Coalition Framework suggests that people engage in coordinated activity to cooperate with each other and compete with other coalitions, based on their shared beliefs and a tendency to demonise their opponents. In some cases, there are commonly accepted ways to interpret the evidence. In others, it is a battle of ideas.
  • Multiple Streams Analysis and Punctuated Equilibrium Theory focus on uncertainty and ambiguity, exploring the potential for policymaker attention to lurch dramatically from one problem or ‘image’ (the way the problem is viewed or understood). They identify the framing strategies – of actors such as ‘entrepreneurs’, ‘venue shoppers’ and ‘monopolists’ – based on a mixture of empirical facts and ‘emotional appeals’.
  • The Narrative Policy Framework combines a discussion of emotion with the identification of narrative strategies. Each narrative has a setting, characters, plot and moral. They can be compared to marketing, as persuasion based more on appealing to an audience’s beliefs (or exploiting their thought processes) than the evidence. People will pay attention to certain narratives because they are boundedly rational, seeking shortcuts to gather sufficient information – and prone to accept simple stories that seem plausible, confirm their biases, exploit their emotions, and/ or come from a source they trust.

In each case, we might see our aim as going beyond the simple phrase: ‘the evidence doesn’t speak for itself’. If ‘fast thinking’ is pervasive in policymaking, then ‘the evidence’ may only be influential if it can be provided in ways that are consistent with the thought processes of certain policymakers – such as by provoking a strong emotional reaction (to confirm or challenge biases), or framing messages in terms that are familiar to (and can be easily processed by) policymakers.

These issues are discussed further in these posts:

Is Evidence-Based Policymaking the same as good policymaking?

Policy Concepts in 1000 Words: The Psychology of Policymaking

And at more length in these papers:

PSA 2014 Cairney Psychology Policymaking 7.4.14

Cairney PSA 2014 EBPM 5.3.14

See also: Joseph Rowntree Foundation, Evidence alone won’t bring about social change

Discover Society (Delaney and Henderson) Risk and Choice in the Scottish Independence debate

4 Comments

Filed under Evidence Based Policymaking (EBPM), public policy

Policy Concepts in 1000 Words: The Psychology of Policymaking

(podcast download)

Psychology is at the heart of policymaking, but the literature on psychology is not always at the heart of policy theory. Most theories identify ‘bounded rationality’ which, on its own, is little more than a truism: people do not have the time, resources and cognitive ability to consider all information, all possibilities, all solutions, or anticipate all consequences of their actions. Consequently, they use informational shortcuts or heuristics – perhaps to produce ‘good-enough’ decisions. This is where psychology comes in, to:

  1. Describe the thought processes that people use to turn a complex world into something simple enough to understand and/ or respond to; and
  2. To compare types of thought process, such as (a) goal-oriented and reasoned, thoughtful behaviour and (b) the intuitive, gut, emotional or other heuristics we use to process and act on information quickly.

Where does policy theory come in? It seeks to situate these processes within a wider examination of policymaking systems and their environments, identifying the role of:

  • A wide range of actors making choices.
  • Institutions, as the rules, norms, and practices that influence behaviour.
  • Policy networks, as the relationships between policymakers and the ‘pressure participants’ with which they consult and negotiate.
  • Ideas – a broad term to describe beliefs, and the extent to which they are shared within groups, organisations, networks and political systems.
  • Context and events, to describe the extent to which a policymaker’s environment is in her control or how it influences her decisions.

Putting these approaches together is not easy. It presents us with an important choice regarding how to treat the role of psychology within explanations of complex policymaking systems – or, at least, on which aspect to focus.

Our first choice is to focus specifically on micro-level psychological processes, to produce hypotheses to test propositions regarding individual thought and action. There are many from which to choose, although from Daniel Kahneman’s Thinking, Fast and Slow (p 20), we can identify a basic distinction between two kinds ‘System 1 operates automatically and quickly, with little or no effort and no sense of voluntary control. System 2 allocates attention to the effortful mental activities that demand it, including complex computations … often associated with the subjective experience of agency, choice and concentration’. Further, system 1 can be related to a series of cognitive shortcuts which develop over time as people learn from experience, including:

  • the ‘availability heuristic’, when people relate the size, frequency or probability of a problem to how easy it is to remember or imagine
  • the ‘representativeness heuristic’, when people overestimate the probability of vivid events
  • ‘prospect theory’, when people value losses more than equivalent gains
  • ‘framing effects’, based on emotional and moral judgements
  • confirmation bias
  • optimism bias, or unrealistic expectations about our aims working out well when we commit to them
  • status quo bias
  • a tendency to use exemplars of social groups to represent general experience; and
  • a ‘need for coherence’ and to establish patterns and causal relationships when they may not exist (see Paul Lewis, p 7).

The ‘availability heuristic’ may also be linked to more recent studies of ‘processing fluency’ – which suggests that people’s decisions are influenced by their familiarity with things; with the ease in which they process information (see Alter and Oppenheimer, 2009). Fluency can take several forms, including conceptual, perceptual, and linguistic. For example, people may pay more attention to an issue or statement if they already possess some knowledge of it and find it easy to understand or recall. They may pay attention to people when their faces seem familiar and find fewer faults with systems they comprehend. They may place more value on things they find familiar, such as their domestic currency, items that they own compared to items they would have to buy, or the stocks of companies with more pronounceable names – even if they are otherwise identical. Or, their ability to imagine things in an abstract or concrete form may relate to their psychological ‘distance’ from it.

Our second choice is to treat these propositions as assumptions, allowing us to build larger (‘meso’ or ‘macro’ level) models that produce other hypotheses. We ask what would happen if these assumptions were true, to allow us to theorise a social system containing huge numbers of people, and/ or focus on the influence of the system or environment in which people make decisions.

These choices are made in different ways in the policy theory literature:

  • The Advocacy Coalition Framework has tested the idea of ‘devil shift’ (coalitions romanticize their own cause and demonise their opponents, misperceiving their power, beliefs and/ or motives) but also makes assumptions about belief systems and prospect theory to build models and test other assumptions.
  • Multiple Streams Analysis and Punctuated Equilibrium Theory focus on uncertainty and ambiguity, exploring the potential for policymaker attention to lurch dramatically from one problem or ‘image’ (the way the problem is viewed or understood). They identify the framing strategies of actors such as ‘entrepreneurs’, ‘venue shoppers’ and ‘monopolists’.
  • Social Construction Theory argues that policymakers make quick, biased, emotional judgements, then back up their actions with selective facts to ‘institutionalize’ their understanding of a policy problem and its solution.
  • The Narrative Policy Framework combines a discussion of emotion with the identification of ‘homo narrans’ (humans as storytellers – in stated contrast to ‘homo economicus’, or humans as rational beings). Narratives are used strategically to reinforce or oppose policy measures. Each narrative has a setting, characters, plot and moral. They can be compared to marketing, as persuasion based more on appealing to an audience’s beliefs (or exploiting their thought processes) than the evidence. People will pay attention to certain narratives because they are boundedly rational, seeking shortcuts to gather sufficient information – and prone to accept simple stories that seem plausible, confirm their biases, exploit their emotions, and/ or come from a source they trust.

These issues are discussed at more length in this paper: PSA 2014 Cairney Psychology Policymaking 7.4.14

33 Comments

Filed under 1000 words, agenda setting, public policy

How Can Policy Theory Have an Impact on Policy Making?

Paul Cairney (2015) ‘How Can Policy Theory Have an Impact on Policy Making?’ Teaching Public Administration, 33, 1, 22-39 PDF

Policymakers and academics often hold different assumptions about the policymaking world based on their different experiences. Academics may enjoy enough distance from the policy process to develop a breadth of knowledge and produce generalisable conclusions across governments, while policymakers/ practitioners such as civil servants may develop in-depth expertise when developing policy for a number of years. In turn, both may learn from each other about how to understand the policymaking world.

Academic-practitioner seminars and short training courses can help further that aim. Yet, there is a major barrier to such conversations: academics and practitioners may have their own language to understand policymaking, and a meaningful conversation may require considerable translation.

To examine these issues, this article relates my attempts, in a series of steps, to turn abstract policy theory into something useful for practitioners.

The first step is to identify a potential disconnect between the starting points for academic-practitioner discussions and policy theories. In the former, we may still use concepts developed to aid policymaking – such as the policy cycle, the ideal of ‘comprehensive rationality’ and the ‘top-down approach’ to implementation – because they aid discussion. In the latter, we have generally moved on from these descriptions of the world, to reflect the policy process’ complexity and our need for new theories to help explain it.

The second is to consider how to make those more realistic, but specialist, scientific concepts as meaningful to practitioners. The article considers the extent to which modern theories can provide straightforward insights to policy practitioners by condensing and articulating its ‘key tenets’.

The third is to consider how insights from those tenets, based largely on what governments do, can be used to recommend what they should do. The article contrasts how they might be used by a ‘top down minded’ government with how they might be used by scholars to recommend action. It focuses in particular on ‘complexity theory’ as an approach which combines policy theory with practical recommendations.

A final step is to consider how we can engage with policymakers to discuss those insights. The article draws on my experience of teaching civil servants in policy training seminars, using these theories to identify complex policymaking systems and encourage ‘reflexivity’ about how to adapt to, and operate within, them.

The article performs a dual role: as a way to explain the policy process in a straightforward way, and as a resource for civil servants engaged in policy training seminars.

Green version

See also: Is Evidence-Based Policymaking the same as good policymaking?

and/ or

Evidence Based Policy Making: If You Want to Inject More Science into Policymaking You Need to Know the Science of Policymaking

6 Comments

Filed under Evidence Based Policymaking (EBPM), public policy, UK politics and policy

Evidence Based Policy Making: If You Want to Inject More Science into Policymaking You Need to Know the Science of Policymaking

‘Evidence-based policy making’ (EBPM) is a vague, aspirational term, rather than a good description of the policy process. It can be interpreted in very different ways. At one extreme is the naïve view that there can and should be a direct and unproblematic link between ‘the evidence’ and policy decisions and outcomes. At the other is the policy-based-evidence view that politics is so corrupt that no decision is based on an appeal to scientific evidence, or so messy that the evidence gets lost somewhere in the political process. A more useful approach is to argue that EBPM is an ideal-type to compare with the real world, in much the same way as we describe a ‘comprehensively rational’ policymaker.  Then, we can draw on the policy literature which provides a wide range of theories to help situate the role of evidence within a complex policymaking system. Evidence may be important but, to identify and gauge its role, we must understand how it fits into the bigger picture in which ‘boundedly rational’ policymakers make choices based on limited information and ambiguity. This takes place in the context of a policy environment which influences how they act and how much control they have over the final outcomes. Images of highly centralised and ‘rational’ policymaking by a small number of actors generally give way to pictures of less predictable or manageable multi-level governance involving many actors.  The implication, particularly in Westminster systems, is that we should change how we see the role of evidence: from focusing on its use by policymakers at the ‘top’, or at a notional point of decision, towards explaining how it is used throughout the political system as a whole.

6 Comments

Filed under Evidence Based Policymaking (EBPM), public policy, UK politics and policy

Is Evidence-Based Policymaking the same as good policymaking?

Evidence based policymaking (EBPM) is a great idea, isn’t it? Who could object to it, apart from the enemies of science? Well, I’m sort-of going to object in two ways, by arguing: that we partly like it so much because it’s a vague idea and we don’t know what it means; and that when we are clearer on its meaning, one type of EBPM seems very problematic indeed.

Carol Weiss gives us a menu of sensible EBPM options from which to choose. Evidence can be used:

  • to inform solutions to a problem identified by policymakers
  • as one of many sources of information used by policymakers, alongside ‘stakeholder’ advice and professional and service user experience
  • as a resource used selectively by politicians, with entrenched positions, to bolster their case
  • as a tool of government, to show it is acting (by setting up a scientific study), or to measure how well policy is working
  • as a source of ‘enlightenment’, shaping how people think over the long term.

In other words, these options provide a description of the use of evidence within an often messy political system: scientists may have a role, but they struggle for attention alongside many other people. This is where a separate definition of EBPM comes in, often as a prescription for the policy process: there should be a much closer link between the process in which scientists identify major policy problems with evidence, and the process in which politicians make policy decisions. We should seek to close the ‘evidence-policy gap’. The evidence should come first and we should bemoan the inability of policymakers to act accordingly.

Most policy science-based studies of EBPM would reject this idea on descriptive grounds – as a rather naïve view about the policy process. In that sense, the call for EBPM is a revival of the idea of ‘comprehensive rationality’ in policymaking – which describes an ‘ideal-type’ and, in one sense, an optimal policy process. We assume that the values of society are reflected in the values of policymakers, and that a small number of policymakers control the policy process from its centre. Then, we highlight the conditions that would have to be met to allow those policymakers to use the government machine to turn those aims into policies – we can separate facts from values, organisations can rank a government’s preferences, the policy process is ‘linear’ and separated into clear stages, and analysis of the world is comprehensive. The point of this ideal-type is that it doesn’t exist. Instead, policy theory is about providing more realistic descriptions of the world.

On that basis, we might argue that scientists should quit moaning about the real world and start adapting to it. Stop bemoaning the pathologies of public policy – and some vague notion of the ‘lack of political will’ – and hoping for something better. If the policy process is messy and unpredictable, be pragmatic about how to engage. Balance the desire to produce a direct evidence-policy effect with a realisation that we need to frame the evidence to make it attractive to actors with very different ideas and incentives to act. Accept that policymakers seek many other legitimate sources of information and knowledge, and do not recognise, in the same way, your evidential hierarchies favouring RCTs and systematic reviews.

Even so, should we still secretly fantasise about the idealistic prescriptive side? Is EBPM an ‘ideal’, or something to aspire to even though it is unrealistic?  Not if it means something akin to comprehensive rationality. Look again at the assumptions – one of which is that a small number of policymakers control the policy process from its centre. In this scenario, EBPM is about closing the evidence policy gap by providing a clear link between scientists and politicians who centralise policymaking and make policy from the top-down with little role for debate, consultation and other forms of knowledge (one might call this ‘leadership’, often in the face of public opinion). This raises a potentially fundamental tension between EBPM and other sources of ‘good’ policymaking. What if the acceptance of one form of EBPM undermines the other roles of government?

A government may legitimately adopt a ‘bottom up’ approach to policymaking and delivery – consulting widely with a range of interest groups and public bodies to inform its aims, and working in partnership with those groups to deliver policy (perhaps by using long term, ‘co-produced’ outcomes rather than top-down and short term targets to measure success). This approach has important benefits – it generates wide ‘ownership’ of a policy solution and allows governments to generate useful feedback on the effects of policy instruments (which is important since, in practice, it may be impossible to separate the effect of an instrument from the effect of the way in which it was implemented).

If so, it would be difficult to maintain a separate EBPM process in which the central government commissions and receives the evidence which directly informs its aims to be carried out elsewhere. If a government is committed to a bottom-up policy style, it seems inevitable that it would adopt the same approach to evidence – sharing it with a wide range of bodies and ‘co-producing’ a response. If so, the use of evidence becomes much less like a linear and simple process, and much more like a complicated and interactive process in which many actors negotiate the practical implications of scientific evidence – considering it alongside other sources of policy relevant information. This has the potential to take us away from the idea of evidence-driven policy, based on external scientific standards and ‘objective’ evidence based on a hierarchy of methods, towards treating evidence as a resource to be used by actors within political systems who draw on different ideas about the hierarchy of evidential sources. As such, ‘the evidence’ is not a resource that is controlled by the scientists producing the information.

From there, we might ask: is this still EBPM? Well, this takes us back to what it means. If it means that a ‘scientific consensus’ should have super-direct policy effects, then no. If it means that scientists provide information to inform the deliberations of policymakers, who claim a legitimate policymaking role, and engage in other forms of ‘good’ policymaking – by consulting widely and generating a degree of societal, governmental and/or practitioner consensus – then yes.

Full paper (quite old): Cairney PSA 2014 EBPM 28.2.14

Full book (less old) Paul Cairney (2016) The Politics of Evidence-based Policymaking (London: Palgrave Pivot) PDF (see also)

See also: Policy Concepts in 1000 Words: ‘Evidence Based Policymaking’

Policy Concepts in 1000 Words: Bounded Rationality and Incrementalism

12 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), public policy, Scottish politics, UK politics and policy

Comparing Theories of the Policy Process: A Brief Guide for Postgraduates

When you do a PhD in policymaking, it is likely that you have to engage significantly with the literature on policy theory. This often prompts bursts of enthusiasm, when some theories seem to be spot on, and periods of frustration, when theories seem to be deficient in some way. For example, they might only explain one part of the ‘policy process’ when you want to explain it all, or they are difficult to operationalise and apply to specific cases. This may be a particular problem for PhDs focused primarily on a substantive case study. You will find that the case study is too complicated to be explained fully by a relatively simple theory designed to be applied across a range of cases. You may then think about what to do, focusing on two main possibilities:

First, should I propose my own theory, which takes insights from other theories but puts them together in a new way, perhaps with a new terminology? My advice is: no (or not unless you are convinced that you are a genius, destined to become the new Sabatier, Kingdon, Ostrom, Schneider, Ingram, Baumgartner or Jones). Don’t confuse a better explanation for your case with a better overall explanation. Your theory is not backed up in the same way by multiple applications and long term refinement.

Second, should I use the insights from a range of established policy theories to inform my case study? My advice is: yes, but be careful. Read this Policy Concepts in 1000 Words: Combining Theories. Then, consider what you are trying to do. In most cases, your aim is to account for a significant part of the policy process. Tanya Heikkila and I describe what that might entail. In Theories of the Policy Process 3rd ed.  we identify (with a lot of input from Chris Weible) what we think are the main elements of the policy process that a theory or case should cover:

  1. “Actors making choices.  We need to simplify a policymaking world which may include thousands of people, into a set of categories and/ or discussion of the key actors involved. Actors can be individuals or collectives, and collectives can range from private companies to interest groups to governments bodies. We also need to account for the ways in which people act; their calculations and motivations. For example, most theories use ‘bounded rationality’ as a springboard for explanation, while others focus on motivations such as beliefs.
  2. Institutions. These are the rules, norms, practices and relationships that influence individual and collective behaviour.  The choices of actors is explained to some extent by their understanding of, and adherence to, such rules. Those rules can be formal and widely understood, such as when enshrined in law or a constitution. Or, they can be informal and only understood in particular organisations. Policies can be considered a subset of the broad concept of institutions, but institutions at one level (e.g. constitutional rules) can also shape the policymaking activities or decisions at another level (e.g. legislation or regulation). Similarly, institutions can establish the types of venues where policy decisions are made and the rules that allow particular types of actors or information and ideas to enter into the policy process.
  3. Networks or subsystems. These are the relationships between actors responsible for policy decisions and the ‘pressure participants’ such as interest groups with which they consult and negotiate.  Senior policymakers delegate responsibility for policy making to bureaucrats, who seek information and advice from groups. Groups exchange information for access to, and potential influence within, government or other collective choice processes. It is through these networks where collective action often emerges in policy processes.  Many theories describe a process in which groups exchange information for access to, and potential influence within, government. Bureaucracies and other public bodies may have particular operating procedures that favour particular sources of evidence and some participants over others.
  4. Ideas. This is a broad term to describe beliefs, or ways of thinking, and the extent to which they are shared within groups, organisations, networks and political systems. It can refer to two intertwined processes. First, shared ideas (knowledge, world views, language) appear to structure political activity when they are almost taken for granted or rarely questioned – for example, as core beliefs, paradigms and monopolies of understanding. Second, new ideas or ways of thinking can be used to prompt actors to rethink their beliefs to some extent – such as when a proposed new solution challenges the way that a problem is framed or understood, and therefore how much attention it receives and how it is solved. So, for example, we may identify relative stability when shared ideas are not questioned and instability when different groups with different beliefs interact.
  5. Policy Context. This is a broad category to describe the extent to which a policymaker’s environment is in her control. It can refer to the often-changing policy conditions that policymakers take into account when identifying problems and deciding how to address them, such as a political system’s: geography, biophysical and demographic profile; economy; and, mass attitudes and behaviour.  It can also refer to a sense of policymaker ‘inheritance’ – of laws, rules, institutions and programs – when they enter office.
  6. Events. Events can be routine and anticipated, such as the elections which produce limited change or introduce new actors with different ideas about policy problems and solutions. Or, they can be unanticipated incidents, including social or natural crises, or major scientific breakthroughs and technological change. Their unpredictability makes them difficult to theorise, and they can often effectively be treated as ‘error’, or as external factors providing an additional source of explanation to a policy theory. Or, they can be incorporated within theories which focus on how actors interpret and respond to events. To some extent, an event is only significant if actors within political systems pay attention to them”.

From there, we can consider how each theory accounts for the nature of these elements, and their interaction, to explain policy dynamics and outcomes. You can find summaries of many of these relevant theories here: https://paulcairney.wordpress.com/1000-words/ and a copy of the 3rd and 4th edition chapters here: Cairney and Heikkila 2014, Heikkila and Cairney 2017.

For what it’s worth, when I co-authored the book Global Tobacco Control (with Studlar and Mamudu), we had chapters on the role of policy theory but structured the book according to those elements of the policy process, not specific theories. I’m not saying that you should take this simple approach. Rather, I am asking you to think about, and explain, why yours is better.

11 Comments

Filed under 1000 words, Academic innovation or navel gazing, agenda setting, public policy

Policy Concepts in 1000 Words: Bounded Rationality and Incrementalism

(podcast download)

Note: I don’t write so much about incrementalism in the 2nd edition. If you would like more of the background, please see see chapter 5 in the 1st ed of Understanding Public Policy

A classic starting point in policy studies is to compare ideal-types (which might be ideals to aspire to) with the real world. The classic example is comprehensive (or synoptic) rationality. The idea is that elected policymakers translate their values into policy in a straightforward manner. They have a clear, coherent and rank-ordered set of policy preferences which neutral organizations carry out on their behalf. We can separate policymaker values from organizational facts. There are clear-cut and ordered stages to the process (aims are identified, the means to achieve those aims are produced and one is selected) and analysis of the policymaking context is comprehensive. This allows policymakers to maximize the benefits of policy to society in much the same way that an individual maximizes her own utility.

Its comparator is ‘bounded rationality’ (coined by Simon) which suggests that policymakers’ ability to make and implement decisions is more problematic. We question our ability to separate values and facts. We note that policymakers have multiple, often unclear, objectives which are difficult to rank in any meaningful way. We wonder if the policy process is so ordered and linear (or if policymakers sometimes select a solution that already exists to a problem defined for them). We know that policymaking organizations have limited knowledge and research capabilities; that they have to use major shortcuts to gather a limited amount of information in a limited time. We know not to seek policymaking perfection, but something that is good enough. We don’t ‘maximize’ – we ‘satisfice’.

We can use this discussion to go down two main paths. The first is empirical/ descriptive. Lindblom’s famous conclusion is that bounded rationality helps cause incrementalism[i]. Organizations use simplifying strategies, such as limiting policy analysis to a small number of policy choices which diverge incrementally from the status quo (based on the argument that it is better to analyse a few issues comprehensively than seek comprehensive coverage of all issues). They use trial and error. Rather than ranking preferences in advance, they test their willingness to trade off one aim for another when they make policy decisions. They use an incremental strategy as a rule of thumb: if a previous policy commanded widespread respect then policymakers recognise the costs (analytical and political) of a significant departure from it.

If we follow this empirical path, we want to know if policymaking, and its outcome, is incremental. This picture of policymaking seemed to be common wisdom for some time: Lindblom’s critics often bemoaned the problems with the outcome, not his analysis. We can also identify related terms such as path dependence, policy succession and inheritance before choice which, albeit in different ways, highlight the dependence of current policy decisions on those made in the past. More recently, punctuated equilibrium theory suggests that incrementalism is not the full story. Rather, we can identify a mix of ‘hyper-incrementalism’ and radical change; a huge number of small changes and a small number of huge changes. Studies of policy diffusion also suggest that bounded rationality often prompts governments to emulate the (often radical) policies of other governments without fully understanding their success. So, we can now identify, in the literature, a common focus on bounded rationality as a starting point, but some very different conclusions about policy change.

The second path is normative/ prescriptive. Lindblom’s early work was often criticised for selling incrementalism as good practice: focusing on radical options is futile if no-one will countenance them anyway; policy in a series of steps reduces serious mistakes; and, existing policy is good if based on wide agreement. It prompted intense debate, focused on the extent to which incrementalism was appropriate when governments had a mandate for, or a need to engage in, radical change. Lindblom’s thinking also changed to some extent, to reflect his diminishing belief that the US political system was pluralistic and therefore a vehicle for policy based on widespread agreement.

This debate took place when Lindblom was contrasting policymaking in the US with the Soviet Union. Incrementalism was partly an antidote to ‘central planning’. While that specific debate has aged, we can still identify a broader concern with the centralization of power. The ideal of comprehensive rationality includes an assumption that power is held centrally by policymakers whose decisions are carried out by neutral bureaucrats or other organizations. In other words, a central decision maker should control the policy process; power should reside in the hands of elites at the top/ the centre at the expense of other actors. This raises debates about the balance between central and local government power, particularly when both have electoral mandates. We should also consider our need to balance authority at the top with local knowledge at the bottom; to balance the delegation of policymaking to people who know best how to do it with the maintenance of a meaningful degree of accountability for the outcomes. For example, you can see this debate play out in studies which apply complexity theory to public policy. In practice, the descriptive/ prescriptive elements of these discussions become blurred. One may argue for particular arrangements not only based on principle but also because they appear to work.

Much of the postwar debate took place in the idiosyncratic US, but incrementalism (or, in some cases, inertia) has been identified as a defining feature in systems such as Japan, Italy and Germany. It is tempting to associate incrementalism with particular types of system, such as ‘consensus democracies’ or systems with diffused power and multiple veto points – in contrast to the ‘majoritarian’ or ‘top-down’ UK. Yet, incrementalism’s key features – bounded rationality; the necessity of bargaining and compromise between actors who have different information, different interests and conflicting views; and, the need to build on past policies – may be ‘universal’.


[i] Policymaking through non-radical steps. Note that the meaning of ‘incrementalism’ is not always clear and that Lindblom is not its only exponent – look out for Wildavsky too.

60 Comments

Filed under 1000 words, public policy, UK politics and policy

Policy and Policymaking in the UK – chapter 2 draft 1

Here is the first draft of the second chapter of my new book:

Chapter 2 20.8.13 Cairney Policy Policymaking UK (direct)

https://www.dropbox.com/s/jdvji823i47c6pv/Chapter%202%2016.8.13%20Cairney%20Policy%20Policymaking%20UK.pdf (dropbox)

Please feel free to comment and share. The introduction and bullet points are at the bottom. Here are some more fiddly things that I have been thinking about when writing the chapter:

1. Is it OK to reference a blog by a think tank like the CPS? It’s not evil or anything, but I think that some people might be put off. Bias is a big thing these days don’t you know.

2. Can I still use the phrase ‘rule of thumb’ when the Daily Mail recently reinforced the myth that it refers to a rule about domestic violence? You’ll have to search that one or take my word for it.

3. At one point, am I providing a big list of concepts without enough of a structure?

4. I succumbed to temptation and used one of the most over-used phrases in social research: complex. Oh, it’s complex. Remember it’s complex. Have you considered complexity? Life’s not simple, don’t you know? I tried to make up for it with a little definition box which differentiates between complex-as-complicated and complex-as-system (with particular properties identified and used to compare with other complex systems). I have also placed (in chapter 1) my flag on the phrase ‘complex government’ – a term used remarkably infrequently given the amount of times those words are used on their own.

5. Does this video (minus the suppository problem) sum up comprehensive rationality nicely? (courtesy of NormBlog – http://normblog.typepad.com/normblog/2013/08/the-suppository-of-all-wisdom.html)

Chapter 2 Policymaking in the UK: What is Policy and How is it Made?

This chapter examines:

  • The meaning of ‘policy’ and ‘public policy’
  • How we categorise, measure and describe public policy
  • The differences between comprehensive and bounded rationality
  • The effect of bounded rationality on policymaking
  • The links between comprehensive rationality and the Westminster model
  • The differences in policymaking styles between the UK and devolved governments

‘Policy’ and ‘public policy’ are difficult to define, but our attempts to give them meaning are important. For example, some definitions refer explicitly to the actions of government, while others identify a wider range of actors. Some talk about the difference between what governments do and what they choose not to do. Some discuss the difference between a range of activities, from the general statement of intent to the actual policy outcome. Overall, these definitions show us that the policy process is complex; the inability of one, concise, definition to capture our field of study demonstrates that complexity.

Even if we can settle on a definition of public policy, we may struggle to measure it. ‘Public policy’ is a collection of policies made at different times and, in many cases, different organisations.  Even when we break policymaking down into particular areas and issues, we find that policy is a collection of different instruments. In some cases, these instruments may be used to form a coherent strategy. For example, high taxes may be combined with public education and regulations on tobacco advertising to discourage smoking. In others, individual instruments may be used by different organisations with insufficient thought about how they will combine. For example, a Department for Education may face problems when coordinating the transition from key educational stages involving different actors (including early years, primary, secondary, and post-compulsory). This is a more regular feature of fragmented policymaking, in which different government departments may be responsible for individual aspects of policy and have different ideas about how to solve the problem. For example, in mental health, a high profile Home Office policy on detaining people with mental illnesses before they commit serious crimes may conflict with a Department of Health focus on challenging mental health stigma. Consequently, the ‘what is policy?’ question has an important practical element. To know what policy is in each area, we must know how to categorise, measure and describe a wide range of government activities.

This question also informs our discussion of how policy is made. The unrealistic idea of a single, coherent, policy strategy can be linked to ‘comprehensive rationality’ in which a single authoritative policymaker is at the heart of the process. This ideal-type is a traditional starting point for policy studies – as a way to highlight the implications of bounded rationality. For example, policymakers may respond to their own limitations by developing shortcuts to gather information, engaging in incremental rather than radical policy change (to address their uncertainty about the effects of their policies), and/ or focusing on some issues at the expense of most others (Simon, 1976; Lindblom 1959; 1979; Baumgartner and Jones, 1993).

A key aspect of comprehensive rationality is that the policy process is linear; there is a clear beginning and end, reached through a series of steps.  Policymaking is also cyclical – the end of one process marks the beginning of another. ‘Policy cycle’ sums up this process. It involves breaking policy down into a series of stages including agenda setting, policy formulation, legitimation, implementation and evaluation. This overall image of policymaking is now less popular in academic research, but there is still considerable attention to individual stages – including the difference between formulation and implementation, the role of bodies other than Parliament in legitimation, and the politics of evaluating the success of policy.

A final important aspect of comprehensive rationality is that it demonstrates important similarities and differences between the UK and other political systems. On the one hand, we can associate its emphasis on sole, central control with the Westminster model’s focus on the centralisation of power in the core executive. In that context, a focus on bounded rationality shows us the practical limits to executive power. On the other, we find that policymakers in all political systems face similar constraints. A focus on bounded rationality shows us that governments are subject to universal policy processes that often produce common patterns of policymaking despite important differences in their institutional design. We can demonstrate these points by comparing the policymaking styles of the UK and devolved governments. In particular, the Scottish system was designed, in part, to diverge from ‘old Westminster’ – but UK and Scottish policy styles are often rather similar.

3 Comments

Filed under agenda setting, public policy, Scottish politics, UK politics and policy