Daily Archives: January 15, 2014

What Works (in a complex policymaking system)?

The Scottish Government and ESRC held an event yesterday to publicise their proposed new What Works centre. Its role is ‘is to deepen the impact of the emergent Scottish approach to public service delivery and reform, by evaluating evidence in delivery of that approach’. As you might expect from a series of presentations (9 in total), the speakers presented ideas which had the potential to vary in meaning and emphasis. Consequently, there seemed to be some potential tensions between things such as:

  1. A government trying to ‘scale up’ from the experience of successful pilots, and a government devolving responsibility to local bodies (such as local authorities), giving them the space to choose how to meet broad Scottish Government aims – unless ‘scaling up’ simply means encouraging ‘best’ or ‘good’ practice. Scaling up may have a different meaning in different academic disciplines and different policy areas. For example, healthcare and public health may be associated more with larger and more uniform policies than ‘community’ based projects.
  2. A focus on learning from successful projects, quite quickly, when the Scottish Government and ESRC are looking for examples of policies that produce good outcomes after, 10, 20 or more years. The former may involve drawing relatively quick conclusions about the success of projects, based on a mixture of their reputations and evidence limited to a small number of years. This is a general feature of ‘lesson-drawing’ or ‘transfer’ from one government to another – borrowers are often very quick to judge the success of lenders, based on some rough and ready indicators.

What struck me in particular was Harry Burns’ (Chief Medical Officer, Scottish Government) emphasis on the importance of complexity, complex systems and ‘complex systems thinking’. As I have found recently, when co-editing a book on complexity and public policy, complexity is a remarkably vague term which can mean very different things to different people. To many, it means something akin to ‘complicated’. To others, to identify a complex system is to identify a very specific set of arguments and properties, including:

  1. A complex system is greater than the sum of its parts; those parts are interdependent – elements interact with each other, share information and combine to produce systemic behaviour.
  2. Some attempts to influence complex systems are dampened (negative feedback) while others are amplified (positive feedback). Small actions can have large effects and large actions can have small effects.
  3. Complex systems are particularly sensitive to initial conditions that produce a long-term momentum or ‘path dependence’.
  4. They exhibit ‘emergence’, or behaviour that results from the interaction between elements at a local level rather than central direction.
  5. They may contain ‘strange attractors’ or demonstrate extended regularities of behaviour which may be interrupted by short bursts of change.

Similarly, the meaning of ‘systems thinking’ is often unclear or unhelpful when we seek to go beyond the instantly intuitive notion of thinking about the broader context of policymaking. Here are some further examples of unresolved issues regarding complexity and public policy:

  • Some people describe the natural and social world (the latter is often the thing that policymakers want, but struggle, to influence) as complex, but neglect to consider the idea that policymaking systems are complex systems. The particular relevance to What Works is that it would be a mistake to recognise the complicated set of problems to be solved by policies, but ignore the complicated set of processes to which the policies themselves are subject. This is often a key feature of wider debates on ‘evidence based policymaking’.
  • One tenet of complexity theory is that law-like behaviour is difficult to identify – so a policy that was successful in one time or place may not have the same effect in another. This places important limits on the idea of ‘scaling up’ from the experience of one study. If scaling-up means encouraging good practice, adapted to local experience, all well and good. If it means ‘rolling out’ a successful policy to many areas, it may be problematic.
  • One solution, of sorts, to such problems, is to accept that we don’t know the individual effects of individual policy instruments. Rather, we introduce a wide range of policy instruments and hope that they interact to produce a positive overall outcome. This sort of approach can be found in areas such as tobacco control (in which there may be six main types of policy, each with several elements), but may be less common or applicable in other policy areas. The complication is that it is difficult to say ‘what works’ if we mean ‘what particular policy instruments work?’.
  • ‘Systems thinking’ may prompt us deal with uncertainty and change by encouraging trial-and-error projects, or pilots, that can provide lessons, and be adopted, amended or rejected, relatively quickly. This encourages us to think about the potentially very limited role of a central government. A lot of the literature uses complexity theory to present almost the opposite idea to the ‘Westminster model’ in which power rests, and should rest, in the hands of a small number of elected policymakers, accountable to the public via Parliament.  Many advocate relying less on central government driven targets, in favour of giving local organisations more freedom to learn from their experience and adapt to their rapidly-changing environment.
  • A lot of the literature uses complexity theory to “challenge particular brands of ‘positivism’ which present a ‘vision of society based on order, laws and progress’ (Geyer and Rihani, 2010, p. 5); to suggest that ‘quantitative and reductionist methodologies’ may be useful to explain topics such as elections with ‘rules and orderly structures’, but not issues that contain unpredictable political events, significant levels of uncertainty and ambiguity (Geyer and Rihani, 2010, pp. 74–5) or factors outside the control of policy makers (Room, 2011)” (excerpt from Cairney 2012 PSR Complexity Theory). This raises the issue of methods and their underlying philosophy. At the event, he general response to a question on methods was the usual idea that we can mix them. Yet, the underlying ideas behind particular approaches may be complementary and others contradictory (see here and here Cairney 2013 PSJ Standing on the shoulder of giants for a wider discussion). A mix of methods is not a solution in itself.

A positive interpretation of these problems is that all governments face them, and some may respond better than others. This is certainly the reputation that the Scottish Government has developed, and this reputation is articulated in the What Works call:

  • The Scottish Government is unusually positioned to respond favourably because successive Scottish Governments have appeared to be much more open to this sort of advice (or, at least, they have engaged in behaviour consistent with it).
  • In particular, they have relied more on ‘partnership working’ and less on stringent performance management regimes linked to targets and punitive measures for not meeting them.
  • The SNP Government in particular (from 2007) signalled a willingness to devolve more responsibility to local authorities, reduce the proportion of ‘ring fenced’ budgets  and develop less-top-down ‘single outcome agreements’ (Remember Alex Salmond in 2007: ‘The days of top-down diktats are over’).
  • This difference of attitude might reflect cultural differences in Scotland’s political system or simply the different policymaking environment in which Scottish Governments operate.  In particular, Scotland is smaller and its policymakers have fewer responsibilities; both factors may allow them to develop quite meaningful horizontal and vertical relationships and rely less on more impersonal and inflexible policy measures.

Related posts:

Policy Concepts in 1000 Words: Policy Transfer and Learning

Policy Concepts in 1000 Words: ‘Evidence Based Policymaking’

Policy Concepts in 1000 Words: Complex Systems

Related paper: Paul Cairney 13.1.14 How Can Policy Theory Inform Policymaking

1 Comment

Filed under Academic innovation or navel gazing, Evidence Based Policymaking (EBPM), public policy, Scottish politics, UK politics and policy, Uncategorized