How does ‘complexity thinking’ improve our understanding of politics and policymaking?

Presentation to ‘A jurisprudence of complexity? Rethinking the relationship between law and society’, University of Lancaster, 25th September 2015

It is customary to describe complexity theory as new, exciting, and interdisciplinary. Its advocates suggest that it offers a new way of seeing the world, a scientific paradigm to replace ‘reductionism’, a way for many academic disciplines to use the same language to explain key processes, and the potential for an impressively broad and rich empirical base. Robert Geyer and I explore these themes in the introduction and conclusion to our edited Handbook on Complexity and Public Policy.

In this short discussion, I present a more critical discussion of these high expectations, examining how they translate into something new for political and policy science, and asking: what does complexity theory offer policy studies? I suggest that its focus on greater interdisciplinarity is potentially misleading, that its theoretical appeal may be more about conceptual consolidation than novelty, and that it may take some time to demonstrate its empirical value in relation to more established theories. We can use this discussion to draw parallels between the study of policy and legal processes.

Interdisciplinarity: a language that spreads across disciplines?

Complexity theory can be sold as a way to encourage interdisciplinarity: if we all have the same theoretical outlook, use the same language, and perhaps even use the same research methods, we can combine disciplinary approaches to tackle major social and environmental problems. Yet, there are three obstacles worth discussing:

First, it is possible that we can only maintain a similar language if it is highly abstract or superficially similar. The danger is that the same words mean different things in each discipline, and that we may not always appreciate major differences in reference points. I often get this impression when speaking with natural scientists about complexity ideas in physics and biology. In some cases, we have found interesting differences in language that are easily resolved – such as that ‘first’ order change means almost no change in policy studies (with reference to Hall) but major change in physics (akin to Hall’s 3^rd order, Kuhnian change) – and others that are more difficult to overcome, such as the meaning of ‘chaos’ and the role of deterministic arguments (which seem more useful to explain natural rather than social systems).

In complexity, a key difference may be in the discussion of ‘emergence’ in the absence of a central brain or central control. In cell biology we can witness completely local interaction without a centre. In politics, there is a centre – central government – and our focus is on emergence despite its role. These are different processes which require different explanation.

Second, at least for me, complexity theory makes most sense when grounded in a well-established literature. As I discuss below, its value to policy studies may be to consolidate a range of concepts rather than provide a completely new way of thinking. The drawback is that, if this process is also necessary in each discipline, the necessity to make specific sense of complexity in each discipline undermines a general understanding: it takes time and training to become sufficiently aware of the relevant literature in each discipline; and, the translation involves using well-established concepts in one discipline that mean little in another.

Third, there are major differences of approach and understanding even within single disciplines. For example, when I wrote a policy focused piece on complexity in 2012, each reviewer noted the profound absence of discussion of particular individuals (such as Prigogine) or schools (such as Santa Fe or Brussels) without noting the same absences. Put me in a room with 5 other political scientists, interested in complexity, and I’m not sure how long it would take us to agree our terms.

A new way of looking at the world?

A lot of complexity’s claims to novelty come from a sense that complexity theory is ‘anti-reductionist’. In some subjects, it marks a major challenge to the old ways of doing science, in which, for example, we establish general laws or study individual parts of larger systems.

In the social sciences, there have been decades-long debates about the use of systems to explain social and political behaviour, and a greater sense of anti-reductionism – and ‘post-positivism’ – for some time. There may be examples in political science in which scholars identify simple regularities or trends in behaviour, but also a tradition of case study analysis to generate rich descriptions of specific decisions and events. It may therefore be more difficult to establish the sense that complexity is a new way of thinking, rather than simply the right way to think.

A newISH way of looking at the world?

Perhaps a more important ‘new’ way of seeing the world comes from complexity theory’s ability to bring together many strands of the political science literature into one framework, with reference to four main components of complex systems and their relevance to, for example, the limits to central government control.

Negative and positive feedback

First, ‘negative and positive feedback’ describes the suggestion that, in complex systems, some inputs of energy are dampened while others are amplified. Thus, small actions can have large effects and large actions can have small effects. This process can be linked to Jones and Baumgartner’s work on information processing. The cognitive ability of policymakers, and their ability to gather information, is limited. They can only pay attention to a small fraction of the issues for which they are responsible. They have to ignore most and promote a few to the top of their agenda. They receive the same amount of information over time, ignoring most for long periods (negative feedback) and paying disproportionate attention to some (positive feedback). Consequently, the controlling capacity of the centre is limited to the small number of issues to which policymakers pay particular attention or energetically seek to solve.

Strange attractors: regularities of behaviour which may be interrupted by short bursts of change

Second, positive and negative feedback extends to other parts of the system. For example, the instructions of central governments may be dampened or amplified by actors responsible for policy delivery. Much depends on the patterns of attention paid by policymakers at the ‘centre’. In theory, they could pay attention to, and influence, any part of the system. However, to do so, they have to ignore most other parts. Consequently, all rules that develop in institutions or policy networks could be challenged at any time, but most tend not to be. In this sense, ‘strange attractors’ describes the tendency for regular patterns of policymaking behaviour to persist in most cases despite the ever-present potential for policy instability (again, this is a feature of Baumgartner and Jones’ work).

Sensitivity to initial conditions and ‘path dependence’

Third, ‘sensitivity to initial conditions’ describes the tendency for events and decisions made in the distant past to contribute to the formation of institutions that influence current practices. When a commitment to a policy is established it produces ‘increasing returns’ over time: as people adapt to, and build ‘institutions’ around, the initial decision, it becomes increasingly costly to choose a different path. Initial choices are reinforced when the rules governing systemic behaviour become established and difficult to change. As a result the bulk of policy is repetitive: most policy decisions are based on legislation that already exists, most public expenditure is devoted to activities that continue by routine, and policy implementation continues long after policymakers have lost interest. ‘Critical juncture’ begins to describe the infrequent and often dramatic series of events or decisions that challenge such routines.

‘Emergence’ from the interaction between elements at a local level

The idea of ‘emergence’, in policy studies, can relate to attempts by central governments to control the system (or, in some cases, encourage ‘localism’). Emergence refers to behaviour which results from local interaction, based on locally define rules, with an emphasis on the extent to which local behaviour takes place despite central government policies or rules.

This concept resonates with the well-established literature on policy implementation and governance. For example, Lipsky frames local behaviour in terms of the limits to which actors can meet central demands, and the extent to which they draw on their own judgement and professional training when interacting with service users. Local actors face so many targets, rules and laws that no public agency or official can be expected to fulfil them all. In fact, many may be too vague or even contradictory, requiring ‘street level bureaucrats’ to choose some over others. Or, central governments may introduce performance measures which limit the discretion of delivery organizations but relate to a small part of government business.

The theme of emergence has also been a key feature of modern accounts of ‘governance’. They examine how governments have sought to respond to limited central control, particularly during the peak of New Public Management (NPM) which describes the application of private business methods to government (including attempts to secure order through hierarchical management structures and targets for public bodies to meet). They often note that central governments struggle to maintain order and, in many cases, have exacerbated their limited control by introducing a wide range of new public service delivery functions which rely on public bodies and organisations in the third and private sectors for their success.

Newish normative advice for policymakers

The same point about newishness can be said for complexity theory’s practical or normative advice: it is justified in a new (and often convincing) way, but the advice itself is not new. Complexity theory’s focus on the lack of central government control on local behaviour and policy outcomes can be linked closely to the need to be pragmatic in government, to act with a sense of realism regarding what policymakers can achieve.

If policymakers deny their reliance on other actors to help them understand and adapt to their policy-making environment, they are doomed to make the same mistakes as their predecessors. Instead, central government policymakers should embrace interdependence, to pursue more pragmatic solutions based on increasing the freedom of local actors to learn and adapt to environmental signals, such as the responses they get from service users. To address the ever-presence of uncertainty, they should make greater use of trial and error policy making. To address the inevitable gap between policymakers’ aims and policy outcomes, they should change their expectations and the way they think about policy success and evaluation.

These are the kinds of recommendations provide by Lindlom 55 years ago (when policymakers were men):

Making policy is at best a very rough process. Neither social scientists, nor politicians, nor public administrators yet know enough about the social world to avoid repeated error in predicting the consequences of policy moves. A wise policy-maker consequently expects that his policies will achieve only part of what he hopes and at the same time will produce unanticipated consequences he would have preferred to avoid. If he proceeds through a succession of incremental changes, he avoids serious lasting mistakes.

Such advice may be used by policymakers in parts of the UK and Scottish Governments, although without reference to the intricacies of complexity theory.

What is its original empirical base?

A large part of our handbook is devoted to chapters which outline new ways of thinking in a range of disciplines, or discuss the benefits of particular methods such as agent based modelling. Some chapters discuss empirical case studies, but without giving us the ability to combine their insights to help produce accumulated knowledge (at least in a straightforward way). For people seeking a payoff in terms of a bank of empirical case studies, complexity theory may seem like a source of as-yet-untapped potential. It is reasonable for scholars to wonder if we will ever get beyond a focus on new thinking and methods, towards a series of studies which can be linked to each other in a meaningful way.

The rise of systematic reviews of the literature perhaps magnify this problem, either because (a) there is now some evidence of accumulated knowledge in more established theories, or (b) the reviews show how difficult it is to accumulate knowledge when empirical studies make only vague reference to a common theory and struggle to ‘operationalise’ key concepts in a way that can be compared meaningfully to other studies (examples include multiple streams analysis – Jones et al, and Cairney and Jones – social construction theory, and the advocacy coalition framework).

Conclusion: what is complexity theory useful for?

And yet we persist with complexity theory perhaps because, if we rejected theories on the basis of the concerns I raise, we would have no theories left.

More positively, for me, it offers a way of thinking about, organising, and explaining empirical studies. For example, Emily St Denny and I are working on a book/ papers in which we use the language of complexity to explore the interaction between new policy solutions/ ideas and the old ways of doing things in policymaking systems. Although it is not easy to compare our results with others, it is valuable to use a complexity framework to provide detailed understanding of cases, and to use multiple viewpoints – zooming in to examine the perception of actors, or zooming out to observe systems and their environments – to understand the same processes.

Instead of leading to the rejection of complexity theory, these concerns prompt us to explore common topics with people in other disciplines carefully, and perhaps to reduce the claims we make about theoretical and empirical novelty while we do so. For example, the way Robert and I describe Thomas Webb’s chapter ties it closely to the themes in politics and policymaking that I discussed:

‘He draws on complexity terms, such as emergence and contingency, to argue that the legal process cannot be boiled down to a set of simple laws and rules to be implemented by government bodies such as the courts. Rather, people interpret rules and interact with each other to produce outcomes that are difficult to predict with reference to the statute book’.

Complexity also helps us think about practical problems, such as our ability to be pragmatic (and, as Robert would say, ‘humble’) or somehow manage or influence emergent behaviour (this is an immediate concern in some discussions, such as discussed by Tara Fenwick in relation to education and the classroom).

In politics, perhaps the biggest practical question relates to traditional notions of democratic accountability: how can governments let go of order and claim to be in control enough to take responsibility for their actions? Perhaps in politics and law, we may start to wonder if the law is actually made by legislators, or in practice, and how we should respond. Complexity theory doesn’t answer the question – and the literature sometimes seems to provide contradictory advice – but it gives us a useful language in which to discuss it.