Policy Concepts in 1000 Words: how do policy theories describe policy change?

The 1000 words and 500 words series already show how important but difficult it is to define and measure policy change. In this post, Leanne Giordono and I dig deeper into the – often confusingly different – ways in which different researchers conceptualise this process. We show why there is such variation and provide a checklist of questions to ask of any description of policy change.

Measuring policy change is more difficult than it looks

The measurement of policy change is important. Most ‘what is policy?’ discussions remind us that there can be a huge difference between policy as a (a)  statement of intent, (b) strategy, (c) collection of tools/ instruments and (d) contributor to policy outcomes.

Policy theories remind us that, while politicians and political parties often promise to sweep into office and produce radical departures from the past, most policy change is minor. There is a major gap between stated intention and actual outcomes, partly because policymakers do not control the policy process for which they are responsible. Instead, they inherit the commitments of their predecessors and make changes at the margins.

The 1000 words and 500 words posts suggest that we address this problem of measurement by identifying the use of a potentially large number of policy instruments or policy tools such as regulation (including legislation) and resources (money and staffing) to accentuate the power at policymaker’s disposal.

Then, they suggest that we tell a story of policy change, focusing on (a) what problem policymakers were trying to solve, and the size of their response in relation to the size of the problem, and (b) the precise nature of specific changes, or how each change contributes to the ‘big picture’.

This recommendation highlights a potentially major problem: as researchers, we can produce very different narratives of policy change from the same pool of evidence, by accentuating some measures and ignoring others, or putting more faith in some data than others.

Three ways to navigate different approaches to imagining and measuring change

Researchers use many different concepts and measures to define and identify policy change. It would be unrealistic – and perhaps unimaginative – to solve this problem with a call for one uniform approach.

Rather, our aim is to help you (a) navigate this diverse field by (b) identifying the issues and concepts that will help you interpret and compare different ways to measure change.

1. Check if people are ‘showing their work’

Pay close attention to how scholars are defining their terms. For example, be careful with incomplete definitions that rely on a reference to evolutionary change (which can mean so many different things) or incremental change (e.g. does an increment mean small or non-radical)? Or, note that frequent distinctions between minor versus major change seem useful, but we are often trying to capture and explain a confusing mixture of both.

2. Look out for different questions

Multiple typologies of change often arise because different theories ask and answer different questions:

• The Advocacy Coalition Framework distinguishes between minor and major change, associating the former with routine ‘policy-oriented learning’, and the latter with changes in core policy beliefs, often caused by a ‘shock’ associated with policy failure or external events.
• Innovation and Diffusion models examine the adoption and non-adoption of a specific policy solution over a specific period of time in multiple jurisdictions as a result of learning, imitation, competition or coercion.
• Classic studies of public expenditure generated four categories to ask if the ‘budgetary process of the United States government is equivalent to a set of temporally stable linear decision rules’. They describe policy change as minor and predictable and explain outliers as deviations from the norm.
• Punctuated Equilibrium Theory identifies a combination of (a) huge numbers of small policy change and (b) small numbers of huge change as the norm, in budgetary and other policy changes.
• Hall distinguishes between (a) routine adjustments to policy instruments, (b) changes in instruments to achieve existing goals, and (c) complete shifts in goals. He compares long periods in which (1) some ideas dominate and institutions do not change, with (2) ‘third order’ change in which a profound sense of failure contributes to a radical shift of beliefs and rules.
• More recent scholarship identifies a range of concepts – including layering, drift, conversion, and displacement – to explain more gradual causes of profound changes to institutions.

These approaches identify a range of possible sources of measures:

1. a combination of policy instruments that add up to overall change
2. the same single change in many places
3. change in relation to one measure, such as budgets
4. a change in ideas, policy instruments and/ or rules.

As such, the potential for confusion is high when we include all such measures under the single banner of ‘policy change’.

3. Look out for different measures

Spot the different ways in which scholars try to ‘operationalize’ and measure policy change, quantitatively and/ or qualitatively, with reference to four main categories.

1. Size can be measured with reference to:

• A comparison of old and new policy positions.
• A change observed in a sample or whole population (using, for example, standard deviations from the mean).
• An ‘ideal’ state, such as an industry or ‘best practice’ standard.

2. Speed describes the amount of change that occurs over a specific interval of time, such as:

• How long it takes for policy to change after a specific event or under specific conditions.
• The duration of time between commencement and completion (often described as ‘sudden’ or ‘gradual’).
• How this speed compares with comparable policy changes in other jurisdictions (often described with reference to ‘leaders’ and ‘laggards’).

3. Direction describes the course of the path from one policy state to another. It is often described in comparison to:

• An initial position in one jurisdiction (such as an expansion or contraction).
• Policy or policy change in other jurisdictions (such as via ‘benchmarking’ or ‘league tables’)
• An ‘ideal’ state (such as with reference to left or right wing aims).

4. Substance relates to policy change in relations to:

• Relatively tangible instruments such as legislation, regulation, or public expenditure.
• More abstract concepts such as in relation to beliefs or goals.

Take home points for students

Be thoughtful when drawing comparisons between applications, drawn from many theoretical traditions, and addressing different research questions.  You can seek clarity by posing three questions:

1. How clearly has the author defined the concept of policy change?
2. How are the chosen theories and research questions likely to influence the author’s operationalization of policy change?
3. How does the author operationalize policy change with respect to size, speed, direction, and/or substance?

However, you should also note that the choice of definition and theory may affect the meaning of measures such as size, speed, direction, and/or substance.