Tag Archives: measuring policy change

Policy Concepts in 1000 Words: Policy Change

Christopher M. Weible & Paul Cairney

Policy change is a central concern of policy research and practice. Some want to explain it. Some want to achieve it.

Explanation begins with the ‘what is policy?’ question, since we cannot observe something without defining it.  However, we soon find that: no single definition can capture all forms of policy change, the absence of policy change is often more important, and important changes can be found in the everyday application of rules and practices related to public policies.  Further, studies often focus on changes in public policies without a focus on societal outcomes or effects.

One pragmatic solution is to define public policies as decisions made by policymakers or policymaking venues such as legislatures, executives, regulatory agencies, courts, national and local governments (and, in some countries, citizen-led policy changes).  Focusing on this type of policy change, two major categories of insights unfold:

  1. Patterns of Policy Change: incrementalism, punctuations, and drift

A focus on decisions suggests that most policymaking venues contribute primarily to incremental policy change, or often show little change from year to year but with the occasional punctuation of major policymaking activity.  This pattern reflects a frequent story about governments doing too much or nothing at all. The logic is that policymaking attention is always limited, so a focus on one issue in any policymaking venue requires minimal focus on others.  Then, when attention shifts, we see instances of major policy change as attempts to compensate (or overcompensate) for what was ignored for too long.

An additional focus on institutions highlights factors such as policy drift, to describe slow and small changes to policies, or to aspects of their design, that accumulate eventually and can have huge impacts on outcomes and society.  These drifts often happen outside the public eye or are overlooked as being negative but trivial.  For example, rising economic inequality in the US resulted from the slow accumulation of policies – related to labor unions, tax structures, and corporate governance – as well as globalization and labor-saving technologies.

  1. Factors Associated with Policy Change

Many factors help us understand instances of policy change. We can separate them analytically (as below) but, in practice, they occur simultaneously or sequentially, and can reinforce or stifle each other.

Context

Context includes history, biophysical conditions, socio-economic conditions, culture, and basic institutional structures (such as a constitution).  For example, historical and geographic conditions are often viewed as funneling or constraining the type of policy decisions made by a government.

Events 

Policymaking venues are often described as being resistant to change or in a state of equilibrium of competing political forces.  As a result, one common explanation for change is a focusing event or shock.  Events by themselves don’t create policy change. Rather, they present an opportunity for people or coalitions to exploit.   Focusing events might include disasters or crises, tragic incidents, a terrorist attack, disruptive changes in technology, or more routine events such as elections. Events may have tangible qualities, but studies tend to highlight the ways in which people frame events to construct their meaning and implications for policy.

Public Opinion 

The relationship between public opinion and policy change is a difficult one to assess.  Some research shows that the preferences of the general public only matter when they coincide with the preferences of the elite or major interest groups.  Or, it matters only when the topic is salient and the public is paying attention. Little evidence suggests that public opinion matters when few are paying attention.  Others describe public opinion as setting the boundaries within which the government operates.

Learning

Learning is a process of updating understandings of the world in response to signals from the environment.  Learning is a political activity rather than simply a technical exercise in which people learn from teachers. Learning could involve becoming aware of the severity of a policy problem, evaluating outcomes to determine if a government intervention works, and learning to trust an opponent and reach compromise. For example, certain types of rules in a collaborative process can shape the ways in which individuals gain new knowledge and change their views about the scientific evidence informing a problem.

Diffusion of Ideas 

Sometimes governments learn from or transfer policies from other governments. For example, in collections of policymaking venues (such as US state governments or EU member states) it is common for one venue to adopt a policy and prompt this policy to spread across other venues in a process of diffusion.  There are many explanations for diffusion including learning, a response to competition, mimicking, and coercion. In each case, the explanation for policy change comes from an external impetus and an internal context.

Champions and Political Associations

All policy change is driven, to some extent, by individual or group agency.  Key players include public policy champions in the form of policy entrepreneurs or in groups of government and/or non-government entities in the form of coalitions, social movements, epistemic communities, and political parties.  In each case, individuals or organizations mobilize resources, capitalize on opportunities, and apply pressure to formulate and adopt public policies.

 

The presence of these factors does not always lead to policy change, and no single study can capture a full explanation of policy change. Instead, many quantitative studies focus on multiple instances of policy change and are often broad in geographic scope or spans of time, while many case study or qualitative studies focus intensely on a very particular instance of policy change. Both approaches are essential.

See also:

Policy in 500 Words: what is public policy and why does it matter?

Policy in 500 Words: how much does policy change?

Policy Concepts in 1000 Words: Policy change and measurement (podcast download)

Policy Concepts in 1000 Words: how do policy theories describe policy change?

 

 

Leave a comment

Filed under 1000 words, public policy

Policy Concepts in 1000 Words: how do policy theories describe policy change?

The 1000 words and 500 words series already show how important but difficult it is to define and measure policy change. In this post, Leanne Giordono and I dig deeper into the – often confusingly different – ways in which different researchers conceptualise this process. We show why there is such variation and provide a checklist of questions to ask of any description of policy change.

Measuring policy change is more difficult than it looks

The measurement of policy change is important. Most ‘what is policy?’ discussions remind us that there can be a huge difference between policy as a (a)  statement of intent, (b) strategy, (c) collection of tools/ instruments and (d) contributor to policy outcomes.

Policy theories remind us that, while politicians and political parties often promise to sweep into office and produce radical departures from the past, most policy change is minor. There is a major gap between stated intention and actual outcomes, partly because policymakers do not control the policy process for which they are responsible. Instead, they inherit the commitments of their predecessors and make changes at the margins.

The 1000 words and 500 words posts suggest that we address this problem of measurement by identifying the use of a potentially large number of policy instruments or policy tools such as regulation (including legislation) and resources (money and staffing) to accentuate the power at policymaker’s disposal.

Then, they suggest that we tell a story of policy change, focusing on (a) what problem policymakers were trying to solve, and the size of their response in relation to the size of the problem, and (b) the precise nature of specific changes, or how each change contributes to the ‘big picture’.

This recommendation highlights a potentially major problem: as researchers, we can produce very different narratives of policy change from the same pool of evidence, by accentuating some measures and ignoring others, or putting more faith in some data than others.

Three ways to navigate different approaches to imagining and measuring change

Researchers use many different concepts and measures to define and identify policy change. It would be unrealistic – and perhaps unimaginative – to solve this problem with a call for one uniform approach.

Rather, our aim is to help you (a) navigate this diverse field by (b) identifying the issues and concepts that will help you interpret and compare different ways to measure change.

  1. Check if people are ‘showing their work’

Pay close attention to how scholars are defining their terms. For example, be careful with incomplete definitions that rely on a reference to evolutionary change (which can mean so many different things) or incremental change (e.g. does an increment mean small or non-radical)? Or, note that frequent distinctions between minor versus major change seem useful, but we are often trying to capture and explain a confusing mixture of both.

  1. Look out for different questions

Multiple typologies of change often arise because different theories ask and answer different questions:

  • The Advocacy Coalition Framework distinguishes between minor and major change, associating the former with routine ‘policy-oriented learning’, and the latter with changes in core policy beliefs, often caused by a ‘shock’ associated with policy failure or external events.
  • Innovation and Diffusion models examine the adoption and non-adoption of a specific policy solution over a specific period of time in multiple jurisdictions as a result of learning, imitation, competition or coercion.
  • Classic studies of public expenditure generated four categories to ask if the ‘budgetary process of the United States government is equivalent to a set of temporally stable linear decision rules’. They describe policy change as minor and predictable and explain outliers as deviations from the norm.
  • Punctuated Equilibrium Theory identifies a combination of (a) huge numbers of small policy change and (b) small numbers of huge change as the norm, in budgetary and other policy changes.
  • Hall distinguishes between (a) routine adjustments to policy instruments, (b) changes in instruments to achieve existing goals, and (c) complete shifts in goals. He compares long periods in which (1) some ideas dominate and institutions do not change, with (2) ‘third order’ change in which a profound sense of failure contributes to a radical shift of beliefs and rules.
  • More recent scholarship identifies a range of concepts – including layering, drift, conversion, and displacement – to explain more gradual causes of profound changes to institutions.

These approaches identify a range of possible sources of measures:

  1. a combination of policy instruments that add up to overall change
  2. the same single change in many places
  3. change in relation to one measure, such as budgets
  4. a change in ideas, policy instruments and/ or rules.

As such, the potential for confusion is high when we include all such measures under the single banner of ‘policy change’.

  1. Look out for different measures

Spot the different ways in which scholars try to ‘operationalize’ and measure policy change, quantitatively and/ or qualitatively, with reference to four main categories.

  1. Size can be measured with reference to:
  • A comparison of old and new policy positions.
  • A change observed in a sample or whole population (using, for example, standard deviations from the mean).
  • An ‘ideal’ state, such as an industry or ‘best practice’ standard.
  1. Speed describes the amount of change that occurs over a specific interval of time, such as:
  • How long it takes for policy to change after a specific event or under specific conditions.
  • The duration of time between commencement and completion (often described as ‘sudden’ or ‘gradual’).
  • How this speed compares with comparable policy changes in other jurisdictions (often described with reference to ‘leaders’ and ‘laggards’).
  1. Direction describes the course of the path from one policy state to another. It is often described in comparison to:
  • An initial position in one jurisdiction (such as an expansion or contraction).
  • Policy or policy change in other jurisdictions (such as via ‘benchmarking’ or ‘league tables’)
  • An ‘ideal’ state (such as with reference to left or right wing aims).
  1. Substance relates to policy change in relations to:
  • Relatively tangible instruments such as legislation, regulation, or public expenditure.
  • More abstract concepts such as in relation to beliefs or goals.

Take home points for students

Be thoughtful when drawing comparisons between applications, drawn from many theoretical traditions, and addressing different research questions.  You can seek clarity by posing three questions:

  1. How clearly has the author defined the concept of policy change?
  2. How are the chosen theories and research questions likely to influence the author’s operationalization of policy change?
  3. How does the author operationalize policy change with respect to size, speed, direction, and/or substance?

However, you should also note that the choice of definition and theory may affect the meaning of measures such as size, speed, direction, and/or substance.

 

7 Comments

Filed under 1000 words, public policy

Policy in 500 Words: how much does policy change?

You should get the impression from 1000 words that most policy changes are small or not radically different from the past: Lindblom identifies incrementalism; punctuated equilibrium  highlights a huge number of small changes and small number of huge changes; the ACF compares routine learning by a dominant coalition to a ‘shock’ which prompts new subsystem and policy dynamics; and multiple streams identifies the conditions (rarely met) for major change.

Yet, I just gave you the impression that we don’t know how to define policy. If we can’t define it well, how can we measure it well enough to come to this conclusion so consistently?

Why is the measurement of policy change important?

We miss a lot if we equate policy with statements rather than outputs/ outcomes. We also miss a lot if we equate policy change with the most visible outputs such as legislation. I list 16 different policy instruments, although they tend to be grouped into smaller categories: focusing on regulation (including legislation) and resources (money and staffing) to accentuate the power at policymaker’s disposal; or regulatory/ distributive/ redistributive to suggest that some policy measures are more difficult to ‘sell’ than others.

We also give a limited picture if we equate change with outputs rather than outcomes, since a key insight from policy studies is that there is generally a gap between policymaker expectations and the actual result.

What are the key issues in measurement?

So, as in defining policy change, we need to make choices about what counts as policy in this instance to measure how much it has changed. For example, I have (a) written on one output as a key exemplar of policy change –  legislation to ban smoking in public places for Scotland, England/ Wales, the UK, and (almost) EU – to show that a government is signalling major changes to come, but also (b) situated that policy instrument within a much broader discussion – of many tobacco policies in the UK and across the globe – to examine the extent to which it is already consistent with a well-established direction of travel.

To make such choices we need to consider:

  • Breadth (to give the ‘big picture’) versus depth (to note important details forensically)
  • How much we expect policy to change, given the size of the problem (a big feature in public health studies, which criticise government inaction)
  • How radical policy change looks from the ‘top’ (at the point of central government choice) or the ‘bottom’ (longer-term delivery of policy by other bodies)
  • What policies mean (what problem were policymakers trying to solve?)
  • How consistent ‘policy’ seems when made of often-contradictory instruments

How do we solve the problem?

The problem is that we can produce very different accounts of policy change from the same pool of evidence, by accentuating some measures and ignoring others, or putting more faith in some data more than others (e.g. during interviews).

500 words p30 UPP

Sometimes, my preferred solution is to compare more than one narrative of policy change. Another is simply to ‘show your work’.

Take home message for students: ‘show your work’ means explaining your logical process and step-by-step choices. Don’t just write that it is difficult to define policy and measure change. Instead, explain how you assess policy change in one important way, why you chose this way, and shine a light on the payoffs to your approach. Read up on how other scholars do it, to learn good practice and how to make your results comparable to theirs. Indeed, part of the benefit of using an established theory, to guide our analysis, is that we can engage in research systematically as a group.

.

.

.

PS Here is the way in which I describe these issues to MPP students writing theory-driven coursework on policy and policy change (using the case study of UK tobacco policy as a guide):

 

6 Comments

Filed under 500 words, public policy, Research design