Policy in 500 Words: how much does policy change?

You should get the impression from 1000 words that most policy changes are small or not radically different from the past: Lindblom identifies incrementalism; punctuated equilibrium  highlights a huge number of small changes and small number of huge changes; the ACF compares routine learning by a dominant coalition to a ‘shock’ which prompts new subsystem and policy dynamics; and multiple streams identifies the conditions (rarely met) for major change.

Yet, I just gave you the impression that we don’t know how to define policy. If we can’t define it well, how can we measure it well enough to come to this conclusion so consistently?

Why is the measurement of policy change important?

We miss a lot if we equate policy with statements rather than outputs/ outcomes. We also miss a lot if we equate policy change with the most visible outputs such as legislation. I list 16 different policy instruments, although they tend to be grouped into smaller categories: focusing on regulation (including legislation) and resources (money and staffing) to accentuate the power at policymaker’s disposal; or regulatory/ distributive/ redistributive to suggest that some policy measures are more difficult to ‘sell’ than others.

We also give a limited picture if we equate change with outputs rather than outcomes, since a key insight from policy studies is that there is generally a gap between policymaker expectations and the actual result.

What are the key issues in measurement?

So, as in defining policy change, we need to make choices about what counts as policy in this instance to measure how much it has changed. For example, I have (a) written on one output as a key exemplar of policy change –  legislation to ban smoking in public places for Scotland, England/ Wales, the UK, and (almost) EU – to show that a government is signalling major changes to come, but also (b) situated that policy instrument within a much broader discussion – of many tobacco policies in the UK and across the globe – to examine the extent to which it is already consistent with a well-established direction of travel.

To make such choices we need to consider:

• Breadth (to give the ‘big picture’) versus depth (to note important details forensically)
• How much we expect policy to change, given the size of the problem (a big feature in public health studies, which criticise government inaction)
• How radical policy change looks from the ‘top’ (at the point of central government choice) or the ‘bottom’ (longer-term delivery of policy by other bodies)
• What policies mean (what problem were policymakers trying to solve?)
• How consistent ‘policy’ seems when made of often-contradictory instruments

How do we solve the problem?

The problem is that we can produce very different accounts of policy change from the same pool of evidence, by accentuating some measures and ignoring others, or putting more faith in some data more than others (e.g. during interviews).

Sometimes, my preferred solution is to compare more than one narrative of policy change. Another is simply to ‘show your work’.

Take home message for students: ‘show your work’ means explaining your logical process and step-by-step choices. Don’t just write that it is difficult to define policy and measure change. Instead, explain how you assess policy change in one important way, why you chose this way, and shine a light on the payoffs to your approach. Read up on how other scholars do it, to learn good practice and how to make your results comparable to theirs. Indeed, part of the benefit of using an established theory, to guide our analysis, is that we can engage in research systematically as a group.

.

.

.

PS Here is the way in which I describe these issues to MPP students writing theory-driven coursework on policy and policy change (using the case study of UK tobacco policy as a guide):