Tag Archives: Policy

Policy Concepts in 1000 Words: Policy Change

Christopher M. Weible & Paul Cairney

Policy change is a central concern of policy research and practice. Some want to explain it. Some want to achieve it.

Explanation begins with the ‘what is policy?’ question, since we cannot observe something without defining it.  However, we soon find that: no single definition can capture all forms of policy change, the absence of policy change is often more important, and important changes can be found in the everyday application of rules and practices related to public policies.  Further, studies often focus on changes in public policies without a focus on societal outcomes or effects.

One pragmatic solution is to define public policies as decisions made by policymakers or policymaking venues such as legislatures, executives, regulatory agencies, courts, national and local governments (and, in some countries, citizen-led policy changes).  Focusing on this type of policy change, two major categories of insights unfold:

  1. Patterns of Policy Change: incrementalism, punctuations, and drift

A focus on decisions suggests that most policymaking venues contribute primarily to incremental policy change, or often show little change from year to year but with the occasional punctuation of major policymaking activity.  This pattern reflects a frequent story about governments doing too much or nothing at all. The logic is that policymaking attention is always limited, so a focus on one issue in any policymaking venue requires minimal focus on others.  Then, when attention shifts, we see instances of major policy change as attempts to compensate (or overcompensate) for what was ignored for too long.

An additional focus on institutions highlights factors such as policy drift, to describe slow and small changes to policies, or to aspects of their design, that accumulate eventually and can have huge impacts on outcomes and society.  These drifts often happen outside the public eye or are overlooked as being negative but trivial.  For example, rising economic inequality in the US resulted from the slow accumulation of policies – related to labor unions, tax structures, and corporate governance – as well as globalization and labor-saving technologies.

  1. Factors Associated with Policy Change

Many factors help us understand instances of policy change. We can separate them analytically (as below) but, in practice, they occur simultaneously or sequentially, and can reinforce or stifle each other.

Context

Context includes history, biophysical conditions, socio-economic conditions, culture, and basic institutional structures (such as a constitution).  For example, historical and geographic conditions are often viewed as funneling or constraining the type of policy decisions made by a government.

Events 

Policymaking venues are often described as being resistant to change or in a state of equilibrium of competing political forces.  As a result, one common explanation for change is a focusing event or shock.  Events by themselves don’t create policy change. Rather, they present an opportunity for people or coalitions to exploit.   Focusing events might include disasters or crises, tragic incidents, a terrorist attack, disruptive changes in technology, or more routine events such as elections. Events may have tangible qualities, but studies tend to highlight the ways in which people frame events to construct their meaning and implications for policy.

Public Opinion 

The relationship between public opinion and policy change is a difficult one to assess.  Some research shows that the preferences of the general public only matter when they coincide with the preferences of the elite or major interest groups.  Or, it matters only when the topic is salient and the public is paying attention. Little evidence suggests that public opinion matters when few are paying attention.  Others describe public opinion as setting the boundaries within which the government operates.

Learning

Learning is a process of updating understandings of the world in response to signals from the environment.  Learning is a political activity rather than simply a technical exercise in which people learn from teachers. Learning could involve becoming aware of the severity of a policy problem, evaluating outcomes to determine if a government intervention works, and learning to trust an opponent and reach compromise. For example, certain types of rules in a collaborative process can shape the ways in which individuals gain new knowledge and change their views about the scientific evidence informing a problem.

Diffusion of Ideas 

Sometimes governments learn from or transfer policies from other governments. For example, in collections of policymaking venues (such as US state governments or EU member states) it is common for one venue to adopt a policy and prompt this policy to spread across other venues in a process of diffusion.  There are many explanations for diffusion including learning, a response to competition, mimicking, and coercion. In each case, the explanation for policy change comes from an external impetus and an internal context.

Champions and Political Associations

All policy change is driven, to some extent, by individual or group agency.  Key players include public policy champions in the form of policy entrepreneurs or in groups of government and/or non-government entities in the form of coalitions, social movements, epistemic communities, and political parties.  In each case, individuals or organizations mobilize resources, capitalize on opportunities, and apply pressure to formulate and adopt public policies.

 

The presence of these factors does not always lead to policy change, and no single study can capture a full explanation of policy change. Instead, many quantitative studies focus on multiple instances of policy change and are often broad in geographic scope or spans of time, while many case study or qualitative studies focus intensely on a very particular instance of policy change. Both approaches are essential.

See also:

Policy in 500 Words: what is public policy and why does it matter?

Policy in 500 Words: how much does policy change?

Policy Concepts in 1000 Words: Policy change and measurement (podcast download)

Policy Concepts in 1000 Words: how do policy theories describe policy change?

 

 

Leave a comment

Filed under 1000 words, public policy

Who can you trust during the coronavirus crisis?

By Paul Cairney and Adam Wellstead, based on this paper.

Trust is essential during a crisis. It is necessary for cooperation. Cooperation helps people coordinate action, to reduce the need for imposition. It helps reduce uncertainty in a complex world. It facilitates social order and cohesiveness. In a crisis, almost-instant choices about who to trust or distrust make a difference between life and death.

Put simply, we need to trust: experts to help us understand and address the problem, governments to coordinate policy and make choices about levels of coercion, and each other to cooperate to minimise infection.

Yet, there are three unresolved problems with understanding trust in relation to coronavirus policy.

  1. What does trust really mean?

Trust is one of those words that could mean everything and nothing. We feel like we understand it intuitively, but would also struggle to define it well enough to explain how exactly it works. For example, in social science, there is some agreement on the need to describe individual motivation, social relationships, and some notion of the ‘public good’:

  • the production of trust helps boost the possibility of cooperation, partly by
  • reducing uncertainty (low information about a problem) and ambiguity (low agreement on how to understand it) when making choices, partly by
  • helping you manage the risk of making yourself vulnerable when relying on others, particularly when
  • people demonstrate trustworthiness by developing a reputation for competence, honesty, and/ or reliability, and
  • you combine cognition and emotion to produce a disposition to trust, and
  • social and political rules facilitate this process, from the formal and well-understood rules governing behaviour to the informal rules and norms shaping behaviour.

As such, trust describes your non-trivial belief in the reliability of other people, organisations, or processes. It facilitates the kinds of behaviour that are essential to an effective response to the coronavirus, in which we need to:

  1. Make judgements about the accuracy of information underpinning our choices to change behaviour (such as from scientific agencies).
  2. Assess the credibility of the people with whom we choose to cooperate or take advice (such as more or less trust in each country’s leadership).
  3. Measure the effectiveness of the governments or political systems to which we pledge our loyalty.

Crucially, in most cases, people need to put their trust in actions or outcomes caused by people they do not know, and the explanation for this kind of trust is very different to trusting people you know.

  1. What does trust look like in policymaking?

Think of trust as a mechanism to boost cooperation and coalition formation, help reduce uncertainty, and minimise the ‘transactions costs’ of cooperation (for example, monitoring behaviour, or producing or enforcing contracts). However, uncertainty is remarkably high because the policy process is not easy to understand. We can try to understand the ‘mechanisms’ of trust, to boost cooperation, with reference to these statements about trustees and the trusted:

  1. Individuals need to find ways to make choices about who to trust and distrust.
  2. However, they must act within a complex policymaking environment in which they have minimal knowledge of what will happen and who will make it happen.
  3. To respond effectively, people seek ways to cooperate with others systematically, such as by establishing formal and informal rules.

People seeking to make and influence policy must act despite uncertainty about the probability of success or risk of failure. In a crisis, it happens almost instantly. People generate beliefs about what they want to happen and how their reliance on others can help it happen. This calculation depends on:

  • Another person or organisation’s reputation for being trustworthy, allowing people the ability to increase certainty when they calculate the risk of engagement.
  • The psychology of trust and perceptions of another actor’s motives. To some extent, people gather information and use logic to determine someone’s competence. However, they also use gut feeling or emotion to help them decide to depend on someone else. They may also trust a particular source if the cognitive load is low, such as because (a) the source is familiar (e.g. a well-known politician or a celebrity, or oft-used source), or (b) the information is not challenging to remember or accept.

If so, facilitators of trust include:

  • People share the same characteristics, such as beliefs, norms, or expectations.
  • Some people have reputations for being reliable, predictable, honest, competent, and/ or relatively selfless.
  • Good experiences of previous behaviour, including repeated interactions that foster rewards and help predict future risk (with face to face contact often described as particularly helpful).
  • People may trust people in a position of authority (or the organisation or office), such as an expert or policymaker (although perhaps the threat of rule enforcement is better understood as a substitute for trust, and in practice it is difficult to spot the difference).

High levels of trust are apparent when effective practices – built on reciprocity, emotional bonds, and/ or positive expectations – become the norms or formalised and written down for all to see and agree. High levels of distrust indicate a need to deter the breach of agreements, by introducing expectations combined with sanctions for not behaving as expected.

  1. Who should you trust?

These concepts do not explain fully why people trust particular people more than others, or help us determine who you should trust during a crisis.

Rather, first, they help us reflect on the ways in which people have been describing their own thought processes (click here, and scroll to ‘Limiting the use of evidence’), such as trusting an expert source because they: (a) have a particular scientific background, (b) have proven to be honest and reliable in the past, (c) represent a wider scientific profession/ community, (d) are part of a systematic policymaking machinery, (e) can be held to account for their actions, (f) are open about the limits to their knowledge, and/or (g) engage critically with information to challenge simplistic rushes to judgement. Overall, note how much trust relates to our minimal knowledge about their research skills, prompting us to rely on an assessment of their character or status to judge their behaviour. In most cases, this is an informal process in which people may not state (or really know) why they trust or distrust someone so readily.

Then, we can reflect on who we trust, and why, and if we should change how we make such calculations during a crisis like the coronavirus. Examples include:

  • A strong identity with a left or right wing cause might prompt us only to trust people from one political party. This thought process may be efficient during elections and debates, but does it work so well during a crisis necessitating so high levels of cross-party cooperation?
  • People may be inclined to ignore advice because they do not trust their government, but maybe (a) high empathy for their vulnerable neighbours, and (b) low certainty about the impact of their actions, should prompt them to trust in government advice unless they have a tangible reason not to (while low empathy helps explain actions such as hoarding).
  • Government policy is based strongly on the extent to which policymakers trust people to do the right thing. Most debates in liberal democracies relate to the idea that (a) people can be trusted, so give advice and keep action voluntary, or cannot be trusted, so make them do the right thing, and that (b) citizens can trust their government. In other words, it must be a reciprocal relationship (see the Tweets in Step 3).

Finally, governments make policy based on limited knowledge and minimal control of the outcomes, and they often respond with trial-and-error strategies. The latter is fine if attention to policy is low and trust in government sufficiently high. However, in countries like the UK and US, each new choice prompts many people to question not only the competence of leaders but also their motivation. This is a worrying development for which everyone should take some responsibility.

See also:

Policy Concepts in 1000 Words: the Institutional Analysis and Development Framework (IAD) and Governing the Commons

The coronavirus and evidence-informed policy analysis (short version)

The coronavirus and evidence-informed policy analysis (long version)

 

5 Comments

Filed under 1000 words, 750 word policy analysis, Public health, public policy

The coronavirus and evidence-informed policy analysis (short version)

The coronavirus feels like a new policy problem that requires new policy analysis. The analysis should be informed by (a) good evidence, translated into (b) good policy. However, don’t be fooled into thinking that either of those things are straightforward. There are simple-looking steps to go from defining a problem to making a recommendation, but this simplicity masks the profoundly political process that must take place. Each step in analysis involves political choices to prioritise some problems and solutions over others, and therefore prioritise some people’s lives at the expense of others.

The very-long version of this post takes us through those steps in the UK, and situates them in a wider political and policymaking context. This post is shorter, and only scratches the surface of analysis.

5 steps to policy analysis

  1. Define the problem.

Perhaps we can sum it up as: (a) the impact of this virus and illness will be a level of death and illness that could overwhelm the population and exceed the capacity of public services, so (b) we need to contain the virus enough to make sure it spreads in the right way at the right time, so (c) we need to encourage and make people change their behaviour (primarily via hygiene and social distancing). However, there are many ways to frame this problem to emphasise the importance of some populations over others, and some impacts over others.

  1. Identify technically and politically feasible solutions.

Solutions are not really solutions: they are policy instruments that address one aspect of the problem, including taxation and spending, delivering public services, funding research, giving advice to the population, and regulating or encouraging changes to social behaviour. Each new instrument contributes an existing mix, with unpredictable and unintended consequences. Some instruments seem technically feasible (they will work as intended if implemented), but will not be adopted unless politically feasible (enough people support their introduction). Or vice versa. This dual requirement rules out a lot of responses.

  1. Use values and goals to compare solutions.

Typical judgements combine: (a) broad descriptions of values such as efficiency, fairness, freedom, security, and human dignity, (b) instrumental goals, such as sustainable policymaking (can we do it, and for how long?), and political feasibility (will people agree to it, and will it make me more or less popular or trusted?), and (c) the process to make choices, such as the extent to which a policy process involves citizens or stakeholders (alongside experts) in deliberation. They combine to help policymakers come to high profile choices (such as the balance between individual freedom and state coercion), and low profile but profound choices (to influence the level of public service capacity, and level of state intervention, and therefore who and how many people will die).

  1. Predict the outcome of each feasible solution.

It is difficult to envisage a way for the UK Government to publicise all of the thinking behind its choices (Step 3) and predictions (Step 4) in a way that would encourage effective public deliberation. People often call for the UK Government to publicise its expert advice and operational logic, but I am not sure how they would separate it from their normative logic about who should live or die, or provide a frank account without unintended consequences for public trust or anxiety. If so, one aspect of government policy is to keep some choices implicit and avoid a lot of debate on trade-offs. Another is to make choices continuously without knowing what their impact will be (the most likely scenario right now).

  1. Make a choice, or recommendation to your client.

Your recommendation or choice would build on these four steps. Define the problem with one framing at the expense of the others. Romanticise some people and not others. Decide how to support some people, and coerce or punish others. Prioritise the lives of some people in the knowledge that others will suffer or die. Do it despite your lack of expertise and profoundly limited knowledge and information. Learn from experts, but don’t assume that only scientific experts have relevant knowledge (decolonise; coproduce). Recommend choices that, if damaging, could take decades to fix after you’ve gone. Consider if a policymaker is willing and able to act on your advice, and if your proposed action will work as intended. Consider if a government is willing and able to bear the economic and political costs. Protect your client’s popularity, and trust in your client, at the same time as protecting lives. Consider if your advice would change if the problem seemed to change. If you are writing your analysis, maybe keep it down to one sheet of paper (in other words, fewer words than in this post up to this point).

Policy analysis is not as simple as these steps suggest, and further analysis of the wider policymaking environment helps describe two profound limitations to simple analytical thought and action.

  1. Policymakers must ignore almost all evidence

The amount of policy relevant information is infinite, and capacity is finite. So, individuals and governments need ways to filter out almost all of it. Individuals combine cognition and emotion to help them make choices efficiently, and governments have equivalent rules to prioritise only some information. They include: define a problem and a feasible response, seek information that is available, understandable, and actionable, and identify credible sources of information and advice. In that context, the vague idea of trusting or not trusting experts is nonsense, and the larger post highlights the many flawed ways in which all people decide whose expertise counts.

  1. They do not control the policy process.

Policymakers engage in a messy and unpredictable world in which no single ‘centre’ has the power to turn a policy recommendation into an outcome.

  • There are many policymakers and influencers spread across a political system. For example, consider the extent to which each government department, devolved governments, and public and private organisations are making their own choices that help or hinder the UK government approach.
  • Most choices in government are made in ‘subsystems’, with their own rules and networks, over which ministers have limited knowledge and influence.
  • The social and economic context, and events, are largely out of their control.

The take home messages (if you accept this line of thinking)

  1. The coronavirus is an extreme example of a general situation: policymakers will always have very limited knowledge of policy problems and control over their policymaking environment. They make choices to frame problems narrowly enough to seem solvable, rule out most solutions as not feasible, make value judgements to try help some more than others, try to predict the results, and respond when the results do not match their hopes or expectations.
  2. This is not a message of doom and despair. Rather, it encourages us to think about how to influence government, and hold policymakers to account, in a thoughtful and systematic way that does not mislead the public or exacerbate the problem we are seeing. No one is helping their government solve the problem by saying stupid shit on the internet (OK, that last bit was a message of despair).

 

Further reading:

The longer report sets out these arguments in much more detail, with some links to further thoughts and developments.

This series of ‘750 words’ posts summarises key texts in policy analysis and tries to situate policy analysis in a wider political and policymaking context. Note the focus on whose knowledge counts, which is not yet a big feature of this crisis.

These series of 500 words and 1000 words posts (with podcasts) summarise concepts and theories in policy studies.

This page on evidence-based policymaking (EBPM) uses those insights to demonstrate why EBPM is  a political slogan rather than a realistic expectation.

These recorded talks relate those insights to common questions asked by researchers: why do policymakers seem to ignore my evidence, and what can I do about it? I’m happy to record more (such as on the topic you just read about) but not entirely sure who would want to hear what.

3 Comments

Filed under 750 word policy analysis, agenda setting, Evidence Based Policymaking (EBPM), Policy learning and transfer, POLU9UK, Prevention policy, Psychology Based Policy Studies, Public health, public policy, Social change, UK politics and policy

The coronavirus and evidence-informed policy analysis (long version)

This is the long version. It is long. Too long to call a blog post. Let’s call it a ‘living document’ that I update and amend as new developments arise (then start turning into a more organised paper). In most cases, I am adding tweets, so the date of the update is embedded. If I add a new section, I will add a date. If you seek specific topics (like ‘herd immunity’), it might be worth doing a search. The short version is shorter.

The coronavirus feels like a new policy problem. Governments already have policies for public health crises, but the level of uncertainty about the spread and impact of this virus seems to be taking it to a new level of policy, media, and public attention. The UK Government’s Prime Minister calls it ‘the worst public health crisis for a generation’.

As such, there is no shortage of opinions on what to do, but there is a shortage of well-considered opinions, producing little consensus. Many people are rushing to judgement and expressing remarkably firm opinions about the best solutions, but their contributions add up to contradictory evaluations, in which:

  • the government is doing precisely the right thing or the completely wrong thing,
  • we should listen to this expert saying one thing or another expert saying the opposite.

Lots of otherwise-sensible people are doing what they bemoan in politicians: rushing to judgement, largely accepting or sharing evidence only if it reinforces that judgement, and/or using their interpretation of any new development to settle scores with their opponents.

Yet, anyone who feels, without uncertainty, that they have the best definition of, and solution to, this problem is a fool. If people are also sharing bad information and advice, they are dangerous fools. Further, as Professor Madley puts it (in the video below), ‘anyone who tells you they know what’s going to happen over the next six months is lying’.

In that context, how can we make sense of public policy to address the coronavirus in a more systematic way?

Studies of policy analysis and policymaking do not solve a policy problem, but they at least give us a language to think it through.

  1. Let’s focus on the UK as an example, and use common steps in policy analysis, to help us think through the problem and how to try to manage it.
  • In each step, note how quickly it is possible to be overwhelmed by uncertainty and ambiguity, even when the issue seems so simple at first.
  • Note how difficult it is to move from Step 1, and to separate Step 1 from the others. It is difficult to define the problem without relating it to the solution (or to the ways in which we will evaluate each solution).
  1. Let’s relate that analysis to research on policymaking, to understand the wider context in which people pay attention to, and try to address, important problems that are largely out of their control.

Throughout, note that I am describing a thought process as simply as I can, not a full examination of relevant evidence. I am highlighting the problems that people face when ‘diagnosing’ policy problems, not trying to diagnose it myself. To do so, I draw initially on common advice from the key policy analysis texts (summaries of the texts that policy analysis students are most likely to read) that simplify the process a little too much. Still, the thought process that it encourages took me hours alone (spread over three days) to produce no real conclusion. Policymakers and advisers, in the thick of this problem, do not have that luxury of time or uncertainty.

See also: Boris Johnson’s address to the nation in full (23.3.20) and press conference transcripts

https://twitter.com/BorisJohnson/status/1246358936585986048

https://twitter.com/BorisJohnson/status/1243496858095411200

https://twitter.com/R_S_P_H/status/1242833029728477188

Step 1 Define the problem

Common advice in policy analysis texts:

  • Provide a diagnosis of a policy problem, using rhetoric and eye-catching data to generate attention.
  • Identify its severity, urgency, cause, and our ability to solve it. Don’t define the wrong problem, such as by oversimplifying.
  • Problem definition is a political act of framing, as part of a narrative to evaluate the nature, cause, size, and urgency of an issue.
  • Define the nature of a policy problem, and the role of government in solving it, while engaging with many stakeholders.
  • ‘Diagnose the undesirable condition’ and frame it as ‘a market or government failure (or maybe both)’.

Coronavirus as a physical problem is not the same as a coronavirus policy problem. To define the physical problem is to identify the nature, spread, and impact of a virus and illness on individuals and populations. To define a policy problem, we identify the physical problem and relate it (implicitly or explicitly) to what we think a government can, and should, do about it. Put more provocatively, it is only a policy problem if policymakers are willing and able to offer some kind of solution.

This point may seem semantic, but it raises a profound question about the capacity of any government to solve a problem like an epidemic, or for governments to cooperate to solve a pandemic. It is easy for an outsider to exhort a government to ‘do something!’ (or ‘ACT NOW!’) and express certainty about what would happen. However, policymakers inside government:

  1. Do not enjoy the same confidence that they know what is happening, or that their actions will have their intended consequences, and
  2. Will think twice about trying to regulate social behaviour under those circumstances, especially when they
  3. Know that any action or inaction will benefit some and punish others.

For example, can a government make people wash their hands? Or, if it restricts gatherings at large events, can it stop people gathering somewhere else, with worse impact? If it closes a school, can it stop children from going to their grandparents to be looked after until it reopens? There are 101 similar questions and, in each case, I reckon the answer is no. Maybe government action has some of the desired impact; maybe not. If you agree, then the question might be: what would it really take to force people to change their behaviour?

See also: Coronavirus has not suspended politics – it has revealed the nature of power (David Runciman)

The answer is: often too much for a government to consider (in a liberal democracy), particularly if policymakers are informed that it will not have the desired impact.

https://twitter.com/AdamJKucharski/status/1238152492178976769

If so, the UK government’s definition of the policy problem will incorporate this implicit question: what can we do if we can influence, but not determine (or even predict well) how people behave?

Uncertainty about the coronavirus plus uncertainty about policy impact

Now, add that general uncertainty about the impact of government to this specific uncertainty about the likely nature and spread of the coronavirus:

https://www.youtube.com/watch?time_continue=350&v=blkDulsgh3Q&feature=emb_logo

A summary of this video suggests:

  • There will be an epidemic (a profound spread to many people in a short space of time), then the problem will be endemic (a long-term, regular feature of life) (see also UK policy on coronavirus COVID-19 assumes that the virus is here to stay).
  • In the absence of a vaccine, the only way to produce ‘herd immunity’ is for most people to be infected and recover

[Note: there is much debate on whether ‘herd immunity’ is or is not government policy. Much of it relates to interpretation, based on levels of trust/distrust in the UK Government, its Prime Minister, and the Prime Minister’s special adviser. I discuss this point below under ‘trial and error policymaking’. See also Who can you trust during the coronavirus crisis? ]

  • The ideal spread involves all well people sharing the virus first, while all vulnerable people (e.g. older, and/or with existing health problems that affect their immune systems) protected in one isolated space, but it won’t happen like that; so, we are trying to minimise damage in the real world.
  • We mainly track the spread via deaths, with data showing a major spike appearing one month later, so the problem may only seem real to most people when it is too late to change behaviour

https://twitter.com/ChrisGiles_/status/1247458186300456960

https://twitter.com/d_spiegel/status/1248157520943857665

https://twitter.com/d_spiegel/status/1247824140645683205

https://twitter.com/EmergMedDr/status/1250039068890726400

See also: Coronavirus: Government expert defends not closing UK schools (BBC, Sir Patrick Vallance 13th March 2020)

https://twitter.com/DrSamSims/status/1247445729439895555

  • The choice in theory is between a rapid epidemic with a high peak, or a slowed-down epidemic over a longer period, but ‘anyone who tells you they know what’s going to happen over the next six months is lying’.
  • Maybe this epidemic will be so memorable as to shift social behaviour, but so much depends on trying to predict (badly) if individuals will actually change (see also Spiegelhalter on communicating risk).

None of this account tells policymakers what to do, but at least it helps them clarify three key aspects of their policy problem:

  1. The impact of this virus and illness could overwhelm the population, to the extent that it causes mass deaths, causes a level of illness that exceeds the capacity of health services to treat, and contributes to an unpredictable amount of social and economic damage.
  2. We need to contain the virus enough to make sure it (a) spreads at the right speed and/or (b) peaks at the right time. The right speed seems to be: a level that allows most people to recover alone, while the most vulnerable are treated well in healthcare settings that have enough capacity. The right time seems to be the part of the year with the lowest demand on health services (e.g. summer is better than winter). In other words, (a) reduce the size of the peak by ‘flattening the curve’, and/or (b) find the right time of year to address the peak, while (c) anticipating more than one peak.

My impression is that the most frequently-expressed aim is (a) …

https://twitter.com/STVNews/status/1238468179036459008

https://twitter.com/DHSCgovuk/status/1238540941717356548

… while the UK Government’s Deputy Chief Medical Officer also seems to be describing (b):

  1. We need to encourage or coerce people to change their behaviour, to look after themselves (e.g. by handwashing) and forsake their individual preferences for the sake of public health (e.g. by self-isolating or avoiding vulnerable people). Perhaps we can foster social trust and empathy to encourage responsible individual action. Perhaps people will only protect others if obliged to do so (compare Stone; Ostrom; game theory).

See also: From across the Ditch: How Australia has to decide on the least worst option for COVID-19 (Prof Tony Blakely on three bad options: (1) the likelihood of ‘elimination’ of the virus before vaccination is low; (2) an 18-month lock-down will help ‘flatten the curve’; (3) ‘to prepare meticulously for allowing the pandemic to wash through society over a period of six or so months. To tool up the production of masks and medical supplies. To learn as quickly as possible which treatments of people sick with COVID-19 saves lives. To work out our strategies for protection of the elderly and those with a chronic condition (for whom the mortality from COVID-19 is much higher’).

https://twitter.com/luciadambruoso/status/1246361265909444608

https://twitter.com/anandMenon1/status/1246712962519310337

From uncertainty to ambiguity

If you are still with me, I reckon you would have worded those aims slightly differently, right? There is some ambiguity about these broad intentions, partly because there is some uncertainty, and partly because policymakers need to set rather vague intentions to generate the highest possible support for them. However, vagueness is not our friend during a crisis involving such high anxiety. Further, they are only delaying the inevitable choices that people need to make to turn a complex multi-faceted problem into something simple enough to describe and manage. The problem may be complex, but our attention focuses only on a small number of aspects, at the expense of the rest. Examples that have arisen, so far, include to accentuate:

  1. The health of the whole population or people who would be affected disproportionately by the illness.
  • For example, the difference in emphasis affects the health advice for the relatively vulnerable (and the balance between exhortation and reassurance)

https://twitter.com/colinrtalbot/status/1238227267471527937?s=09

https://twitter.com/hacscot/status/1240588827829436416?s=09

https://twitter.com/lisatrigg/status/1249670660802187266

 

  1. Inequalities in relation to health, socio-economic status (e.g. income, gender, race, ethnicity), or the wider economy.
  • For example, restrictive measures may reduce the risk of harm to some, but increase the burden on people with no savings or reliable sources of income.
  • For example, some people are hoarding large quantities of home and medical supplies that (a) other people cannot afford, and (b) some people cannot access, despite having higher need.
  • For example, social distancing will limit the spread of the virus (see the nascent evidence), but also produce highly unequal forms of social isolation that increase the risk of domestic abuse (possibly exacerbated by school closures) and undermine wellbeing. Or, there will be major policy changes, such as to the rules to detain people under mental health legislation, regarding abortion, or in relation to asylum (note: some of these tweets are from the US, partly because I’m seeing more attention to race – and the consequence of systematic racism on the socioeconomic inequalities so important to COVID-19 mortality – than in the UK).

See also: COVID-19: how the UK’s economic model contributes towards a mismanagement of the crisis (Carolina Alves and Farwa Sial 30.3.20),

Economic downturn and wider NHS disruption likely to hit health hard – especially health of most vulnerable (Institute for Fiscal Studies 9.4.20),

Don’t be fooled: Britain’s coronavirus bailout will make the rich richer still (Christine Berry 13.4.20)

https://twitter.com/closethepaygap/status/1244579870392422400

https://twitter.com/heyDejan/status/1238944695260233728?s=09

https://twitter.com/TimothyNoah1/status/1240375741809938433

https://twitter.com/politicshome/status/1249236632009691136?s=09

 

https://twitter.com/NPR/status/1246837779474120705?s=09

https://twitter.com/povertyscholar/status/1246487621230092294

https://twitter.com/Yamiche/status/1248028548998344708

https://twitter.com/MalindaSmith/status/1247281226274107392

https://twitter.com/Jas_Athwal/status/1248875273568878592?s=09

https://twitter.com/GKBhambra/status/1248874500764073989

cc

https://twitter.com/sunny_hundal/status/1247454112762990592

https://twitter.com/olivernmoody/status/1248260326140805125

https://twitter.com/boodleoops/status/1246717497308577792

https://twitter.com/boodleoops/status/1246717497308577792

https://twitter.com/MarioLuisSmall/status/1239879542094925825

https://twitter.com/kevinstoneUWE/status/1240000285046640645?s=09

https://twitter.com/colinimckay/status/1240721797731045378?s=09

https://twitter.com/heytherehurley/status/1242113416103432195

https://twitter.com/stellacreasy/status/1244022413865648128

https://twitter.com/NIOgov/status/1246482663738871811

https://twitter.com/refugeecouncil/status/1243842703680471040

https://twitter.com/libertyhq/status/1248173788598013953

https://twitter.com/TheLancet/status/1246039259880054784

https://twitter.com/profhrs/status/1247572112061222914

https://twitter.com/HumzaYousaf/status/1248262165657722885

  • For example, governments cannot ignore the impact of their actions on the economy, however much they emphasise mortality, health, and wellbeing. Most high-profile emphasis was initially on the fate of large and small businesses, and people with mortgages, but a long period of crisis will a tip the balance from low income to unsustainable poverty (even prompting Iain Duncan Smith to propose policy change), and why favour people who can afford a mortgage over people scraping the money together for rent?
  1. A need for more communication and exhortation, or for direct action to change behaviour.
  2. The short term (do everything possible now) or long term (manage behaviour over many months).
  1. How to maintain trust in the UK government when (a) people are more or less inclined to trust a the current part of government and general trust may be quite low, and (b) so many other governments are acting differently from the UK.

https://twitter.com/DrSophieHarman/status/1238893265782530059

https://twitter.com/Sander_vdLinden/status/1242168652180475906?s=09

https://twitter.com/policyatkings/status/1248318259029516289

  • For example, note the visible presence of the Prime Minister, but also his unusually high deference to unelected experts such as (a) UK Government senior scientists providing direct advice to ministers and the public, and (b) scientists drawing on limited information to model behaviour and produce realistic scenarios (we can return to the idea of ‘evidence-based policymaking’ later). This approach is not uncommon with epidemics/ pandemics (LD was then the UK Government’s Chief Medical Officer):

https://twitter.com/AndyBurnhamGM/status/1239153510903619584

  • For example, note how often people are second guessing and criticising the UK Government position (and questioning the motives of Conservative ministers).

See also: Coronavirus: meet the scientists who are now household names

  1. How policy in relation to the coronavirus relates to other priorities (e.g. Brexit, Scottish independence, trade, education, culture)

7. Who caused, or who is exacerbating, the problem? The answers to such questions helps determine which populations are most subject to policy intervention.

  • For example, people often try to lay blame for viruses on certain populations, based on their nationality, race, ethnicity, sexuality, or behaviour (e.g. with HIV).
  • For example, the (a) association between the coronavirus and China and Chinese people (e.g. restrict travel to/ from China; e.g. exacerbate racism), initially overshadowed (b) the general role of international travellers (e.g. place more general restrictions on behaviour), and (c) other ways to describe who might be responsible for exacerbating a crisis.

See also: ‘Othering the Virus‘ by Marius Meinhof

Under ‘normal’ policymaking circumstances, we would expect policymakers to resolve this ambiguity by exercising power to set the agenda and make choices that close off debate. Attention rises at first, a choice is made, and attention tends to move on to something else. With the coronavirus, attention to many different aspects of the problem has been lurching remarkably quickly. The definition of the policy problem often seems to be changing daily or hourly, and more quickly than the physical problem. It will also change many more times, particularly when attention to each personal story of illness or death prompts people to question government policy every hour. If the policy problem keeps changing in these ways, how could a government solve it?

Step 2 Identify technically and politically feasible solutions

Common advice in policy analysis texts:

  • Identify the relevant and feasible policy solutions that your audience/ client might consider.
  • Explain potential solutions in sufficient detail to predict the costs and benefits of each ‘alternative’ (including current policy).
  • Provide ‘plausible’ predictions about the future effects of current/ alternative policies.
  • Identify many possible solutions, then select the ‘most promising’ for further analysis.
  • Identify how governments have addressed comparable problems, and a previous policy’s impact.

Policy ‘solutions’ are better described as ‘tools’ or ‘instruments’, largely because (a) it is rare to expect them to solve a problem, and (b) governments use many instruments (in different ways, at different times) to make policy, including:

  1. Public expenditure (e.g. to boost spending for emergency care, crisis services, medical equipment)
  2. Economic incentives and disincentives (e.g. to reduce the cost of business or borrowing, or tax unhealthy products)
  3. Linking spending to entitlement or behaviour (e.g. social security benefits conditional on working or seeking work, perhaps with the rules modified during crises)
  4. Formal regulations versus voluntary agreements (e.g. making organisations close, or encouraging them to close)
  5. Public services: universal or targeted, free or with charges, delivered directly or via non-governmental organisations
  6. Legal sanctions (e.g. criminalising reckless behaviour)
  7. Public education or advertising (e.g. as paid adverts or via media and social media)
  8. Funding scientific research, and organisations to advise on policy
  9. Establishing or reforming policymaking units or departments
  10. Behavioural instruments, to ‘nudge’ behaviour (seemingly a big feature in the UK , such as on how to encourage handwashing).

As a result, what we call ‘policy’ is really a complex mix of instruments adopted by one or more governments. A truism in policy studies is that it is difficult to define or identify exactly what policy is because (a) each new instrument adds to a pile of existing measures (with often-unpredictable consequences), and (b) many instruments designed for individual sectors tend, in practice, to intersect in ways that we cannot always anticipate. When you think through any government response to the coronavirus, note how every measure is connected to many others.

Further, it is a truism in public policy that there is a gap between technical and political feasibility: the things that we think will be most likely to work as intended if implemented are often the things that would receive the least support or most opposition. For example:

  1. Redistributing income and wealth to reduce socio-economic inequalities (e.g. to allay fears about the impact of current events on low-income and poverty) seems to be less politically feasible than distributing public services to deal with the consequences of health inequalities.
  2. Providing information and exhortation seems more politically feasible than the direct regulation of behaviour. Indeed, compared to many other countries, the UK Government seems reluctant to introduce ‘quarantine’ style measures to restrict behaviour.

Under ‘normal’ circumstances, governments may be using these distinctions as simple heuristics to help them make modest policy changes while remaining sufficiently popular (or at least looking competent). If so, they are adding or modifying policy instruments during individual ‘windows of opportunity’ for specific action, or perhaps contributing to the sense of incremental change towards an ambitious goal.

Right now, we may be pushing the boundaries of what seems possible, since crises – and the need to address public anxiety – tend to change what seems politically feasible. However, many options that seem politically feasible may not be possible (e.g. to buy a lot of extra medical/ technology capacity quickly), or may not work as intended (e.g. to restrict the movement of people). Think of technical and political feasibility as necessary but insufficient on their own, which is a requirement that rules out a lot of responses.

https://twitter.com/CairneyPaul/status/1244970044351791104

https://twitter.com/ChrisCEOHopson/status/1249617980859744256?s=09

Step 3 Use value-based criteria and political goals to compare solutions

Common advice in policy analysis texts:

  • Typical value judgements relate to efficiency, equity and fairness, the trade-off between individual freedom and collective action, and the extent to which a policy process involves citizens in deliberation.
  • Normative assessments are based on values such as ‘equality, efficiency, security, democracy, enlightenment’ and beliefs about the preferable balance between state, communal, and market/ individual solutions
  • ‘Specify the objectives to be attained in addressing the problem and the criteria  to  evaluate  the  attainment  of  these  objectives  as  well as  the  satisfaction  of  other  key  considerations  (e.g.,  equity,  cost, equity, feasibility)’.
  • ‘Effectiveness, efficiency, fairness, and administrative efficiency’ are common.
  • Identify (a) the values to prioritise, such as ‘efficiency’, ‘equity’, and ‘human dignity’, and (b) ‘instrumental goals’, such as ‘sustainable public finance or political feasibility’, to generate support for solutions.
  • Instrumental questions may include: Will this intervention produce the intended outcomes? Is it easy to get agreement and maintain support? Will it make me popular, or diminish trust in me even further?

Step 3 is the most simple-looking but difficult task. Remember that it is a political, not technical, process. It is also a political process that most people would like to avoid doing (at least publicly) because it involves making explicit the ways in which we prioritise some people over others. Public policy is the choice to help some people and punish or refuse to help others (and includes the choice to do nothing).

Policy analysis texts describe a relatively simple procedure of identifying criteria and producing a table (with a solution in each row, and criteria in each column) to compare the trade-offs between each solution. However, these criteria are notoriously difficult to define, and people resolve that problem by exercising power to decide what each term means, and whose interests should be served when they resolve trade-offs. For example, see Stone on whose needs come first, who benefits from each definition of fairness, and how technical-looking processes such as ‘cost benefit analysis’ mask political choices.

Right now, the most obvious and visible trade-off, accentuated in the UK, is between individual freedom and collective action, or the balance between state, communal, and market/ individual solutions. In comparison with many countries (and China and Italy in particular), the UK Government seems to be favouring individual action over state quarantine measures. However, most trade-offs are difficult to categorise

  1. What should be the balance between efforts to minimise the deaths of some (generally in older populations) and maximise the wellbeing of others? This is partly about human dignity during crisis, how we treat different people fairly, and the balance of freedom and coercion.
  2. How much should a government spend to keep people alive using intensive case or expensive medicines, when the money could be spent improving the lives of far more people? This is partly about human dignity, the relative efficiency of policy measures, and fairness.

If you are like me, you don’t really want to answer such questions (indeed, even writing them looks callous). If so, one way to resolve them is to elect policymakers to make such choices on our behalf (perhaps aided by experts in moral philosophy, or with access to deliberative forums). To endure, this unusually high level of deference to elected ministers requires some kind of reciprocal act:

https://twitter.com/devisridhar/status/1240648925998178304

See also: We must all do everything in our power to protect lives (UK Secretary of State for Health and Social Care)

Still, I doubt that governments are making reportable daily choices with reference to a clear and explicit view of what the trade-offs and priorities should be, because their choices are about who will die, and their ability to predict outcomes is limited.

See also: Media experts despair at Boris Johnson’s coronavirus campaign (Sonia Sodha)

Step 4 Predict the outcome of each feasible solution.

Common advice in policy analysis texts:

  • Focus on the outcomes that key actors care about (such as value for money), and quantify and visualise your predictions if possible. Compare the pros and cons of each solution, such as how much of a bad service policymakers will accept to cut costs.
  • ‘Assess the outcomes of the policy options in light of the criteria and weigh trade-offs between the advantages and disadvantages of the options’.
  • Estimate the cost of a new policy, in comparison with current policy, and in relation to factors such as savings to society or benefits to certain populations. Use your criteria and projections to compare each alternative in relation to their likely costs and benefits.
  • Explain potential solutions in sufficient detail to predict the costs and benefits of each ‘alternative’ (including current policy).
  • Short deadlines dictate that you use ‘logic and theory, rather than systematic empirical evidence’ to make predictions efficiently.
  • Monitoring is crucial because it is difficult to predict policy success, and unintended consequences are inevitable. Try to measure the outcomes of your solution, while noting that evaluations are contested.

It is difficult to envisage a way for the UK Government to publicise the thinking behind its choices (Step 3) and predictions (Step 4) in a way that would encourage effective public deliberation, rather than a highly technical debate between a small number of academics:

Further, people often call for the UK Government to publicise its expert advice and operational logic, but I am not sure how they would separate it from their normative logic, or provide a frank account without unintended consequences for public trust or anxiety. If so, government policy involves (a) to keep some choices implicit to avoid a lot of debate on trade-offs, and (b) to make general statements about choices when they do not know what their impact will be.

Step 5 Make a recommendation to your client

Common advice in policy analysis texts:

  • Examine your case through the eyes of a policymaker. Keep it simple and concise.
  • Make a preliminary recommendation to inform an iterative process, drawing feedback from clients and stakeholder groups
  • Client-oriented advisors identify the beliefs of policymakers and tailor accordingly.
  • ‘Unless your client asks you not to do so, you should explicitly recommend one policy’

I now invite you to make a recommendation (step 5) based on our discussion so far (steps 1-4). Define the problem with one framing at the expense of the others. Romanticise some people and not others. Decide how to support some people, and coerce or punish others. Prioritise the lives of some people in the knowledge that others will suffer or die. Do it despite your lack of expertise and profoundly limited knowledge and information. Learn from experts, but don’t assume that only scientific experts have relevant knowledge (decolonise; coproduce). Recommend choices that, if damaging, could take decades to fix after you’ve gone. Consider if a policymaker is willing and able to act on your advice, and if your proposed action will work as intended. Consider if a government is willing and able to bear the economic and political costs. Protect your client’s popularity, and trust in your client, at the same time as protecting lives. Consider if your advice would change if the problem would seem to change. If you are writing your analysis, maybe keep it down to one sheet of paper (and certainly far fewer words than in this post). Better you than me.

Please now watch this video before I suggest that things are not so simple.

Would that policy analysis were so simple

Imagine writing policy analysis in an imaginary world, in which there is a single powerful ‘rational’ policymaker at the heart of government, making policy via an orderly series of stages.

cycle and cycle spirograph 18.2.20

Your audience would be easy to identify at each stage, your analysis would be relatively simple, and you would not need to worry about what happens after you make a recommendation for policy change (since the selection of a solution would lead to implementation).  You could adopt a simple 5 step policy analysis method, use widely-used tools such as cost-benefit analysis to compare solutions, and know where the results would feed into the policy process.

Studies of policy analysts describe how unrealistic this expectation tends to be (Radin, Brans, Thissen).

Table for coronavirus 750

For example, there are many policymakers, analysts, influencers, and experts spread across political systems, and engaging with 101 policy problems simultaneously, which suggests that it is not even clear how everyone fits together and interacts in what we call (for the sake of simplicity) ‘the policy process’.

Instead, we can describe real world policymaking with reference to two factors.

The wider policymaking environment: 1. Limiting the use of evidence

First, policymakers face ‘bounded rationality’, in which they only have the ability to pay attention to a tiny proportion of available facts, are unable to separate those facts from their values (since we use our beliefs to evaluate the meaning of facts), struggle to make clear and consistent choices, and do not know what impact they will have. The consequences can include:

  • Limited attention, and lurches of attention. Policymakers can only pay attention to a tiny proportion of their responsibilities, and policymaking organizations struggle to process all policy-relevant information. They prioritize some issues and information and ignore the rest.
  • Power and ideas. Some ways of understanding and describing the world dominate policy debate, helping some actors and marginalizing others.
  • Beliefs and coalitions. Policymakers see the world through the lens of their beliefs. They engage in politics to turn their beliefs into policy, form coalitions with people who share them, and compete with coalitions who don’t.
  • Dealing with complexity. They engage in ‘trial-and-error strategies’ to deal with uncertain and dynamic environments (see the new section on trial-and-error- at the end).
  • Framing and narratives. Policy audiences are vulnerable to manipulation when they rely on other actors to help them understand the world. People tell simple stories to persuade their audience to see a policy problem and its solution in a particular way.
  • The social construction of populations. Policymakers draw on quick emotional judgements, and social stereotypes, to propose benefits to some target populations and punishments for others.
  • Rules and norms. Institutions are the formal rules and informal understandings that represent a way to narrow information searches efficiently to make choices quickly.
  • Learning. Policy learning is a political process in which actors engage selectively with information, not a rational search for truth.

Evidence-based or expert-informed policymaking

Put simply, policymakers cannot oversee a simple process of ‘evidence-based policymaking’. Rather, to all intents and purposes:

  1. They need to find ways to ignore most evidence so that they can focus disproportionately on some. Otherwise, they will be unable to focus well enough to make choices. The cognitive and organisational shortcuts, described above, help them do it almost instantly.
  2. They also use their experience to help them decide – often very quickly – what evidence is policy-relevant under the circumstances. Relevance can include:
  • How it relates to the policy problem as they define it (Step 1).
  • If it relates to a feasible solution (Step 2).
  • If it is timely, available, understandable, and actionable.
  • If it seems credible, such as from groups representing wider populations, or from people they trust.
  1. They use a specific shortcut: relying on expertise.

However, the vague idea of trusting or not trusting experts is a nonsense, largely because it is virtually impossible to set a clear boundary between relevant/irrelevant experts and find a huge consensus on (exactly) what is happening and what to do. Instead, in political systems, we define the policy problem or find other ways to identify the most relevant expertise and exclude other sources of knowledge.

In the UK Government’s case, it appears to be relying primarily on expertise from its own general scientific advisers, medical and public health advisers, and – perhaps more controversially – advisers on behavioural public policy.

box 7.1

Right now, it is difficult to tell exactly how and why it relies on each expert (at least when the expert is not in a clearly defined role, in which case it would be irresponsible not to consider their advice). Further, there are regular calls on Twitter for ministers to be more open about their decisions.

See also: Coronavirus: do governments ever truly listen to ‘the science’?

However, don’t underestimate the problems of identifying why we make choices, then justifying one expert or another (while avoiding pointless arguments), or prioritising one form of advice over another. Look, for example, at the kind of short-cuts that intelligent people use, which seem sensible enough, but would receive much more intense scrutiny if presented in this way by governments:

  • Sophisticated speculation by experts in a particular field, shared widely (look at the RTs), but questioned by other experts in another field:
  • Experts in one field trusting certain experts in another field based on personal or professional interaction:
  • Experts in one field not trusting a government’s approach based on its use of one (of many) sources of advice:
  • Experts representing a community of experts, criticising another expert (Prof John Ashton), for misrepresenting the amount of expert scepticism of government experts (yes, I am trying to confuse you):
  • Expert debate on how well policymakers are making policy based on expert advice
  • Finding quite-sensible ways to trust certain experts over others, such as because they can be held to account in some way (and may be relatively worried about saying any old shit on the internet):

There are many more examples in which the shortcut to expertise is fine, but not particularly better than another shortcut (and likely to include a disproportionately high number of white men with STEM backgrounds).

Update: of course, they are better than the volume trumps expertise approach:

See also:

Further, in each case, we may be receiving this expert advice via many other people, and by the time it gets to us the meaning is lost or reversed (or there is some really sophisticated expert analysis of something rumoured – not demonstrated – to be true):

For what it’s worth, I tend to favour experts who:

(a) establish the boundaries of their knowledge, (b) admit to high uncertainty about the overall problem:

(c) (in this case) make it clear that they are working on scenarios, not simple prediction

(d) examine critically the too-simple ideas that float around, such as the idea that the UK Government should emulate ‘what works’ somewhere else

(e) situate their own position (in Prof Sridhar’s case, for mass testing) within a broader debate

See also:

See also: Prof Sir John Bell (4.3.20) on why an accurate antibody test is at least one month away and these exchanges on the problems with test ‘accuracy’:

(f) use their expertise on governance to highlight problems with thoughtless criticism

However, note that most of these experts are from a very narrow social background, and from very narrow scientific fields (first in modelling, then likely in testing), despite the policy problem being largely about (a) who, and how many people, a government should try to save, and (b) how far a government should go to change behaviour to do it (Update 2.4.20: I wrote that paragraph before adding so many people to the list). It is understandable to defer in this way during a crisis, but it also contributes to a form of ‘depoliticisation’ that masks profound choices that benefit some people and leave others vulnerable to harm.

See also: COVID-19: a living systematic map of the evidence

See also: To what extent does evidence support decision making during infectious disease outbreaks? A scoping literature review

See also: Covid-19: why is the UK government ignoring WHO’s advice? (British Medical Journal editorial)

See also: Coronavirus: just 2,000 NHS frontline workers tested so far

See also: ‘What’s important is social distancing’ coronavirus testing ‘is a side issue’, says Deputy Chief Medical Officer [Professor Jonathan Van-Tam talks about the important distinction between a currently available test to see if someone has contracted the virus (an antigen test) and a forthcoming test to see if someone has had and recovered from COVID-19 (an antibody test)]. The full interview is here (please feel free to ignore the editorialising of the uploader):

See also: Why is Germany able to test for coronavirus so much more than the UK? (which is mostly a focus on Germany’s innovation and partly on the UK (Public Health England) focus on making sure its test is reliable, in the context of ‘coronavirus tests produced at great speed which have later proven to be inaccurate’ (such as one with a below-30% accuracy rate, which is worse than not testing at all). Compare with The Coronavirus Hit Germany And The UK Just Days Apart But The Countries Have Responded Differently. Here’s How and the Opinion piece ‘A public inquiry into the UK’s coronavirus response would find a litany of failures

See also: Rights and responsibilities in the Coronavirus pandemic

See also: UK police warned against ‘overreach’ in use of virus lockdown powers (although note that there is no UK police force and that Scotland has its own legal system) and Coronavirus: extra police powers risk undermining public trust (Alex Oaten and Chris Allen)

See also (Calderwood resigned as CMO that night):

See also: Social Licensing of Privacy-Encroaching Policies to Address the COVID-19 Pandemic (U.K.) (research on public opinion)

The wider policymaking environment: 2. Limited control

Second, policymakers engage in a messy and unpredictable world in which no single ‘centre’ has the power to turn a policy recommendation into an outcome. I normally use the following figure to think through the nature of a complex and unwieldy policymaking environment of which no ‘centre’ of government has full knowledge or control.

image policy process round 2 25.10.18

It helps us identify (further) the ways in which we can reject the idea that the UK Prime Minister and colleagues can fully understand and solve policy problems:

Actors. The environment contains many policymakers and influencers spread across many levels and types of government (‘venues’).

For example, consider how many key decisions that (a) have been made by organisations not in the UK central government, and (b) are more or less consistent with its advice, including:

  • Devolved governments announcing their own healthcare and public health responses (although the level of UK coordination seems more significant than the level of autonomy).
  • Public sector employers initiating or encouraging at-home working (and many Universities moving quickly from in-person to online teaching)
  • Private organisations cancelling cultural and sporting events.

Context and events. Policy solutions relate to socioeconomic context and events which can be impossible to ignore and out of the control of policymakers. The coronavirus, and its impact on so many aspects on population health and wellbeing, is an extreme example of this problem.

Networks, Institutions, and Ideas. Policymakers and influencers operate in subsystems (specialist parts of political systems). They form networks or coalitions built on the exchange of resources or facilitated by trust underpinned by shared beliefs or previous cooperation. Many different parts of government have practices driven by their own formal and informal rules. Formal rules are often written down or known widely. Informal rules are the unwritten rules, norms and practices that are difficult to understand, and may not even be understood in the same way by participants. Political actors relate their analysis to shared understandings of the world – how it is, and how it should be – which are often so established as to be taken for granted. These dominant frames of reference establish the boundaries of the political feasibility of policy solutions.  These kinds of insights suggest that most policy decisions are considered, made, and delivered in the name of – but not in the full knowledge of – government ministers.

Trial and error policymaking in complex policymaking systems (17.3.20)

There are many ways to conceptualise this policymaking environment, but few theories provide specific advice on what to do, or how to engage effectively in it. One notable exception is the general advice that comes from complexity theory, including:

  • Law-like behaviour is difficult to identify – so a policy that was successful in one context may not have the same effect in another.
  • Policymaking systems are difficult to control; policy makers should not be surprised when their policy interventions do not have the desired effect.
  • Policy makers in the UK have been too driven by the idea of order, maintaining rigid hierarchies and producing top-down, centrally driven policy strategies.  An attachment to performance indicators, to monitor and control local actors, may simply result in policy failure and demoralised policymakers.
  • Policymaking systems or their environments change quickly. Therefore, organisations must adapt quickly and not rely on a single policy strategy.

On this basis, there is a tendency in the literature to encourage the delegation of decision-making to local actors:

  1. Rely less on central government driven targets, in favour of giving local organisations more freedom to learn from their experience and adapt to their rapidly-changing environment.
  2. To deal with uncertainty and change, encourage trial-and-error projects, or pilots, that can provide lessons, or be adopted or rejected, relatively quickly.
  3. Encourage better ways to deal with alleged failure by treating ‘errors’ as sources of learning (rather than a means to punish organisations) or setting more realistic parameters for success/ failure (although see this example and this comment).
  4. Encourage a greater understanding, within the public sector, of the implications of complex systems and terms such as ‘emergence’ or ‘feedback loops’.

In other words, this literature, when applied to policymaking, tends to encourage a movement from centrally driven targets and performance indicators towards a more flexible understanding of rules and targets by local actors who are more able to understand and adapt to rapidly-changing local circumstances.

[See also: Complex systems and systems thinking]

Now, just imagine the UK Government taking that advice right now. I think it is fair to say that it would be condemned continuously (even more so than right now). Maybe that is because it is the wrong way to make policy in times of crisis. Maybe it is because too few people are willing and able to accept that the role of a small group of people at the centre of government is necessarily limited, and that effective policymaking requires trial-and-error rather than a single, fixed, grand strategy to be communicated to the public. The former highlights policy that changes with new information and perspective. The latter highlights errors of judgement, incompetence, and U-turns. In either case, the advice is changing as estimates of the coronavirus’ impact change:

I think this tension, in the way that we understand UK government, helps explain some of the criticism that it faces when changing its advice to reflect changes in its data or advice. This criticism becomes intense when people also question the competence or motives of ministers (and even people reporting the news) more generally, leading to criticism that ranges from mild to outrageous:

For me, this casual reference to a government policy to ‘cull the heard of the weak’ is outrageous, but you can find much worse on Twitter. It reflects wider debate on whether ‘herd immunity’ is or is not government policy. Much of it relates to interpretation of government statements, based on levels of trust/distrust in the UK Government, its Prime Minister and Secretaries of State, and the Prime Minister’s special adviser

However, I think that some of it is also about:

1. Wilful misinterpretation (particularly on Twitter). For example, in the early development and communication of policy, Boris Johnson was accused (in an irresponsibly misleading way) of advocating for herd immunity rather than restrictive measures.

See: Here is the transcript of what Boris Johnson said on This Morning about the new coronavirus (Full Fact)

full fact coronavirus

Below is one of the most misleading videos of its type. Look at how it cuts each segment into a narrative not provided by ministers or their advisors (see also this stinker):

See also:

2. The accentuation of a message not being emphasised by government spokespeople.

See for example this interview, described by Sky News (13.3.20) as: The government’s chief scientific adviser Sir Patrick Vallance has told Sky News that about 60% of people will need to become infected with coronavirus in order for the UK to enjoy “herd immunity”. You might be forgiven for thinking that he was on Sky extolling the virtues of a strategy to that end (and expressing sincere concerns on that basis). This was certainly the write-up in respected papers like the FT (UK’s chief scientific adviser defends ‘herd immunity’ strategy for coronavirus). Yet, he was saying nothing of the sort. Rather, when prompted, he discussed herd immunity in relation to the belief that COVID-19 will endure long enough to become as common as seasonal flu.

The same goes for Vallance’s interview on the same day (13.3.20) during Radio 4’s Today programme (transcribed by the Spectator, which calls Vallance the author, and gives it the headlineHow ‘herd immunity’ can help fight coronavirusas if it is his main message). The Today Programme also tweeted only 30 seconds to single out that brief exchange:

Yet, clearly his overall message – in this and other interviews – was that some interventions (e.g. staying at home; self-isolating with symptoms) would have bigger effects than others (e.g. school closures; prohibiting mass gatherings) during the ‘flattening of the peak’ strategy (‘What we don’t want is everybody to end up getting it in a short period of time so that we swamp and overwhelm NHS services’). Rather than describing ‘herd immunity’ as a strategy, he is really describing how to deal with its inevitability (‘Well, I think that we will end up with a number of people getting it’).

See also: British government wants UK to acquire coronavirus ‘herd immunity’, writes Robert Peston (12.3.20) and live debates (and reports grasping at straws) on whether or not ‘herd immunity’ was the goal of the UK government:

See also: Why weren’t we ready? (Harry Lambert) which is a good exemplar of the ‘U turn’ argument, and compare with the evidence to the Health and Social Care Committee (CMO Whitty, DCMO Harries) that it describes.

A more careful forensic analysis (such as this one) will try to relate each government choice to the ways in which key advisory bodies (such as the New and Emerging Respiratory Virus Threats Advisory Group, NERVTAG) received and described evidence on the current nature of the problem:

See also: Special Report: Johnson listened to his scientists about coronavirus – but they were slow to sound the alarm (Reuters)

Some aspects may also be clearer when there is systematic qualitative interview data on which to draw. Right now, there are bits and pieces of interviews sandwiched between whopping great editorial discussions (e.g. FT Alphaville Imperial’s Neil Ferguson: “We don’t have a clear exit strategy”; compare with the more useful Let’s flatten the coronavirus confusion curve) or confused accounts by people speaking to someone who has spoken to someone else (e.g. Buzzfeed Even The US Is Doing More Coronavirus Tests Than The UK. Here Are The Reasons Why).

See also: other rabbit holes are available

[OK, that proved to be a big departure from the trial-and-error discussion. Here we are, back again]

In some cases, maybe people are making the argument that trial-and-error is the best way to respond quickly, and adapt quickly, in a crisis but that the UK Government version is not what, say, the WHO thinks of as good kind of adaptive response. It is not possible to tell, at least from the general ways in which they justify acting quickly.

See also the BBC’s provocative question (which I expect to be replaced soon):

Compare with:

The take home messages

  1. The coronavirus is an extreme example of a general situation: policymakers will always have very limited knowledge of policy problems and control over their policymaking environment. They make choices to frame problems narrowly enough to seem solvable, rule out most solutions as not feasible, make value judgements to try help some more than others, try to predict the results, and respond when the results to not match their hopes or expectations.
  2. This is not a message of doom and despair. Rather, it encourages us to think about how to influence government, and hold policymakers to account, in a thoughtful and systematic way that does not mislead the public or exacerbate the problem we are seeing.

Further reading, until I can think of a better conclusion:

This series of ‘750 words’ posts summarises key texts in policy analysis and tries to situate policy analysis in a wider political and policymaking context. Note the focus on whose knowledge counts, which is not yet a big feature of this crisis.

These series of 500 words and 1000 words posts (with podcasts) summarise concepts and theories in policy studies.

This page on evidence-based policymaking (EBPM) uses those insights to demonstrate why EBPM is  a political slogan rather than a realistic expectation.

These recorded talks relate those insights to common questions asked by researchers: why do policymakers seem to ignore my evidence, and what can I do about it? I’m happy to record more (such as on the topic you just read about) but not entirely sure who would want to hear what.

See also: Advisers, Governments and why blunders happen? (Colin Talbot)

See also: Why we might disagree about … Covid-19 (Ruth Dixon and Christopher Hood)

See also: Pandemic Science and Politics (Daniel Sarewitz)

See also: We knew this would happen. So why weren’t we ready? (Steve Bloomfield)

See also: Europe’s coronavirus lockdown measures compared (Politico)

.

.

.

.

.

7 Comments

Filed under 750 word policy analysis, agenda setting, Evidence Based Policymaking (EBPM), Policy learning and transfer, POLU9UK, Prevention policy, Psychology Based Policy Studies, Public health, public policy, Social change, UK politics and policy

Policy Analysis in 750 Words: what you need as an analyst versus policymaking reality

This post forms one part of the Policy Analysis in 750 words series overview. Note for the eagle eyed: you are not about to experience déjà vu. I’m just using the same introduction.

When describing ‘the policy sciences’, Lasswell distinguishes between:

  1. ‘knowledge of the policy process’, to foster policy studies (the analysis of policy)
  2. ‘knowledge in the process’, to foster policy analysis (analysis for policy)

The lines between each approach are blurry, and each element makes less sense without the other. However, the distinction is crucial to help us overcome the major confusion associated with this question:

Does policymaking proceed through a series of stages?

The short answer is no.

The longer answer is that you can find about 40 blog posts (of 500 and 1000 words) which compare (a) a stage-based model called the policy cycle, and (b) the many, many policy concepts and theories that describe a far messier collection of policy processes.

cycle

In a nutshell, most policy theorists reject this image because it oversimplifies a complex policymaking system. The image provides a great way to introduce policy studies, and serves a political purpose, but it does more harm than good:

  1. Descriptively, it is profoundly inaccurate (unless you imagine thousands of policy cycles interacting with each other to produce less orderly behaviour and less predictable outputs).
  2. Prescriptively, it gives you rotten advice about the nature of your policymaking task (for more on these points, see this chapter, article, article, and series).

Why does the stages/ policy cycle image persist? Two relevant explanations

 

  1. It arose from a misunderstanding in policy studies

In another nutshell, Chris Weible and I argue (in a secret paper) that the stages approach represents a good idea gone wrong:

  • If you trace it back to its origins, you will find Lasswell’s description of decision functions: intelligence, recommendation, prescription, invocation, application, appraisal and termination.
  • These functions correspond reasonably well to a policy cycle’s stages: agenda setting, formulation, legitimation, implementation, evaluation, and maintenance, succession or termination.
  • However, Lasswell was imagining functional requirements, while the cycle seems to describe actual stages.

In other words, if you take Lasswell’s list of what policy analysts/ policymakers need to do, multiple it by the number of actors (spread across many organisations or venues) trying to do it, then you get the multi-centric policy processes described by modern theories. If, instead, you strip all that activity down into a single cycle, you get the wrong idea.

  1. It is a functional requirement of policy analysis

This description should seem familiar, because the classic policy analysis texts appear to describe a similar series of required steps, such as:

  1. define the problem
  2. identify potential solutions
  3. choose the criteria to compare them
  4. evaluate them in relation to their predicted outcomes
  5. recommend a solution
  6. monitor its effects
  7. evaluate past policy to inform current policy.

However, these texts also provide a heavy dose of caution about your ability to perform these steps (compare Bardach, Dunn, Meltzer and Schwartz, Mintrom, Thissen and Walker, Weimer and Vining)

In addition, studies of policy analysis in action suggest that:

  • an individual analyst’s need for simple steps, to turn policymaking complexity into useful heuristics and pragmatic strategies,

should not be confused with

What you need versus what you can expect

Overall, this discussion of policy studies and policy analysis reminds us of a major difference between:

  1. Functional requirements. What you need from policymaking systems, to (a) manage your task (the 5-8 step policy analysis) and (b) understand and engage in policy processes (the simple policy cycle).
  2. Actual processes and outcomes. What policy concepts and theories tell us about bounded rationality (which limit the comprehensiveness of your analysis) and policymaking complexity (which undermines your understanding and engagement in policy processes).

Of course, I am not about to provide you with a solution to these problems.

Still, this discussion should help you worry a little bit less about the circular arguments you will find in key texts: here are some simple policy analysis steps, but policymaking is not as ‘rational’ as the steps suggest, but (unless you can think of an alternative) there is still value in the steps, and so on.

See also:

The New Policy Sciences

2 Comments

Filed under 750 word policy analysis, agenda setting, public policy

Can A Government Really Take Control Of Public Policy?

This post first appeared on the MIHE blog to help sell my book.

During elections, many future leaders give the impression that they will take control of public policy. They promise major policy change and give little indication that anything might stand in their way.

This image has been a major feature of Donald Trump’s rhetoric on his US Presidency. It has also been a feature of campaigns for the UK withdrawal from the European Union (‘Brexit’) to allow its leaders to take back control of policy and policymaking. According to this narrative, Brexit would allow (a) the UK government to make profound changes to immigration and spending, and (b) Parliament and the public to hold the UK government directly to account, in contrast to a distant EU policy process less subject to direct British scrutiny.

Such promises are built on the false image of a single ‘centre’ of government, in which a small number of elected policymakers take responsibility for policy outcomes. This way of thinking is rejected continuously in the modern literature. Instead, policymaking is ‘multi-centric’: responsibility for policy outcomes is spread across many levels and types of government (‘centres’), and shared with organisations outside of government, to the extent that it is not possible to simply know who is in charge and to blame. This arrangement helps explain why leaders promise major policy change but most outcomes represent a minor departure from the status quo.

Some studies of politics relate this arrangement to the choice to share power across many centres. In the US, a written constitution ensures power sharing across different branches (executive, legislative, judicial) and between federal and state or local jurisdictions. In the UK, central government has long shared power with EU, devolved, and local policymaking organisations.

However, policy theories show that most aspects of multi-centric governance are necessary. The public policy literature provides many ways to describe such policy processes, but two are particularly useful.

The first approach is to explain the diffusion of power with reference to an enduring logic of policymaking, as follows:

  • The size and scope of the state is so large that it is always in danger of becoming unmanageable. Policymakers manage complexity by breaking the state’s component parts into policy sectors and sub-sectors, with power spread across many parts of government.
  • Elected policymakers can only pay attention to a tiny proportion of issues for which they are responsible. They pay attention to a small number and ignore the rest. They delegate policymaking responsibility to other actors such as bureaucrats, often at low levels of government.
  • At this level of government and specialisation, bureaucrats rely on specialist organisations for information and advice. Those organisations trade that information/advice and other resources for access to, and influence within, the government.
  • Most public policy is conducted primarily through small and specialist ‘policy communities’ that process issues at a level of government not particularly visible to the public, and with minimal senior policymaker involvement.

This description suggests that senior elected politicians are less important than people think, their impact on policy is questionable, and elections may not provide major changes in policy. Most decisions are taken in their name but without their intervention.

A second, more general, approach is to show that elected politicians deal with such limitations by combining cognition and emotion to make choices quickly. Although such action allows them to be decisive, they occur within a policymaking environment over which governments have limited control. Government bureaucracies only have the coordinative capacity to direct policy outcomes in a small number of high priority areas. In most other cases, policymaking is spread across many venues, each with their own rules, networks, ways of seeing the world, and ways of responding to socio-economic factors and events.

In that context, we should always be sceptical when election candidates and referendum campaigners (or, in many cases, leaders of authoritarian governments) make such promises about political leadership and government control.

A more sophisticated knowledge of policy processes allows us to identify the limits to the actions of elected policymakers, and develop a healthier sense of pragmatism about the likely impact of government policy. The question of our age is not: how can governments take back control? Rather, it is: how can we hold policymakers to account in a complex system over which they have limited knowledge and even less control?

Leave a comment

Filed under public policy, UK politics and policy

Policy Analysis in 750 words: Beryl Radin, B (2019) Policy Analysis in the Twenty-First Century

Please see the Policy Analysis in 750 words series overview before reading the summary. As usual, the 750-word description is more for branding than accuracy.

Beryl Radin (2019) Policy Analysis in the Twenty-First Century (Routledge)

Radin cover 2019

The basic relationship between a decision-maker (the client) and an analyst has moved from a two-person encounter to an extremely complex and diverse set of interactions’ (Radin, 2019: 2).

Many texts in this series continue to highlight the client-oriented nature of policy analysis (Weimer and Vining), but within a changing policy process that has altered the nature of that relationship profoundly.

This new policymaking environment requires new policy analysis skills and training (see Mintrom), and limits the applicability of classic 8-step (or 5-step) policy analysis techniques (2019: 82).

We can use Radin’s work to present two main stories of policy analysis:

  1. The old ways of making policy resembled a club, or reflected a clear government hierarchy, involving:
  • a small number of analysts, generally inside government (such as senior bureaucrats, scientific experts, and – in particular- economists),
  • giving technical or factual advice,
  • about policy formulation,
  • to policymakers at the heart of government,
  • on the assumption that policy problems would be solved via analysis and action.
  1. Modern policy analysis is characterised by a more open and politicised process in which:
  • many analysts, inside and outside government,
  • compete to interpret facts, and give advice,
  • about setting the agenda, and making, delivering, and evaluating policy,
  • across many policymaking venues,
  • often on the assumption that governments have a limited ability to understand and solve complex policy problems.

As a result, the client-analyst relationship is increasingly fluid:

In previous eras, the analyst’s client was a senior policymaker, the main focus was on the analyst-client relationship, and ‘both analysts and clients did not spend much time or energy thinking about the dimensions of the policy environment in which they worked’ (2019: 59). Now, in a multi-centric policymaking environment:

  1. It is tricky to identify the client.
  • We could imagine the client to be someone paying for the analysis, someone affected by its recommendations, or all policy actors with the ability to act on the advice (2019: 10).
  • If there is ‘shared authority’ for policymaking within one political system, a ‘client’ (or audience) may be a collection of policymakers and influencers spread across a network containing multiple types of government, non-governmental actors, and actors responsible for policy delivery (2019: 33).
  • The growth in international cooperation also complicates the idea of a single client for policy advice (2019: 33-4)
  • This shift may limit the ‘face-to-face encounters’ that would otherwise provide information for – and perhaps trust in – the analyst (2019: 2-3).
  1. It is tricky to identify the analyst
  • Radin (2019: 9-25) traces, from the post-war period in the US, a major expansion of policy analysts, from the notional centre of policymaking in federal government towards analysts spread across many venues, inside government (across multiple levels, ‘policy units’, and government agencies) and congressional committees, and outside government (such as in influential think tanks).
  • Policy analysts can also be specialist external companies contracted by organisations to provide advice (2019: 37-8).
  • This expansion shifted the image of many analysts, from a small number of trusted insiders towards many being treated as akin to interest groups selling their pet policies (2019: 25-6).
  • The nature – and impact – of policy analysis has always been a little vague, but now it seems more common to suggest that ‘policy analysts’ may really be ‘policy advocates’ (2019: 44-6).
  • As such, they may now have to work harder to demonstrate their usefulness (2019: 80-1) and accept that their analysis will have a limited impact (2019: 82, drawing on Weiss’ discussion of ‘enlightenment’).

Consequently, the necessary skills of policy analysis have changed:

Although many people value systematic policy analysis (and many rely on economists), an effective analyst does not simply apply economic or scientific techniques to analyse a problem or solution, or rely on one source of expertise or method, as if it were possible to provide ‘neutral information’ (2019: 26).

Indeed, Radin (2019: 31; 48) compares the old ‘acceptance that analysts would be governed by the norms of neutrality and objectivity’ with

(a) increasing calls to acknowledge that policy analysis is part of a political project to foster some notion of public good or ‘public interest’, and

(b)  Stone’s suggestion that the projection of reason and neutrality is a political strategy.

In other words, the fictional divide between political policymakers and neutral analysts is difficult to maintain.

Rather, think of analysts as developing wider skills to operate in a highly political environment in which the nature of the policy issue is contested, responsibility for a policy problem is unclear, and it is not clear how to resolve major debates on values and priorities:

  • Some analysts will be expected to see the problem from the perspective of a specific client with a particular agenda.
  • Other analysts may be valued for their flexibility and pragmatism, such as when they acknowledge the role of their own values, maintain or operate within networks, communicate by many means, and supplement ‘quantitative data’ with ‘hunches’ when required (2019: 2-3; 28-9).

Radin (2019: 21) emphasises a shift in skills and status

The idea of (a) producing new and relatively abstract ideas, based on high control over available information, at the top of a hierarchical organisation, makes way for (b) developing the ability to:

  • generate a wider understanding of organisational and policy processes, reflecting the diffusion of power across multiple policymaking venues
  • identify a map of stakeholders,
  • manage networks of policymakers and influencers,
  • incorporate ‘multiple and often conflicting perspectives’,
  • make and deliver more concrete proposals (2019: 59-74), while recognising
  • the contested nature of information, and the practices sued to gather it, even during multiple attempts to establish the superiority of scientific evidence (2019: 89-103),
  • the limits to a government’s ability to understand and solve problems (2019: 95-6),
  • the inescapable conflict over trade-offs between values and goals, which are difficult to resolve simply by weighting each goal (2019: 105-8; see Stone), and
  • do so flexibly, to recognise major variations in problem definition, attention and networks across different policy sectors and notional ‘stages’ of policymaking (2019: 75-9; 84).

Radin’s (2019: 48) overall list of relevant skills include:

  1. ‘Case study methods, Cost- benefit analysis, Ethical analysis, Evaluation, Futures analysis, Historical analysis, Implementation analysis, Interviewing, Legal analysis, Microeconomics, Negotiation, mediation, Operations research, Organizational analysis, Political feasibility analysis, Public speaking, Small- group facilitation, Specific program knowledge, Statistics, Survey research methods, Systems analysis’

They develop alongside analytical experience and status, from the early career analyst trying to secure or keep a job, to the experienced operator looking forward to retirement (2019: 54-5)

A checklist for policy analysts

Based on these skills requirements, the contested nature of evidence, and the complexity of the policymaking environment, Radin (2019: 128-31) produces a 4-page checklist of – 91! – questions for policy analysts.

For me, it serves two main functions:

  1. It is a major contrast to the idea that we can break policy analysis into a mere 5-8 steps (rather, think of these small numbers as marketing for policy analysis students, akin to 7-minute abs)
  2. It presents policy analysis as an overwhelming task with absolutely no guarantee of policy impact.

To me, this cautious, eyes-wide-open, approach is preferable to the sense that policy analysts can change the world if they just get the evidence and the steps right.

Further Reading:

  1. Iris Geva-May (2005) ‘Thinking Like a Policy Analyst. Policy Analysis as a Clinical Profession’, in Geva-May (ed) Thinking Like a Policy Analyst. Policy Analysis as a Clinical Profession (Basingstoke: Palgrave)

Although the idea of policy analysis may be changing, Geva-May (2005: 15) argues that it remains a profession with its own set of practices and ways of thinking. As with other professions (like medicine), it would be unwise to practice policy analysis without education and training or otherwise learning the ‘craft’ shared by a policy analysis community (2005: 16-17). For example, while not engaging in clinical diagnosis, policy analysts can draw on 5-step process to diagnose a policy problem and potential solutions (2005: 18-21). Analysts may also combine these steps with heuristics to determine the technical and political feasibility of their proposals (2005: 22-5), as they address inevitable uncertainty and their own bounded rationality (2005: 26-34; see Gigerenzer on heuristics). As with medicine, some aspects of the role – such as research methods – can be taught in graduate programmes, while others may be better suited to on the job learning (2005: 36-40). If so, it opens up the possibility that there are many policy analysis professions to reflect different cultures in each political system (and perhaps the venues within each system).

  1. Vining and Weimar’s take on the distinction between policy analysis and policy process research

 

10 Comments

Filed under 750 word policy analysis, public policy

Policy Concepts in 1000 Words: how do policy theories describe policy change?

The 1000 words and 500 words series already show how important but difficult it is to define and measure policy change. In this post, Leanne Giordono and I dig deeper into the – often confusingly different – ways in which different researchers conceptualise this process. We show why there is such variation and provide a checklist of questions to ask of any description of policy change.

Measuring policy change is more difficult than it looks

The measurement of policy change is important. Most ‘what is policy?’ discussions remind us that there can be a huge difference between policy as a (a)  statement of intent, (b) strategy, (c) collection of tools/ instruments and (d) contributor to policy outcomes.

Policy theories remind us that, while politicians and political parties often promise to sweep into office and produce radical departures from the past, most policy change is minor. There is a major gap between stated intention and actual outcomes, partly because policymakers do not control the policy process for which they are responsible. Instead, they inherit the commitments of their predecessors and make changes at the margins.

The 1000 words and 500 words posts suggest that we address this problem of measurement by identifying the use of a potentially large number of policy instruments or policy tools such as regulation (including legislation) and resources (money and staffing) to accentuate the power at policymaker’s disposal.

Then, they suggest that we tell a story of policy change, focusing on (a) what problem policymakers were trying to solve, and the size of their response in relation to the size of the problem, and (b) the precise nature of specific changes, or how each change contributes to the ‘big picture’.

This recommendation highlights a potentially major problem: as researchers, we can produce very different narratives of policy change from the same pool of evidence, by accentuating some measures and ignoring others, or putting more faith in some data than others.

Three ways to navigate different approaches to imagining and measuring change

Researchers use many different concepts and measures to define and identify policy change. It would be unrealistic – and perhaps unimaginative – to solve this problem with a call for one uniform approach.

Rather, our aim is to help you (a) navigate this diverse field by (b) identifying the issues and concepts that will help you interpret and compare different ways to measure change.

  1. Check if people are ‘showing their work’

Pay close attention to how scholars are defining their terms. For example, be careful with incomplete definitions that rely on a reference to evolutionary change (which can mean so many different things) or incremental change (e.g. does an increment mean small or non-radical)? Or, note that frequent distinctions between minor versus major change seem useful, but we are often trying to capture and explain a confusing mixture of both.

  1. Look out for different questions

Multiple typologies of change often arise because different theories ask and answer different questions:

  • The Advocacy Coalition Framework distinguishes between minor and major change, associating the former with routine ‘policy-oriented learning’, and the latter with changes in core policy beliefs, often caused by a ‘shock’ associated with policy failure or external events.
  • Innovation and Diffusion models examine the adoption and non-adoption of a specific policy solution over a specific period of time in multiple jurisdictions as a result of learning, imitation, competition or coercion.
  • Classic studies of public expenditure generated four categories to ask if the ‘budgetary process of the United States government is equivalent to a set of temporally stable linear decision rules’. They describe policy change as minor and predictable and explain outliers as deviations from the norm.
  • Punctuated Equilibrium Theory identifies a combination of (a) huge numbers of small policy change and (b) small numbers of huge change as the norm, in budgetary and other policy changes.
  • Hall distinguishes between (a) routine adjustments to policy instruments, (b) changes in instruments to achieve existing goals, and (c) complete shifts in goals. He compares long periods in which (1) some ideas dominate and institutions do not change, with (2) ‘third order’ change in which a profound sense of failure contributes to a radical shift of beliefs and rules.
  • More recent scholarship identifies a range of concepts – including layering, drift, conversion, and displacement – to explain more gradual causes of profound changes to institutions.

These approaches identify a range of possible sources of measures:

  1. a combination of policy instruments that add up to overall change
  2. the same single change in many places
  3. change in relation to one measure, such as budgets
  4. a change in ideas, policy instruments and/ or rules.

As such, the potential for confusion is high when we include all such measures under the single banner of ‘policy change’.

  1. Look out for different measures

Spot the different ways in which scholars try to ‘operationalize’ and measure policy change, quantitatively and/ or qualitatively, with reference to four main categories.

  1. Size can be measured with reference to:
  • A comparison of old and new policy positions.
  • A change observed in a sample or whole population (using, for example, standard deviations from the mean).
  • An ‘ideal’ state, such as an industry or ‘best practice’ standard.
  1. Speed describes the amount of change that occurs over a specific interval of time, such as:
  • How long it takes for policy to change after a specific event or under specific conditions.
  • The duration of time between commencement and completion (often described as ‘sudden’ or ‘gradual’).
  • How this speed compares with comparable policy changes in other jurisdictions (often described with reference to ‘leaders’ and ‘laggards’).
  1. Direction describes the course of the path from one policy state to another. It is often described in comparison to:
  • An initial position in one jurisdiction (such as an expansion or contraction).
  • Policy or policy change in other jurisdictions (such as via ‘benchmarking’ or ‘league tables’)
  • An ‘ideal’ state (such as with reference to left or right wing aims).
  1. Substance relates to policy change in relations to:
  • Relatively tangible instruments such as legislation, regulation, or public expenditure.
  • More abstract concepts such as in relation to beliefs or goals.

Take home points for students

Be thoughtful when drawing comparisons between applications, drawn from many theoretical traditions, and addressing different research questions.  You can seek clarity by posing three questions:

  1. How clearly has the author defined the concept of policy change?
  2. How are the chosen theories and research questions likely to influence the author’s operationalization of policy change?
  3. How does the author operationalize policy change with respect to size, speed, direction, and/or substance?

However, you should also note that the choice of definition and theory may affect the meaning of measures such as size, speed, direction, and/or substance.

 

7 Comments

Filed under 1000 words, public policy

A general theory of public policy

This is a placeholder for future work and discussion. It tails off at the end.

People sometimes talk about a ‘general theory’ of public policy to put in our minds a comparison with the physical sciences. Usually, the punchline is that there are ‘no general theories of public policy that are not bounded by space or time’ (p21). There may be some reference to the accumulation of knowledge or wisdom in policy studies, but based rarely on the understanding that policy studies contain the equivalent of general laws (I can only think of one possible exception).

This outcome is not too surprising in the social sciences, in which context really matters and we would expect a lot of variation in policy, policymaking, and outcomes.

On the other hand, we still need a way to communicate our findings, relate them to other studies, compare them, and wonder what it all adds up to. Few people go as far as expressing the sense that every study is unique (to the point of non-comparability) and that every description of policymaking does not compare to another.

In other words, we may be looking for a happy medium, to reject the idea of general laws but encourage – when appropriate or necessary – enough of a sense of common outlook and experience to help us communicate with each other (without descending too quickly into heated debate on our cross-purposes). Or, we can at least tell a story of policy studies and invite others to learn from, or challenge, its insights.

In my case, there are two examples in which it is necessary to project some sense of a common and initially-not-too-complicated story:

  1. When describing policy theory insights to students, on the assumption that it may be their gateway to more reading.

It is possible to choose how many words to devote to each topic, including 500 Words, 1000 Words, a 9000 word Understanding Public Policy chapter, more in the source material, and even more if students start to ‘snowball’.

It is also possible, if you have a clearly defined audience, to introduce some level of uncertainty about these descriptions and their limitations.

For example, I try to describe ‘the policy process’ in 500 words and 1000 words, but in the context of a wider discussion of images of the policy process.

Circle image policy process 24.10.18

It is also possible to provide more context, such as in this kind of introductory box, coupled with 12 things to know about studying public policy

Introduction box

(from Chapter 1)

You can also get into the idea that my story is one of many, particularly after students have invested in many versions of that story by the end of an introductory book

conclusion box

(from Chapter 13)

  1. When describing these insights to people – from other disciplines or professions – who do not have the time, inclination, or frame of reference to put in that kind of work.

In this case, one presentation or article may be the limit. People may want to know the answer to a question – e.g. Why don’t policymakers listen to your evidence?rather than hear all about the explanation for the answer.

You do your best, and then – if there is time – you talk about what you missed out.

For example, in this talk, the first question was: why didn’t you mention the role of power?

 

A general theory or a general understanding? Two key issues

That was a long-winded introduction to a more philosophical point about what we might want from general theories. My impression is that you might be seeking one of these two possibilities:

  1. To use theories and concepts to describe material reality. In producing a general theory, we seek a general understanding of the ways in which the real world works. If so, we may focus primarily on how well these concepts describe the world, and the extent to which we can produce methods to produce systematic and consistent findings. The lack of a general theory denotes too much complexity and context.
  2. To use theories and concepts to represent a useful story. In producing a general understanding, we focus on the ways in which people generate and communicate their understanding. If so, we may focus more on how people come together to produce and share meaning through concepts. The lack of a general theory could reflect the lack of agreement on how to study policymaking. Or, the presence of a general understanding could represent the exercise of power, to set the agenda and limit scholarly attention to a small number of theories.

I describe this distinction in the following audio clip, produced halfway through a run with the dogs, while jetlagged. The large gap in the middle happens when I am trying to see if the voice to text is working well enough for me to copy/paste it here (no).

Key examples of the exercise of power include:

  1. The act of dismissing an individual, social group, or population by undermining the value of their knowledge or claim to knowledge (discussed in power and knowledge and Chapter 3).
  2. Ongoing discussions about how we deal with (a) a relatively new focus (among the most-established policy theories) on policy studies in countries in the Global South, given that (b) the dominant interpretations of policymaking come from experiences in the Global North.

box 13.4 part 1box 13.4 part 2

So, if you read these posts or Chapter 13 you will find a story of a general understanding of policy followed, almost immediately, by a list of reasons for why you should engage with it critically and perhaps not accept it. I’m setting your agenda but also reminding you that I’m doing it.

That’s it really. To be continued.

Leave a comment

Filed under 1000 words, Academic innovation or navel gazing, agenda setting, public policy, Storytelling

Understanding Public Policy 2nd edition

All going well, it will be out in November 2019. We are now at the proofing stage.

I have included below the summaries of the chapters (and each chapter should also have its own entry (or multiple entries) in the 1000 Words and 500 Words series).

2nd ed cover

titlechapter 1chapter 2chapter 3chapter 4.JPG

chapter 5

chapter 6chapter 7.JPG

chapter 8

chapter 9

chapter 10

chapter 11

chapter 12

chapter 13

 

2 Comments

Filed under 1000 words, 500 words, agenda setting, Evidence Based Policymaking (EBPM), Policy learning and transfer, public policy

Policy in 500 Words: The advocacy coalition framework

Here is the ACF story.

People engage in politics to turn their beliefs into policy. They form advocacy coalitions with people who share their beliefs, and compete with other coalitions. The action takes place within a subsystem devoted to a policy issue, and a wider policymaking process that provides constraints and opportunities to coalitions.

The policy process contains multiple actors and levels of government. It displays a mixture of intensely politicized disputes and routine activity. There is much uncertainty about the nature and severity of policy problems. The full effects of policy may be unclear for over a decade. The ACF sums it up in the following diagram:

acf

Policy actors use their beliefs to understand, and seek influence in, this world. Beliefs about how to interpret the cause of and solution to policy problems, and the role of government in solving them, act as a glue to bind actors together within coalitions.

If the policy issue is technical and humdrum, there may be room for routine cooperation. If the issue is highly charged, then people romanticise their own cause and demonise their opponents.

The outcome is often long-term policymaking stability and policy continuity because the ‘core’ beliefs of coalitions are unlikely to shift and one coalition may dominate the subsystem for long periods.

There are two main sources of change.

  1. Coalitions engage in policy learning to remain competitive and adapt to new information about policy. This process often produces minor change because coalitions learn on their own terms. They learn how to retain their coalition’s strategic advantage and use the information they deem most relevant.
  2. ‘Shocks’ affect the positions of coalitions within subsystems. Shocks are the combination of events and coalition responses. External shocks are prompted by events including the election of a new government with different ideas, or the effect of socioeconomic change. Internal shocks are prompted by policy failure. Both may prompt major change as members of one coalition question their beliefs in the light of new evidence. Or, another coalition may adapt more readily to its new policy environment and exploit events to gain competitive advantage.

The ACF began as the study of US policymaking, focusing largely on environmental issues. It has changed markedly to reflect the widening of ACF scholarship to new policy areas, political systems, and methods.

For example, the flow diagram’s reference to the political system’s long term coalition opportunity structures is largely the response to insights from comparative international studies:

  • A focus on the ‘degree of consensus needed for major policy change’ reflects applications in Europe that highlighted the important of proportional electoral systems
  • A focus on the ‘openness of the political system’ partly reflects applications to countries without free and fair elections, and/ or systems that do not allow people to come together easily as coalitions to promote policy change.

As such, like all theories in this series, the ACF discusses elements that it would treat as (a) universally applicable, such as the use of beliefs to address bounded rationality, and (b) context-specific, such as the motive and opportunity of specific people to organize collectively to translate their beliefs into policy.

See also:

The 500 and 1000 Words series

Why Advocacy Coalitions Matter and How to Think about Them

Three lessons from a comparison of fracking policy in the UK and Switzerland

Bonus material

Scottish Independence and the Devil Shift

Image source: Weible, Heikkila, Ingold, and Fischer (2016: 6)

https://twitter.com/amwellstead/status/1095011852915011586

 

 

 

14 Comments

Filed under 500 words, public policy

Evidence-informed policymaking: context is everything

I thank James Georgalakis for inviting me to speak at the inaugural event of IDS’ new Evidence into Policy and Practice Series, and the audience for giving extra meaning to my story about the politics of ‘evidence-based based policymaking’. The talk (using powerpoint) and Q&A is here:

https://www.facebook.com/idsuk/videos/364796097654832/

 

James invited me to respond to some of the challenges raised to my talk – in his summary of the event – so here it is.

I’m working on a ‘show, don’t tell’ approach, leaving some of the story open to interpretation. As a result, much of the meaning of this story – and, in particular, the focus on limiting participation – depends on the audience.

For example, consider the impact of the same story on audiences primarily focused on (a) scientific evidence and policy, or (b) participation and power.

Normally, when I talk about evidence and policy, my audience is mostly people with scientific or public health backgrounds asking why do policymakers ignore scientific evidence? I am usually invited to ruffle feathers, mostly by challenging a – remarkably prevalent – narrative that goes like this:

  • We know what the best evidence is, since we have produced it with the best research methods (the ‘hierarchy of evidence’ argument).
  • We have evidence on the nature of the problem and the most effective solutions (the ‘what works’ argument).
  • Policymakers seems to be ignoring our evidence or failing to act proportionately (the ‘evidence-policy barriers’ argument).
  • Or, they cherry-pick evidence to suit their agenda (the ‘policy based evidence’ argument).

In that context, I suggest that there are many claims to policy-relevant knowledge, policymakers have to ignore most information before making choices, and they are not in control of the policy process for which they are ostensibly in charge.

Limiting participation as a strategic aim

Then, I say to my audience that – if they are truly committed to maximising the use of scientific evidence in policy – they will need to consider how far they will go to get what they want. I use the metaphor of an ethical ladder in which each rung offers more influence in exchange for dirtier hands: tell stories and wait for opportunities, or demonise your opponents, limit participation, and humour politicians when they cherry-pick to reinforce emotional choices.

It’s ‘show don’t tell’ but I hope that the take-home point for most of the audience is that they shouldn’t focus so much on one aim – maximising the use of scientific evidence – to the detriment of other important aims, such as wider participation in politics beyond a reliance on a small number of experts. I say ‘keep your eyes on the prize’ but invite the audience to reflect on which prizes they should seek, and the trade-offs between them.

Limited participation – and ‘windows of opportunity’ – as an empirical finding

NASA launch

I did suggest that most policymaking happens away from the sphere of ‘exciting’ and ‘unruly’ politics. Put simply, people have to ignore almost every issue almost all of the time. Each time they focus their attention on one major issue, they must – by necessity – ignore almost all of the others.

For me, the political science story is largely about the pervasiveness of policy communities and policymaking out of the public spotlight.

The logic is as follows. Elected policymakers can only pay attention to a tiny proportion of their responsibilities. They delegate the rest to bureaucrats at lower levels of government. Bureaucrats lack specialist knowledge, and rely on other actors for information and advice. Those actors trade information for access. In many cases, they develop effective relationships based on trust and a shared understanding of the policy problem.

Trust often comes from a sense that everyone has proven to be reliable. For example, they follow norms or the ‘rules of the game’. One classic rule is to contain disputes within the policy community when actors don’t get what they want: if you complain in public, you draw external attention and internal disapproval; if not, you are more likely to get what you want next time.

For me, this is key context in which to describe common strategic concerns:

  • Should you wait for a ‘window of opportunity’ for policy change? Maybe. Or, maybe it will never come because policymaking is largely insulated from view and very few issues reach the top of the policy agenda.
  • Should you juggle insider and outsider strategies? Yes, some groups seem to do it well and it is possible for governments and groups to be in a major standoff in one field but close contact in another. However, each group must consider why they would do so, and the trade-offs between each strategy. For example, groups excluded from one venue may engage (perhaps successfully) in ‘venue shopping’ to get attention from another. Or, they become discredited within many venues if seen as too zealous and unwilling to compromise. Insider/outsider may seem like a false dichotomy to experienced and well-resourced groups, who engage continuously, and are able to experiment with many approaches and use trial-and-error learning. It is a more pressing choice for actors who may have only one chance to get it right and do not know what to expect.

Where is the power analysis in all of this?

image policy process round 2 25.10.18

I rarely use the word power directly, partly because – like ‘politics’ or ‘democracy’ – it is an ambiguous term with many interpretations (see Box 3.1). People often use it without agreeing its meaning and, if it means everything, maybe it means nothing.

However, you can find many aspects of power within our discussion. For example, insider and outsider strategies relate closely to Schattschneider’s classic discussion in which powerful groups try to ‘privatise’ issues and less powerful groups try to ‘socialise’ them. Agenda setting is about using resources to make sure issues do, or do not, reach the top of the policy agenda, and most do not.

These aspects of power sometimes play out in public, when:

  • Actors engage in politics to turn their beliefs into policy. They form coalitions with actors who share their beliefs, and often romanticise their own cause and demonise their opponents.
  • Actors mobilise their resources to encourage policymakers to prioritise some forms of knowledge or evidence over others (such as by valuing scientific evidence over experiential knowledge).
  • They compete to identify the issues most worthy of our attention, telling stories to frame or define policy problems in ways that generate demand for their evidence.

However, they are no less important when they play out routinely:

  • Governments have standard operating procedures – or institutions – to prioritise some forms of evidence and some issues routinely.
  • Many policy networks operate routinely with few active members.
  • Certain ideas, or ways of understanding the world and the nature of policy problems within it, becomes so dominant that they are unspoken and taken for granted as deeply held beliefs. Still, they constrain or facilitate the success of new ‘evidence based’ policy solutions.

In other words, the word ‘power’ is often hidden because the most profound forms of power often seem to be hidden.

In the context of our discussion, power comes from the ability to define some evidence as essential and other evidence as low quality or irrelevant, and therefore define some people as essential or irrelevant. It comes from defining some issues as exciting and worthy of our attention, or humdrum, specialist and only relevant to experts. It is about the subtle, unseen, and sometimes thoughtless ways in which we exercise power to harness people’s existing beliefs and dominate their attention as much as the transparent ways in which we mobilise resources to publicise issues. Therefore, to ‘maximise the use of evidence’ sounds like an innocuous collective endeavour, but it is a highly political and often hidden use of power.

See also:

I discussed these issues at a storytelling workshop organised by the OSF:

listening-new-york-1-11-16

See also:

Policy in 500 Words: Power and Knowledge

The politics of evidence-based policymaking

Palgrave Communications: The politics of evidence-based policymaking

Using evidence to influence policy: Oxfam’s experience

The UK government’s imaginative use of evidence to make policy

https://youtu.be/A7qYj8nRkYg

 

4 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), Policy learning and transfer, Psychology Based Policy Studies, public policy, Storytelling

Taking lessons from policy theory into practice: 3 examples

Notes for ANZSOG/ ANU Crawford School/ UNSW Canberra workshop. Powerpoint here. The recording of the lecture (skip to 2m30) and Q&A is here (right click to download mp3 or dropbox link):

The context for this workshop is the idea that policy theories could be more helpful to policymakers/ practitioners if we could all communicate more effectively with each other. Academics draw general and relatively abstract conclusions from multiple cases. Practitioners draw very similar conclusions from rich descriptions of direct experience in a smaller number of cases. How can we bring together their insights and use a language that we all understand? Or, more ambitiously, how can we use policy theory-based insights to inform the early career development training that civil servants and researchers receive?

The first step is to translate policy theories into a non-technical language by trying to speak with an audience beyond our immediate peers (see for example Practical Lessons from Policy Theories).

However, translation is not enough. A second crucial step is to consider how policymakers and practitioners are likely to make sense of theoretical insights when they apply them to particular aims or responsibilities. For example:

  1. Central government policymakers may accept the descriptive accuracy of policy theories emphasising limited central control, but not the recommendation that they should let go, share power, and describe their limits to the public.
  2. Scientists may accept key limitations to ‘evidence based policymaking’ but reject the idea that they should respond by becoming better storytellers or more manipulative operators.
  3. Researchers and practitioners struggle to resolve hard choices when combining evidence and ‘coproduction’ while ‘scaling up’ policy interventions. Evidence choice is political choice. Can we do more than merely encourage people to accept this point?

I discuss these examples below because they are closest to my heart (especially example 1). Note throughout that I am presenting one interpretation about: (1) the most promising insights, and (2) their implications for practice. Other interpretations of the literature and its implications are available. They are just a bit harder to find.

Example 1: the policy cycle endures despite its descriptive inaccuracy

cycle

The policy cycle does not describe and explain the policy process well:

  • If we insist on keeping the cycle metaphor, it is more accurate to see the process as a huge set of policy cycles that connect with each other in messy and unpredictable ways.
  • The cycle approach also links strongly to the idea of ‘comprehensive rationality’ in which a small group of policymakers and analysts are in full possession of the facts and full control of the policy process. They carry out their aims through a series of stages.

Policy theories provide more descriptive and explanatory usefulness. Their insights include:

  • Limited choice. Policymakers inherit organisations, rules, and choices. Most ‘new’ choice is a revision of the old.
  • Limited attention. Policymakers must ignore almost all of the policy problems for which they are formally responsible. They pay attention to some, and delegate most responsibility to civil servants. Bureaucrats rely on other actors for information and advice, and they build relationships on trust and information exchange.
  • Limited central control. Policy may appear to be made at the ‘top’ or in the ‘centre’, but in practice policymaking responsibility is spread across many levels and types of government (many ‘centres’). ‘Street level’ actors make policy as they deliver. Policy outcomes appear to ‘emerge’ locally despite central government attempts to control their fate.
  • Limited policy change. Most policy change is minor, made and influenced by actors who interpret new evidence through the lens of their beliefs. Well-established beliefs limit the opportunities of new solutions. Governments tend to rely on trial-and-error, based on previous agreements, rather than radical policy change based on a new agenda. New solutions succeed only during brief and infrequent windows of opportunity.

However, the cycle metaphor endures because:

  • It provides a simple model of policymaking with stages that map onto important policymaking functions.
  • It provides a way to project policymaking to the public. You know how we make policy, and that we are in charge, so you know who to hold to account.

In that context, we may want to be pragmatic about our advice:

  1. One option is via complexity theory, in which scholars generally encourage policymakers to accept and describe their limits:
  • Accept routine error, reduce short-term performance management, engage more in trial and error, and ‘let go’ to allow local actors the flexibility to adapt and respond to their context.
  • However, would a government in the Westminster tradition really embrace this advice? No. They need to balance (a) pragmatic policymaking, and (b) an image of governing competence.
  1. Another option is to try to help improve an existing approach.

Further reading (blog posts):

The language of complexity does not mix well with the language of Westminster-style accountability

Making Sense of Policymaking: why it’s always someone else’s fault and nothing ever changes

Two stories of British politics: the Westminster model versus Complex Government

Example 2: how to deal with a lack of ‘evidence based policymaking’

I used to read many papers on tobacco policy, with the same basic message: we have the evidence of tobacco harm, and evidence of which solutions work, but there is an evidence-policy gap caused by too-powerful tobacco companies, low political will, and pathological policymaking. These accounts are not informed by theories of policymaking.

I then read Oliver et al’s paper on the lack of policy theory in health/ environmental scholarship on the ‘barriers’ to the use of evidence in policy. Very few articles rely on policy concepts, and most of the few rely on the policy cycle. This lack of policy theory is clear in their description of possible solutions – better communication, networking, timing, and more science literacy in government – which does not describe well the need to respond to policymaker psychology and a complex policymaking environment.

So, I wrote The Politics of Evidence-Based Policymaking and one zillion blog posts to help identify the ways in which policy theories could help explain the relationship between evidence and policy.

Since then, the highest demand to speak about the book has come from government/ public servant, NGO, and scientific audiences outside my discipline. The feedback is generally that: (a) the book’s description sums up their experience of engagement with the policy process, and (b) maybe it opens up discussion about how to engage more effectively.

But how exactly do we turn empirical descriptions of policymaking into practical advice?

For example, scientist/ researcher audiences want to know the answer to a question like: Why don’t policymakers listen to your evidence? and so I focus on three conversation starters:

  1. they have a broader view on what counts as good evidence (see ANZSOG description)
  2. they have to ignore almost all information (a nice way into bounded rationality and policymaker psychology)
  3. they do not understand or control the process in which they seek to use evidence (a way into ‘the policy process’)

Cairney 2017 image of the policy process

We can then consider many possible responses in the sequel What can you do when policymakers ignore your evidence?

Examples include:

  • ‘How to do it’ advice. I compare tips for individuals (from experienced practitioners) with tips based on policy concepts. They are quite similar-looking tips – e.g. find out where the action is, learn the rules, tell good stories, engage allies, seek windows of opportunity – but I describe mine as 5 impossible tasks!
  • Organisational reform. I describe work with the European Commission Joint Research Centre to identify 8 skills or functions of an organisation bringing together the supply/demand of knowledge.
  • Ethical dilemmas. I use key policy theories to ask people how far they want to go to privilege evidence in policy. It’s fun to talk about these things with the type of scientist who sees any form of storytelling as manipulation.

Further reading:

Is Evidence-Based Policymaking the same as good policymaking?

A 5-step strategy to make evidence count

Political science improves our understanding of evidence-based policymaking, but does it produce better advice?

Principles of science advice to government: key problems and feasible solutions

Example 3: how to encourage realistic evidence-informed policy transfer

This focus on EBPM is useful context for discussions of ‘policy learning’ and ‘policy transfer’, and it was the focus of my ANZOG talk entitled (rather ambitiously) ‘teaching evidence-based policy to fly’.

I’ve taken a personal interest in this one because I’m part of a project – called IMAJINE – in which we have to combine academic theory and practical responses. We are trying to share policy solutions across Europe rather than explain why few people share them!

For me, the context is potentially overwhelming:

So, when we start to focus on sharing lessons, we will have three things to discover:

  1. What is the evidence for success, and from where does it come? Governments often project success without backing it up.
  2. What story do policymakers tell about the problem they are trying to solve, the solutions they produced, and why? Two different governments may be framing and trying to solve the same problem in very different ways.
  3. Was the policy introduced in a comparable policymaking system? People tend to focus on political system comparability (e.g. is it unitary or federal?), but I think the key is in policymaking system comparability (e.g. what are the rules and dominant ideas?).

To be honest, when one of our external assessors asked me how well I thought I would do, we both smiled because the answer may be ‘not very’. In other words, the most practical lesson may be the hardest to take, although I find it comforting: the literature suggests that policymakers might ignore you for 20 years then suddenly become very (but briefly) interested in your work.

 

The slides are a bit wonky because I combined my old ppt to the Scottish Government with a new one for UNSW Paul Cairney ANU Policy practical 22 October 2018

I wanted to compare how I describe things to (1) civil servants (2) practitioners/ researcher (3) me, but who has the time/ desire to listen to 3 powerpoints in one go? If the answer is you, let me know and we’ll set up a Zoom call.

2 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), IMAJINE, Policy learning and transfer

The Politics of Evidence-Based Policymaking: ANZSOG talks

This post introduces a series of related talks on ‘the politics of evidence-based policymaking’ (EBPM) that I’m giving as part of larger series of talks during this ANZOG-funded/organised trip.

The EBPM talks begin with a discussion of the same three points: what counts as evidence, why we must ignore most of it (and how), and the policy process in which policymakers use some of it. However, the framing of these points, and the ways in which we discuss the implications, varies markedly by audience. So, in this post, I provide a short discussion of the three points, then show how the audience matters (referring to the city as a shorthand for each talk).

The overall take-home points are highly practical, in the same way that critical thinking has many practical applications (in other words, I’m not offering a map, toolbox, or blueprint):

  • If you begin with (a) the question ‘why don’t policymakers use my evidence?’ I like to think you will end with (b) the question ‘why did I ever think they would?’.
  • If you begin by taking the latter as (a) a criticism of politics and policymakers, I hope you will end by taking it as (b) a statement of the inevitability of the trade-offs that must accompany political choice.
  • We may address these issues by improving the supply and use of evidence. However, it is more important to maintain the legitimacy of the politicians and political systems in which policymakers choose to ignore evidence. Technocracy is no substitute for democracy.

3 ways to describe the use of evidence in policymaking

  1. Discussions of the use of evidence in policy often begin as a valence issue: who wouldn’t want to use good evidence when making policy?

However, it only remains a valence issue when we refuse to define evidence and justify what counts as good evidence. After that, you soon see the political choices emerge. A reference to evidence is often a shorthand for scientific research evidence, and good often refers to specific research methods (such as randomised control trials). Or, you find people arguing very strongly in the almost-opposite direction, criticising this shorthand as exclusionary and questioning the ability of scientists to justify claims to superior knowledge. Somewhere in the middle, we find that a focus on evidence is a good way to think about the many forms of information or knowledge on which we might make decisions, including: a wider range of research methods and analyses, knowledge from experience, and data relating to the local context with which policy would interact.

So, what begins as a valence issue becomes a gateway to many discussions about how to understand profound political choices regarding: how we make knowledge claims, how to ‘co-produce’ knowledge via dialogue among many groups, and the relationship between choices about evidence and governance.

  1. It is impossible to pay attention to all policy relevant evidence.

There is far more information about the world than we are able to process. A focus on evidence gaps often gives way to the recognition that we need to find effective ways to ignore most evidence.

There are many ways to describe how individuals combine cognition and emotion to limit their attention enough to make choices, and policy studies (to all intents and purposes) describe equivalent processes – described, for example, as ‘institutions’ or rules – in organisations and systems.

One shortcut between information and choice is to set aims and priorities; to focus evidence gathering on a small number of problems or one way to define a problem, and identify the most reliable or trustworthy sources of evidence (often via evidence ‘synthesis’). Another is to make decisions quickly by relying on emotion, gut instinct, habit, and existing knowledge or familiarity with evidence.

Either way, agenda setting and problem definition are political processes that address uncertainty and ambiguity. We gather evidence to reduce uncertainty, but first we must reduce ambiguity by exercising power to define the problem we seek to solve.

  1. It is impossible to control the policy process in which people use evidence.

Policy textbooks (well, my textbook at least!) provide a contrast between:

  • The model of a ‘policy cycle’ that sums up straightforward policymaking, through a series of stages, over which policymakers have clear control. At each stage, you know where evidence fits in: to help define the problem, generate solutions, and evaluate the results to set the agenda for the next cycle.
  • A more complex ‘policy process’, or policymaking environment, of which policymakers have limited knowledge and even less control. In this environment, it is difficult to know with whom engage, the rules of engagement, or the likely impact of evidence.

Overall, policy theories have much to offer people with an interest in evidence-use in policy, but primarily as a way to (a) manage expectations, to (b) produce more realistic strategies and less dispiriting conclusions. It is useful to frame our aim as to analyse the role of evidence within a policy process that (a) we don’t quite understand, rather than (b) we would like to exist.

The events themselves

Below, you will find a short discussion of the variations of audience and topic. I’ll update and reflect on this discussion (in a revised version of this post) after taking part in the events.

Social science and policy studies: knowledge claims, bounded rationality, and policy theory

For Auckland and Wellington A, I’m aiming for an audience containing a high proportion of people with a background in social science and policy studies. I describe the discussion as ‘meta’ because I am talking about how I talk about EBPM to other audiences, then inviting discussion on key parts of that talk, such as how to conceptualise the policy process and present conceptual insights to people who have no intention of deep dives into policy theory.

I often use the phrase ‘I’ve read it, so you don’t have to’ partly as a joke, but also to stress the importance of disciplinary synthesis when we engage in interdisciplinary (and inter-professional) discussion. If so, it is important to discuss how to produce such ‘synthetic’ accounts.

I tend to describe key components of a policymaking environment quickly: many policy makers and influencers spread across many levels and types of government, institutions, networks, socioeconomic factors and events, and ideas. However, each of these terms represents a shorthand to describe a large and diverse literature. For example, I can describe an ‘institution’ in a few sentences, but the study of institutions contains a variety of approaches.

Background post: I know my audience, but does my other audience know I know my audience?

Academic-practitioner discussions: improving the use of research evidence in policy

For Wellington B and Melbourne, the audience is an academic-practitioner mix. We discuss ways in which we can encourage the greater use of research evidence in policy, perhaps via closer collaboration between suppliers and users.

Discussions with scientists: why do policymakers ignore my evidence?

Sydney UNSW focuses more on researchers in scientific fields (often not in social science).  I frame the question in a way that often seems central to scientific researcher interest: why do policymakers seem to ignore my evidence, and what can I do about it?

Then, I tend to push back on the idea that the fault lies with politics and policymakers, to encourage researchers to think more about the policy process and how to engage effectively in it. If I’m trying to be annoying, I’ll suggest to a scientific audience that they see themselves as ‘rational’ and politicians as ‘irrational’. However, the more substantive discussion involves comparing (a) ‘how to make an impact’ advice drawn from the personal accounts of experienced individuals, giving advice to individuals, and (b) the sort of advice you might draw from policy theories which focus more on systems.

Background post: What can you do when policymakers ignore your evidence?

https://youtu.be/L77Y7wynqXY

https://www.youtube.com/watch?v=Nk6g-xh-NN8&feature=youtu.be

Early career researchers: the need to build ‘impact’ into career development

Canberra UNSW is more focused on early career researchers. I think this is the most difficult talk because I don’t rely on the same joke about my role: to turn up at the end of research projects to explain why they failed to have a non-academic impact.  Instead, my aim is to encourage intelligent discussion about situating the ‘how to’ advice for individual researchers into a wider discussion of policymaking systems.

Similarly, Brisbane A and B are about how to engage with practitioners, and communicate well to non-academic audiences, when most of your work and training is about something else entirely (such as learning about research methods and how to engage with the technical language of research).

Background posts:

What can you do when policymakers ignore your evidence? Tips from the ‘how to’ literature from the science community

What can you do when policymakers ignore your evidence? Encourage ‘knowledge management for policy’

See also:

  1. A similar talk at LSHTM (powerpoint and audio)

https://twitter.com/CairneyPaul/status/1065310239329853440

2. European Health Forum Gastein 2018 ‘Policy in Evidence’ (from 6 minutes)

https://webcasting.streamdis.eu/Mediasite/Play/8143157d976146b4afd297897c68be5e1d?catalog=62e4886848394f339ff678a494afd77f21&playFrom=126439&autoStart=true

 

See also:

Evidence-based policymaking and the new policy sciences

 

8 Comments

Filed under Evidence Based Policymaking (EBPM)

Managing expectations about the use of evidence in policy

Notes for the #transformURE event hosted by Nuffield, 25th September 2018

I like to think that I can talk with authority on two topics that, much like a bottle of Pepsi and a pack of Mentos, you should generally keep separate:

  1. When talking at events on the use of evidence in policy, I say that you need to understand the nature of policy and policymaking to understand the role of evidence in it.
  2. When talking with students, we begin with the classic questions ‘what is policy?’ and ‘what is the policy process’, and I declare that we don’t know the answer. We define policy to show the problems with all definitions of policy, and we discuss many models and theories that only capture one part of the process. There is no ‘general theory’ of policymaking.

The problem, when you put together those statements, is that you need to understand the role of evidence within a policy process that we don’t really understand.

It’s an OK conclusion if you just want to declare that the world is complicated, but not if you seek ways to change it or operate more effectively within it.

Put less gloomily:

  • We have ways to understand key parts of the policy process. They are not ready-made to help us understand evidence use, but we can use them intelligently.
  • Most policy theories exist to explain policy dynamics, not to help us adapt effectively to them, but we can derive general lessons with often-profound implications.

Put even less gloomily, it is not too difficult to extract/ synthesise key insights from policy theories, explain their relevance, and use them to inform discussions about how to promote your preferred form of evidence use.

The only remaining problem is that, although the resultant advice looks quite straightforward, it is far easier said than done. The proposed actions are more akin to the Labours of Hercules than [PAC: insert reference to something easier].

They include:

  1. Find out where the ‘action’ is, so that you can find the right audience for your evidence. Why? There are many policymakers and influencers spread across many levels and types of government.
  2. Learn and follow the ‘rules of the game’. Why? Each policymaking venue has its own rules of engagement and evidence gathering, and the rules are often informal and unwritten.
  3. Gain access to ‘policy networks’. Why? Most policy is processed at a low level of government, beyond the public spotlight, between relatively small groups of policymakers and influencers. They build up trust as they work together, learning who is reliable and authoritative, and converging on how to use evidence to understand the nature and solution to policy problems.
  4. Learn the language. Why? Each venue has its own language to reflect dominant ideas, beliefs, or ways to understand a policy problem. In some arenas, there is a strong respect for a ‘hierarchy’ of evidence. In others, they key reference point may be value for money. In some cases, the language reflects the closing-off of some policy solutions (such as redistributing resources from one activity to another).
  5. Exploit windows of opportunity. Why? Events, and changes in socioeconomic conditions, often prompt shifts of attention to policy issues. ‘Policy entrepreneurs’ lie in wait for the right time to exploit a shift in the motive and opportunity of a policymaker to pay attention to and try to solve a problem.

So far so good, until you consider the effort it would take to achieve any of these things: you may need to devote the best part of your career to these tasks with no guarantee of success.

Put more positively, it is better to be equipped with these insights, and to appreciate the limits to our actions, than to think we can use top tips to achieve ‘research impact’ in a more straightforward way.

Kathryn Oliver and I describe these ‘how to’ tips in this post and, in this article in Political Studies Review, use a wider focus on policymaking environments to produce a more realistic sense of what individual researchers – and research-producing organisations – could achieve.

There is some sensible-enough advice out there for individuals – produce good evidence, communicate it well, form relationships with policymakers, be available, and so on – but I would exercise caution when it begins to recommend being ‘entrepreneurial’. The opportunities to be entrepreneurial are not shared equally, most entrepreneurs fail, and we can likely better explain their success with reference to their environment than their skill.

hang-in-there-baby

3 Comments

Filed under agenda setting, Evidence Based Policymaking (EBPM), public policy, UK politics and policy

Why don’t policymakers listen to your evidence?

Since 2016, my most common academic presentation to interdisciplinary scientist/ researcher audiences is a variant of the question, ‘why don’t policymakers listen to your evidence?’

I tend to provide three main answers.

1. Many policymakers have many different ideas about what counts as good evidence

Few policymakers know or care about the criteria developed by some scientists to describe a hierarchy of scientific evidence. For some scientists, at the top of this hierarchy is the randomised control trial (RCT) and the systematic review of RCTs, with expertise much further down the list, followed by practitioner experience and service user feedback near the bottom.

Yet, most policymakers – and many academics – prefer a wider range of sources of information, combining their own experience with information ranging from peer reviewed scientific evidence and the ‘grey’ literature, to public opinion and feedback from consultation.

While it may be possible to persuade some central government departments or agencies to privilege scientific evidence, they also pursue other key principles, such as to foster consensus driven policymaking or a shift from centralist to localist practices.

Consequently, they often only recommend interventions rather than impose one uniform evidence-based position. If local actors favour a different policy solution, we may find that the same type of evidence may have more or less effect in different parts of government.

2. Policymakers have to ignore almost all evidence and almost every decision taken in their name

Many scientists articulate the idea that policymakers and scientists should cooperate to use the best evidence to determine ‘what works’ in policy (in forums such as INGSA, European Commission, OECD). Their language is often reminiscent of 1950s discussions of the pursuit of ‘comprehensive rationality’ in policymaking.

The key difference is that EBPM is often described as an ideal by scientists, to be compared with the more disappointing processes they find when they engage in politics. In contrast, ‘comprehensive rationality’ is an ideal-type, used to describe what cannot happen, and the practical implications of that impossibility.

The ideal-type involves a core group of elected policymakers at the ‘top’, identifying their values or the problems they seek to solve, and translating their policies into action to maximise benefits to society, aided by neutral organisations gathering all the facts necessary to produce policy solutions. Yet, in practice, they are unable to: separate values from facts in any meaningful way; rank policy aims in a logical and consistent manner; gather information comprehensively, or possess the cognitive ability to process it.

Instead, Simon famously described policymakers addressing ‘bounded rationality’ by using ‘rules of thumb’ to limit their analysis and produce ‘good enough’ decisions. More recently, punctuated equilibrium theory uses bounded rationality to show that policymakers can only pay attention to a tiny proportion of their responsibilities, which limits their control of the many decisions made in their name.

More recent discussions focus on the ‘rational’ short cuts that policymakers use to identify good enough sources of information, combined with the ‘irrational’ ways in which they use their beliefs, emotions, habits, and familiarity with issues to identify policy problems and solutions (see this post on the meaning of ‘irrational’). Or, they explore how individuals communicate their narrow expertise within a system of which they have almost no knowledge. In each case, ‘most members of the system are not paying attention to most issues most of the time’.

This scarcity of attention helps explain, for example, why policymakers ignore most issues in the absence of a focusing event, policymaking organisations make searches for information which miss key elements routinely, and organisations fail to respond to events or changing circumstances proportionately.

In that context, attempts to describe a policy agenda focusing merely on ‘what works’ are based on misleading expectations. Rather, we can describe key parts of the policymaking environment – such as institutions, policy communities/ networks, or paradigms – as a reflection of the ways in which policymakers deal with their bounded rationality and lack of control of the policy process.

3. Policymakers do not control the policy process (in the way that a policy cycle suggests)

Scientists often appear to be drawn to the idea of a linear and orderly policy cycle with discrete stages – such as agenda setting, policy formulation, legitimation, implementation, evaluation, policy maintenance/ succession/ termination – because it offers a simple and appealing model which gives clear advice on how to engage.

Indeed, the stages approach began partly as a proposal to make the policy process more scientific and based on systematic policy analysis. It offers an idea of how policy should be made: elected policymakers in central government, aided by expert policy analysts, make and legitimise choices; skilful public servants carry them out; and, policy analysts assess the results with the aid of scientific evidence.

Yet, few policy theories describe this cycle as useful, while most – including the advocacy coalition framework , and the multiple streams approach – are based on a rejection of the explanatory value of orderly stages.

Policy theories also suggest that the cycle provides misleading practical advice: you will generally not find an orderly process with a clearly defined debate on problem definition, a single moment of authoritative choice, and a clear chance to use scientific evidence to evaluate policy before deciding whether or not to continue. Instead, the cycle exists as a story for policymakers to tell about their work, partly because it is consistent with the idea of elected policymakers being in charge and accountable.

Some scholars also question the appropriateness of a stages ideal, since it suggests that there should be a core group of policymakers making policy from the ‘top down’ and obliging others to carry out their aims, which does not leave room for, for example, the diffusion of power in multi-level systems, or the use of ‘localism’ to tailor policy to local needs and desires.

Now go to:

What can you do when policymakers ignore your evidence?

Further Reading

The politics of evidence-based policymaking

The politics of evidence-based policymaking: maximising the use of evidence in policy

Images of the policy process

How to communicate effectively with policymakers

Special issue in Policy and Politics called ‘Practical lessons from policy theories’, which includes how to be a ‘policy entrepreneur’.

See also the 750 Words series to explore the implications for policy analysis

16 Comments

Filed under Evidence Based Policymaking (EBPM), Psychology Based Policy Studies, Public health, public policy

Policy in 500 words: uncertainty versus ambiguity

In policy studies, there is a profound difference between uncertainty and ambiguity:

  • Uncertainty describes a lack of knowledge or a worrying lack of confidence in one’s knowledge.
  • Ambiguity describes the ability to entertain more than one interpretation of a policy problem.

Both concepts relate to ‘bounded rationality’: policymakers do not have the ability to process all information relevant to policy problems. Instead, they employ two kinds of shortcut:

  • ‘Rational’. Pursuing clear goals and prioritizing certain sources of information.
  • ‘Irrational’. Drawing on emotions, gut feelings, deeply held beliefs, and habits.

I make an artificially binary distinction, uncertain versus ambiguous, and relate it to another binary, rational versus irrational, to point out the pitfalls of focusing too much on one aspect of the policy process:

  1. Policy actors seek to resolve uncertainty by generating more information or drawing greater attention to the available information.

Actors can try to solve uncertainty by: (a) improving the quality of evidence, and (b) making sure that there are no major gaps between the supply of and demand for evidence. Relevant debates include: what counts as good evidence?, focusing on the criteria to define scientific evidence and their relationship with other forms of knowledge (such as practitioner experience and service user feedback), and what are the barriers between supply and demand?, focusing on the need for better ways to communicate.

  1. Policy actors seek to resolve ambiguity by focusing on one interpretation of a policy problem at the expense of another.

Actors try to solve ambiguity by exercising power to increase attention to, and support for, their favoured interpretation of a policy problem. You will find many examples of such activity spread across the 500 and 1000 words series:

A focus on reducing uncertainty gives the impression that policymaking is a technical process in which people need to produce the best evidence and deliver it to the right people at the right time.

In contrast, a focus on reducing ambiguity gives the impression of a more complicated and political process in which actors are exercising power to compete for attention and dominance of the policy agenda. Uncertainty matters, but primarily to describe the role of a complex policymaking system in which no actor truly understands where they are or how they should exercise power to maximise their success.

Further reading:

For a longer discussion, see Fostering Evidence-informed Policy Making: Uncertainty Versus Ambiguity (PDF)

Or, if you fancy it in French: Favoriser l’élaboration de politiques publiques fondées sur des données probantes : incertitude versus ambiguïté (PDF)

Framing

The politics of evidence-based policymaking

To Bridge the Divide between Evidence and Policy: Reduce Ambiguity as Much as Uncertainty

How to communicate effectively with policymakers: combine insights from psychology and policy studies

Here is the relevant opening section in UPP:

p234 UPP ambiguity

27 Comments

Filed under 500 words, agenda setting, Evidence Based Policymaking (EBPM), public policy, Storytelling

The global study of governance and public policy

Cairney GPP Iran

I was invited by Dr Emamian from the Governance and Policy Think Tank to deliver this short lecture at the first ‘governance and public policy conference’ in Iran. I was unable to attend, so recorded a set of short video presentations supplemented by blog discussion. The topics to be covered include the importance of a scholarly network for policy studies, the need for a set of core policy concepts to act as a technical language for that network, and the need to apply that language to explain shifts in government and regulation towards ‘regulatory governance’.

Please note that my choice to record the videos in my garden (while I look up) seemed good at the time, for some very good reasons that I won’t get into. However, you will see that I become increasingly cold and annoyed at being cold. I can only apologize for my face and the fact that I was too cold to remember to put on my professional voice.

Using shared concepts in a scholarly network of policy researchers

Our aim may be to produce a global network of policy scholars, in two main ways:

  1. To foster meetings and discussion, such as via this conference and others such as the ICPP and ECPR
  2. To make sure that we are talking about the same thing. Most of the theories to which I refer are based on studies of countries like the US and UK. Their prominence contributes to a ‘global north’ perspective which can be useful in the abstract but with uncertain applicability across the globe.

For example, when considering the applicability of US-inspired theories, think about their taken-for-granted assumptions about the nature of a political system, in which leaders in many levels and types of government are elected regularly, there is a constitution guaranteeing a division of powers across legislative-executive-judicial branches and between federal/subnational levels, and people describe a ‘pluralist’ system in which many groups mobilise and counter-mobilise to influence policy.

What happens when we stop taking this political context for granted? Do these theories remain as relevant?

Which concepts do we use?

I describe two main abstract concepts then invite you to think about how to apply them in more concrete circumstances.

  1. Bounded rationality, not comprehensive rationality.

No-one can understand fully the world in which we live. Individuals can only understand and pay attention to a tiny part of key aspects of the world such as political systems.

Indeed, a handy phrase to remember is that almost all people must ignore almost everything almost all of the time.

Yet, they must make choices despite uncertainty, perhaps by adopting ‘fast and frugal’ heuristics. In other words, we may see all human choices as flawed when compared with an ideal of perfect decision-making. On the other hand, we may marvel at the ways in which humans make often-good choices despite their limitations.

Individual policymakers use two short-cuts to gather enough information to make choices:

  • ‘Rational’, in which they adopt measures to ensure that they have good enough information to inform decisions. For example, they prioritise certain written sources of information and draw on people they consider to be experts.
  • ‘Irrational’, in which they rely on things like gut instinct, habit, and emotion to make snap decisions.

In that context, policy scholarship involves the study of how people make and influence those choices. One part is about the role of evidence, in which people produce information to reduce uncertainty about the nature of the world. However, the more important study is of how people understand the world in the first place. As policy scholars, we focus on ambiguity, to describe the many ways in which people choose to understand the same problems, and the exercise of power to influence those choices.

  1. A complex policymaking environment, not a policy cycle.

Things get more complicated when we move from the analysis of (a) key individuals to (b) the interaction between many individuals and organisations in a complex policymaking ‘system’ or ‘environment’. Policy scholars describe this environment in many different ways, using different concepts, but we can identify a core set of terms on which to focus:

  • Actors. There are many policy influencers and policymakers in many authoritative venues spread across many levels and types of government.
  • Institutions. Each venue has its own rules, including the formal, written-down, and easy to understand rules, versus the informal norms, cultures, and practices which are difficult to identify and describe.
  • Networks. Policymakers and influencers form relationships based on factors such as trust, authority, and the exchange of resources such as information and support.
  • Ideas. People communicate their beliefs, about policy problems and potential solutions, within a wider understanding of the world (often described as a paradigm or hegemony). Some of that understanding is taken-for-granted and not described, and people limit their analysis and argument according to the ways in which they think other people see the world.
  • Socioeconomic context and events. Policymakers often have to respond to policy conditions and events over which they have limited control, such geographic, demographic, and economic factors. These factors help produce non-routine ‘events’ alongside more predictable events such as elections (or other means to ensure a change of government).

In that context, policy scholarship focuses on producing theories to explain what happens when policymakers have limited control over their political systems and policymaking environments.

How far do these concepts travel?

As you can see, these concepts are widely applicable because they are abstract. What happens when we try to apply them to specific countries or case studies? For example:

  1. We talk about policymakers using cognitive, moral, and emotional shortcuts, but those shortcuts can vary profoundly across the globe.
  2. Each political system has a different collection of authoritative venues, formal and informal rules of politics, networks of power, ways to describe how the world works and should work, and socioeconomic context.

This is where our global network becomes valuable, to help us describe how we make sense of the same concepts in very different ways, and consider the extent to which such discussions are comparable.

Example: how do governments address an ‘era of governance’?

One way to foster such discussion is to consider how governments address the limits to their powers. These limits are described in many different ways, from a focus on ‘complexity’ and policy outcomes which ‘emerge’ from local activity (despite attempts by central governments to control outcomes), to a focus on the shift from ‘government’ to ‘governance’.

As policy scholars, we can make several useful distinctions to describe these dynamics, such as to separate an actual shift in policymaking from government to governance, versus a shift in the way we now describe government.

Or, we can separate how governments can, do, and should address the limits to their powers.

I’d say that most policy scholarship focuses on how governments operate: how they actually address problems and what are the – intended and unintended – consequences.

However, these studies are trying to describe the tensions between what governments can do, given the limits I describe, and what they think they should do, given their position of authority and their need to describe their success.

For example, some systems may be more conducive to the support for ‘polycentric governance’, in which many authoritative venues cooperate to address problems, while others are built on the idea of central control and the concentration of authority in a small group of actors.

Therefore, the study of actual policymaking and outcomes will vary markedly according to the ways in which government actors feel they need to assert an image of control over a policy environment which is almost immune to control.

Perhaps an ‘era of governance’ describes some recognition by many governments that they need to find new ways to address their limited control over policy outcomes, both domestically and globally. However, an enduring theme in political science and policy studies is that we do not explain policymaking well if we restrict our attention to the ‘rational’ decisions of a small number of actors. Let’s not make too many assumptions about their power and motive.

Further reading:

The New Policy Sciences

How to Communicate Effectively with Policymakers

The Politics of Evidence Based Policymaking

Comparison of Theories of the Policy Process

Key policy theories and concepts in 1000 words.

Leave a comment

Filed under public policy

What do we need to know about the politics of evidence-based policymaking?

Today, I’m helping to deliver a new course – Engaging Policymakers Training Programme – piloted by the Alliance for Useful Evidence and the UCL. Right now, it’s for UCL staff (and mostly early career researchers). My bit is about how we can better understand the policy process so that we can engage in it more effectively.  I have reproduced the brief guide below (for my two 2-hour sessions as part of a wider block). If anyone else is delivering something similar, please let me know. We could compare notes. 

This module will be delivered in two parts to combine theory and practice

Part 1: What do we need to know about the politics of evidence-based policymaking?

Policy theories provide a wealth of knowledge about the role of evidence in policymaking systems. They prompt us to understand and respond to two key dynamics:

  1. Policymaker psychology. Policymakers combine rational and irrational shortcuts to gather information and make good enough decisions quickly. To appeal to rational shortcuts and minimise cognitive load, we reduce uncertainty by providing syntheses of the available evidence. To appeal to irrational shortcuts and engage emotional interest, we reduce ambiguity by telling stories or framing problems in specific ways.
  2. Complex policymaking environments. These processes take place in the context of a policy environment out of the control of individual policymakers. Environments consist of: many actors in many levels and types of government; engaging with institutions and networks, each with their own informal and formal rules; responding to socioeconomic conditions and events; and, learning how to engage with dominant ideas or beliefs about the nature of the policy problem. In other words, there is no policy cycle or obvious stage in which to get involved.

In this seminar, we discuss how to respond effectively to these dynamics. We focus on unresolved issues:

  1. Effective engagement with policymakers requires storytelling skills, but do we possess them?
  2. It requires a combination of evidence and emotional appeals, but is it ethical to do more than describe the evidence?
  3. The absence of a policy cycle, and presence of an ever-shifting context, requires us to engage for the long term, to form alliances, learn the rules, and build up trust in the messenger. However, do we have and how should we invest the time?

The format will be relatively informal. Cairney will begin by making some introductory points (not a powerpoint driven lecture) and encourage participants to relate the three questions to their research and engagement experience.

Gateway to further reading:

  • Paul Cairney and Richard Kwiatkowski (2017) ‘How to communicate effectively with policymakers: combine insights from psychology and policy studies’, Palgrave Communications
  • Paul Cairney and Kathryn Oliver (2017) ‘Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy?’ Health Research Policy and Systems (HARPS), DOI: 10.1186/s12961-017-0192-x
  • Paul Cairney, Kathryn Oliver, and Adam Wellstead (2016) ‘To Bridge the Divide between Evidence and Policy: Reduce Ambiguity as Much as Uncertainty’, Public Administration Review, Early View (forthcoming) DOI:10.1111/puar.12555 PDF

Part 2: How can we respond pragmatically and effectively to the politics of EBPM?

In this seminar, we move from abstract theory and general advice to concrete examples and specific strategies. Each participant should come prepared to speak about their research and present a theoretically informed policy analysis in 3 minutes (without the aid of powerpoint). Their analysis should address:

  1. What policy problem does my research highlight?
  2. What are the most technically and politically feasible solutions?
  3. How should I engage in the policy process to highlight these problems and solutions?

After each presentation, each participant should be prepared to ask questions about the problem raised and the strategy to engage. Finally, to encourage learning, we will reflect on the memorability and impact of presentations.

Powerpoint: Paul Cairney A4UE UCL 2017

1 Comment

Filed under Evidence Based Policymaking (EBPM), Psychology Based Policy Studies, public policy

How to write theory-driven policy analysis

Writing theory driven policy analysis 10.11.17

(or right click to download this lecture which accompanies my MPP)

Here is a guide to writing theory-driven policy analysis. Your aim is to identify a policy problem and solution, know your audience, and account for the complexity of policymaking.

At first, it may seem like daunting task to put together policy analysis and policy theory. On its own, policy analysis seems difficult but relatively straightforward: use evidence to identify and measure a policy problem, compare the merits of one or more solution, and make a recommendation on the steps to take us from policy to action.

However, policy process research tells us that people will engage emotionally with that evidence, and that policymakers operate in a complex system of which they have very limited knowledge and control.

So, how can we produce a policy analysis paper to which people will pay attention, and respond positively and effectively, under such circumstances? I focus on developing the critical analysis that will help you produce effective and feasible analysis. To do so, I show how policy analysis forms part of a collection of exercises to foster analysis informed by theory and reflection.

Aims of this document:

  1. Describe the context. There are two fields of study – theory and analysis – which do not always speak to each other. Theory can inform analysis, but it is not always clear how. I show the payoff to theory-driven policy analysis and the difference between it and regular analysis. Note the two key factors that policy analysis should address: your audience will engage emotionally with your analysis, and the feasibility of your solutions depends on the complexity of the policy environment.
  2. Describe how the coursework helps you combine policy theory and policy analysis. Policy analysis is one of four tasks. There is a reflection, to let you ‘show your work’; how your knowledge of policy theory guides your description of a problem and feasible solutions. The essay allows you to expand on theory, to describe how and why policy changes (and therefore what a realistic policy analysis would look like). The blogs encourage new communication skills. In one, you explore how you would expect a policy maker or influencer to sell the recommendations in your policy analysis. In another, you explain complex concepts to a non-academic audience.

Background notes.

I have written this document as if part of a book to be called Teaching Public Policy and co-authored with Dr Emily St Denny.

For that audience, I have two aims: (1) to persuade policy scholars-as-teachers to adopt this kind of coursework in their curriculum; and, (2) to show students how to complete it effectively.

If you prefer shorter advice, see Writing a policy paper and blog post and Writing an essay on politics, policymaking, and policy change.

If you are interested in more background reading, see: The New Policy Sciences (by Paul Cairney and Chris Weible) which describes the need to combine policy theory-driven research with policy analysis; and, Practical Lessons from Policy Theories which describes eight attempts by scholars to translate policy theory into lessons that can be used for policy analysis.

The theories make more sense if you have read the corresponding 1000 Words posts (based on Cairney, 2012). Some of the forthcoming text will look familiar if you read my blog because I am consolidating several individual posts into an overall discussion.

I’m not quite there yet (the chapter is a first draft, a bit scrappy at times, and longer than a chapter should be), so all comments welcome (in the comments bit).

Writing theory driven policy analysis 10.11.17

Cairney 2017 image of the policy process

8 Comments

Filed under public policy