Category Archives: Prevention policy

The coronavirus and evidence-informed policy analysis (short version)

The coronavirus feels like a new policy problem that requires new policy analysis. The analysis should be informed by (a) good evidence, translated into (b) good policy. However, don’t be fooled into thinking that either of those things are straightforward. There are simple-looking steps to go from defining a problem to making a recommendation, but this simplicity masks the profoundly political process that must take place. Each step in analysis involves political choices to prioritise some problems and solutions over others, and therefore prioritise some people’s lives at the expense of others.

The very-long version of this post takes us through those steps in the UK, and situates them in a wider political and policymaking context. This post is shorter, and only scratches the surface of analysis.

5 steps to policy analysis

  1. Define the problem.

Perhaps we can sum it up as: (a) the impact of this virus and illness will be a level of death and illness that could overwhelm the population and exceed the capacity of public services, so (b) we need to contain the virus enough to make sure it spreads in the right way at the right time, so (c) we need to encourage and make people change their behaviour (primarily via hygiene and social distancing). However, there are many ways to frame this problem to emphasise the importance of some populations over others, and some impacts over others.

  1. Identify technically and politically feasible solutions.

Solutions are not really solutions: they are policy instruments that address one aspect of the problem, including taxation and spending, delivering public services, funding research, giving advice to the population, and regulating or encouraging changes to social behaviour. Each new instrument contributes an existing mix, with unpredictable and unintended consequences. Some instruments seem technically feasible (they will work as intended if implemented), but will not be adopted unless politically feasible (enough people support their introduction). Or vice versa. This dual requirement rules out a lot of responses.

  1. Use values and goals to compare solutions.

Typical judgements combine: (a) broad descriptions of values such as efficiency, fairness, freedom, security, and human dignity, (b) instrumental goals, such as sustainable policymaking (can we do it, and for how long?), and political feasibility (will people agree to it, and will it make me more or less popular or trusted?), and (c) the process to make choices, such as the extent to which a policy process involves citizens or stakeholders (alongside experts) in deliberation. They combine to help policymakers come to high profile choices (such as the balance between individual freedom and state coercion), and low profile but profound choices (to influence the level of public service capacity, and level of state intervention, and therefore who and how many people will die).

  1. Predict the outcome of each feasible solution.

It is difficult to envisage a way for the UK Government to publicise all of the thinking behind its choices (Step 3) and predictions (Step 4) in a way that would encourage effective public deliberation. People often call for the UK Government to publicise its expert advice and operational logic, but I am not sure how they would separate it from their normative logic about who should live or die, or provide a frank account without unintended consequences for public trust or anxiety. If so, one aspect of government policy is to keep some choices implicit and avoid a lot of debate on trade-offs. Another is to make choices continuously without knowing what their impact will be (the most likely scenario right now).

  1. Make a choice, or recommendation to your client.

Your recommendation or choice would build on these four steps. Define the problem with one framing at the expense of the others. Romanticise some people and not others. Decide how to support some people, and coerce or punish others. Prioritise the lives of some people in the knowledge that others will suffer or die. Do it despite your lack of expertise and profoundly limited knowledge and information. Learn from experts, but don’t assume that only scientific experts have relevant knowledge (decolonise; coproduce). Recommend choices that, if damaging, could take decades to fix after you’ve gone. Consider if a policymaker is willing and able to act on your advice, and if your proposed action will work as intended. Consider if a government is willing and able to bear the economic and political costs. Protect your client’s popularity, and trust in your client, at the same time as protecting lives. Consider if your advice would change if the problem seemed to change. If you are writing your analysis, maybe keep it down to one sheet of paper (in other words, fewer words than in this post up to this point).

Policy analysis is not as simple as these steps suggest, and further analysis of the wider policymaking environment helps describe two profound limitations to simple analytical thought and action.

  1. Policymakers must ignore almost all evidence

The amount of policy relevant information is infinite, and capacity is finite. So, individuals and governments need ways to filter out almost all of it. Individuals combine cognition and emotion to help them make choices efficiently, and governments have equivalent rules to prioritise only some information. They include: define a problem and a feasible response, seek information that is available, understandable, and actionable, and identify credible sources of information and advice. In that context, the vague idea of trusting or not trusting experts is nonsense, and the larger post highlights the many flawed ways in which all people decide whose expertise counts.

  1. They do not control the policy process.

Policymakers engage in a messy and unpredictable world in which no single ‘centre’ has the power to turn a policy recommendation into an outcome.

  • There are many policymakers and influencers spread across a political system. For example, consider the extent to which each government department, devolved governments, and public and private organisations are making their own choices that help or hinder the UK government approach.
  • Most choices in government are made in ‘subsystems’, with their own rules and networks, over which ministers have limited knowledge and influence.
  • The social and economic context, and events, are largely out of their control.

The take home messages (if you accept this line of thinking)

  1. The coronavirus is an extreme example of a general situation: policymakers will always have very limited knowledge of policy problems and control over their policymaking environment. They make choices to frame problems narrowly enough to seem solvable, rule out most solutions as not feasible, make value judgements to try help some more than others, try to predict the results, and respond when the results do not match their hopes or expectations.
  2. This is not a message of doom and despair. Rather, it encourages us to think about how to influence government, and hold policymakers to account, in a thoughtful and systematic way that does not mislead the public or exacerbate the problem we are seeing. No one is helping their government solve the problem by saying stupid shit on the internet (OK, that last bit was a message of despair).

 

Further reading:

The longer report sets out these arguments in much more detail, with some links to further thoughts and developments.

This series of ‘750 words’ posts summarises key texts in policy analysis and tries to situate policy analysis in a wider political and policymaking context. Note the focus on whose knowledge counts, which is not yet a big feature of this crisis.

These series of 500 words and 1000 words posts (with podcasts) summarise concepts and theories in policy studies.

This page on evidence-based policymaking (EBPM) uses those insights to demonstrate why EBPM is  a political slogan rather than a realistic expectation.

These recorded talks relate those insights to common questions asked by researchers: why do policymakers seem to ignore my evidence, and what can I do about it? I’m happy to record more (such as on the topic you just read about) but not entirely sure who would want to hear what.

3 Comments

Filed under 750 word policy analysis, agenda setting, Evidence Based Policymaking (EBPM), Policy learning and transfer, POLU9UK, Prevention policy, Psychology Based Policy Studies, Public health, public policy, Social change, UK politics and policy

The coronavirus and evidence-informed policy analysis (long version)

This is the long version. It is long. Too long to call a blog post. Let’s call it a ‘living document’ that I will update and amend as new developments arise. In most cases, I am adding tweets, so the date of the update is embedded. If I add a new section, I will add a date. The short version is shorter.

The coronavirus feels like a new policy problem. Governments already have policies for public health crises, but the level of uncertainty about the spread and impact of this virus seems to be taking it to a new level of policy, media, and public attention. The UK Government’s Prime Minister calls it ‘the worst public health crisis for a generation’.

As such, there is no shortage of opinions on what to do, but there is a shortage of well-considered opinions, producing little consensus. Many people are rushing to judgement and expressing remarkably firm opinions about the best solutions, but their contributions add up to contradictory evaluations, in which:

  • the government is doing precisely the right thing or the completely wrong thing,
  • we should listen to this expert saying one thing or another expert saying the opposite.

Lots of otherwise-sensible people are doing what they bemoan in politicians: rushing to judgement, largely accepting or sharing evidence only if it reinforces that judgement, and/or using their interpretation of any new development to settle scores with their opponents.

Yet, anyone who feels, without uncertainty, that they have the best definition of, and solution to, this problem is a fool. If people are also sharing bad information and advice, they are dangerous fools. Further, as Professor Madley puts it (in the video below), ‘anyone who tells you they know what’s going to happen over the next six months is lying’.

In that context, how can we make sense of public policy to address the coronavirus in a more systematic way?

Studies of policy analysis and policymaking do not solve a policy problem, but they at least give us a language to think it through.

  1. Let’s focus on the UK as an example, and use common steps in policy analysis, to help us think through the problem and how to try to manage it.
  • In each step, note how quickly it is possible to be overwhelmed by uncertainty and ambiguity, even when the issue seems so simple at first.
  • Note how difficult it is to move from Step 1, and to separate Step 1 from the others. It is difficult to define the problem without relating it to the solution (or to the ways in which we will evaluate each solution).
  1. Let’s relate that analysis to research on policymaking, to understand the wider context in which people pay attention to, and try to address, important problems that are largely out of their control.

Throughout, note that I am describing a thought process as simply as I can, not a full examination of relevant evidence. I am highlighting the problems that people face when ‘diagnosing’ policy problems, not trying to diagnose it myself. To do so, I draw initially on common advice from the key policy analysis texts (summaries of the texts that policy analysis students are most likely to read) that simplify the process a little too much. Still, the thought process that it encourages took me hours alone (spread over three days) to produce no real conclusion. Policymakers and advisers, in the thick of this problem, do not have that luxury of time or uncertainty.

See also: Boris Johnson’s address to the nation in full (23.3.20) and press conference transcripts

Step 1 Define the problem

 

Common advice in policy analysis texts:

  • Provide a diagnosis of a policy problem, using rhetoric and eye-catching data to generate attention.
  • Identify its severity, urgency, cause, and our ability to solve it. Don’t define the wrong problem, such as by oversimplifying.
  • Problem definition is a political act of framing, as part of a narrative to evaluate the nature, cause, size, and urgency of an issue.
  • Define the nature of a policy problem, and the role of government in solving it, while engaging with many stakeholders.
  • ‘Diagnose the undesirable condition’ and frame it as ‘a market or government failure (or maybe both)’.

Coronavirus as a physical problem is not the same as a coronavirus policy problem. To define the physical problem is to identify the nature, spread, and impact of a virus and illness on individuals and populations. To define a policy problem, we identify the physical problem and relate it (implicitly or explicitly) to what we think a government can, and should, do about it. Put more provocatively, it is only a policy problem if policymakers are willing and able to offer some kind of solution.

This point may seem semantic, but it raises a profound question about the capacity of any government to solve a problem like an epidemic, or for governments to cooperate to solve a pandemic. It is easy for an outsider to exhort a government to ‘do something!’ (or ‘ACT NOW!’) and express certainty about what would happen. However, policymakers inside government:

  1. Do not enjoy the same confidence that they know what is happening, or that their actions will have their intended consequences, and
  2. Will think twice about trying to regulate social behaviour under those circumstances, especially when they
  3. Know that any action or inaction will benefit some and punish others.

For example, can a government make people wash their hands? Or, if it restricts gatherings at large events, can it stop people gathering somewhere else, with worse impact? If it closes a school, can it stop children from going to their grandparents to be looked after until it reopens? There are 101 similar questions and, in each case, I reckon the answer is no. Maybe government action has some of the desired impact; maybe not. If you agree, then the question might be: what would it really take to force people to change their behaviour?

See also: Coronavirus has not suspended politics – it has revealed the nature of power (David Runciman)

The answer is: often too much for a government to consider (in a liberal democracy), particularly if policymakers are informed that it will not have the desired impact.

If so, the UK government’s definition of the policy problem will incorporate this implicit question: what can we do if we can influence, but not determine (or even predict well) how people behave?

Uncertainty about the coronavirus plus uncertainty about policy impact

Now, add that general uncertainty about the impact of government to this specific uncertainty about the likely nature and spread of the coronavirus:

A summary of this video suggests:

  • There will be an epidemic (a profound spread to many people in a short space of time), then the problem will be endemic (a long-term, regular feature of life) (see also UK policy on coronavirus COVID-19 assumes that the virus is here to stay).
  • In the absence of a vaccine, the only way to produce ‘herd immunity’ is for most people to be infected and recover

[Note: there is much debate on whether ‘herd immunity’ is or is not government policy. Much of it relates to interpretation, based on levels of trust/distrust in the UK Government, its Prime Minister, and the Prime Minister’s special adviser. I discuss this point below under ‘trial and error policymaking’. See also Who can you trust during the coronavirus crisis? ]

  • The ideal spread involves all well people sharing the virus first, while all vulnerable people (e.g. older, and/or with existing health problems that affect their immune systems) protected in one isolated space, but it won’t happen like that; so, we are trying to minimise damage in the real world.
  • We mainly track the spread via deaths, with data showing a major spike appearing one month later, so the problem may only seem real to most people when it is too late to change behaviour.
  • A lot of the spread will happen inside homes, where the role of government is minimal (compared to public places). So, for example, the impact of school closures could be good (isolation) or make things worse (children spreading the virus to vulnerable relatives) (see also ‘we don’t know [if the UKG decision not to close schools] was brilliant or catastrophic’). [Update 18.3.20: as it turned out, the First Minister’s argument for closing Scottish schools was that there were too few teachers available).
  • The choice in theory is between a rapid epidemic with a high peak, or a slowed-down epidemic over a longer period, but ‘anyone who tells you they know what’s going to happen over the next six months is lying’.
  • Maybe this epidemic will be so memorable as to shift social behaviour, but so much depends on trying to predict (badly) if individuals will actually change (see also Spiegelhalter on communicating risk).

None of this account tells policymakers what to do, but at least it helps them clarify three key aspects of their policy problem:

  1. The impact of this virus and illness could overwhelm the population, to the extent that it causes mass deaths, causes a level of illness that exceeds the capacity of health services to treat, and contributes to an unpredictable amount of social and economic damage.
  2. We need to contain the virus enough to make sure it (a) spreads at the right speed and/or (b) peaks at the right time. The right speed seems to be: a level that allows most people to recover alone, while the most vulnerable are treated well in healthcare settings that have enough capacity. The right time seems to be the part of the year with the lowest demand on health services (e.g. summer is better than winter). In other words, (a) reduce the size of the peak by ‘flattening the curve’, and/or (b) find the right time of year to address the peak, while (c) anticipating more than one peak.

My impression is that the most frequently-expressed aim is (a) …

… while the UK Government’s Deputy Chief Medical Officer also seems to be describing (b):

  1. We need to encourage or coerce people to change their behaviour, to look after themselves (e.g. by handwashing) and forsake their individual preferences for the sake of public health (e.g. by self-isolating or avoiding vulnerable people). Perhaps we can foster social trust and empathy to encourage responsible individual action. Perhaps people will only protect others if obliged to do so (compare Stone; Ostrom; game theory).

From uncertainty to ambiguity

If you are still with me, I reckon you would have worded those aims slightly differently, right? There is some ambiguity about these broad intentions, partly because there is some uncertainty, and partly because policymakers need to set rather vague intentions to generate the highest possible support for them. However, vagueness is not our friend during a crisis involving such high anxiety. Further, they are only delaying the inevitable choices that people need to make to turn a complex multi-faceted problem into something simple enough to describe and manage. The problem may be complex, but our attention focuses only on a small number of aspects, at the expense of the rest. Examples that have arisen, so far, include to accentuate:

  1. The health of the whole population or people who would be affected disproportionately by the illness.
  • For example, the difference in emphasis affects the health advice for the relatively vulnerable (and the balance between exhortation and reassurance)

  1. Inequalities in relation to health, socio-economic status (e.g. income, gender, race, ethnicity), or the wider economy.
  • For example, restrictive measures may reduce the risk of harm to some, but increase the burden on people with no savings or reliable sources of income.
  • For example, some people are hoarding large quantities of home and medical supplies that (a) other people cannot afford, and (b) some people cannot access, despite having higher need.
  • For example, social distancing will limit the spread of the virus (see the nascent evidence), but also produce highly unequal forms of social isolation that increase the risk of domestic abuse (possibly exacerbated by school closures) and undermine wellbeing. Or, there will be major policy changes, such as to the rules to detain people under mental health legislation, regarding abortion, or in relation to asylum.

  • For example, governments cannot ignore the impact of their actions on the economy, however much they emphasise mortality, health, and wellbeing. Most high-profile emphasis was initially on the fate of large and small businesses, and people with mortgages, but a long period of crisis will a tip the balance from low income to unsustainable poverty (even prompting Iain Duncan Smith to propose policy change), and why favour people who can afford a mortgage over people scraping the money together for rent?

  1. A need for more communication and exhortation, or for direct action to change behaviour.
  2. The short term (do everything possible now) or long term (manage behaviour over many months).

  1. How to maintain trust in the UK government when (a) people are more or less inclined to trust a the current part of government and general trust may be quite low, and (b) so many other governments are acting differently from the UK.

  • For example, note the visible presence of the Prime Minister, but also his unusually high deference to unelected experts such as (a) UK Government senior scientists providing direct advice to ministers and the public, and (b) scientists drawing on limited information to model behaviour and produce realistic scenarios (we can return to the idea of ‘evidence-based policymaking’ later). This approach is not uncommon with epidemics/ pandemics (LD was then the UK Government’s Chief Medical Officer):

  • For example, note how often people are second guessing and criticising the UK Government position (and questioning the motives of Conservative ministers).
  1. How policy in relation to the coronavirus relates to other priorities (e.g. Brexit, Scottish independence, trade, education, culture)

7. Who caused, or who is exacerbating, the problem? The answers to such questions helps determine which populations are most subject to policy intervention.

  • For example, people often try to lay blame for viruses on certain populations, based on their nationality, race, ethnicity, sexuality, or behaviour (e.g. with HIV).
  • For example, the (a) association between the coronavirus and China and Chinese people (e.g. restrict travel to/ from China; e.g. exacerbate racism), initially overshadowed (b) the general role of international travellers (e.g. place more general restrictions on behaviour), and (c) other ways to describe who might be responsible for exacerbating a crisis.

See also: ‘Othering the Virus‘ by Marius Meinhof

Under ‘normal’ policymaking circumstances, we would expect policymakers to resolve this ambiguity by exercising power to set the agenda and make choices that close off debate. Attention rises at first, a choice is made, and attention tends to move on to something else. With the coronavirus, attention to many different aspects of the problem has been lurching remarkably quickly. The definition of the policy problem often seems to be changing daily or hourly, and more quickly than the physical problem. It will also change many more times, particularly when attention to each personal story of illness or death prompts people to question government policy every hour. If the policy problem keeps changing in these ways, how could a government solve it?

Step 2 Identify technically and politically feasible solutions

 

Common advice in policy analysis texts:

  • Identify the relevant and feasible policy solutions that your audience/ client might consider.
  • Explain potential solutions in sufficient detail to predict the costs and benefits of each ‘alternative’ (including current policy).
  • Provide ‘plausible’ predictions about the future effects of current/ alternative policies.
  • Identify many possible solutions, then select the ‘most promising’ for further analysis.
  • Identify how governments have addressed comparable problems, and a previous policy’s impact.

Policy ‘solutions’ are better described as ‘tools’ or ‘instruments’, largely because (a) it is rare to expect them to solve a problem, and (b) governments use many instruments (in different ways, at different times) to make policy, including:

  1. Public expenditure (e.g. to boost spending for emergency care, crisis services, medical equipment)
  2. Economic incentives and disincentives (e.g. to reduce the cost of business or borrowing, or tax unhealthy products)
  3. Linking spending to entitlement or behaviour (e.g. social security benefits conditional on working or seeking work, perhaps with the rules modified during crises)
  4. Formal regulations versus voluntary agreements (e.g. making organisations close, or encouraging them to close)
  5. Public services: universal or targeted, free or with charges, delivered directly or via non-governmental organisations
  6. Legal sanctions (e.g. criminalising reckless behaviour)
  7. Public education or advertising (e.g. as paid adverts or via media and social media)
  8. Funding scientific research, and organisations to advise on policy
  9. Establishing or reforming policymaking units or departments
  10. Behavioural instruments, to ‘nudge’ behaviour (seemingly a big feature in the UK , such as on how to encourage handwashing).

As a result, what we call ‘policy’ is really a complex mix of instruments adopted by one or more governments. A truism in policy studies is that it is difficult to define or identify exactly what policy is because (a) each new instrument adds to a pile of existing measures (with often-unpredictable consequences), and (b) many instruments designed for individual sectors tend, in practice, to intersect in ways that we cannot always anticipate. When you think through any government response to the coronavirus, note how every measure is connected to many others.

Further, it is a truism in public policy that there is a gap between technical and political feasibility: the things that we think will be most likely to work as intended if implemented are often the things that would receive the least support or most opposition. For example:

  1. Redistributing income and wealth to reduce socio-economic inequalities (e.g. to allay fears about the impact of current events on low-income and poverty) seems to be less politically feasible than distributing public services to deal with the consequences of health inequalities.
  2. Providing information and exhortation seems more politically feasible than the direct regulation of behaviour. Indeed, compared to many other countries, the UK Government seems reluctant to introduce ‘quarantine’ style measures to restrict behaviour.

Under ‘normal’ circumstances, governments may be using these distinctions as simple heuristics to help them make modest policy changes while remaining sufficiently popular (or at least looking competent). If so, they are adding or modifying policy instruments during individual ‘windows of opportunity’ for specific action, or perhaps contributing to the sense of incremental change towards an ambitious goal.

Right now, we may be pushing the boundaries of what seems possible, since crises – and the need to address public anxiety – tend to change what seems politically feasible. However, many options that seem politically feasible may not be possible (e.g. to buy a lot of extra medical/ technology capacity quickly), or may not work as intended (e.g. to restrict the movement of people). Think of technical and political feasibility as necessary but insufficient on their own, which is a requirement that rules out a lot of responses.

Step 3 Use value-based criteria and political goals to compare solutions

 

Common advice in policy analysis texts:

  • Typical value judgements relate to efficiency, equity and fairness, the trade-off between individual freedom and collective action, and the extent to which a policy process involves citizens in deliberation.
  • Normative assessments are based on values such as ‘equality, efficiency, security, democracy, enlightenment’ and beliefs about the preferable balance between state, communal, and market/ individual solutions
  • ‘Specify the objectives to be attained in addressing the problem and the criteria  to  evaluate  the  attainment  of  these  objectives  as  well as  the  satisfaction  of  other  key  considerations  (e.g.,  equity,  cost, equity, feasibility)’.
  • ‘Effectiveness, efficiency, fairness, and administrative efficiency’ are common.
  • Identify (a) the values to prioritise, such as ‘efficiency’, ‘equity’, and ‘human dignity’, and (b) ‘instrumental goals’, such as ‘sustainable public finance or political feasibility’, to generate support for solutions.
  • Instrumental questions may include: Will this intervention produce the intended outcomes? Is it easy to get agreement and maintain support? Will it make me popular, or diminish trust in me even further?

Step 3 is the most simple-looking but difficult task. Remember that it is a political, not technical, process. It is also a political process that most people would like to avoid doing (at least publicly) because it involves making explicit the ways in which we prioritise some people over others. Public policy is the choice to help some people and punish or refuse to help others (and includes the choice to do nothing).

Policy analysis texts describe a relatively simple procedure of identifying criteria and producing a table (with a solution in each row, and criteria in each column) to compare the trade-offs between each solution. However, these criteria are notoriously difficult to define, and people resolve that problem by exercising power to decide what each term means, and whose interests should be served when they resolve trade-offs. For example, see Stone on whose needs come first, who benefits from each definition of fairness, and how technical-looking processes such as ‘cost benefit analysis’ mask political choices.

Right now, the most obvious and visible trade-off, accentuated in the UK, is between individual freedom and collective action, or the balance between state, communal, and market/ individual solutions. In comparison with many countries (and China and Italy in particular), the UK Government seems to be favouring individual action over state quarantine measures. However, most trade-offs are difficult to categorise

  1. What should be the balance between efforts to minimise the deaths of some (generally in older populations) and maximise the wellbeing of others? This is partly about human dignity during crisis, how we treat different people fairly, and the balance of freedom and coercion.
  2. How much should a government spend to keep people alive using intensive case or expensive medicines, when the money could be spent improving the lives of far more people? This is partly about human dignity, the relative efficiency of policy measures, and fairness.

If you are like me, you don’t really want to answer such questions (indeed, even writing them looks callous). If so, one way to resolve them is to elect policymakers to make such choices on our behalf (perhaps aided by experts in moral philosophy, or with access to deliberative forums). To endure, this unusually high level of deference to elected ministers requires some kind of reciprocal act:

See also: We must all do everything in our power to protect lives (UK Secretary of State for Health and Social Care)

Still, I doubt that governments are making reportable daily choices with reference to a clear and explicit view of what the trade-offs and priorities should be, because their choices are about who will die, and their ability to predict outcomes is limited.

See also: Media experts despair at Boris Johnson’s coronavirus campaign (Sonia Sodha)

Step 4 Predict the outcome of each feasible solution.

 

Common advice in policy analysis texts:

  • Focus on the outcomes that key actors care about (such as value for money), and quantify and visualise your predictions if possible. Compare the pros and cons of each solution, such as how much of a bad service policymakers will accept to cut costs.
  • ‘Assess the outcomes of the policy options in light of the criteria and weigh trade-offs between the advantages and disadvantages of the options’.
  • Estimate the cost of a new policy, in comparison with current policy, and in relation to factors such as savings to society or benefits to certain populations. Use your criteria and projections to compare each alternative in relation to their likely costs and benefits.
  • Explain potential solutions in sufficient detail to predict the costs and benefits of each ‘alternative’ (including current policy).
  • Short deadlines dictate that you use ‘logic and theory, rather than systematic empirical evidence’ to make predictions efficiently.
  • Monitoring is crucial because it is difficult to predict policy success, and unintended consequences are inevitable. Try to measure the outcomes of your solution, while noting that evaluations are contested.

It is difficult to envisage a way for the UK Government to publicise the thinking behind its choices (Step 3) and predictions (Step 4) in a way that would encourage effective public deliberation, rather than a highly technical debate between a small number of academics:

Further, people often call for the UK Government to publicise its expert advice and operational logic, but I am not sure how they would separate it from their normative logic, or provide a frank account without unintended consequences for public trust or anxiety. If so, government policy involves (a) to keep some choices implicit to avoid a lot of debate on trade-offs, and (b) to make general statements about choices when they do not know what their impact will be.

Step 5 Make a recommendation to your client

 

Common advice in policy analysis texts:

  • Examine your case through the eyes of a policymaker. Keep it simple and concise.
  • Make a preliminary recommendation to inform an iterative process, drawing feedback from clients and stakeholder groups
  • Client-oriented advisors identify the beliefs of policymakers and tailor accordingly.
  • ‘Unless your client asks you not to do so, you should explicitly recommend one policy’

I now invite you to make a recommendation (step 5) based on our discussion so far (steps 1-4). Define the problem with one framing at the expense of the others. Romanticise some people and not others. Decide how to support some people, and coerce or punish others. Prioritise the lives of some people in the knowledge that others will suffer or die. Do it despite your lack of expertise and profoundly limited knowledge and information. Learn from experts, but don’t assume that only scientific experts have relevant knowledge (decolonise; coproduce). Recommend choices that, if damaging, could take decades to fix after you’ve gone. Consider if a policymaker is willing and able to act on your advice, and if your proposed action will work as intended. Consider if a government is willing and able to bear the economic and political costs. Protect your client’s popularity, and trust in your client, at the same time as protecting lives. Consider if your advice would change if the problem would seem to change. If you are writing your analysis, maybe keep it down to one sheet of paper (and certainly far fewer words than in this post). Better you than me.

Please now watch this video before I suggest that things are not so simple.

Would that policy analysis were so simple

Imagine writing policy analysis in an imaginary world, in which there is a single powerful ‘rational’ policymaker at the heart of government, making policy via an orderly series of stages.

cycle and cycle spirograph 18.2.20

Your audience would be easy to identify at each stage, your analysis would be relatively simple, and you would not need to worry about what happens after you make a recommendation for policy change (since the selection of a solution would lead to implementation).  You could adopt a simple 5 step policy analysis method, use widely-used tools such as cost-benefit analysis to compare solutions, and know where the results would feed into the policy process.

Studies of policy analysts describe how unrealistic this expectation tends to be (Radin, Brans, Thissen).

Table for coronavirus 750

For example, there are many policymakers, analysts, influencers, and experts spread across political systems, and engaging with 101 policy problems simultaneously, which suggests that it is not even clear how everyone fits together and interacts in what we call (for the sake of simplicity) ‘the policy process’.

Instead, we can describe real world policymaking with reference to two factors.

The wider policymaking environment: 1. Limiting the use of evidence

First, policymakers face ‘bounded rationality’, in which they only have the ability to pay attention to a tiny proportion of available facts, are unable to separate those facts from their values (since we use our beliefs to evaluate the meaning of facts), struggle to make clear and consistent choices, and do not know what impact they will have. The consequences can include:

  • Limited attention, and lurches of attention. Policymakers can only pay attention to a tiny proportion of their responsibilities, and policymaking organizations struggle to process all policy-relevant information. They prioritize some issues and information and ignore the rest.
  • Power and ideas. Some ways of understanding and describing the world dominate policy debate, helping some actors and marginalizing others.
  • Beliefs and coalitions. Policymakers see the world through the lens of their beliefs. They engage in politics to turn their beliefs into policy, form coalitions with people who share them, and compete with coalitions who don’t.
  • Dealing with complexity. They engage in ‘trial-and-error strategies’ to deal with uncertain and dynamic environments (see the new section on trial-and-error- at the end).
  • Framing and narratives. Policy audiences are vulnerable to manipulation when they rely on other actors to help them understand the world. People tell simple stories to persuade their audience to see a policy problem and its solution in a particular way.
  • The social construction of populations. Policymakers draw on quick emotional judgements, and social stereotypes, to propose benefits to some target populations and punishments for others.
  • Rules and norms. Institutions are the formal rules and informal understandings that represent a way to narrow information searches efficiently to make choices quickly.
  • Learning. Policy learning is a political process in which actors engage selectively with information, not a rational search for truth.

Evidence-based or expert-informed policymaking

Put simply, policymakers cannot oversee a simple process of ‘evidence-based policymaking’. Rather, to all intents and purposes:

  1. They need to find ways to ignore most evidence so that they can focus disproportionately on some. Otherwise, they will be unable to focus well enough to make choices. The cognitive and organisational shortcuts, described above, help them do it almost instantly.
  2. They also use their experience to help them decide – often very quickly – what evidence is policy-relevant under the circumstances. Relevance can include:
  • How it relates to the policy problem as they define it (Step 1).
  • If it relates to a feasible solution (Step 2).
  • If it is timely, available, understandable, and actionable.
  • If it seems credible, such as from groups representing wider populations, or from people they trust.
  1. They use a specific shortcut: relying on expertise.

However, the vague idea of trusting or not trusting experts is a nonsense, largely because it is virtually impossible to set a clear boundary between relevant/irrelevant experts and find a huge consensus on (exactly) what is happening and what to do. Instead, in political systems, we define the policy problem or find other ways to identify the most relevant expertise and exclude other sources of knowledge.

In the UK Government’s case, it appears to be relying primarily on expertise from its own general scientific advisers, medical and public health advisers, and – perhaps more controversially – advisers on behavioural public policy.

box 7.1

Right now, it is difficult to tell exactly how and why it relies on each expert (at least when the expert is not in a clearly defined role, in which case it would be irresponsible not to consider their advice). Further, there are regular calls on Twitter for ministers to be more open about their decisions.

However, don’t underestimate the problems of identifying why we make choices, then justifying one expert or another (while avoiding pointless arguments), or prioritising one form of advice over another. Look, for example, at the kind of short-cuts that intelligent people use, which seem sensible enough, but would receive much more intense scrutiny if presented in this way by governments:

  • Sophisticated speculation by experts in a particular field, shared widely (look at the RTs), but questioned by other experts in another field:

  • Experts in one field trusting certain experts in another field based on personal or professional interaction:

  • Experts in one field not trusting a government’s approach based on its use of one (of many) sources of advice:

  • Experts representing a community of experts, criticising another expert, for misrepresenting the amount of expert scepticism of government experts (yes, I am trying to confuse you):

  • Expert debate on how well policymakers are making policy based on expert advice

  • Finding quite-sensible ways to trust certain experts over others, such as because they can be held to account in some way (and may be relatively worried about saying any old shit on the internet):

There are many more examples in which the shortcut to expertise is fine, but not particularly better than another shortcut (and likely to include a disproportionately high number of white men with STEM backgrounds).

Update: of course, they are better than the volume trumps expertise approach:

Further, in each case, we may be receiving this expert advice via many other people, and by the time it gets to us the meaning is lost or reversed (or there is some really sophisticated expert analysis of something rumoured – not demonstrated – to be true):

For what it’s worth, I tend to favour experts who:

(a) establish the boundaries of their knowledge, (b) admit to high uncertainty about the overall problem:

(c) (in this case) make it clear that they are working on scenarios, not simple prediction

(d) examine critically the too-simple ideas that float around, such as the idea that the UK Government should emulate ‘what works’ somewhere else

(e) situate their own position (in Prof Sridhar’s case, for mass testing) within a broader debate

However, note that most of these experts are from a very narrow social background, and from very narrow scientific fields (first in modelling, then likely in testing), despite the policy problem being largely about (a) who, and how many people, a government should try to save, and (b) how far a government should go to change behaviour to do it. It is understandable to defer in this way during a crisis, but it also contributes to a form of ‘depoliticisation’ that masks profound choices that benefit some people and leave others vulnerable to harm.

See also: COVID-19: a living systematic map of the evidence

See also: To what extent does evidence support decision making during infectious disease outbreaks? A scoping literature review

See also: Rights and responsibilities in the Coronavirus pandemic

The wider policymaking environment: 2. Limited control

Second, policymakers engage in a messy and unpredictable world in which no single ‘centre’ has the power to turn a policy recommendation into an outcome. I normally use the following figure to think through the nature of a complex and unwieldy policymaking environment of which no ‘centre’ of government has full knowledge or control.

image policy process round 2 25.10.18

It helps us identify (further) the ways in which we can reject the idea that the UK Prime Minister and colleagues can fully understand and solve policy problems:

Actors. The environment contains many policymakers and influencers spread across many levels and types of government (‘venues’).

For example, consider how many key decisions that (a) have been made by organisations not in the UK central government, and (b) are more or less consistent with its advice, including:

  • Devolved governments announcing their own healthcare and public health responses.
  • Public sector employers initiating or encouraging at-home working (and many Universities moving quickly from in-person to online teaching)
  • Private organisations cancelling cultural and sporting events.

Context and events. Policy solutions relate to socioeconomic context and events which can be impossible to ignore and out of the control of policymakers. The coronavirus, and its impact on so many aspects on population health and wellbeing, is an extreme example of this problem.

Networks, Institutions, and Ideas. Policymakers and influencers operate in subsystems (specialist parts of political systems). They form networks or coalitions built on the exchange of resources or facilitated by trust underpinned by shared beliefs or previous cooperation. Many different parts of government have practices driven by their own formal and informal rules. Formal rules are often written down or known widely. Informal rules are the unwritten rules, norms and practices that are difficult to understand, and may not even be understood in the same way by participants. Political actors relate their analysis to shared understandings of the world – how it is, and how it should be – which are often so established as to be taken for granted. These dominant frames of reference establish the boundaries of the political feasibility of policy solutions.  These kinds of insights suggest that most policy decisions are considered, made, and delivered in the name of – but not in the full knowledge of – government ministers.

Trial and error policymaking in complex policymaking systems (17.3.20)

There are many ways to conceptualise this policymaking environment, but few theories provide specific advice on what to do, or how to engage effectively in it. One notable exception is the general advice that comes from complexity theory, including:

  • Law-like behaviour is difficult to identify – so a policy that was successful in one context may not have the same effect in another.
  • Policymaking systems are difficult to control; policy makers should not be surprised when their policy interventions do not have the desired effect.
  • Policy makers in the UK have been too driven by the idea of order, maintaining rigid hierarchies and producing top-down, centrally driven policy strategies.  An attachment to performance indicators, to monitor and control local actors, may simply result in policy failure and demoralised policymakers.
  • Policymaking systems or their environments change quickly. Therefore, organisations must adapt quickly and not rely on a single policy strategy.

On this basis, there is a tendency in the literature to encourage the delegation of decision-making to local actors:

  1. Rely less on central government driven targets, in favour of giving local organisations more freedom to learn from their experience and adapt to their rapidly-changing environment.
  2. To deal with uncertainty and change, encourage trial-and-error projects, or pilots, that can provide lessons, or be adopted or rejected, relatively quickly.
  3. Encourage better ways to deal with alleged failure by treating ‘errors’ as sources of learning (rather than a means to punish organisations) or setting more realistic parameters for success/ failure (although see this example and this comment).
  4. Encourage a greater understanding, within the public sector, of the implications of complex systems and terms such as ‘emergence’ or ‘feedback loops’.

In other words, this literature, when applied to policymaking, tends to encourage a movement from centrally driven targets and performance indicators towards a more flexible understanding of rules and targets by local actors who are more able to understand and adapt to rapidly-changing local circumstances.

[See also: Complex systems and systems thinking]

Now, just imagine the UK Government taking that advice right now. I think it is fair to say that it would be condemned continuously (even more so than right now). Maybe that is because it is the wrong way to make policy in times of crisis. Maybe it is because too few people are willing and able to accept that the role of a small group of people at the centre of government is necessarily limited, and that effective policymaking requires trial-and-error rather than a single, fixed, grand strategy to be communicated to the public. The former highlights policy that changes with new information and perspective. The latter highlights errors of judgement, incompetence, and U-turns. In either case, the advice is changing as estimates of the coronavirus’ impact change:

I think this tension, in the way that we understand UK government, helps explain some of the criticism that it faces when changing its advice to reflect changes in its data or advice. This criticism becomes intense when people also question the competence or motives of ministers (and even people reporting the news) more generally, leading to criticism that ranges from mild to outrageous:

For me, this casual reference to a government policy to ‘cull the heard of the weak’ is outrageous, but you can find much worse on Twitter. It reflects wider debate on whether ‘herd immunity’ is or is not government policy. Much of it relates to interpretation of government statements, based on levels of trust/distrust in the UK Government, its Prime Minister and Secretaries of State, and the Prime Minister’s special adviser

However, I think that some of it is also about:

1. Wilful misinterpretation (particularly on Twitter). For example, in the early development and communication of policy, Boris Johnson was accused (in an irresponsibly misleading way) of advocating for herd immunity rather than restrictive measures.

See: Here is the transcript of what Boris Johnson said on This Morning about the new coronavirus (Full Fact)

full fact coronavirus

Below is one of the most misleading videos of its type. Look at how it cuts each segment into a narrative not provided by ministers or their advisors (see also this stinker):

2. The accentuation of a message not being emphasised by government spokespeople.

See for example this interview, described by Sky News as: The government’s chief scientific adviser Sir Patrick Vallance has told Sky News that about 60% of people will need to become infected with coronavirus in order for the UK to enjoy “herd immunity”. You might be forgiven for thinking that he was on Sky extolling the virtues of a strategy to that end (and expressing sincere concerns on that basis). Yet, he was saying nothing of the sort. Rather, when prompted, he discussed herd immunity in relation to the belief that COVID-19 will endure long enough to become as common as seasonal flu.

 

See also British government wants UK to acquire coronavirus ‘herd immunity’, writes Robert Peston and live debates on whether or not ‘herd immunity’ was the goal of the UK government:

A more careful forensic analysis will try to relate each government choice to the ways in which key advisory bodies (such as the New and Emerging Respiratory Virus Threats Advisory Group, NERVTAG) received and described evidence on the current nature of the problem:

In some cases, maybe people are making the argument that trial-and-error is the best way to respond quickly, and adapt quickly, in a crisis but that the UK Government version is not what, say, the WHO thinks of as good kind of adaptive response. It is not possible to tell, at least from the general ways in which they justify acting quickly.

See also the BBC’s provocative question (which I expect to be replaced soon):

The take home messages

  1. The coronavirus is an extreme example of a general situation: policymakers will always have very limited knowledge of policy problems and control over their policymaking environment. They make choices to frame problems narrowly enough to seem solvable, rule out most solutions as not feasible, make value judgements to try help some more than others, try to predict the results, and respond when the results to not match their hopes or expectations.
  2. This is not a message of doom and despair. Rather, it encourages us to think about how to influence government, and hold policymakers to account, in a thoughtful and systematic way that does not mislead the public or exacerbate the problem we are seeing.

Further reading, until I can think of a better conclusion:

This series of ‘750 words’ posts summarises key texts in policy analysis and tries to situate policy analysis in a wider political and policymaking context. Note the focus on whose knowledge counts, which is not yet a big feature of this crisis.

These series of 500 words and 1000 words posts (with podcasts) summarise concepts and theories in policy studies.

This page on evidence-based policymaking (EBPM) uses those insights to demonstrate why EBPM is  a political slogan rather than a realistic expectation.

These recorded talks relate those insights to common questions asked by researchers: why do policymakers seem to ignore my evidence, and what can I do about it? I’m happy to record more (such as on the topic you just read about) but not entirely sure who would want to hear what.

See also: Advisers, Governments and why blunders happen? (Colin Talbot)

See also: Pandemic Science and Politics (Daniel Sarewitz)

 

 

4 Comments

Filed under 750 word policy analysis, agenda setting, Evidence Based Policymaking (EBPM), Policy learning and transfer, POLU9UK, Prevention policy, Psychology Based Policy Studies, Public health, public policy, Social change, UK politics and policy

Policy Analysis in 750 Words: complex systems and systems thinking

This post forms one part of the Policy Analysis in 750 words series overview and connects to previous posts on complexity. The first 750 words tick along nicely, then there is a picture of a cat hanging in there baby to signal where it can all go wrong.

There are a million-and-one ways to describe systems and systems thinking. These terms are incredibly useful, but also at risk of meaning everything and therefore nothing (compare with planning and consultation).

Let’s explore how the distinction between policy studies and policy analysis can help us clarify the meaning of ‘complex systems’ and ‘systems thinking’ in policymaking.

For example, how might we close a potentially large gap between these two stories?

  1. Systems thinking in policy analysis.
  • Avoid the unintended consequences of too-narrow definitions of problems and processes (systems thinking, not simplistic thinking).
  • If we engage in systems thinking effectively, we can understand systems well enough to control, manage, or influence them.
  1. The study of complex policymaking systems.
  • Policy emerges from complex systems in the absence of: (a) central government control and often (b) policymaker awareness.
  • We need to acknowledge these limitations properly, to accept our limitations, and avoid the mechanistic language of ‘policy levers’ which exaggerate human or government control.

Six meanings of complex systems in policy and policymaking

Let’s begin by trying to clarify many meanings of complex system and relate them to systems thinking storylines.

For example, you will encounter three different meanings of complex system in this series alone, and each meaning presents different implications for systems thinking:

  1. A complex policymaking system

Policy outcomes seem to ‘emerge’ from policymaking systems in the absence of central government control. As such, we should rely less on central government driven targets (in favour of local discretion to adapt to environments), encourage trial-and-error learning, and rethink the ways in which we think about government ‘failure’ (see, for example, Hallsworth on ‘system stewardship’, the OECD on ‘Systemic Thinking for Policy Making‘, and this thread)

  • Systems thinking is about learning and adapting to the limits to policymaker control.

  1. Complex policy problems

Dunn (2017:  73) describes the interdependent nature of problems:

Subjectively experienced problems – crime, poverty, unemployment, inflation, energy, pollution, health, security – cannot be decomposed into independent subsets without running the risk of producing an approximately right solution to the wrong problem. A key characteristic of systems of problems is that the whole is greater – that is, qualitatively different – than the simple sum of its parts” (contrast with Meltzer and Schwartz on creating a ‘boundary’ to make problems seem solveable).

  • Systems thinking is about addressing policy problems holistically.
  1. Complex policy mixes

What we call ‘policy’ is actually a collection of policy instruments. Their overall effect is ‘non-linear’, difficult to predict, and subject to emergent outcomes, rather than cumulative (compare with Lindblom’s hopes for incrementalist change).

This point is crucial to policy analysis: does it involve a rethink of all instruments, or merely add a new instrument to the pile?

  • Systems thinking is about anticipating the disproportionate effect of a new policy instrument.

These three meanings are joined by at least three more (from Munro and Cairney on energy systems):

  1. Socio-technical systems (Geels)

Used to explain the transition from unsustainable to sustainable energy systems.

  • Systems thinking is about identifying the role of new technologies, protected initially in a ‘niche’, and fostered by a supportive ‘social and political environment’.
  1. Socio-ecological systems (Ostrom)

Used to explain how and why policy actors might cooperate to manage finite resources.

  • Systems thinking is about identifying the conditions under which actors develop layers of rules to foster trust and cooperation.
  1. The metaphor of systems

Used by governments – rather loosely – to indicate an awareness of the interconnectedness of things.

  • Systems thinking is about projecting the sense that (a) policy and policymaking is complicated, but (b) governments can still look like they are in control.

Four more meanings of systems thinking

Now, let’s compare these storylines with a small sample of wider conceptions of systems thinking:

  1. The old way of establishing order from chaos

Based on the (now-diminished) faith in science and rational management techniques to control the natural world for human benefit (compare Hughes and Hughes on energy with Checkland on ‘hard’ v ‘soft’ systems approaches, then see What you need as an analyst versus policymaking reality and Radin on the old faith in rationalist governing systems).

  • Systems thinking was about the human ability to turn potential chaos into well-managed systems (such as ‘large technical systems’ to distribute energy)
  1. The new way of accepting complexity but seeking to make an impact

Based on the idea that we can identify ‘leverage points’, or the places that help us ‘intervene in a system’ (see Meadows then compare with Arnold and Wade).

  • Systems thinking is about the human ability to use a small shift in a system to produce profound changes in that system.
  1. A way to rethink cause-and-effect

Based on the idea that current research methods are too narrowly focused on linearity rather than the emergent properties of systems of behaviour (for example, Rutter et al on how to analyse the cumulative effect of public health interventions).

  • Systems thinking is about rethinking the ways in which governments, funders, or professions conduct policy-relevant research on social behaviour.
  1. A way of thinking about ourselves

Embrace the limits to human cognition, and accept that all understandings of complex systems are limited.

  • Systems thinking is about developing the ‘wisdom’ and ‘humility’ to accept our limited knowledge of the world.

hang-in-there-baby

 

How can we clarify systems thinking and use it effectively in policy analysis?

Now, imagine you are in a room of self-styled systems thinkers, and that no-one has yet suggested a brief conversation to establish what you all mean by systems thinking. I reckon you can make a quick visual distinction by seeing who looks optimistic.

I’ll be the morose-looking guy sitting in the corner, waiting to complain about ambiguity, so you would probably be better off sitting next to Luke Craven who still ‘believes in the power of systems thinking’.

If you can imagine some amalgam of these pessimistic/ optimistic positions, perhaps the conversation would go like this:

  1. Reasons to expect some useful collaboration.

Some of these 10 discussions seem to complement each other. For example:

  • We can use 3 and 9 to reject one narrow idea of ‘evidence-based policymaking’, in which the focus is on (a) using experimental methods to establish cause and effect in relation to one policy instrument, without showing (b) the overall impact on policy and outcomes (e.g. compare FNP with more general ‘families’ policy).
  • 1-3 and 10 might be about the need for policy analysts to show humility when seeking to understand and influence complex policy problems, solutions, and policymaking systems.

In other words, you could define systems thinking in relation to the need to rethink the ways in which we understand – and try to address – policy problems. If so, you can stop here and move on to the next post. There is no benefit to completing this post.

  1. Reasons to expect the same old frustrating discussions based on no-one defining terms well enough (collectively) to collaborate effectively (beyond using the same buzzwords).

Although all of these approaches use the language of complex systems and systems thinking, note some profound differences:

Holding on versus letting go.

  • Some are about intervening to take control of systems or, at least, make a disproportionate difference from a small change.
  • Some are about accepting our inability to understand, far less manage, these systems.

Talking about different systems.

  • Some are about managing policymaking systems, and others about social systems (or systems of policy problems), without making a clear connection between both endeavours.

For example, if you use approach 9 to rethink societal cause-and-effect, are you then going to pretend that you can use approach 7 to do something about it? Or, will our group have a difficult discussion about the greater likelihood of 6 (metaphorical policymaking) in the context of 1 (the inability of governments to control the policymaking systems we need to solve the problems raised by 9).

In that context, the reason that I am sitting in the corner, looking so morose, is that too much collective effort goes into (a) restating, over and over and over again, the potential benefits of systems thinking, leaving almost no time for (b) clarifying systems thinking well enough to move on to these profound differences in thinking. Systems thinking has not even helped us solve these problems with systems thinking.

See also:

Why systems thinkers and data scientists should work together to solve social challenges

4 Comments

Filed under 750 word policy analysis, Evidence Based Policymaking (EBPM), Prevention policy, public policy, UKERC

Prevenir es mejor que curar, entonces, ¿por qué no hacemos más?

Serie: El proceso de las políticas públicas.

Paul Cairney, Profesor de Política y Políticas Públicas en la Universidad de Stirling, Escocia. Enlace a texto original en inglés.

Esta publicación proporciona una amplia cantidad de antecedentes de mi plática en la Escuela de Gobierno de Australia y Nueva Zelanda (ANZSOG, por sus siglas en inglés), la cual se titula “Prevenir es mejor que curar, entonces, ¿por qué no hacemos más?” [en inglés] Si lo lees todo, es una lectura larga. Si no, es una lectura corta antes de la lectura larga. Aquí está la descripción de la plática:

“¿Te suena familiar? Comienza una nueva administración en el gobierno, la cual promete cambiar el equilibrio en las políticas sociales y de salud, – de costosos remedios y atención de alta dependencia o complejidad, a prevención e intervención temprana-. Se comprometen a una mejor formulación de políticas públicas; y dicen que la entrega de políticas y programas se hará de forma coordinada, delegando responsabilidades a nivel local y enfocándose en resultados a largo plazo en lugar de soluciones a corto plazo; y que garantizarán que la política se base en evidencia. Y luego todo se vuelve demasiado difícil y el ciclo comienza nuevamente, dejando a su paso algunos especialistas exhaustos y desilusionados. ¿Por qué sucede esto repetidamente, en diferentes países y con gobiernos de diferentes doctrinas, incluso con la mejor voluntad del mundo?

  • De acuerdo con la pregunta verás que no estoy sugiriendo que todas las políticas públicas de prevención o intervención temprana fallen. Por el contrario, utilizo teorías de políticas públicas para proporcionar una explicación general de la brecha significativa entre las expectativas (realistas) expresadas en las estrategias de prevención y los resultados reales. Luego se puede discutir sobre cómo disminuir esa brecha.
  • También verás la frase “incluso con la mejor voluntad del mundo”, que considero clave para esta plática. Nadie necesita que yo ensaye las formas comunes y generalmente vagas de explicar las políticas de prevención fallidas, incluida la “intratabilidad” [en inglés] de los problemas de políticas públicas o la “patología” [en inglés] de las mismas. Más bien, demuestro que tales políticas públicas pueden “fracasar” incluso cuando existe un acuerdo franco y amplio entre las partes sobre la necesidad de pasar del diseño de políticas reactivas a más preventivas. También sugiero que la explicación general del fracaso (baja “voluntad política”) a menudo es perjudicial para posibilidades de éxito en el futuro.
  • Comencemos por definir la política pública de prevención y la formulación de políticas públicas.

Cuando los gobiernos se involucran en la “prevención”, buscan:

  1. Reformar las políticas públicas

La política pública de prevención es realmente un conjunto de políticas diseñadas para intervenir lo antes posible en la vida de las personas para mejorar su bienestar y reducir las desigualdades o la demanda de servicios agudos. El objetivo es pasar de los servicios públicos reactivos a los preventivos, interviniendo de manera temprana en la vida de las personas para abordar una amplia gama de problemas de largo alcance, incluidos el crimen y el comportamiento antisocial, la mala salud y los comportamientos no saludables, el bajo nivel educativo, el desempleo y la baja empleabilidad, antes de que se vuelvan demasiado severos.

  1. Reformar la formulación de la política pública

La formulación de políticas públicas preventivas describe las formas en que los gobiernos reforman sus prácticas para apoyar las políticas de prevención, incluido el compromiso de:

  • “Unir” a departamentos y servicios gubernamentales para resolver “problemas intratables” que trascienden áreas.
  • Producir objetivos a largo plazo para obtener mayores resultados a través de otorgar mayor responsabilidad en el diseño del servicio a los organismos públicos locales, las partes interesadas, las “comunidades” y los usuarios del servicio
  • Reducir los objetivos a corto plazo en favor de resultados a largo plazo.
  1. Asegurar que la política pública es “basada en evidencia”

Tres razones generales por las cuales las políticas públicas de “prevención” nunca parecen tener éxito.

  1. Los formuladores de política pública no saben el significado de la prevención

Expresan un compromiso con la prevención antes de definirla completamente. Cuando comienzan a dar sentido a la prevención, descubren lo difícil que es perseguirla y las elecciones controvertidas que esto implica (ver también incertidumbre versus ambigüedad)

  1. Se involucran en un sistema de formulación de políticas públicas que es demasiado complejo para controlarse

Intentan compartir la responsabilidad entre varios actores y coordinan acciones para direccionar los resultados de las políticas públicas. Sin embargo, no poseen la capacidad de diseñar dichas relaciones y controlar los resultados de las políticas públicas.

Sin embargo, también deben demostrarle al electorado que tienen el control y descubrir lo difícil que es localizar y centralizar las políticas públicas.

  1. No pueden y no quieren producir la “formulación de la política pública basada en evidencia”

Los formuladores buscan atajos cognitivos (y sus equivalentes organizacionales) para recopilar suficiente información para tomar decisiones “suficientemente buenas”. Cuando buscan evidencia sobre la prevención, descubren que es irregular, poco concluyente y a menudo contraria a sus creencias, y no una “bala mágica” para ayudar a justificar las elecciones.

A lo largo de este proceso, su compromiso con la política pública de prevención puede ser sincero, pero no se materializa. No articulan completamente lo que significa prevención ni aprecian la dimensión de dicha tarea. Cuando intentan ofrecer estrategias de prevención, se enfrentan a varios problemas que por sí solos parecerían desalentadores. Muchos de los problemas que tratan de “prevenir” son “intratables” o difíciles de definir y aparentemente imposibles de resolver, como la pobreza, el desempleo, las viviendas de baja calidad y la falta de ellas, el crimen y las desigualdades en salud y educación. Se enfrentan a elecciones difíciles sobre cuán lejos deberían llegar para cambiar el equilibrio entre el Estado y el mercado, redistribuir la riqueza y los ingresos, distribuir recursos públicos e intervenir en la vida de las personas para cambiar su comportamiento y sus formas de pensar. Su enfoque en el largo plazo se enfrenta a una gran competencia por problemas de políticas públicas cortoplacistas más destacados que los impulsan a mantener servicios públicos “reactivos”. Su deseo puro de “localizar” la formulación de políticas, a menudo cede el paso a la política electoral nacional, en la que los gobiernos centrales se enfrentan a la presión para formular políticas públicas desde “arriba” y ser decisivos. Su búsqueda de políticas “basadas en evidencia” a menudo revela una falta de evidencia sobre qué intervenciones políticas funcionan y la medida en que se pueden “expandir” con éxito.

Un mal diagnostico por parte de los encargados de la formulación de la política pública y actores influyentes hará que los problemas no se resuelvan

  • Si los actores con poder en las políticas públicas hacen la suposición simplista de que un problema es causado por cuestiones que no son vitales para el Estado, darán malos consejos.
  • Si los nuevos formuladores realmente piensan que el problema fue la falta de compromiso y la competencia de sus predecesores, comenzarán con las mismas esperanzas sobre el impacto que pueden tener, solo para desencantarse cuando vean la diferencia entre sus objetivos abstractos y los resultados del mundo real.
  • La mala explicación del éxito limitado contribuye en gran medida a observar (a) un período inicial de entusiasmo y actividad, reemplazado por (b) desencanto e inactividad, y (c) la repetición de este ciclo.

Agreguemos más detalles a estas explicaciones generales:

  1. ¿Qué hace que la prevención sea tan difícil de definir?

Cuando se ve como un eslogan simple, “prevención” parece un objetivo intuitivamente atractivo. Puede generar un consenso entre los partidos políticos, reuniendo grupos de la “izquierda”, buscando reducir las desigualdades, y de la “derecha”, buscando reducir la inactividad económica y el costo de servicios.

Tal consenso es superficial e ilusorio. Al hacer una estrategia detallada, la prevención está abierta a muchas interpretaciones por parte de muchos formuladores de políticas públicas. Imagina los muchos tipos de políticas de prevención y formulación de políticas que podríamos producir:

 

     1. ¿Qué problema tratamos de resolver?

La formulación de políticas públicas de prevención representa una solución heroica a varias crisis: grandes desigualdades, servicios públicos con recursos insuficientes y un gobierno disfuncional.

 

     2. ¿En qué medidas debemos centrarnos?

¿En qué desigualdades debemos concentrarnos principalmente? Riqueza, ocupación y empleo, ingresos, raza, etnia, género, sexualidad, discapacidad, salud mental.

¿De acuerdo a cuál medida de desigualdad? Económica, salud, comportamiento saludable, educación, bienestar, castigo.

     3. ¿En qué solución deberíamos centrarnos?

Para reducir la pobreza y las desigualdades socioeconómicas, mejorar la calidad de vida nacional, reducir los costos de los servicios públicos o aumentar la relación precio-calidad.

     4. ¿Qué “herramientas” o instrumentos de política debemos utilizar?

¿Políticas redistributivas para abordar las causas “estructurales” de pobreza y desigualdad?

O tal vez políticas centradas en el individuo para: (a) aumentar la “resistencia” mental de los usuarios de servicios públicos, (b) obligar o (c) exhortar a las personas a cambiar su comportamiento. 

     5. ¿Cómo se interviene lo antes posible en la vida de las personas?

Prevención primaria. Concentrándose en toda la población para evitar que ocurra un problema invirtiendo de forma temprana o modificando el entorno social o físico. Similar a la vacunación del total de la población.

Prevención secundaria. Enfocándose en los grupos en riesgo para identificar un problema en una etapa temprana con el objetivo de minimizar el daño.

Prevención terciaria. Concentrándose en los grupos afectados para evitar que un problema empeore.

 

     6. ¿Cómo se alcanza la “formulación de políticas públicas basada en evidencia”? 3 modelos ideales (en preparación).

¿Usando ensayos controlados aleatorios y revisión sistemática para identificar las mejores intervenciones?

¿Narrativas para compartir las mejores prácticas de gobernanza?

¿Métodos de “mejora” para experimentar a menor escala y compartir las mejores prácticas?

 

     7. ¿Cómo se relaciona la recopilación de evidencia con la formulación de políticas públicas a largo plazo? 

¿Una estrategia nacional impulsa resultados a largo plazo?

¿El gobierno central produce acuerdos u objetivos para las autoridades locales?

 

  1. ¿La formulación de políticas públicas preventivas es una filosofía o un profundo proceso de reforma?

¿Qué tan serios son los gobiernos nacionales (sobre el localismo, los servicios públicos impulsados por los usuarios del servicio y la formulación de políticas integrales u holísticas), cuando los responsables del resultado son políticos electos?

 

  1. ¿Cuál es la naturaleza de la intervención del Estado?

Puede ser punitivo o de apoyo. Ver: ¿Cómo harían Lisa Simpson y Monty Burns una política social progresista? [en inglés]

 

  1. Tomar “decisiones difíciles”: ¿Qué problemas surgen cuando la política se enfrenta a la formulación de políticas públicas?

 

Cuando los formuladores de políticas se mueven desde un amplia filosofía y lenguaje hacia políticas y prácticas específicas, encuentran una serie de obstáculos, que incluyen:

La escala de la tarea se vuelve abrumadora y no se adapta a los ciclos electorales.

Desarrollar políticas públicas y reformar su formulación lleva tiempo, su efecto puede tardar una generación en verse.

 

Existe competencia por los recursos para la formulación de las políticas públicas, tales como la atención y el dinero.

La prevención es general, a largo plazo y de poca importancia. Compite contra los principales problemas a corto plazo que los políticos se sienten obligados a resolver primero.

La prevención es similar a la inversión de capital sin garantía de retorno sobre la inversión. Las reducciones en los fondos de “lucha contra incendios”, “servicios de primera línea” para solventar las iniciativas de prevención, son difíciles de vender. Los gobiernos invierten en pequeñas acciones, y la inversión es vulnerable cuando se necesita dinero rápidamente para financiar crisis en el servicio público.

 

Los beneficios son difíciles de ver y medir.

Los impactos a corto plazo son difíciles de medir, los impactos a largo plazo son difíciles de atribuir a una sola intervención, y la prevención no necesariamente implica ahorrar dinero (ni proporciona ahorros “canjeables”).

Las políticas reactivas tienen un impacto más visible, como reducir los tiempos de espera en el hospital o aumentar el número de maestros u oficiales de policía.

 

Los problemas son “intratables”.

Llegar a la “causa raíz” de los problemas no es sencillo; los formuladores de políticas públicas a menudo no tienen certeza de la causa de los problemas o el efecto de sus soluciones. Pocos aspectos de la prevención en la política social se asemejan a la prevención de enfermedades, en la que se conocen las causas de muchas enfermedades, así como sus formas de detección y prevención.

 

La gestión del desempeño no conduce a la prevención.

Los sistemas de gestión del desempeño alientan a los administradores del sector público a considerar servicios cuyos objetivos sean medibles a corto plazo, sobre aquellos compartidos con socios de prestación de servicios públicos o referentes al bienestar de sus pobladores.

La gestión del desempeño consiste en establecer prioridades cuando los gobiernos tienen demasiados objetivos que cumplir. Cuando los gobiernos centrales alientan a los órganos de gobierno locales a formar asociaciones a largo plazo para abordar las desigualdades y cumplir los objetivos a corto plazo, lo último es lo primero.

 

Los gobiernos enfrentan grandes dilemas éticos.

Las elecciones políticas coexisten con juicios normativos sobre el papel del Estado y la responsabilidad personal, a menudo socavando acuerdos entre partidos políticos.

 

Un aspecto de la prevención puede debilitar al otro.

Una visión cínica de las iniciativas de prevención es que representan una solución política rápida en lugar de una solución significativa a largo plazo:

  • Los gobiernos centrales describen la prevención como la solución a los costos del sector público. A la vez, delegan la responsabilidad de la formulación de políticas públicas y reducen los presupuestos de los organismos públicos subnacionales.
  • Luego los organismos públicos de acuerdo a la urgencia priorizan sus responsabilidades legales.

 

Alguien debe rendir cuentas.

Si todos están involucrados en la formulación y elaboración de políticas públicas, no queda claro quién puede será responsable de los resultados. Esto es incompatible con la responsabilidad democrática al estilo de “Westminster” en donde se sabe quién es responsable y, por lo tanto, a quién culpar o reconocerle el buen desempeño.

 

     3. La evidencia no es una “bala mágica”

 

En una serie de pláticas [en inglés], identifico las razones por las cuales la “formulación de políticas públicas basada en evidencia” (EBPM) [en inglés] no describe bien el proceso de la política pública.

En otras publicaciones también sugiero que es más difícil para la evidencia “ganar la batalla” [en inglés] en las extensas áreas de la política de prevención en comparación con campos más específicos, por ejemplo el control del tabaco.

En general, una regla simple sobre EBPM es que nunca hay una panacea que sustituya al juicio. La política se trata de tomar decisiones que beneficien a algunos mientras que otros pierden. Puedes usar la evidencia para ayudar a comprender esas opciones, pero no para producir una solución “técnica”.

Una regla adicional con los problemas “intratables” es que la evidencia no es lo suficientemente buena como para generar claridad sobre la causa del problema. O simplemente encuentras cosas que no quieres saber.

La intervención temprana en las “políticas públicas familiares” parece ser un buen candidato para este último, por tres razones principales:

 

  1. Muy pocas intervenciones cumplen con los más altos estándares de evidencia

Hay dos tipos principales de intervenciones relevantes “basadas en evidencia” en este campo [en inglés].

Los primeros son “proyectos de intervención familiar” (FIPs, por sus siglas en inglés). En general, se centran en familias de bajos ingresos a menudo de padres solteros, en riesgo de desalojo y vinculados a comportamientos antisociales. Dichos proyectos proporcionan dos formas de intervención:

  • Apoyo intensivo las 24 horas del día, los 7 días de la semana. Los programas incluyen grupos y actividades después de la escuela (para niños) y clases de habilidades (para padres). En algunos casos también consideran tratamiento para las adicciones o la depresión. Dicho tratamiento se lleva a cabo en alojamientos destinados para este fin con reglas estrictas sobre acceso y comportamiento.
  • Un modelo de apoyo y capacitación.

 

La evidencia del éxito proviene de la evaluación más un contrafáctico: esta intervención es costosa, pero se cree que habría costado mucho más dinero y esfuerzo si no se hubiese intervenido. En general, no existe un ensayo controlado aleatorio (RCT, por sus siglas en inglés) para establecer la causa de los mejores resultados, o demostrar que esos resultados no habrían sucedido sin esta intervención.

El segundo son proyectos transferidos de otros países (principalmente los Estados Unidos de América. y Australia) en función de su exitosa reputación que se basa en la evidencia de los RCTs. Hay más evidencia cuantitativa de éxito, pero aún es difícil saber si el proyecto puede transferirse de manera efectiva y si su éxito puede replicarse en otro país con impulsores, problemas y servicios políticos muy diferentes.

 

  1. La evidencia sobre la “expansión” de la prevención primaria es relativamente débil

 Kenneth Dodge [en inglés] (2009) resume un problema general:

  • Hay pocos ejemplos de proyectos efectivos que especialistas llevan a cabo a “a escala”.
  • Existen problemas importantes en torno a la “fidelidad” al proyecto original cuando se amplía (incluida la necesidad de supervisar una expansión de profesionales bien capacitados)
  • Es difícil predecir el efecto de un programa, que se mostró prometedor cuando se aplicó a una determinada población, a una nueva y diferente.

 

  1. La evidencia sobre la intervención temprana secundaria también es débil

 Este punto sobre diferentes poblaciones con diferentes motivaciones se demuestra en un estudio (publicado en 2014) por Stephen Scott y otros [en inglés], acerca de dos intervenciones de Incredible Years para abordar los “síntomas de trastorno de oposición desafiante y los rasgos de personalidad antisocial” en niños de 3 a 7 años (para una discusión más amplia de tales programas, ver Fundamentos para la vida: ¿qué funciona para apoyar la interacción entre padres e hijos en los primeros años? [en inglés], publicado por la Early Intervention Foundation (Fundación de Intervención Temprana)).

Destacan un dilema clásico en la intervención temprana: la evidencia de efectividad solo es clara cuando los niños han sido remitidos clínicamente (“enfoque indicado”), pero no está claro cuando los niños han sido identificados como de alto riesgo utilizando predictores socioeconómicos (“enfoque selectivo”):

 

Un enfoque indicado es más sencillo de administrar, ya que hay menos niños con problemas graves, son más fáciles de identificar y sus padres generalmente están preparados para participar en el tratamiento; sin embargo, los problemas podrían ya estar demasiado arraigados para tratarlos. Por el contrario, un enfoque selectivo se centra en casos menos severos, pero debido a que los problemas están menos establecidos se debe evaluar a poblaciones enteras y algunos casos desarrollarán problemas graves.

 

Para nuestros propósitos, esto podría representar la forma más inconveniente de evidencia sobre intervención temprana: se podría intervenir temprano con respaldo limitado de evidencia que resulte probablemente exitoso o se podría tener una probabilidad mucho mayor de éxito cuando se interviene más tarde, en otras palabras, cuando se está acabando de tiempo para llamarlo ‘intervención temprana’.

Conclusión: Un vago consenso no sustituye la elección política.

Los gobiernos comienzan con la sensación de que han encontrado la solución a muchos problemas, solo para descubrir que tienen que tomar y defender elecciones altamente “políticas”.

Por ejemplo, considera el uso “creativo” de evidencia del gobierno del Reino Unido para hacer una política familiar [en inglés]. En pocas palabras, el gobierno eligió actuar rápido y a la ligera con la evidencia, demonizando a 117,000 familias para proporcionarle cobertura política a una redistribución de recursos hacia proyectos de intervención familiar.

Con justa razón, se podría objetar este estilo de política. Sin embargo, también se tendría que producir una alternativa factible.

Por ejemplo, el Gobierno escocés ha adoptado un enfoque diferente (quizás más cercano a lo que se esperaría en Nueva Zelanda), pero aún necesita producir y defender una narrativa acerca de sus elecciones. El gobierno de Escocia enfrenta casi las mismas limitaciones que el Reino Unido, su auto descripción hacia un “cambio decisivo” hacia la prevención [en inglés], no lo es.

Después de todo, la prevención no es diferente de cualquier otra área de política pública, excepto que ha demostrado ser mucho más complicada y difícil de mantener que la mayoría de las demás. La prevención es parte de un lenguaje excelente pero no una panacea para los problemas de política pública.

 

Otras lecturas:

Prevención [en inglés]

 

Vea también:

¿Qué haces cuando el 20% de la población causa el 80% de sus problemas? Posiblemente nada [en inglés].

Política de intervención temprana, desde “familias con problemas” hasta “personas nombradas”: problemas con la evidencia y encuadre de problemas [en inglés]

 

Traductores

Anette Bonifant Cisneros anette.bonifant@york.ac.uk

Juan Guillermo Vieira jgvieiras@unal.edu.co

Leave a comment

Filed under Evidence Based Policymaking (EBPM), Políticas Públicas, Prevention policy

Public health policy: assumptions and expectations

Rather misleadingly, this very draft paper is called The Politics of Evidence-based ‘Health in All Policies’It’s for Integrating Science and Politics for Public Health, convened by Patrick Fafard and Adèle Cassola at the Global Strategy Lab.

The most interesting section, for me, is the attempt to sense check the following list of assumptions/ expectations that I associate with public health studies of public policy. Unless stated otherwise, this list is based on literature reviews and documentary analysis underpinning studies of tobacco policy and prevention policy (Cairney and St Denny, 2020), as well as more impressionistic reflections from peer-reviewing many papers on this topic and attending relevant conferences (usually to speak to practitioners about the politics of EBPM). I am relying primarily on (a) the sense, often described in qualitative research, of a ‘saturation point’ to feel confident that more research will not unearth more categories, than (b) counting the frequency of term-use in each category, or (c) network analysis to identify the nature of a self-defined public health profession or community. As such, the focus is on the assumptions that scholars in this field often seem to take for granted, and often do not feel the need to explain. Its purpose is logical and conditional: if these are the assumptions, these are the expectations.

On that basis, I present a common public health narrative of the policy problem, how to understand it, and the processes necessary to address it:

  • Focus on preventing ill health rather than treating it when it becomes too severe.
  • Distinguish between types of prevention: primary (focus on the whole population to stop a problem occurring by investing early and/or modifying the social or physical environment); secondary (focus on at-risk groups to identify a problem at a very early stage to minimise harm); tertiary (focus on affected groups to stop a problem getting worse)
  • Focus on the social determinants of health inequalities, defined by the WHO (2019) as ‘the unfair and avoidable differences in health status’ that are ‘shaped by the distribution of money, power and resources’ and ‘the conditions in which people are born, grow, live, work and age’.
  • Promote ‘upstream’ measures designed to influence the health of the whole population (or health inequalities) rather than ‘downstream’ measures targeting individuals (although we discussed some debate/ confusion about the meaning of upstream).
  • Use scientific evidence to identify the nature of problems and most effective solutions.
  • Define scientific evidence in a particular way, such as in relation to a ‘hierarchy’ in which (a) the systematic review of randomised control trials often represents the gold standard, and (b) systems modelling plays a key role. Or, in fewer cases, challenge that hierarchy energetically.
  • Promote major policymaking reforms, including a focus on holistic or joined-up government, since the responsibility for health improvement goes well beyond health departments.  Prevention (or preventive policymaking) is a classic term, and ‘health in all policies’ (HIAP) is currently a key term.
  • Focus strongly on the role of industry as ‘vested interests’ causing public health problems (the ‘commercial determinants of health’) and, often, the lack of political will to regulate commercial activity.
  • Treat public health and prevention as a form of social protection (new category after PHE). Often, actors describe a moral imperative to intervene (in which case, the opposite argument relates to individual responsibility and opposition to the ‘nanny state’ – see also Cairney et al, 2012 on ‘secular morality’).
  • Use tobacco control as a model for other specific issues (e.g. alcohol use, obesity, salt) and the prevention agenda more generally (Studlar and Cairney, 2019).
  • Focus on identifying policy changes that represent a ‘win-win’ scenario in which all parties benefit from the policy outcome (in terms of their health), rather than identifying political winners and losers from the policy choice itself (new category – Baum et al, 2014).

Such assumptions underpin expectations for the role of government, and provide a frame of reference for assessing the overall direction of policy (such as for ‘prevention’). Please let me know if there is a big missing category, or one of them doesn’t seem quite right.

Leave a comment

Filed under Prevention policy, Public health, public policy, tobacco policy

Prevention is better than cure, so why aren’t we doing more of it?

This post provides a generous amount of background for my ANZSOG talk Prevention is better than cure, so why aren’t we doing more of it? If you read all of it, it’s a long read. If not, it’s a short read before the long read. Here is the talk’s description:

‘Does this sound familiar? A new government comes into office, promising to shift the balance in social and health policy from expensive remedial, high dependency care to prevention and early intervention. They commit to better policy-making; they say they will join up policy and program delivery, devolving responsibility to the local level and focusing on long term outcomes rather than short term widgets; and that they will ensure policy is evidence-based.  And then it all gets too hard, and the cycle begins again, leaving some exhausted and disillusioned practitioners in its wake. Why does this happen repeatedly, across different countries and with governments of different persuasions, even with the best will in the world?’ 

  • You’ll see from the question that I am not suggesting that all prevention or early intervention policies fail. Rather, I use policy theories to provide a general explanation for a major gap between the (realistic) expectations expressed in prevention strategies and the actual outcomes. We can then talk about how to close that gap.
  • You’ll also see the phrase ‘even with the best will in the world’, which I think is key to this talk. No-one needs me to rehearse the usually-vague and often-stated ways to explain failed prevention policies, including the ‘wickedness’ of policy problems, or the ‘pathology’ of public policy. Rather, I show that such policies may ‘fail’ even when there is wide and sincere cross-party agreement about the need to shift from reactive to more prevention policy design. I also suggest that the general explanation for failure – low ‘political will’ – is often damaging to the chances for future success.
  • Let’s start by defining prevention policy and policymaking.

When engaged in ‘prevention’, governments seek to:

  1. Reform policy.

Prevention policy is really a collection of policies designed to intervene as early as possible in people’s lives to improve their wellbeing and reduce inequalities and/or demand for acute services. The aim is to move from reactive to preventive public services, intervening earlier in people’s lives to address a wide range of longstanding problems – including crime and anti-social behaviour, ill health and unhealthy behaviour, low educational attainment, unemployment and low employability – before they become too severe.

  1. Reform policymaking.

Preventive policymaking describes the ways in which governments reform their practices to support prevention policy, including a commitment to:

  • ‘join up’ government departments and services to solve ‘wicked problems’ that transcend one area
  • give more responsibility for service design to local public bodies, stakeholders, ‘communities’ and service users produce long term aims for outcomes, and
  • reduce short term performance targets in favour of long term outcomes agreements.
  1. Ensure that policy is ‘evidence based’.

Three general reasons why ‘prevention’ policies never seem to succeed.

  1. Policymakers don’t know what prevention means.

They express a commitment to prevention before defining it fully. When they start to make sense of prevention, they find out how difficult it is to pursue, and how many controversial choices it involves (see also uncertainty versus ambiguity)

  1. They engage in a policymaking system that is too complex to control.

They try to share responsibility with many actors and coordinate action to direct policy outcomes, without the ability to design those relationships and control policy outcomes.

Yet, they also need to demonstrate to the electorate that they are in control, and find out how difficult it is to localise and centralise policy.

  1. They are unable and unwilling to produce ‘evidence based policymaking’.

Policymakers seek cognitive shortcuts (and their organisational equivalents) to gather enough information to make ‘good enough’ decisions. When they seek evidence on prevention, they find that it is patchy, inconclusive, often counter to their beliefs, and not a ‘magic bullet’ to help justify choices.

Throughout this process, their commitment to prevention policy can be sincere but unfulfilled. They do not articulate fully what prevention means or appreciate the scale of their task. When they try to deliver prevention strategies, they face several problems that, on their own, would seem daunting. Many of the problems they seek to ‘prevent’ are ‘wicked’, or difficult to define and seemingly impossible to solve, such as poverty, unemployment, low quality housing and homelessness, crime, and health and education inequalities. They face stark choices on how far they should go to shift the balance between state and market, redistribute wealth and income, distribute public resources, and intervene in people’s lives to change their behaviour and ways of thinking. Their focus on the long term faces major competition from more salient short-term policy issues that prompt them to maintain ‘reactive’ public services. Their often-sincere desire to ‘localise’ policymaking often gives way to national electoral politics, in which central governments face pressure to make policy from the ‘top’ and be decisive. Their pursuit of ‘evidence based’ policymaking often reveals a lack of evidence about which policy interventions work and the extent to which they can be ‘scaled up’ successfully.

These problems will not be overcome if policy makers and influencers misdiagnose them

  • If policy influencers make the simplistic assumption that this problem is caused by low political they will provide bad advice.
  • If new policymakers truly think that the problem was the low commitment and competence of their predecessors, they will begin with the same high hopes about the impact they can make, only to become disenchanted when they see the difference between their abstract aims and real world outcomes.
  • Poor explanation of limited success contributes to the high potential for (a) an initial period of enthusiasm and activity, replaced by (b) disenchantment and inactivity, and (c) for this cycle to be repeated without resolution.

Let’s add more detail to these general explanations:

1. What makes prevention so difficult to define?

When viewed as a simple slogan, ‘prevention’ seems like an intuitively appealing aim. It can generate cross-party consensus, bringing together groups on the ‘left’, seeking to reduce inequalities, and on the ‘right’, seeking to reduce economic inactivity and the cost of services.

Such consensus is superficial and illusory. When making a detailed strategy, prevention is open to many interpretations by many policymakers. Imagine the many types of prevention policy and policymaking that we could produce:

  1. What problem are we trying to solve?

Prevention policymaking represents a heroic solution to several crises: major inequalities, underfunded public services, and dysfunctional government.

  1. On what measures should we focus?

On which inequalities should we focus primarily? Wealth, occupation, income, race, ethnicity, gender, sexuality, disability, mental health.

On which measures of inequality? Economic, health, healthy behaviour, education attainment, wellbeing, punishment.

  1. On what solution should we focus?

To reduce poverty and socioeconomic inequalities, improve national quality of life, reduce public service costs, or increase value for money

  1. Which ‘tools’ or policy instruments should we use?

Redistributive policies to address ‘structural’ causes of poverty and inequality?

Or, individual-focused policies to: (a) boost the mental ‘resilience’ of public service users, (b) oblige, or (c) exhort people to change behaviour.

  1. How do we intervene as early as possible in people’s lives?

Primary prevention. Focus on the whole population to stop a problem occurring by investing early and/or modifying the social or physical environment. Akin to whole-population immunizations.

Secondary prevention. Focus on at-risk groups to identify a problem at a very early stage to minimise harm.

Tertiary prevention. Focus on affected groups to stop a problem getting worse.

  1. How do we pursue ‘evidence based policymaking’? 3 ideal-types

Using randomised control trials and systematic review to identify the best interventions?

Storytelling to share best governance practice?

‘Improvement’ methods to experiment on a small scale and share best practice?

  1. How does evidence gathering connect to long-term policymaking?

Does a national strategy drive long-term outcomes?

Does central government produce agreements with or targets for local authorities?

  1. Is preventive policymaking a philosophy or a profound reform process?

How serious are national governments – about localism, service user-driven public services, and joined up or holistic policymaking – when their elected policymakers are held to account for outcomes?

  1. What is the nature of state intervention?

It may be punitive or supportive. See: How would Lisa Simpson and Monty Burns make progressive social policy?

2.     Making ‘hard choices’: what problems arise when politics meets policymaking?

When policymakers move from idiom and broad philosophy towards specific policies and practices, they find a range of obstacles, including:

The scale of the task becomes overwhelming, and not suited to electoral cycles.

Developing policy and reforming policymaking takes time, and the effect may take a generation to see.

There is competition for policymaking resources such as attention and money.

Prevention is general, long-term, and low salience. It competes with salient short-term problems that politicians feel compelled to solve first.

Prevention is akin to capital investment with no guarantee of a return. Reductions in funding ‘fire-fighting’, ‘frontline’ services to pay for prevention initiatives, are hard to sell. Governments invest in small steps, and investment is vulnerable when money is needed quickly to fund public service crises.

The benefits are difficult to measure and see.

Short-term impacts are hard to measure, long-term impacts are hard to attribute to a single intervention, and prevention does not necessarily save money (or provide ‘cashable’ savings’).

Reactive policies have a more visible impact, such as to reduce hospital waiting times or increase the number of teachers or police officers.

Problems are ‘wicked’.

Getting to the ‘root causes’ of problems is not straightforward; policymakers often have no clear sense of the cause of problems or effect of solutions. Few aspects of prevention in social policy resemble disease prevention, in which we know the cause of many diseases, how to screen for them, and how to prevent them in a population.

Performance management is not conducive to prevention.

Performance management systems encourage public sector managers to focus on their services’ short-term and measurable targets over shared aims with public service partners or the wellbeing of their local populations.

Performance management is about setting priorities when governments have too many aims to fulfil. When central governments encourage local governing bodies to form long-term partnerships to address inequalities and meet short-term targets, the latter come first.

Governments face major ethical dilemmas.

Political choices co-exist with normative judgements concerning the role of the state and personal responsibility, often undermining cross-party agreement.

One aspect of prevention may undermine the other.

A cynical view of prevention initiatives is that they represent a quick political fix rather than a meaningful long-term solution:

  • Central governments describe prevention as the solution to public sector costs while also delegating policymaking responsibility to, and reducing the budgets of, local public bodies.
  • Then, public bodies prioritise their most pressing statutory responsibilities.

Someone must be held to account.

If everybody is involved in making and shaping policy, it becomes unclear who can be held to account over the results. This outcome is inconsistent with Westminster-style democratic accountability in which we know who is responsible and therefore who to praise or blame.

3.      ‘The evidence’ is not a ‘magic bullet’

In a series of other talks, I identify the reasons why ‘evidence based policymaking’ (EBPM) does not describe the policy process well.

Elsewhere, I also suggest that it is more difficult for evidence to ‘win the day’ in the broad area of prevention policy compared to the more specific field of tobacco control.

Generally speaking, a good simple rule about EBPM is that there is never a ‘magic bullet’ to take the place of judgement. Politics is about making choices which benefit some while others lose out. You can use evidence to help clarify those choices, but not produce a ‘technical’ solution.

A further rule with ‘wicked’ problems is that the evidence is not good enough even to generate clarity about the cause of the problem. Or, you simply find out things you don’t want to hear.

Early intervention in ‘families policies’ seems to be a good candidate for the latter, for three main reasons:

  1. Very few interventions live up to the highest evidence standards

There are two main types of relevant ‘evidence based’ interventions in this field.

The first are ‘family intervention projects’ (FIPs). They generally focus on low income, often lone parent, families at risk of eviction linked to factors such as antisocial behaviour, and provide two forms of intervention:

  • intensive 24/7 support, including after school clubs for children and parenting skills classes, and treatment for addiction or depression in some cases, in dedicated core accommodation with strict rules on access and behaviour
  • an outreach model of support and training.

The evidence of success comes from evaluation plus a counterfactual: this intervention is expensive, but we think that it would have cost far more money and heartache if we had not intervened. There is generally no randomised control trial (RCT) to establish the cause of improved outcomes, or demonstrate that those outcomes would not have happened without this intervention.

The second are projects imported from other countries (primarily the US and Australia) based on their reputation for success built on RCT evidence. There is more quantitative evidence of success, but it is still difficult to know if the project can be transferred effectively and if its success can be replicated in another country with a very different political drivers, problems, and services.

2. The evidence on ‘scaling up’ for primary prevention is relatively weak

Kenneth Dodge (2009) sums up a general problem:

  • there are few examples of taking effective specialist projects ‘to scale’
  • there are major issues around ‘fidelity’ to the original project when you scale up (including the need to oversee a major expansion in well-trained practitioners)
  • it is difficult to predict the effect of a programme, which showed promise when applied to one population, to a new and different population.

3. The evidence on secondary early intervention is also weak

This point about different populations with different motivations is demonstrated in a more recent (published 2014) study by Stephen Scott et al of two Incredible Years interventions – to address ‘oppositional defiant disorder symptoms and antisocial personality character traits’ in children aged 3-7 (for a wider discussion of such programmes see the Early Intervention Foundation’s Foundations for life: what works to support parent child interaction in the early years?).

They highlight a classic dilemma in early intervention: the evidence of effectiveness is only clear when children have been clinically referred (‘indicated approach’), but unclear when children have been identified as high risk using socioeconomic predictors (‘selective approach’):

An indicated approach is simpler to administer, as there are fewer children with severe problems, they are easier to identify, and their parents are usually prepared to engage in treatment; however, the problems may already be too entrenched to treat. In contrast, a selective approach targets milder cases, but because problems are less established, whole populations have to be screened and fewer cases will go on to develop serious problems.

For our purposes, this may represent the most inconvenient form of evidence on early intervention: you can intervene early on the back of very limited evidence of likely success, or have a far higher likelihood of success when you intervene later, when you are running out of time to call it ‘early intervention’.

Conclusion: vague consensus is no substitute for political choice

Governments begin with the sense that they have found the solution to many problems, only to find that they have to make and defend highly ‘political’ choices.

For example, see the UK government’s ‘imaginative’ use of evidence to make families policy. In a nutshell, it chose to play fast and loose with evidence, and demonise 117000 families, to provide political cover to a redistribution of resources to family intervention projects.

We can, with good reason, object to this style of politics. However, we would also have to produce a feasible alternative.

For example, the Scottish Government has taken a different approach (perhaps closer to what one might often expect in New Zealand), but it still needs to produce and defend a story about its choices, and it faces almost the same constraints as the UK. It’s self-described ‘decisive shift’ to prevention was no a decisive shift to prevention.

Overall, prevention is no different from any other policy area, except that it has proven to be much more complicated and difficult to sustain than most others. Prevention is part of an excellent idiom but not a magic bullet for policy problems.

Further reading:

Prevention

See also

What do you do when 20% of the population causes 80% of its problems? Possibly nothing.

Early intervention policy, from ‘troubled families’ to ‘named persons’: problems with evidence and framing ‘valence’ issues

 

 

2 Comments

Filed under Evidence Based Policymaking (EBPM), Prevention policy, Public health, public policy

The UK government’s imaginative use of evidence to make policy

This post describes a new article published in British Politics (Open Access). Please find:

(1) A super-exciting video/audio powerpoint I use for a talk based on the article

(2) The audio alone (link)

(3) The powerpoint to download, so that the weblinks work (link) or the ppsx/ presentation file in case you are having a party (link)

(4) A written/ tweeted discussion of the main points

In retrospect, I think the title was too subtle and clever-clever. I wanted to convey two meanings: imaginative as a euphemism for ridiculous/ often cynical and to argue that a government has to be imaginative with evidence. The latter has two meanings: imaginative (1) in the presentation and framing of evidence-informed agenda, and (2) when facing pressure to go beyond the evidence and envisage policy outcomes.

So I describe two cases in which its evidence-use seems cynical, when:

  1. Declaring complete success in turning around the lives of ‘troubled families’
  2. Exploiting vivid neuroscientific images to support ‘early intervention’

Then I describe more difficult cases in which supportive evidence is not clear:

  1. Family intervention project evaluations are of limited value and only tentatively positive
  2. Successful projects like FNP and Incredible Years have limited applicability or ‘scalability’

As scientists, we can shrug our shoulders about the uncertainty, but elected policymakers in government have to do something. So what do they do?

At this point of the article it will look like I have become an apologist for David Cameron’s government. Instead, I’m trying to demonstrate the value of comparing sympathetic/ unsympathetic interpretations and highlight the policy problem from a policymaker’s perspective:

Cairney 2018 British Politics discussion section

I suggest that they use evidence in a mix of ways to: describe an urgent problem, present an image of success and governing competence, and provide cover for more evidence-informed long term action.

The result is the appearance of top-down ‘muscular’ government and ‘a tendency for policy to change as is implemented, such as when mediated by local authority choices and social workers maintaining a commitment to their professional values when delivering policy’

I conclude by arguing that ‘evidence-based policy’ and ‘policy-based evidence’ are political slogans with minimal academic value. The binary divide between EBP/ PBE distracts us from more useful categories which show us the trade-offs policymakers have to make when faced with the need to act despite uncertainty.

Cairney British Politics 2018 Table 1

As such, it forms part of a far wider body of work …

In both cases, the common theme is that, although (1) the world of top-down central government gets most attention, (2) central governments don’t even know what problem they are trying to solve, far less (3) how to control policymaking and outcomes.

In that wider context, it is worth comparing this talk with the one I gave at the IDS (which, I reckon is a good primer for – or prequel to – the UK talk):

See also:

Early intervention policy, from ‘troubled families’ to ‘named persons’: problems with evidence and framing ‘valence’ issues

Why doesn’t evidence win the day in policy and policymaking?

(found by searching for early intervention)

See also:

Here’s why there is always an expectations gap in prevention policy

Social investment, prevention and early intervention: a ‘window of opportunity’ for new ideas?

(found by searching for prevention)

Powerpoint for guest lecture: Paul Cairney UK Government Evidence Policy

5 Comments

Filed under Evidence Based Policymaking (EBPM), POLU9UK, Prevention policy, UK politics and policy