The coronavirus feels like a new policy problem that requires new policy analysis. The analysis should be informed by (a) good evidence, translated into (b) good policy. However, don’t be fooled into thinking that either of those things are straightforward. There are simple-looking steps to go from defining a problem to making a recommendation, but this simplicity masks the profoundly political process that must take place. Each step in analysis involves political choices to prioritise some problems and solutions over others, and therefore prioritise some people’s lives at the expense of others.
The very-long version of this post takes us through those steps in the UK, and situates them in a wider political and policymaking context. This post is shorter, and only scratches the surface of analysis.
5 steps to policy analysis
Define the problem.
Perhaps we can sum it up as: (a) the impact of this virus and illness will be a level of death and illness that could overwhelm the population and exceed the capacity of public services, so (b) we need to contain the virus enough to make sure it spreads in the right way at the right time, so (c) we need to encourage and make people change their behaviour (primarily via hygiene and social distancing). However, there are many ways to frame this problem to emphasise the importance of some populations over others, and some impacts over others.
Identify technically and politically feasible solutions.
Solutions are not really solutions: they are policy instruments that address one aspect of the problem, including taxation and spending, delivering public services, funding research, giving advice to the population, and regulating or encouraging changes to social behaviour. Each new instrument contributes an existing mix, with unpredictable and unintended consequences. Some instruments seem technically feasible (they will work as intended if implemented), but will not be adopted unless politically feasible (enough people support their introduction). Or vice versa. This dual requirement rules out a lot of responses.
Use values and goals to compare solutions.
Typical judgements combine: (a) broad descriptions of values such as efficiency, fairness, freedom, security, and human dignity, (b) instrumental goals, such as sustainable policymaking (can we do it, and for how long?), and political feasibility (will people agree to it, and will it make me more or less popular or trusted?), and (c) the process to make choices, such as the extent to which a policy process involves citizens or stakeholders (alongside experts) in deliberation. They combine to help policymakers come to high profile choices (such as the balance between individual freedom and state coercion), and low profile but profound choices (to influence the level of public service capacity, and level of state intervention, and therefore who and how many people will die).
Predict the outcome of each feasible solution.
It is difficult to envisage a way for the UK Government to publicise all of the thinking behind its choices (Step 3) and predictions (Step 4) in a way that would encourage effective public deliberation. People often call for the UK Government to publicise its expert advice and operational logic, but I am not sure how they would separate it from their normative logic about who should live or die, or provide a frank account without unintended consequences for public trust or anxiety. If so, one aspect of government policy is to keep some choices implicit and avoid a lot of debate on trade-offs. Another is to make choices continuously without knowing what their impact will be (the most likely scenario right now).
Make a choice, or recommendation to your client.
Your recommendation or choice would build on these four steps. Define the problem with one framing at the expense of the others. Romanticise some people and not others. Decide how to support some people, and coerce or punish others. Prioritise the lives of some people in the knowledge that others will suffer or die. Do it despite your lack of expertise and profoundly limited knowledge and information. Learn from experts, but don’t assume that only scientific experts have relevant knowledge (decolonise; coproduce). Recommend choices that, if damaging, could take decades to fix after you’ve gone. Consider if a policymaker is willing and able to act on your advice, and if your proposed action will work as intended. Consider if a government is willing and able to bear the economic and political costs. Protect your client’s popularity, and trust in your client, at the same time as protecting lives. Consider if your advice would change if the problem seemed to change. If you are writing your analysis, maybe keep it down to one sheet of paper (in other words, fewer words than in this post up to this point).
Policy analysis is not as simple as these steps suggest, and further analysis of the wider policymaking environment helps describe two profound limitations to simple analytical thought and action.
Policymakers must ignore almost all evidence
The amount of policy relevant information is infinite, and capacity is finite. So, individuals and governments need ways to filter out almost all of it. Individuals combine cognition and emotion to help them make choices efficiently, and governments have equivalent rules to prioritise only some information. They include: define a problem and a feasible response, seek information that is available, understandable, and actionable, and identify credible sources of information and advice. In that context, the vague idea of trusting or not trusting experts is nonsense, and the larger post highlights the many flawed ways in which all people decide whose expertise counts.
They do not control the policy process.
Policymakers engage in a messy and unpredictable world in which no single ‘centre’ has the power to turn a policy recommendation into an outcome.
There are many policymakers and influencers spread across a political system. For example, consider the extent to which each government department, devolved governments, and public and private organisations are making their own choices that help or hinder the UK government approach.
Most choices in government are made in ‘subsystems’, with their own rules and networks, over which ministers have limited knowledge and influence.
The social and economic context, and events, are largely out of their control.
The take home messages (if you accept this line of thinking)
The coronavirus is an extreme example of a general situation: policymakers will always have very limited knowledge of policy problems and control over their policymaking environment. They make choices to frame problems narrowly enough to seem solvable, rule out most solutions as not feasible, make value judgements to try help some more than others, try to predict the results, and respond when the results do not match their hopes or expectations.
This is not a message of doom and despair. Rather, it encourages us to think about how to influence government, and hold policymakers to account, in a thoughtful and systematic way that does not mislead the public or exacerbate the problem we are seeing. No one is helping their government solve the problem by saying stupid shit on the internet (OK, that last bit was a message of despair).
Further reading:
The longer report sets out these arguments in much more detail, with some links to further thoughts and developments.
This series of ‘750 words’ posts summarises key texts in policy analysis and tries to situate policy analysis in a wider political and policymaking context. Note the focus on whose knowledge counts, which is not yet a big feature of this crisis.
These series of 500 words and 1000 words posts (with podcasts) summarise concepts and theories in policy studies.
This is the long version. It is long. Too long to call a blog post. Let’s call it a ‘living document’ that I will update and amend as new developments arise. In most cases, I am adding tweets, so the date of the update is embedded. If I add a new section, I will add a date. The short version is shorter.
The coronavirus feels like a new policy problem. Governments already have policies for public health crises, but the level of uncertainty about the spread and impact of this virus seems to be taking it to a new level of policy, media, and public attention. The UK Government’s Prime Minister calls it ‘the worst public health crisis for a generation’.
As such, there is no shortage of opinions on what to do, but there is a shortage of well-considered opinions, producing little consensus. Many people are rushing to judgement and expressing remarkably firm opinions about the best solutions, but their contributions add up to contradictory evaluations, in which:
the government is doing precisely the right thing or the completely wrong thing,
we should listen to this expert saying one thing or another expert saying the opposite.
Lots of otherwise-sensible people are doing what they bemoan in politicians: rushing to judgement, largely accepting or sharing evidence only if it reinforces that judgement, and/or using their interpretation of any new development to settle scores with their opponents.
Yet, anyone who feels, without uncertainty, that they have the best definition of, and solution to, this problem is a fool. If people are also sharing bad information and advice, they are dangerous fools. Further, as Professor Madley puts it (in the video below), ‘anyone who tells you they know what’s going to happen over the next six months is lying’.
In that context, how can we make sense of public policy to address the coronavirus in a more systematic way?
Studies of policy analysis and policymaking do not solve a policy problem, but they at least give us a language to think it through.
In each step, note how quickly it is possible to be overwhelmed by uncertainty and ambiguity, even when the issue seems so simple at first.
Note how difficult it is to move from Step 1, and to separate Step 1 from the others. It is difficult to define the problem without relating it to the solution (or to the ways in which we will evaluate each solution).
Let’s relate that analysis to research on policymaking, to understand the wider context in which people pay attention to, and try to address, important problems that are largely out of their control.
Throughout, note that I am describing a thought process as simply as I can, not a full examination of relevant evidence. I am highlighting the problems that people face when ‘diagnosing’ policy problems, not trying to diagnose it myself. To do so, I draw initially on common advice from the key policy analysis texts (summaries of the texts that policy analysis students are most likely to read) that simplify the process a little too much. Still, the thought process that it encourages took me hours alone (spread over three days) to produce no real conclusion. Policymakers and advisers, in the thick of this problem, do not have that luxury of time or uncertainty.
— Boris Johnson #StayHomeSaveLives (@BorisJohnson) March 27, 2020
In our latest guest blog, Jonny Pearson-Stuttard, RSPH Trustee and Public Health Doctor @imperialcollege sets out what we know about the spread of coronavirus to date, and why the Government has taken the measures it hashttps://t.co/XM7zKKjwtE
Provide a diagnosis of a policy problem, using rhetoric and eye-catching data to generate attention.
Identify its severity, urgency, cause, and our ability to solve it. Don’t define the wrong problem, such as by oversimplifying.
Problem definition is a political act of framing, as part of a narrative to evaluate the nature, cause, size, and urgency of an issue.
Define the nature of a policy problem, and the role of government in solving it, while engaging with many stakeholders.
‘Diagnose the undesirable condition’ and frame it as ‘a market or government failure (or maybe both)’.
Coronavirus as a physical problem is not the same as a coronavirus policy problem. To define the physical problem is to identify the nature, spread, and impact of a virus and illness on individuals and populations. To define a policy problem, we identify the physical problem and relate it (implicitly or explicitly) to what we think a government can, and should, do about it. Put more provocatively, it is only a policy problem if policymakers are willing and able to offer some kind of solution.
This point may seem semantic, but it raises a profound question about the capacity of any government to solve a problem like an epidemic, or for governments to cooperate to solve a pandemic. It is easy for an outsider to exhort a government to ‘do something!’ (or ‘ACT NOW!’) and express certainty about what would happen. However, policymakers inside government:
Do not enjoy the same confidence that they know what is happening, or that their actions will have their intended consequences, and
Will think twice about trying to regulate social behaviour under those circumstances, especially when they
Know that any action or inaction will benefit some and punish others.
For example, can a government make people wash their hands? Or, if it restricts gatherings at large events, can it stop people gathering somewhere else, with worse impact? If it closes a school, can it stop children from going to their grandparents to be looked after until it reopens? There are 101 similar questions and, in each case, I reckon the answer is no. Maybe government action has some of the desired impact; maybe not. If you agree, then the question might be: what would it really take to force people to change their behaviour?
The answer is: often too much for a government to consider (in a liberal democracy), particularly if policymakers are informed that it will not have the desired impact.
A couple of key takeaways from our analysis of early COVID-19 dynamics in Wuhan:
1. We estimated that the control measures introduced – unprecedented interventions that will have had a huge social and psychological toll – reduced transmission by around 55% in space of 2 weeks 1/
If so, the UK government’s definition of the policy problem will incorporate this implicit question: what can we do if we can influence, but not determine (or even predict well) how people behave?
Uncertainty about the coronavirus plus uncertainty about policy impact
Now, add that general uncertainty about the impact of government to this specific uncertainty about the likely nature and spread of the coronavirus:
The ideal spread involves all well people sharing the virus first, while all vulnerable people (e.g. older, and/or with existing health problems that affect their immune systems) protected in one isolated space, but it won’t happen like that; so, we are trying to minimise damage in the real world.
We mainly track the spread via deaths, with data showing a major spike appearing one month later, so the problem may only seem real to most people when it is too late to change behaviour.
A lot of the spread will happen inside homes, where the role of government is minimal (compared to public places). So, for example, the impact of school closures could be good (isolation) or make things worse (children spreading the virus to vulnerable relatives) (see also ‘we don’t know [if the UKG decision not to close schools] was brilliant or catastrophic’). [Update 18.3.20: as it turned out, the First Minister’s argument for closing Scottish schools was that there were too few teachers available).
The choice in theory is between a rapid epidemic with a high peak, or a slowed-down epidemic over a longer period, but ‘anyone who tells you they know what’s going to happen over the next six months is lying’.
Maybe this epidemic will be so memorable as to shift social behaviour, but so much depends on trying to predict (badly) if individuals will actually change (see also Spiegelhalter on communicating risk).
None of this account tells policymakers what to do, but at least it helps them clarify three key aspects of their policy problem:
The impact of this virus and illness could overwhelm the population, to the extent that it causes mass deaths, causes a level of illness that exceeds the capacity of health services to treat, and contributes to an unpredictable amount of social and economic damage.
We need to contain the virus enough to make sure it (a) spreads at the right speed and/or (b) peaks at the right time. The right speed seems to be: a level that allows most people to recover alone, while the most vulnerable are treated well in healthcare settings that have enough capacity. The right time seems to be the part of the year with the lowest demand on health services (e.g. summer is better than winter). In other words, (a) reduce the size of the peak by ‘flattening the curve’, and/or (b) find the right time of year to address the peak, while (c) anticipating more than one peak.
My impression is that the most frequently-expressed aim is (a) …
Yesterday we entered the Delay phase of our #COVID_19uk Action Plan. @UKScienceChief explained why this is important.
It allows us to #FlattenTheCurve, which means reducing the impact in the short-term to ensure our health & care system can effectively protect vulnerable people pic.twitter.com/1I45C3v38V
— Department of Health and Social Care (@DHSCgovuk) March 13, 2020
… while the UK Government’s Deputy Chief Medical Officer also seems to be describing (b):
Dr Jenny Harries, Deputy Chief Medical Officer, came into Downing Street to answer some of the most commonly asked questions on coronavirus. pic.twitter.com/KCdeHsaz6a
We need to encourage or coerce people to change their behaviour, to look after themselves (e.g. by handwashing) and forsake their individual preferences for the sake of public health (e.g. by self-isolating or avoiding vulnerable people). Perhaps we can foster social trust and empathy to encourage responsible individual action. Perhaps people will only protect others if obliged to do so (compare Stone; Ostrom; game theory).
From uncertainty to ambiguity
If you are still with me, I reckon you would have worded those aims slightly differently, right? There is some ambiguity about these broad intentions, partly because there is some uncertainty, and partly because policymakers need to set rather vague intentions to generate the highest possible support for them. However, vagueness is not our friend during a crisis involving such high anxiety. Further, they are only delaying the inevitable choices that people need to make to turn a complex multi-faceted problem into something simple enough to describe and manage. The problem may be complex, but our attention focuses only on a small number of aspects, at the expense of the rest. Examples that have arisen, so far, include to accentuate:
The health of the whole population or people who would be affected disproportionately by the illness.
For example, the difference in emphasis affects the health advice for the relatively vulnerable (and the balance between exhortation and reassurance)
This is me. I stand by my judgement that 'at risk' people should cocoon ourselves as much as possible.
There is no need for lots of us to die to create “herd immunity”.
Many lives could be saved by proper guidance and action now.
— Prof. Colin Talbot (Cocooned Not Isolated) (@colinrtalbot) March 12, 2020
Leading surgeon reassures older people and those with health problems, saying scare stories are "completely and utterly wrong" @RCSEdhttps://t.co/2kgeUCfNes
Inequalities in relation to health, socio-economic status (e.g. income, gender, race, ethnicity), or the wider economy.
For example, restrictive measures may reduce the risk of harm to some, but increase the burden on people with no savings or reliable sources of income.
For example, some people are hoarding large quantities of home and medical supplies that (a) other people cannot afford, and (b) some people cannot access, despite having higher need.
For example, social distancing will limit the spread of the virus (see the nascent evidence), but also produce highly unequal forms of social isolation that increase the risk of domestic abuse (possibly exacerbated by school closures) and undermine wellbeing. Or, there will be major policy changes, such as to the rules to detain people under mental health legislation, regarding abortion, or in relation to asylum.
COVID-19 has brought new focus to women’s continued inequality. Without a gendered response to both the health and economic crises, gender inequality will be further cemented. Read more on the blog: https://t.co/zYxSFpUTNE
“The epidemic has had a huge impact on domestic violence,” said Wan. “According to our statistics, 90% of the causes of violence are related to the COVID-19 epidemic.” https://t.co/xswemtf548
I just asked a DC cop what he’s noticed since the coronavirus sent people home. “More domestic violence,” he said, without missing a beat. https://t.co/kv9zH5VNj1
What the coronavirus bill will do: https://t.co/qoBdKKr64H Mental Health Act – detention implemented using just one doctor’s opinion (not 2) & AMHP, & temporarily allow extension or removal of time limits to allow for greater flexibility where services are less able to respond
English obviously, but fascinating that have issued an explicitly ethical framework for COVID decisions re mental health and incapacity. Can Scotland do same? https://t.co/WccPntZOwf
WOW – government has relaxed restrictions on WHERE abortions can take place, temporary inclusion of 'the home' as a legal site for abortion: https://t.co/Vw714fWXEM
BREAKING NEWS!!!! The Home Office have confirmed that ALL evictions and terminations of asylum support have been paused for 3 months. Find out more and read the letter from Home Office Minister Chris Philp confirming this on our website at: https://t.co/KDlVr4PHyP
For example, governments cannot ignore the impact of their actions on the economy, however much they emphasise mortality, health, and wellbeing. Most high-profile emphasis was initially on the fate of large and small businesses, and people with mortgages, but a long period of crisis will a tip the balance from low income to unsustainable poverty (even prompting Iain Duncan Smith to propose policy change), and why favour people who can afford a mortgage over people scraping the money together for rent?
So…. Govt income protection package includes….. 1. 80% of wage costs up to £2500 2. Deferred VAT. 3. £7 billion uplift to Universal Credit and Woring Tax crdit. 4. £1 billion to cover 30% of house rental costs. 5. Self employed to get same as sickness benefit payments.
Coronavirus and inequalities. Here's the evidence that low and middle income families will be severely hit by a reduction in income if workplaces shut, they have to take time off sick or need money to respond to the crisis. It will add to their other problems #Marmot2020https://t.co/Zf1HNQOFyB
Exactly right from @TorstenBell. Increasing SSP, out-of-work benefits, and Universal Credit/tax credit rates is quick, easy & the right thing to do economically and socially. Basic income/UBI/helicopter money is a distraction. https://t.co/Ma1jDCbNns
Here’s Big Issue editor @PauldMcNamee on how you can help Big Issue vendors as we navigate the coronavirus pandemic. Stick with us. https://t.co/jnykEhai3e
Cleaners, porters and catering staff at Lewisham Hospital walked out today after not being paid or given protective clothing – in a hospital that has had confirmed cases of coronavirus. This is what they had to say.https://t.co/qunYVJi8Nxpic.twitter.com/qzdu8LLbOg
I've said it before and I'll say it until you've all obeyed: follow @WomensBudgetGrp because they are one of the UK's most important women's orgs, doing all the gender policy work our govt won't. Here they are on the gendered impacts of #COVID19. https://t.co/6iTc0CGePZ
Almost every Black person who has been through the education system can tell you a tale about a teacher who unfairly penalised them/saw them as disruptive/continually underestimated them & more.
So many students are going to be at the mercy of teachers with unchecked racial bias https://t.co/dQzEQ35Nka
📢BREAKING: the government has changed its approach and is now introducing a FULL ban on evictions, giving much-needed protection for renters at this critical time during the #coronavirus crisis.https://t.co/iu8spPOAaq
A need for more communication and exhortation, or for direct action to change behaviour.
The short term (do everything possible now) or long term (manage behaviour over many months).
The Imperial College COVID report is being discussed. But a major takeaway from it will likely survive discussion: the human cost of a pure mitigation strategy is inacceptable, whilst a pure suppression strategy is unsustainable over time (thread)
How to maintain trust in the UK government when (a) people are more or less inclined to trust a the current part of government and general trust may be quite low, and (b) so many other governments are acting differently from the UK.
2. … poor communication and CRUCIALLY effective community mobilisation to not overwhelm 111 & minimise herd immunity hysteria. You have a PM no-one trusts (even those who voted for him) you MUST build trust in other ways
— Sander van der Linden (@Sander_vdLinden) March 23, 2020
For example, note the visible presence of the Prime Minister, but also his unusually high deference to unelected experts such as (a) UK Government senior scientists providing direct advice to ministers and the public, and (b) scientists drawing on limited information to model behaviour and produce realistic scenarios (we can return to the idea of ‘evidence-based policymaking’ later). This approach is not uncommon with epidemics/ pandemics (LD was then the UK Government’s Chief Medical Officer):
So we agreed that Liam Donaldson would front all media activity & that I would take a backseat. It was the right decision. From then on, everything became a bit calmer. And my time as Health Secretary was spent less on fighting media storms & more on plans for the vaccine …
For example, note how often people are second guessing and criticising the UK Government position (and questioning the motives of Conservative ministers).
For example, people often try to lay blame for viruses on certain populations, based on their nationality, race, ethnicity, sexuality, or behaviour (e.g. with HIV).
For example, the (a) association between the coronavirus and China and Chinese people (e.g. restrict travel to/ from China; e.g. exacerbate racism), initially overshadowed (b) the general role of international travellers (e.g. place more general restrictions on behaviour), and (c) other ways to describe who might be responsible for exacerbating a crisis.
For social scientists wondering “what can I do now?” here’s a challenge:@cp_roth@LukasHenselEcon & others ran a survey with 2500 Italians yday & found that:
Under ‘normal’ policymaking circumstances, we would expect policymakers to resolve this ambiguity by exercising power to set the agenda and make choices that close off debate. Attention rises at first, a choice is made, and attention tends to move on to something else. With the coronavirus, attention to many different aspects of the problem has been lurching remarkably quickly. The definition of the policy problem often seems to be changing daily or hourly, and more quickly than the physical problem. It will also change many more times, particularly when attention to each personal story of illness or death prompts people to question government policy every hour. If the policy problem keeps changing in these ways, how could a government solve it?
@alexwickham doing fine work as a journalist again. Gets right into Government somehow and tells people what is going on.
10 Days That Changed Britain: "Heated" Debate Between Scientists Forced Boris Johnson To Act On Coronavirus https://t.co/hDLEAPT3Z0
Public expenditure (e.g. to boost spending for emergency care, crisis services, medical equipment)
Economic incentives and disincentives (e.g. to reduce the cost of business or borrowing, or tax unhealthy products)
Linking spending to entitlement or behaviour (e.g. social security benefits conditional on working or seeking work, perhaps with the rules modified during crises)
Formal regulations versus voluntary agreements (e.g. making organisations close, or encouraging them to close)
Public services: universal or targeted, free or with charges, delivered directly or via non-governmental organisations
As a result, what we call ‘policy’ is really a complex mix of instruments adopted by one or more governments. A truism in policy studies is that it is difficult to define or identify exactly what policy is because (a) each new instrument adds to a pile of existing measures (with often-unpredictable consequences), and (b) many instruments designed for individual sectors tend, in practice, to intersect in ways that we cannot always anticipate. When you think through any government response to the coronavirus, note how every measure is connected to many others.
Further, it is a truism in public policy that there is a gap between technical and political feasibility: the things that we think will be most likely to work as intended if implemented are often the things that would receive the least support or most opposition. For example:
Redistributing income and wealth to reduce socio-economic inequalities (e.g. to allay fears about the impact of current events on low-income and poverty) seems to be less politically feasible than distributing public services to deal with the consequences of health inequalities.
Providing information and exhortation seems more politically feasible than the direct regulation of behaviour. Indeed, compared to many other countries, the UK Government seems reluctant to introduce ‘quarantine’ style measures to restrict behaviour.
Under ‘normal’ circumstances, governments may be using these distinctions as simple heuristics to help them make modest policy changes while remaining sufficiently popular (or at least looking competent). If so, they are adding or modifying policy instruments during individual ‘windows of opportunity’ for specific action, or perhaps contributing to the sense of incremental change towards an ambitious goal.
Right now, we may be pushing the boundaries of what seems possible, since crises – and the need to address public anxiety – tend to change what seems politically feasible. However, many options that seem politically feasible may not be possible (e.g. to buy a lot of extra medical/ technology capacity quickly), or may not work as intended (e.g. to restrict the movement of people). Think of technical and political feasibility as necessary but insufficient on their own, which is a requirement that rules out a lot of responses.
Step 3 Use value-based criteria and political goals to compare solutions
Typical value judgements relate to efficiency, equity and fairness, the trade-off between individual freedom and collective action, and the extent to which a policy process involves citizens in deliberation.
Normative assessments are based on values such as ‘equality, efficiency, security, democracy, enlightenment’ and beliefs about the preferable balance between state, communal, and market/ individual solutions
‘Specify the objectives to be attained in addressing the problem and the criteria to evaluate the attainment of these objectives as well as the satisfaction of other key considerations (e.g., equity, cost, equity, feasibility)’.
‘Effectiveness, efficiency, fairness, and administrative efficiency’ are common.
Identify (a) the values to prioritise, such as ‘efficiency’, ‘equity’, and ‘human dignity’, and (b) ‘instrumental goals’, such as ‘sustainable public finance or political feasibility’, to generate support for solutions.
Instrumental questions may include: Will this intervention produce the intended outcomes? Is it easy to get agreement and maintain support? Will it make me popular, or diminish trust in me even further?
Step 3 is the most simple-looking but difficult task. Remember that it is a political, not technical, process. It is also a political process that most people would like to avoid doing (at least publicly) because it involves making explicit the ways in which we prioritise some people over others. Public policy is the choice to help some people and punish or refuse to help others (and includes the choice to do nothing).
Policy analysis texts describe a relatively simple procedure of identifying criteria and producing a table (with a solution in each row, and criteria in each column) to compare the trade-offs between each solution. However, these criteria are notoriously difficult to define, and people resolve that problem by exercising power to decide what each term means, and whose interests should be served when they resolve trade-offs. For example, see Stone on whose needs come first, who benefits from each definition of fairness, and how technical-looking processes such as ‘cost benefit analysis’ mask political choices.
Right now, the most obvious and visible trade-off, accentuated in the UK, is between individual freedom and collective action, or the balance between state, communal, and market/ individual solutions. In comparison with many countries (and China and Italy in particular), the UK Government seems to be favouring individual action over state quarantine measures. However, most trade-offs are difficult to categorise
What should be the balance between efforts to minimise the deaths of some (generally in older populations) and maximise the wellbeing of others? This is partly about human dignity during crisis, how we treat different people fairly, and the balance of freedom and coercion.
How much should a government spend to keep people alive using intensive case or expensive medicines, when the money could be spent improving the lives of far more people? This is partly about human dignity, the relative efficiency of policy measures, and fairness.
If you are like me, you don’t really want to answer such questions (indeed, even writing them looks callous). If so, one way to resolve them is to elect policymakers to make such choices on our behalf (perhaps aided by experts in moral philosophy, or with access to deliberative forums). To endure, this unusually high level of deference to elected ministers requires some kind of reciprocal act:
"I hope the UK government will be transparent about its decision-making; willing to listen to NHS staff concerns; humble in learning from other countries’ experiences; and pro-active in building relationships with them."https://t.co/CYUyvij2bK
I agree. There is a need to show that divergent opinons in the public health/virology expert sector have been heard, debates have been had and conclusions explained. This is what I need as a citizen. Also casting the public not a bog roll stowing mob is not helpful or kind. https://t.co/g61Nypcqlc
The Guardian calls this document a “secret” briefing from Public Health England. At a time of national crisis there is no place for secrecy from health experts. If you want public support, share your data, scenarios, and forecasts. Now. https://t.co/O8BpDlCJ7H
I am glad Johnson has listened, but we shouldn't have to drag him kicking and screaming to these decisions. A daily update is a basic step. Transparency, honesty, compassion are vital in this time of a global crisis! no more secret briefings PM.https://t.co/eMxZnMehUp
The CSA and CMO say they will publish the models underlying their strategy on Covid-19. Sharing the data and models is important for accountability, testing and learning. https://t.co/rOuJWwy93i
Dear Boris – Number 10 needs a professional communications operation, immediately. (Open letter to the Prime Minister. Britain has some great comms specialists. He needs to hire one of them urgently) https://t.co/8w6MBYHHbm
To all those academics & journalists defending govt longer-term plan- Have you actually seen it yourself? Do you understand triggers for different interventions? Our actions so far do not align with WHO recommendations.
Still, I doubt that governments are making reportable daily choices with reference to a clear and explicit view of what the trade-offs and priorities should be, because their choices are about who will die, and their ability to predict outcomes is limited.
Focus on the outcomes that key actors care about (such as value for money), and quantify and visualise your predictions if possible. Compare the pros and cons of each solution, such as how much of a bad service policymakers will accept to cut costs.
‘Assess the outcomes of the policy options in light of the criteria and weigh trade-offs between the advantages and disadvantages of the options’.
Estimate the cost of a new policy, in comparison with current policy, and in relation to factors such as savings to society or benefits to certain populations. Use your criteria and projections to compare each alternative in relation to their likely costs and benefits.
Explain potential solutions in sufficient detail to predict the costs and benefits of each ‘alternative’ (including current policy).
Short deadlines dictate that you use ‘logic and theory, rather than systematic empirical evidence’ to make predictions efficiently.
Monitoring is crucial because it is difficult to predict policy success, and unintended consequences are inevitable. Try to measure the outcomes of your solution, while noting that evaluations are contested.
It is difficult to envisage a way for the UK Government to publicise the thinking behind its choices (Step 3) and predictions (Step 4) in a way that would encourage effective public deliberation, rather than a highly technical debate between a small number of academics:
Ferguson et al (link) simulate outbreak response. Positive: They show suppression (lockdown R0<1) is essential as mitigation (R0>1, “flattening the curve”) results in massive hospital overload and many dead. BUT 1/3 (review attached)https://t.co/srbBS7F1s5pic.twitter.com/qbEymBdOqm
I’m conscious that lots of people would like to see and run the pandemic simulation code we are using to model control measures against COVID-19. To explain the background – I wrote the code (thousands of lines of undocumented C) 13+ years ago to model flu pandemics…
Further, people often call for the UK Government to publicise its expert advice and operational logic, but I am not sure how they would separate it from their normative logic, or provide a frank account without unintended consequences for public trust or anxiety. If so, government policy involves (a) to keep some choices implicit to avoid a lot of debate on trade-offs, and (b) to make general statements about choices when they do not know what their impact will be.
Examine your case through the eyes of a policymaker. Keep it simple and concise.
Make a preliminary recommendation to inform an iterative process, drawing feedback from clients and stakeholder groups
Client-oriented advisors identify the beliefs of policymakers and tailor accordingly.
‘Unless your client asks you not to do so, you should explicitly recommend one policy’
I now invite you to make a recommendation (step 5) based on our discussion so far (steps 1-4). Define the problem with one framing at the expense of the others. Romanticise some people and not others. Decide how to support some people, and coerce or punish others. Prioritise the lives of some people in the knowledge that others will suffer or die. Do it despite your lack of expertise and profoundly limited knowledge and information. Learn from experts, but don’t assume that only scientific experts have relevant knowledge (decolonise; coproduce). Recommend choices that, if damaging, could take decades to fix after you’ve gone. Consider if a policymaker is willing and able to act on your advice, and if your proposed action will work as intended. Consider if a government is willing and able to bear the economic and political costs. Protect your client’s popularity, and trust in your client, at the same time as protecting lives. Consider if your advice would change if the problem would seem to change. If you are writing your analysis, maybe keep it down to one sheet of paper (and certainly far fewer words than in this post). Better you than me.
Please now watch this video before I suggest that things are not so simple.
Would that policy analysis were so simple
Imagine writing policy analysis in an imaginary world, in which there is a single powerful ‘rational’ policymaker at the heart of government, making policy via an orderly series of stages.
Your audience would be easy to identify at each stage, your analysis would be relatively simple, and you would not need to worry about what happens after you make a recommendation for policy change (since the selection of a solution would lead to implementation). You could adopt a simple 5 step policy analysis method, use widely-used tools such as cost-benefit analysis to compare solutions, and know where the results would feed into the policy process.
Studies of policy analysts describe how unrealistic this expectation tends to be (Radin, Brans, Thissen).
For example, there are many policymakers, analysts, influencers, and experts spread across political systems, and engaging with 101 policy problems simultaneously, which suggests that it is not even clear how everyone fits together and interacts in what we call (for the sake of simplicity) ‘the policy process’.
Instead, we can describe real world policymaking with reference to two factors.
The wider policymaking environment: 1. Limiting the use of evidence
First, policymakers face ‘bounded rationality’, in which they only have the ability to pay attention to a tiny proportion of available facts, are unable to separate those facts from their values (since we use our beliefs to evaluate the meaning of facts), struggle to make clear and consistent choices, and do not know what impact they will have. The consequences can include:
Limited attention, and lurches of attention. Policymakers can only pay attention to a tiny proportion of their responsibilities, and policymaking organizations struggle to process all policy-relevant information. They prioritize some issues and information and ignore the rest.
Power and ideas. Some ways of understanding and describing the world dominate policy debate, helping some actors and marginalizing others.
Beliefs and coalitions. Policymakers see the world through the lens of their beliefs. They engage in politics to turn their beliefs into policy, form coalitions with people who share them, and compete with coalitions who don’t.
Dealing with complexity. They engage in ‘trial-and-error strategies’ to deal with uncertain and dynamic environments (see the new section on trial-and-error- at the end).
Framing and narratives. Policy audiences are vulnerable to manipulation when they rely on other actors to help them understand the world. People tell simple stories to persuade their audience to see a policy problem and its solution in a particular way.
The social construction of populations. Policymakers draw on quick emotional judgements, and social stereotypes, to propose benefits to some target populations and punishments for others.
Rules and norms. Institutions are the formal rules and informal understandings that represent a way to narrow information searches efficiently to make choices quickly.
Learning. Policy learning is a political process in which actors engage selectively with information, not a rational search for truth.
Evidence-based or expert-informed policymaking
Put simply, policymakers cannot oversee a simple process of ‘evidence-based policymaking’. Rather, to all intents and purposes:
They need to find ways to ignore most evidence so that they can focus disproportionately on some. Otherwise, they will be unable to focus well enough to make choices. The cognitive and organisational shortcuts, described above, help them do it almost instantly.
They also use their experience to help them decide – often very quickly – what evidence is policy-relevant under the circumstances. Relevance can include:
How it relates to the policy problem as they define it (Step 1).
If it relates to a feasible solution (Step 2).
If it is timely, available, understandable, and actionable.
If it seems credible, such as from groups representing wider populations, or from people they trust.
They use a specific shortcut: relying on expertise.
However, the vague idea of trusting or not trusting experts is a nonsense, largely because it is virtually impossible to set a clear boundary between relevant/irrelevant experts and find a huge consensus on (exactly) what is happening and what to do. Instead, in political systems, we define the policy problem or find other ways to identify the most relevant expertise and exclude other sources of knowledge.
In the UK Government’s case, it appears to be relying primarily on expertise from its own general scientific advisers, medical and public health advisers, and – perhaps more controversially – advisers on behavioural public policy.
Right now, it is difficult to tell exactly how and why it relies on each expert (at least when the expert is not in a clearly defined role, in which case it would be irresponsible not to consider their advice). Further, there are regular calls on Twitter for ministers to be more open about their decisions.
Key point from @jameswilsdon 'It is problematic if political choices are being made and then the science advice system has to front them up. There needs to be a clearer sense of where science advice ends and political judgement begins.'https://t.co/TjLCJDZijO via @timeshighered
However, don’t underestimate the problems of identifying why we make choices, then justifying one expert or another (while avoiding pointless arguments), or prioritising one form of advice over another. Look, for example, at the kind of short-cuts that intelligent people use, which seem sensible enough, but would receive much more intense scrutiny if presented in this way by governments:
Sophisticated speculation by experts in a particular field, shared widely (look at the RTs), but questioned by other experts in another field:
2. This all assumes I'm correct in what I think the govt are doing and why. I could be wrong – and wouldn't be surprised. But it looks to me like. . .
— Professor Ian Donald (@iandonald_psych) March 13, 2020
As many have said, it would be good to get an official version of this, with acknowledged uncertainties and sources of evidence https://t.co/jxgoysYb3L
But what happened is that they have as a group fallen into a logical error in their attempts to model what will bring this epidemic under control. They have not appreciated that the answer to this question is adaptive behavior change. 3/17
It would be really helpful to project risk of covid death with and without mitigation strategies? Possible to map with inside / outside projections (ie what we gain/ don’t gain with current measures ?)
Experts in one field trusting certain experts in another field based on personal or professional interaction:
Lots of concern about UK's approach to #COVID19. I'm not an epidemiologist or a virologist (=> can't judge the detail) but I knew Patrick Vallance before he was famous and I believe he is a man of integrity. Same for Chris Whitty. Science, not politics, is driving their thinking.
Experts in one field not trusting a government’s approach based on its use of one (of many) sources of advice:
Why is UK government listening to the ‘nudge unit’ on the pandemic, and not expert epidemiologists and the WHO? You would think the ‘anti-experts’ approach would have at least on this occasion, with so many lives at risk, given way to a scientific approach https://t.co/QZIicXYpsj
Experts representing a community of experts, criticising another expert, for misrepresenting the amount of expert scepticism of government experts (yes, I am trying to confuse you):
The Chief Medical Officer @CMO_England and his team have the 100% support and backing of the Public Health community. Every DPH I know thinks he is doing an amazing job in difficult circumstances Sorry but JRA is just demonstrating he is out of touch on this https://t.co/ExmOjEgum0
Expert debate on how well policymakers are making policy based on expert advice
Disagree.
Not much audible consensus amongst scientists anywhere for UK approach. Science can only illuminate value judgements yet now used a shield for determining them. UK science advice has always been characterised by old boys, political operators. Blurring is concerning. https://t.co/iBt07QfvqH
Finding quite-sensible ways to trust certain experts over others, such as because they can be held to account in some way (and may be relatively worried about saying any old shit on the internet):
My current approach to making sense of conflicting expert opinion on #coronavirus: no expert is infallible, but some are accountable and others are not, and I will value the opinions of those who are accountable above the opinions of those who are not.
There are many more examples in which the shortcut to expertise is fine, but not particularly better than another shortcut (and likely to include a disproportionately high number of white men with STEM backgrounds).
Update: of course, they are better than the volume trumps expertise approach:
This meme is spreading (you could say, in a not very funny joke, that it has gone viral). The WHO Director-General did not say this (brief thread). https://t.co/3eMfy70tKZ
For what it’s worth, I tend to favour experts who:
(a) establish the boundaries of their knowledge, (b) admit to high uncertainty about the overall problem:
After having spent considerable time thinking how to mitigate and manage this pandemic, and analysing the available data. I failed to identify the best course of action. Even worse, I'm not sure there is such a thing as an acceptable solution to the problem we are facing. (2/12)
(c) (in this case) make it clear that they are working on scenarios, not simple prediction
I am deeply uncomfortable with the message that UK is actively pursuing ‘herd immunity’ as the main COVID-19 strategy. Our group’s scenario modelling has focused on reducing two main things: peak healthcare demand and deaths… 1/
(d) examine critically the too-simple ideas that float around, such as the idea that the UK Government should emulate ‘what works’ somewhere else
It's easy to say 'let's just do what Wuhan did', but the measures there have involved a change to daily life that really has been unimaginable in scale and impact. And as we've seen, China cannot sustain them indefinitely. 3/
A lot of my colleagues in the @LSHTM modelling centre (@cmmid_lshtm) have been working extremely hard to help expand the COVID-19 evidence base over the past two months. I'd like to take a moment to highlight some of their work… 1/
8. There's no gotcha-ism. Updating your models and predictions in light of new evidence and new inferential methods and insightful counterpoints from colleagues isn't a sign of weakness, it's *doing science*.
(e) situate their own position (in Prof Sridhar’s case, for mass testing) within a broader debate
Scientific community is well-intentioned but split in two camps: one argues why sacrifice short-term social/economic well-being if everyone will get virus regardless, & other which says we have to buy time in short-term & save lives now while figuring out exit plan.
How much effort does your govt want to put into suppressing this outbreak? There is no quick fix or easy solution. S.Korea & Germany show what huge govt effort, planning, strong leadership, & doing utmost to protect population look like. Do everything v. do minimum.
Been saying 3 objectives for weeks. Not to attack anyone, but to highlight what we have learned so far: 1. Testing, tracing, isolating 2. Protect health workers with PPE & testing 3. Buy time for NHS
However, note that most of these experts are from a very narrow social background, and from very narrow scientific fields (first in modelling, then likely in testing), despite the policy problem being largely about (a) who, and how many people, a government should try to save, and (b) how far a government should go to change behaviour to do it. It is understandable to defer in this way during a crisis, but it also contributes to a form of ‘depoliticisation’ that masks profound choices that benefit some people and leave others vulnerable to harm.
He says there is not as much scrutiny as we might normally wish and says concerns raised about human rights, the length of powers and need for safeguards should be heeded in Westminster. He also commits to legislate for reporting requirements for use of powers by SG 4/5
The wider policymaking environment: 2. Limited control
Second, policymakers engage in a messy and unpredictable world in which no single ‘centre’ has the power to turn a policy recommendation into an outcome. I normally use the following figure to think through the nature of a complex and unwieldy policymaking environment of which no ‘centre’ of government has full knowledge or control.
It helps us identify (further) the ways in which we can reject the idea that the UK Prime Minister and colleagues can fully understand and solve policy problems:
Actors. The environment contains many policymakers and influencers spread across many levels and types of government (‘venues’).
For example, consider how many key decisions that (a) have been made by organisations not in the UK central government, and (b) are more or less consistent with its advice, including:
Devolved governments announcing their own healthcare and public health responses.
Public sector employers initiating or encouraging at-home working (and many Universities moving quickly from in-person to online teaching)
Private organisations cancelling cultural and sporting events.
There’s some coverage today suggesting Scotland proposing different policy to rest of UK on over 70s. This isn’t so. The policy of social distancing, not isolation, set out here by @jasonleitch is the policy all 4 nations have been discussing at COBR – and will do so again today. https://t.co/D89nwUDZTb
Context and events. Policy solutions relate to socioeconomic context and events which can be impossible to ignore and out of the control of policymakers. The coronavirus, and its impact on so many aspects on population health and wellbeing, is an extreme example of this problem.
Networks, Institutions, and Ideas. Policymakers and influencers operate in subsystems (specialist parts of political systems). They form networks or coalitions built on the exchange of resources or facilitated by trust underpinned by shared beliefs or previous cooperation. Many different parts of government have practices driven by their own formal and informal rules. Formal rules are often written down or known widely. Informal rules are the unwritten rules, norms and practices that are difficult to understand, and may not even be understood in the same way by participants. Political actors relate their analysis to shared understandings of the world – how it is, and how it should be – which are often so established as to be taken for granted. These dominant frames of reference establish the boundaries of the political feasibility of policy solutions. These kinds of insights suggest that most policy decisions are considered, made, and delivered in the name of – but not in the full knowledge of – government ministers.
Trial and error policymaking in complex policymaking systems (17.3.20)
One way of viewing the UK's COVID-19 policy is that it changed to reflect changing evidence. That is fair; it's both how science-guided policy *should* work, and how I think the govt's advisors are seeing it, as per the Imperial paper. But… 1/
There are many ways to conceptualise this policymaking environment, but few theories provide specific advice on what to do, or how to engage effectively in it. One notable exception is the general advice that comes from complexity theory, including:
Law-like behaviour is difficult to identify – so a policy that was successful in one context may not have the same effect in another.
Policymaking systems are difficult to control; policy makers should not be surprised when their policy interventions do not have the desired effect.
Policy makers in the UK have been too driven by the idea of order, maintaining rigid hierarchies and producing top-down, centrally driven policy strategies. An attachment to performance indicators, to monitor and control local actors, may simply result in policy failure and demoralised policymakers.
Policymaking systems or their environments change quickly. Therefore, organisations must adapt quickly and not rely on a single policy strategy.
On this basis, there is a tendency in the literature to encourage the delegation of decision-making to local actors:
Rely less on central government driven targets, in favour of giving local organisations more freedom to learn from their experience and adapt to their rapidly-changing environment.
To deal with uncertainty and change, encourage trial-and-error projects, or pilots, that can provide lessons, or be adopted or rejected, relatively quickly.
Encourage better ways to deal with alleged failure by treating ‘errors’ as sources of learning (rather than a means to punish organisations) or setting more realistic parameters for success/ failure (although see this example and this comment).
Encourage a greater understanding, within the public sector, of the implications of complex systems and terms such as ‘emergence’ or ‘feedback loops’.
In other words, this literature, when applied to policymaking, tends to encourage a movement from centrally driven targets and performance indicators towards a more flexible understanding of rules and targets by local actors who are more able to understand and adapt to rapidly-changing local circumstances.
Now, just imagine the UK Government taking that advice right now. I think it is fair to say that it would be condemned continuously (even more so than right now). Maybe that is because it is the wrong way to make policy in times of crisis. Maybe it is because too few people are willing and able to accept that the role of a small group of people at the centre of government is necessarily limited, and that effective policymaking requires trial-and-error rather than a single, fixed, grand strategy to be communicated to the public. The former highlights policy that changes with new information and perspective. The latter highlights errors of judgement, incompetence, and U-turns. In either case, the advice is changing as estimates of the coronavirus’ impact change:
I think this tension, in the way that we understand UK government, helps explain some of the criticism that it faces when changing its advice to reflect changes in its data or advice. This criticism becomes intense when people also question the competence or motives of ministers (and even people reporting the news) more generally, leading to criticism that ranges from mild to outrageous:
Incredible detail in this FT story: up until last week, the UK was basing its coronavirus control policy on a model based on hospitalisation rates for 😲a different disease😲 with lower rates of intensive care need than coronavirus pic.twitter.com/7rJYh9sqg2
Laura Kuenssberg says (BBC) that, “The science has changed.” This is not true. The science has been the same since January. What has changed is that govt advisors have at last understood what really took place in China and what is now taking place in Italy. It was there to see.
We can’t keep changing our #COVID19 control policies whenever the results of the “mathematical modelling” change. We need to implement standard WHO-approved epidemic control policies hard and fast, as well as providing more support to frontline NHS staff. https://t.co/HAM9OqbmqW
There may be perfectly valid or at least debatable reasons for each but obfuscation does not help public to understand uncertainty around decisions. In other words, not communicating rationale = incompetence (as in incompetent in terms of state craft, not nec individual decision)
One wonders if Brit leaders have decided that the ultimate way to cut national budgets is to cull the herd of the weak, those who require costly NHS care, and pray for "herd immunity" among the rest. Cruel, cost effective #COVID19 strategy?@richardhorton1
For me, this casual reference to a government policy to ‘cull the heard of the weak’ is outrageous, but you can find much worse on Twitter. It reflects wider debate on whether ‘herd immunity’ is or is not government policy. Much of it relates to interpretation of government statements, based on levels of trust/distrust in the UK Government, its Prime Minister and Secretaries of State, and the Prime Minister’s special adviser
I have enormous respect for the SAGE team and scientific advisors trying to understand the situation & inform the UK's response. If this article is accurate & partisan hacks were deliberately sacrificing lives based on their information, its scandalous. A week ago I was saying… https://t.co/WYsHbj6o0a
If you read the whole article you will see that Dominic Cummings has been, for the last 10 days, the most zealous advocate of a tough lockdown. Which is what his critics seem to want. The world is not black and white
1. Wilful misinterpretation (particularly on Twitter). For example, in the early development and communication of policy, Boris Johnson was accused (in an irresponsibly misleading way) of advocating for herd immunity rather than restrictive measures.
Below is one of the most misleading videos of its type. Look at how it cuts each segment into a narrative not provided by ministers or their advisors (see also this stinker):
The herd immunity strategy would’ve likely caused hundreds of thousands of deaths. They even told us so.
— Shamil Esq, Bring on 2020! (@Shamils18) March 26, 2020
2. The accentuation of a message not being emphasised by government spokespeople.
See for example this interview, described by Sky News as: The government’s chief scientific adviser Sir Patrick Vallance has told Sky News that about 60% of people will need to become infected with coronavirus in order for the UK to enjoy “herd immunity”. You might be forgiven for thinking that he was on Sky extolling the virtues of a strategy to that end (and expressing sincere concerns on that basis). Yet, he was saying nothing of the sort. Rather, when prompted, he discussed herd immunity in relation to the belief that COVID-19 will endure long enough to become as common as seasonal flu.
A more careful forensic analysis will try to relate each government choice to the ways in which key advisory bodies (such as the New and Emerging Respiratory Virus Threats Advisory Group, NERVTAG) received and described evidence on the current nature of the problem:
For anyone who thinks it was all obvious in January and February reading these minutes is a sobering experience. What comes over is the real uncertainty about what could be foretold from the Chinese experience and the ease with which the disease could be transmitted.4/n
In some cases, maybe people are making the argument that trial-and-error is the best way to respond quickly, and adapt quickly, in a crisis but that the UK Government version is not what, say, the WHO thinks of as good kind of adaptive response. It is not possible to tell, at least from the general ways in which they justify acting quickly.
Dr Michael J Ryan, Executive Director at WHO. An off the cuff answer to a question at today's virtual press conference. Inspiring stuff! pic.twitter.com/Q4EUs8V1dG
The coronavirus is an extreme example of a general situation: policymakers will always have very limited knowledge of policy problems and control over their policymaking environment. They make choices to frame problems narrowly enough to seem solvable, rule out most solutions as not feasible, make value judgements to try help some more than others, try to predict the results, and respond when the results to not match their hopes or expectations.
This is not a message of doom and despair. Rather, it encourages us to think about how to influence government, and hold policymakers to account, in a thoughtful and systematic way that does not mislead the public or exacerbate the problem we are seeing.
Further reading, until I can think of a better conclusion:
This series of ‘750 words’ posts summarises key texts in policy analysis and tries to situate policy analysis in a wider political and policymaking context. Note the focus on whose knowledge counts, which is not yet a big feature of this crisis.
These series of 500 words and 1000 words posts (with podcasts) summarise concepts and theories in policy studies.
#politvirus Public Health has always been #political because it’s actions impact on politics, economics, commercial interests, personal freedoms – this becomes most obvious in crisis – it will be key to analyse the political responses to #Covid_19 if we want to be better prepared https://t.co/JkUZrVeAxv
An assessment of the Government's response to date – written by Chair of Global Health at Edinburgh University..Prof Devi Sridhar https://t.co/N31QtFmQ2p
This is a really important paper. Partisanship is a huge influence on timing of state public health measures- Republican governors and Trump majorities slow adoption of measures. This might have big mortality effects in a few weeks. https://t.co/BEOAM69aSw
One reason Germany has so many ventilators (and intensive care beds) given in The Times: Not just more money in the system but design of hospital payment rates through the insurance system has driven up ICU investment be hospital managers pic.twitter.com/7R062IJI2k
I wish the late great Mick Moran were still around – it feels like the next chapter of his analysis of the modern British state urgently needs to be written. https://t.co/ffxegGKVCu
I’m writing a book about @ExtinctionR. Here are some thoughts about today’s controversy. 1. This may or may not be a legit XR group. 2. That may matter because it may be done in order to smear XR & climate activism generally 1/n https://t.co/NyQhbv53a3
Cautionary words for anyone tempted to say "this must be good for the climate" or, worse, "this shows we can tackle climate change".
COVID19 is a re-framing of the climate issues – a dramatically changed context for the response – but those climate issues haven't gone away. https://t.co/gixVwnk6gq
‘Policy analysis’ describes the identification of a policy problem and possible solutions.
Classic models of policy analysis are client-oriented. Most texts identify the steps that a policy analysis should follow, from identifying a problem and potential solutions, to finding ways to predict and evaluate the impact of each solution. Each text describes this process in different ways, as outlined in Boxes 1-5. However, for the most part, they follow the same five steps:
Define a policy problem identified by your client.
Identify technically and politically feasible solutions.
Use value-based criteria and political goals to compare solutions.
Predict the outcome of each feasible solution.
Make a recommendation to your client.
Further, they share the sense that analysts need to adapt pragmatically to a political environment. Assume that your audience is not an experienced policy analyst. Assume a political environment in which there is limited attention or time to consider problems, and some policy solutions will be politically infeasible. Describe the policy problem for your audience: to help them see it as something worthy of their energy. Discuss a small number of possible solutions, the differences between them, and their respective costs and benefits. Keep it short with the aid of visual techniques that sum up the issue concisely, to minimise cognitive load and make the problem seem solvable.
Box 1. Bardach (2012) A Practical Guide for Policy Analysis
‘Define the problem’. Provide a diagnosis of a policy problem, using rhetoric and eye-catching data to generate attention.
‘Assemble some evidence’. Gather relevant data efficiently.
‘Construct the alternatives’. Identify the relevant and feasible policy solutions that your audience might consider.
‘Select the criteria’. Typical value judgements relate to efficiency, equity and fairness, the trade-off between individual freedom and collective action, and the extent to which a policy process involves citizens in deliberation.
‘Project the outcomes’. Focus on the outcomes that key actors care about (such as value for money), and quantify and visualise your predictions if possible.
‘Confront the trade-offs’. Compare the pros and cons of each solution, such as how much of a bad service policymakers will accept to cut costs.
‘Decide’. Examine your case through the eyes of a policymaker.
‘Tell your story’. Identify your target audience and tailor your case. Weigh up the benefits of oral versus written presentation. Provide an executive summary. Focus on coherence and clarity. Keep it simple and concise. Avoid jargon.
Box 2. Dunn (2017) Public Policy Analysis
What is the policy problem to be solved? Identify its severity, urgency, cause, and our ability to solve it. Don’t define the wrong problem, such as by oversimplifying.
What effect will each potential policy solution have? ‘Forecasting’ methods can help provide ‘plausible’ predictions about the future effects of current/ alternative policies.
Which solutions should we choose, and why? Normative assessments are based on values such as ‘equality, efficiency, security, democracy, enlightenment’ and beliefs about the preferable balance between state, communal, and market/ individual solutions (2017: 6; 205).
What were the policy outcomes? ‘Monitoring is crucial because it is difficult to predict policy success, and unintended consequences are inevitable (2017: 250).
Did the policy solution work as intended? Did it improve policy outcomes? Try to measure the outcomes your solution, while noting that evaluations are contested (2017: 332-41).
Box 3. Meltzer and Schwartz (2019) Policy Analysis as Problem Solving
‘Define the problem’. Problem definition is a political act of framing, as part of a narrative to evaluate the nature, cause, size, and urgency of an issue.
‘Identify potential policy options (alternatives) to address the problem’. Identify many possible solutions, then select the ‘most promising’ for further analysis (2019: 65).
‘Specify the objectives to be attained in addressing the problem and the criteria to evaluate the attainment of these objectives as well as the satisfaction of other key considerations (e.g., equity, cost, equity, feasibility)’.
‘Assess the outcomes of the policy options in light of the criteria and weigh trade-offs between the advantages and disadvantages of the options’.
‘Arrive at a recommendation’. Make a preliminary recommendation to inform an iterative process, drawing feedback from clients and stakeholder groups (2019: 212).
‘Engage in problem definition’. Define the nature of a policy problem, and the role of government in solving it, while engaging with many stakeholders (2012: 3; 58-60).
‘Propose alternative responses to the problem’. Identify how governments have addressed comparable problems, and a previous policy’s impact (2012: 21).
‘Choose criteria for evaluating each alternative policy response’. ‘Effectiveness, efficiency, fairness, and administrative efficiency’ are common (2012: 21).
‘Project the outcomes of pursuing each policy alternative’. Estimate the cost of a new policy, in comparison with current policy, and in relation to factors such as savings to society or benefits to certain populations.
‘Identify and analyse trade-offs among alternatives’. Use your criteria and projections to compare each alternative in relation to their likely costs and benefits.
‘Report findings and make an argument for the most appropriate response’. Client-oriented advisors identify the beliefs of policymakers and tailor accordingly (2012: 22).
Box 5 Weimer and Vining (2017) Policy Analysis: Concepts and Practice
‘Write to Your Client’. Having a client such as an elected policymaker requires you to address the question they ask, by their deadline, in a clear and concise way that they can understand (and communicate to others) quickly (2017: 23; 370-4).
‘Understand the Policy Problem’. First, ‘diagnose the undesirable condition’. Second, frame it as ‘a market or government failure (or maybe both)’.
‘Be Explicit About Values’ (and goals). Identify (a) the values to prioritise, such as ‘efficiency’, ‘equity’, and ‘human dignity’, and (b) ‘instrumental goals’, such as ‘sustainable public finance or political feasibility’, to generate support for solutions.
‘Specify Concrete Policy Alternatives’. Explain potential solutions in sufficient detail to predict the costs and benefits of each ‘alternative’ (including current policy).
‘Predict and Value Impacts’. Short deadlines dictate that you use ‘logic and theory, rather than systematic empirical evidence’ to make predictions efficiently (2017: 27)
‘Consider the Trade-Offs’. Each alternatives will fulfil certain goals more than others. Produce a summary table to make value-based choices about trade-offs (2017: 356-8).
‘Make a Recommendation’. ‘Unless your client asks you not to do so, you should explicitly recommend one policy’ (2017: 28).
Incisive essay from @bailabomba on studying the use of research evidence through critical perspectives that center the marginalized. There is so too much good stuff in here to summarize via twitter (you should just read it). But let me point out a few things that resonated (1/n) https://t.co/nIahyIjwBo
Research and policy analysis for marginalized groups
For Doucet (2019: 1), it begins by describing the William T. Grant Foundation’s focus on improving the ‘use of research evidence’ (URE), and the key questions that we should ask when improving URE:
For what purposes do policymakers find evidence useful?
For example, usefulness could be defined by the researchers providing evidence, the policymakers using it, the stakeholders involved in coproduction, or the people affected by research and policy (compare with Bacchi, Stone and Who should be involved in the process of policy analysis?).
How do critical theories inform these questions? (compare with T. Smith)
First, they remind us that so-called ‘rational’ policy processes have incorporated research evidence to help:
‘maintain power hierarchies and accept social inequity as a given. Indeed, research has been historically and contemporaneously (mis)used to justify a range of social harms from enslavement, colonial conquest, and genocide, to high-stakes testing, disproportionality in child welfare services, and “broken windows” policing’ (Doucet, 2019: 2)
Second, they help us redefine usefulness in relation to:
‘how well research evidence communicates the lived experiences of marginalized groups so that the understanding of the problem and its response is more likely to be impactful to the community in the ways the community itself would want’ (Doucet, 2019: 3)
In that context, potential responses include to:
Recognise the ways in which research and policy combine to reproduce the subordination of social groups.
Specific mechanism include: judging marginalised groups harshly according to ‘Western, educated, industrialized, rich and democratic’ norms (‘WEIRD’)
Reject the idea that scientific research can be seen as objective or neutral (and that researchers are beyond reproach for their role in subordination).
Give proper recognition to ‘experiential knowledge’ and ‘transdiciplinary approaches’ to knowledge production, rather than privileging scientific knowledge.
Commit to social justice, to help ‘eliminate oppressions and to emancipate and empower marginalized groups’, such as by disrupting ‘the policies and practices that disproportionately harm marginalized groups’ (2019: 5-7)
Develop strategies to ‘center race’, ‘democratize’ research production, and ‘leverage’ transdisciplinary methods (including poetry, oral history and narrative, art, and discourse analysis – compare with Lorde) (2019: 10-22)
Policy analysis in a ‘racialized polity’
A key way to understand these processes is to use, and improve, policy theories to explain the dynamics and impacts of a racialized political system. For example, ‘policy feedback theory’ (PFT) draws on elements from historical institutionalism and SCPD to identify the rules, norms, and practices that reinforce subordination.
In particular, Michener’s (2019: 424) ‘Policy Feedback in a Racialized Polity’ develops a ‘racialized feedback framework (RFF)’ to help explain the ‘unrelenting force with which racism and White supremacy have pervaded social, economic, and political institutions in the United States’. Key mechanisms include (2019: 424-6):
‘Channelling resources’, in which the rules, to distribute government resources, benefit some social groups and punish others.
Examples include: privileging White populations in social security schemes and the design/ provision of education, and punishing Black populations disproportionately in prisons (2019: 428-32).
‘Generating interests’, in which ‘racial stratification’ is a key factor in the power of interest groups (and balance of power in them).
‘Shaping interpretive schema’, in which race is a lens through which actors understand, interpret, and seek to solve policy problems.
The ways in which centralization (making policy at the federal level) or decentralization influence policy design.
For example, the ‘historical record’ suggests that decentralization is more likely to ‘be a force of inequality than an incubator of power for people of color’ (2019: 433).
Insufficient attention to race and racism: what are the implications for policy analysis?
One potential consequence of this lack of attention to race, and the inequalities caused by racism in policy, is that we place too much faith in the vague idea of ‘pragmatic’ policy analysis.
Throughout the 750 words series, you will see me refer generally to the benefits of pragmatism:
In that context, pragmatism relates to the idea that policy analysis consists of ‘art and craft’, in which analysts assess what is politically feasible if taking a low-risk client-oriented approach.
In this context, pragmatism may be read as a euphemism for conservatism and status quo protection.
In other words, other posts in the series warn against too-high expectations for entrepreneurial and systems thinking approaches to major policy change, but they should not be read as an excuse to reject ambitious plans for much-needed changes to policy and policy analysis (compare with Meltzer and Schwartz, who engage with this dilemma in client-oriented advice).
This post forms one part of the Policy Analysis in 750 words series overview and connects to previous posts on complexity.The first 750 words tick along nicely, then there is a picture of a cat hanging in there baby to signal where it can all go wrong.
There are a million-and-one ways to describe systems and systems thinking. These terms are incredibly useful, but also at risk of meaning everything and therefore nothing (compare with planning and consultation).
We need to acknowledge these limitations properly, to accept our limitations, and avoid the mechanistic language of ‘policy levers’ which exaggerate human or government control.
Complex systems thinking could be the future of policymaking.
Six meanings of complex systems in policy and policymaking
Let’s begin by trying to clarify many meanings of complex system and relate them to systems thinking storylines.
For example, you will encounter three different meanings of complex system in this series alone, and each meaning presents different implications for systems thinking:
Policy outcomes seem to ‘emerge’ from policymaking systems in the absence of central government control. As such, we should rely less on central government driven targets (in favour of local discretion to adapt to environments), encourage trial-and-error learning, and rethink the ways in which we think about government ‘failure’ (see, for example, Hallsworth on ‘system stewardship’, the OECD on ‘Systemic Thinking for Policy Making‘, and this thread)
Systems thinking is about learning and adapting to the limits to policymaker control.
Dunn (2017: 73) describes the interdependent nature of problems:
“Subjectively experienced problems – crime, poverty, unemployment, inflation, energy, pollution, health, security – cannot be decomposed into independent subsets without running the risk of producing an approximately right solution to the wrong problem. A key characteristic of systems of problems is that the whole is greater – that is, qualitatively different – than the simple sum of its parts” (contrast with Meltzer and Schwartz on creating a ‘boundary’ to make problems seem solveable).
Systems thinking is about addressing policy problems holistically.
Used to explain the transition from unsustainable to sustainable energy systems.
Systems thinking is about identifying the role of new technologies, protected initially in a ‘niche’, and fostered by a supportive ‘social and political environment’.
Systems thinking is about projecting the sense that (a) policy and policymaking is complicated, but (b) governments can still look like they are in control.
Four more meanings of systems thinking
Now, let’s compare these storylines with a small sample of wider conceptions of systems thinking:
Systems thinking was about the human ability to turn potential chaos into well-managed systems (such as ‘large technical systems’ to distribute energy)
The new way of accepting complexity but seeking to make an impact
Based on the idea that we can identify ‘leverage points’, or the places that help us ‘intervene in a system’ (see Meadows then compare with Arnold and Wade).
Systems thinking is about the human ability to use a small shift in a system to produce profound changes in that system.
A way to rethink cause-and-effect
Based on the idea that current research methods are too narrowly focused on linearity rather than the emergent properties of systems of behaviour (for example, Rutter et al on how to analyse the cumulative effect of public health interventions).
Systems thinking is about rethinking the ways in which governments, funders, or professions conduct policy-relevant research on social behaviour.
How can we clarify systems thinking and use it effectively in policy analysis?
Now, imagine you are in a room of self-styled systems thinkers, and that no-one has yet suggested a brief conversation to establish what you all mean by systems thinking. I reckon you can make a quick visual distinction by seeing who looks optimistic.
I’ll be the morose-looking guy sitting in the corner, waiting to complain about ambiguity, so you would probably be better off sitting next to Luke Craven who still ‘believes in the power of systems thinking’.
If you can imagine some amalgam of these pessimistic/ optimistic positions, perhaps the conversation would go like this:
Reasons to expect some useful collaboration.
Some of these 10 discussions seem to complement each other. For example:
We can use 3 and 9 to reject one narrow idea of ‘evidence-based policymaking’, in which the focus is on (a) using experimental methods to establish cause and effect in relation to one policy instrument, without showing (b) the overall impact on policy and outcomes (e.g. compare FNP with more general ‘families’ policy).
1-3 and 10 might be about the need for policy analysts to show humility when seeking to understand and influence complex policy problems, solutions, and policymaking systems.
In other words, you could define systems thinking in relation to the need to rethink the ways in which we understand – and try to address – policy problems. If so, you can stop here and move on to the next post. There is no benefit to completing this post.
Reasons to expect the same old frustrating discussions based on no-one defining terms well enough (collectively) to collaborate effectively (beyond using the same buzzwords).
Although all of these approaches use the language of complex systems and systems thinking, note some profound differences:
Holding on versus letting go.
Some are about intervening to take control of systems or, at least, make a disproportionate difference from a small change.
Some are about accepting our inability to understand, far less manage, these systems.
Talking about different systems.
Some are about managing policymaking systems, and others about social systems (or systems of policy problems), without making a clear connection between both endeavours.
For example, if you use approach 9 to rethink societal cause-and-effect, are you then going to pretend that you can use approach 7 to do something about it? Or, will our group have a difficult discussion about the greater likelihood of 6 (metaphorical policymaking) in the context of 1 (the inability of governments to control the policymaking systems we need to solve the problems raised by 9).
In that context, the reason that I am sitting in the corner, looking so morose, is that too much collective effort goes into (a) restating, over and over and over again, the potential benefits of systems thinking, leaving almost no time for (b) clarifying systems thinking well enough to move on to these profound differences in thinking. Systems thinking has not even helped us solve these problems with systems thinking.
Throughout this series you may notice three different conceptions about the scope of policy analysis:
‘Ex ante’ (before the event) policy analysis. Focused primarily on defining a problem, and predicting the effect of solutions, to inform current choice (as described by Meltzer and Schwartz and Thissen and Walker).
‘Ex post’ (after the event) policy analysis. Focused primarily on monitoring and evaluating that choice, perhaps to inform future choice (as described famously by Weiss).
Some combination of both, to treat policy analysis as a continuous (never-ending) process (as described by Dunn).
As usual, these are not hard-and-fast distinctions, but they help us clarify expectations in relation to different scenarios.
The impact of old-school ex ante policy analysis
Radin provides a valuable historical discussion of policymaking with the following elements:
a small number of analysts, generally inside government (such as senior bureaucrats, scientific experts, and – in particular- economists),
giving technical or factual advice,
about policy formulation,
to policymakers at the heart of government,
on the assumption that policy problems would be solved via analysis and action.
This kind of image signals an expectation for high impact: policy analysts face low competition, enjoy a clearly defined and powerful audience, and their analysis is expected to feed directly into choice.
Radin goes on to describe a much different, modern policy environment: more competition, more analysts spread across and outside government, with a less obvious audience, and – even if there is a client – high uncertainty about where the analysis fits into the bigger picture.
Yet, the impetus to seek high and direct impact remains.
This combination of shifting conditions but unshifting hopes/ expectations helps explain a lot of the pragmatic forms of policy analysis you will see in this series, including:
Keep it catchy, gather data efficiently, tailor your solutions to your audience, and tell a good story (Bardach)
Speak with an audience in mind, highlight a well-defined problem and purpose, project authority, use the right form of communication, and focus on clarity, precision, conciseness, and credibility ( Smith)
Address your client’s question, by their chosen deadline, in a clear and concise way that they can understand (and communicate to others) quickly (Weimer and Vining)
Client-oriented advisors identify the beliefs of policymakers and anticipate the options worth researching (Mintrom)
Identify your client’s resources and motivation, such as how they seek to use your analysis, the format of analysis they favour (make it ‘concise’ and ‘digestible’), their deadline, and their ability to make or influence the policies you might suggest (Meltzer and Schwartz).
‘Advise strategically’, to help a policymaker choose an effective solution within their political context (Thissen and Walker).
Focus on producing ‘policy-relevant knowledge’ by adapting to the evidence-demands of policymakers and rejecting a naïve attachment to ‘facts speaking for themselves’ or ‘knowledge for its own sake’ (Dunn).
The impact of research and policy evaluation
Many of these recommendations are familiar to scientists and researchers, but generally in the context of far lower expectations about their likely impact, particularly if those expectations are informed by policy studies (compare Oliver & Cairney with Cairney & Oliver).
In that context, Weiss’ work is a key reference point. It gives us a menu of ways in which policymakers might use policy evaluation (and research evidence more widely):
to inform solutions to a problem identified by policymakers
as one of many sources of information used by policymakers, alongside ‘stakeholder’ advice and professional and service user experience
as a resource used selectively by politicians, with entrenched positions, to bolster their case
as a tool of government, to show it is acting (by setting up a scientific study), or to measure how well policy is working
as a source of ‘enlightenment’, shaping how people think over the long term (compare with this discussion of ‘evidence based policy’ versus ‘policy based evidence’).
In other words, researchers may have a role, but they struggle (a) to navigate the politics of policy analysis, (b) find the right time to act, and (c) to secure attention, in competition with many other policy actors.
The potential for a form of continuous impact
Dunn suggests that the idea of ‘ex ante’ policy analysis is misleading, since policymaking is continuous, and evaluations of past choices inform current choices. Think of each policy analysis steps as ‘interdependent’, in which new knowledge to inform one step also informs the other four. For example, routine monitoring helps identify compliance with regulations, if resources and services reach ‘target groups’, if money is spent correctly, and if we can make a causal link between the policy solutions and outcomes. Its impact is often better seen as background information with intermittent impact.
Key conclusions to bear in mind
The demand for information from policy analysts may be disproportionately high when policymakers pay attention to a problem, and disproportionately low when they feel that they have addressed it.
Common advice for policy analysts and researchers often looks very similar: keep it concise, tailor it to your audience, make evidence ‘policy relevant’, and give advice (don’t sit on the fence). However, unless researchers are prepared to act quickly, to gather data efficiently (not comprehensively), to meet a tight brief for a client, they are not really in the impact business described by most policy analysis texts.
A lot of routine, continuous, impact tends to occur out of the public spotlight, based on rules and expectations that most policy actors take for granted.
When describing ‘the policy sciences’, Lasswell distinguishes between:
‘knowledge of the policy process’, to foster policy studies (the analysis of policy)
‘knowledge in the process’, to foster policy analysis (analysis for policy)
The lines between each approach are blurry, and each element makes less sense without the other. However, the distinction is crucial to help us overcome the major confusion associated with this question:
Does policymaking proceed through a series of stages?
The short answer is no.
The longer answer is that you can find about 40 blog posts (of 500 and 1000 words) which compare (a) a stage-based model called the policy cycle, and (b) the many, many policy concepts and theories that describe a far messier collection of policy processes.
In a nutshell, most policy theorists reject this image because it oversimplifies a complex policymaking system. The image provides a great way to introduce policy studies, and serves a political purpose, but it does more harm than good:
Prescriptively, it gives you rotten advice about the nature of your policymaking task (for more on these points, see this chapter, article, article, and series).
Why does the stages/ policy cycle image persist? Two relevant explanations
It arose from a misunderstanding in policy studies
In another nutshell, Chris Weible and I argue (in a secret paper) that the stages approach represents a good idea gone wrong:
If you trace it back to its origins, you will find Lasswell’s description of decision functions: intelligence, recommendation, prescription, invocation, application, appraisal and termination.
These functions correspond reasonably well to a policy cycle’s stages: agenda setting, formulation, legitimation, implementation, evaluation, and maintenance, succession or termination.
However, Lasswell was imagining functional requirements, while the cycle seems to describe actual stages.
In other words, if you take Lasswell’s list of what policy analysts/ policymakers need to do, multiple it by the number of actors (spread across many organisations or venues) trying to do it, then you get the multi-centric policy processes described by modern theories. If, instead, you strip all that activity down into a single cycle, you get the wrong idea.
It is a functional requirement of policy analysis
This description should seem familiar, because the classic policy analysis texts appear to describe a similar series of required steps, such as:
define the problem
identify potential solutions
choose the criteria to compare them
evaluate them in relation to their predicted outcomes
In addition, studies of policy analysis in action suggest that:
an individual analyst’sneed for simple steps, to turn policymaking complexity into useful heuristics and pragmatic strategies,
should not be confused with
what actually happens when many policy analysts, influencers, and policymakers interact in policy processes (see Radin, and Brans, Geva-May, and Howlett).
What you need versus what you can expect
Overall, this discussion of policy studies and policy analysis reminds us of a major difference between:
Functional requirements. What you need from policymaking systems, to (a) manage your task (the 5-8 step policy analysis) and (b) understand and engage in policy processes (the simple policy cycle).
Actual processes and outcomes. What policy concepts and theories tell us about bounded rationality (which limit the comprehensiveness of your analysis) and policymaking complexity (which undermines your understanding and engagement in policy processes).
Of course, I am not about to provide you with a solution to these problems.
These posts introduce you to key concepts in the study of public policy. They are all designed to turn a complex policymaking world into something simple enough to understand. Some of them focus on small parts of the system. Others present ambitious ways to explain the system as a whole. The wide range of concepts should give you a sense of a variety of studies out there, but my aim is to show you that these studies have common themes.
