Professor Claudio Radaelli introduces the first article – ‘Occupy the semantic space!’ – to be published in the Journal of European Public Policy Special Issue ‘The Politics of Policy Analysis’. Radaelli analyses the regulatory reform agenda of international organizations to shine a light on the language of depoliticization. He highlights a tendency for policymakers to use the phrase ‘Better Regulation’ as a tool to describe policy activities as self-evident, common sense, or natural (who would not want regulation to be better?). This approach helps to insulate current approaches from debate. Such cases studies highlight the need for policy actors to challenge attempts to ‘occupy the semantic space’.
What do ‘better regulation’, ‘policy coherence’, ‘agile governance’, ‘smart cities’, and ‘social value judgements’ have in common? They are all part of our contemporary language of governance. Policymakers use them every day. International organizations publish indicators on the progress made by individual countries in achieving better, coherent, agile governance. But, there is something else.
Look at the semantics
Semantically, these conceptual entities have something important in common. It is difficult to object to language that points to something naturally desirable. Who can argue for worse regulation or policy incoherence? The whole semantic space is kind of already taken, occupied by the dominant language of governance. Then, you either talk within that language or you do not find semantic space to explore, argue for, and organise alternatives. In a recent article, I explore what happens with this language of governance.
I explore in detail better regulation as policy reform agenda. This appears at first glance unquestionable, universally desirable. Yet, the content of better regulation is actually assembled in distinctive ways – such as the pivotal role of economics as justification for regulatory choice, the concerns about excessive regulatory burdens, the imperative to use regulation to stimulate innovation. Again, I am not saying these are wrong concepts. But they are one of the ways we can reason about regulation, not the only one. Instead, with better regulation, it looks like there is no other way.
As political theorist Michael Freeden would say, concepts are assembled in morphologies that make up an ideology. I use ideology not in the sense that this reform agenda is ideological or false consciousness. Ideology, in this case, is how concepts are assembled and work together.
A semantic double act
So, how do concepts work together? First, the adoption of better regulation language limits semantic fragmentation within large coalitions for reforms, for example it keeps together the delegates of the Regulatory Policy Committee of the OECD. Imagine a semantic big tent where all delegates can say ‘we are all for better regulation’ whilst at the same time muting the difference between those of us who want to cut regulation and those who care more about the quality of regulation than its quantity.
This is the first move of the semantic act: all concepts are essentially contestable, but here, in this language, they appear de-contested. The second move is to erect a semantic wall that leaves no space to those outside. There is no semantic room for those who disagree with better regulation, only the absurdity of asking for ‘worse’ regulation. It is a bit like saying ‘here, we are all liberals’ (although policy disagreements exist within the liberal front) and vehemently discrediting how the concept of freedom is understood by libertarians.
Not just language
It is not just a story about language. It is a story about how dominant policy coalitions shield internal conflict (by de-contesting concepts) and make it difficult to build alternative agendas.
I extend the analysis to other domains, such as policy coherence – a morphology of concepts that has been proved analytically flawed, yet it still seduces policy-makers and generates guidance documents of international organizations like the United Nations. In certain domains, these semantic constructions obfuscate winners and losers (as in the case of smart cities), in others they do not provide the correct basis for taking decisions (such as social value judgements).
So what?
In terms of policy practice, to understand how polysemy works brings in transparency. It allows a more diverse dialogue about the advantages and limitations of reform agendas, without obfuscating practice under generically attractive labels.
Providers of public management executive training should be able to discuss the tools they teach by opening up the semantic horizon, considering concepts that allow for an open discussion with practitioners. For policy entrepreneurs who want to contest dominant language, the pathway is the following: show the fragility of the intellectual foundations of certain morphologies of concepts, expose internal ambiguity camouflaged by decontestation, gain a discursive level-playing-field, re-configure polysemy in ways that are more transparent and inclusive.
Looking critically into the language that is taken for granted in international organizations, governments, and many schools of public policy is a valuable task. Unveiling and exposing the double act can empower alternative coalitions but also benefit the members of the dominant coalition willing to reduce ambiguity and increase transparency in the connection between language and practice. To expose ambiguity helps a dominant coalition to move forward – for example the OECD has carried out a project on moving beyond the classic perimeter of better regulation, discussing four beliefs systems.
And what about us, policy researchers? In the end, all concepts are contestable: policy researchers can contribute to keep this important door (to contestation) open. The identification and critical discussion of dominant language offers citizens the possibility to discuss what is really ‘better’ and ‘for whom’.
Claudio M. Radaelli (2023) ‘Occupy the semantic space! Opening up the language of better regulation’, Journal of European Public Policy, https://doi.org/10.1080/13501763.2023.2181852 (Special Issue: The politics of policy analysis: theoretical insights on real world problems)
Could policy theories help to understand and facilitate the pursuit of equity (or reduction of unfair inequalities)?
We are producing a series of literature reviews to help answer that question, beginning with the study of equity policy and policymaking in health, education, and gender research.
Each field has a broadly similar focus. Most equity researchers challenge the ‘neoliberal’ approaches to policy that favour low state action in favour of individual responsibility and market forces. They seek ‘social justice’ approaches, favouring far greater state intervention to address the social and economic causes of unfair inequalities, via redistributive or regulatory measures. They seek policymaking reforms to reflect the fact that most determinants of inequalities are not contained to one policy sector and cannot be solved in policy ‘silos’. Rather, equity policy initiatives should be mainstreamed via collaboration across (and outside of) government. Each field also projects a profound sense of disenchantment with limited progress, including a tendency to describe a too-large gap between their aspirations and actual policy outcomes. They describe high certainty about what needs to happen, but low confidence that equity advocates have the means to achieve it (or to persuade powerful politicians to change course).
Policy theories could offer some practical insights for equity research, but not always offer the lessons that some advocates seek. In particular, health equity researchers seek to translate insights on policy processes into a playbook for action, such as to frame policy problems to generate more attention to inequalities, secure high-level commitment to radical change, and improve the coherence of cross-cutting policy measures. Yet, policy theories are more likely to identify the dominance of unhelpful policy frames, the rarity of radical change, and the strong rationale for uncoordinated policymaking across a large number of venues. Rather than fostering technical fixes with a playbook, they encourage more engagement with the inescapable dilemmas and trade-offs inherent to policy choice. This focus on contestation (such as when defining and addressing policy problems) is more of a feature of education and gender equity research.
While we ask what policy theories have to offer other disciplines, in fact the most useful lessons emerge from cross-disciplinary insights. They highlight two very different approaches to transformational political change. One offers the attractive but misleading option of radical change through non-radical action, by mainstreaming equity initiatives into current arrangements and using a toolbox to make continuous progress. Yet, each review highlights a tendency for radical aims to be co-opted and often used to bolster the rules and practices that protect the status quo. The other offers radical change through overtly political action, fostering continuous contestation to keep the issue high on the policy agenda and challenge co-option. There is no clear step-by-step playbook for this option, since political action in complex policymaking systems is necessarily uncertain and often unrewarding. Still, insights from policy theories and equity research shows that grappling with these challenges is inescapable.
Ultimately, we conclude that advocates of profound social transformation are wasting each other’s time if they seek short-cuts and technical fixes to enduring political problems. Supporters of policy equity should be cautious about any attempt to turn a transformational political project into a technical process containing a ‘toolbox’ or ‘playbook’.
You can read the original research in Policy & Politics:
Paul Cairney, Emily St.Denny, Sean Kippin, and Heather Mitchell (2022) ‘Lessons from policy theories for the pursuit of equity in health, education, and gender policy’, Policy and Politicshttps://doi.org/10.1332/030557321X16487239616498
My contribution to this interdisciplinary academic-practitioner discussion is to present insights from political science and policy process research, which required me to define some terms (background) before identifying three cautionary messages.
However, note the verb/noun distinction, and common architectural metaphor, to distinguish between the (a) act of design, and (b) the output (e.g. the blueprints).
In terms of the outputs, tools can be defined narrowly as policy instruments – including tax/spending, regulations, staff and other resources for delivery, information sharing, ‘nudging’, etc. – or more widely to include the processes involved in their formulation (such as participatory and deliberative). Therefore, we could be describing:
A highly centralized process, involving very few people, to produce the equivalent of a blueprint.
A decentralized, and perhaps uncoordinated, process involving many people, built on the principle that to seek a blueprint would be to miss the point of participation and deliberation.
Policymaking research tends to focus on
(1) measuring policy change with reference to the ‘policy mix’ of these tools/ instruments, and generally showing that most policy change is minor (and some is major) (link1, link2, link3, link4), and/ or
(2) how to understand the complex policymaking systems or environments in which policy design processes take place.
These studies are the source of my messages of doom.
Three cautionary messages about new policy design
There is a major gap between the act of policy design and actual policies and policy processes. This issue led to the decline of old policy design studies in the 1980s.
While ‘new policy design’ scholars seek to reinvigorate the field, the old issues serve as a cautionary tale, reminding us that (1) policy design is not new, and (2) its decline did not relate to the lack of sophisticated skills or insights among policy designers.
In other words, these old problems will not simply be solved by modern scientific, methodological, or policy design advances. Rather, I encourage policy designers to pay particular attention to:
1. The gap between functional requirements and real world policymaking.
Policy analysts and designers often focus on what they need, or require to get their job done or produce the outcomes they seek.
Policy process researchers identify the major, inevitable, gaps between those requirements and actual policy processes (to the extent that the link between design and policy is often difficult to identify).
2. The strong rationale for the policy processes that undermine policy design.
Policy processes – and their contribution to policy mixes – may seem incoherent from a design perspective. However, they make sense to the participants involved.
Some relate to choice, including to share responsibility for instruments across many levels or types of government (without focusing on how those responsibilities will connect or be coordinated).
Some result from necessity, to delegate responsibility to many policy communities spread across government, each with their own ways to define and address problems (without the ability to know how those responsibilities will be connected).
3. The policy analysis and design dilemmas that cannot be solved by design methods alone.
When seen from the ‘top down’, design problems often relate to the perceived lack of delivery or follow-through in relation to agreed high level design outputs (great design, poor delivery).
When seen from the ‘bottom up’, they represent legitimate ways to incorporate local stakeholder and citizen perspectives. This process will inevitably produce a gap between different sources and outputs of design, making it difficult to separate poor delivery (bad?) from deviation (good?).
Such dynamics are solved via political choice rather than design processes and techniques.
You can hear my presentation below (it took a while to get going because I wasn’t sure who could hear me):
Notes on the workshop discussion
The workshop discussion prompted us initially to consider how many different people would define it. The range of responses included seeing policy design as:
a specific process with specific tools to produce a well-defined output (applied to specific areas conducive to design methods)
a more general philosophy or way of thinking about things like policy issues (compare with systems thinking)
a means to encourage experimentation (such as to produce a prototype policy instrument, use it, and reflect or learn about its impact) or change completely how people think about an issue
the production of a policy solution, or one part of a large policy mix
a niche activity in one unit of government, or something mainstreamed across governments
something done in government, or inside and outside of government
producing something new (like writing on a blank sheet of paper), adding to a pile of solutions, or redesigning what exists
primarily a means to empower people to tell their story, or as a means to improve policy advocacy (as in discussions of narrative/ storytelling)
something done with authoritative policymakers like government ministers (in other words, people with the power to make policy changes after they participate in design processes) or given to them (in other words, the same people but as the audience for the outcomes of design)
These definitions matter since they have very different implications for policy and practice. Take, for example, the link – made by Professor Liz Richardson – between policy design and the idea of evidence-based policymaking, to consider two very different scenarios:
A minister is directly involved in policy design processes. They use design thinking to revisit how they think about a policy problem (and target populations), seek to foster participation and deliberation, and use that process – perhaps continuously – to consider how to reconcile very different sources of evidence (including, say, new data from randomized control trials and powerful stories from citizens, stakeholders, service users). I reckon that this kind of scenario would be in the minds of people who describe policy design optimistically.
A minister is the intended audience of a report on the outcomes of policy design. You assume that their thoughts on a policy problem are well-established. There is no obvious way for them to reconcile different sources of policy-relevant evidence. Crucially, the fruits of your efforts have made a profound impact on the people involved but, for the minister, the outcome is just one of too-many sources of information (likely produced too soon before or after they want to consider the issue).
The second scenario is closer to the process that I describe in the main post, although policy studies would warn against seeing someone like a government minister as authoritative in the sense that they reside in the centre of government. Rather, studies of multi-centric policymaking remind us that there are many possible centres spread across political systems. If so, policy design – according to approaches like the IAD – is about ways to envisage a much bigger context in which design success depends on the participation and agreement of a large number of influential actors (who have limited or no ability to oblige others to cooperate).
James Nicholls, Wulf Livingston, Andy Perkins, Beth Cairns, Rebecca Foster, Kirsten M. A. Trayner, Harry R. Sumnall, Tracey Price, Paul Cairney, Josh Dumbrell, and Tessa Parkes (2022) ‘Drug Consumption Rooms and Public Health Policy: Perspectives of Scottish Strategic Decision-Makers’, International Journal of Environmental Research and Public Health, 19(11), 6575; https://doi.org/10.3390/ijerph19116575
Q: if stakeholders in Scotland express high support for drug consumption rooms, and many policymakers in Scotland seem sympathetic, why is there so little prospect of policy change?
My summary of the article’s answer is as follows:
Although stakeholders support DCRs almost unanimously, they do not support them energetically.
They see this solution as one part of a much larger package rather than a magic bullet. They are not sure of the cost-effectiveness in relation to other solutions, and can envisage some potential users not using them.
The existing evidence on their effectiveness is not persuasive for people who (1) adhere to a hierarchy of evidence which prioritizes evidence from randomized control trials or (2) advocate alternative ways to use evidence.
There are competing ways to frame this policy solution. It suggests that there are some unresolved issues among stakeholders which have not yet come to the fore (since the lack of need to implement something specific reduces the need to engage with a more concrete problem definition).
This method invites local policymakers and practitioners to try out new solutions, work with stakeholders and service users during delivery, reflect on the results, and use this learning to design the next iteration. This is a pragmatic, small-scale, approach that appeals to the (small-c conservative) Scottish Government, which uses pilots to delay major policy changes, and is keen on its image as not too centralist and quite collaboration minded.
3. This approach is not politically feasible in this case.
Some factors suggest that the general argument has almost been won, including positive informal feedback from policymakers, and increasingly sympathetic media coverage (albeit using problematic ways to describe drug use).
However, this level of support is not enough to support experimentation. Drug consumption rooms would need a far stronger steer from the Scottish Government.
In this case, it can’t experiment now and decide later. It needs to make a strong choice (with inevitable negative blowback) and stay the course, knowing that one failed political experiment could set back progress for years.
4. The multi-level policymaking system is not conducive to overcoming these obstacles.
The issue of drugs policy is often described as a public health – and therefore devolved – issue politically (and in policy circles)
However, the legal/ formal division of responsibilities suggests that UK government consent is necessary and not forthcoming.
It is possible that the Scottish Government could take a chance and act alone. Indeed, the example of smoking in public places showed that it shifted its position after a slow start (it described the issue as reserved to the UK took charge of its own legislation, albeit with UK support).
However, the Scottish Government seems unwilling to take that chance, partly because it has been stung by legal challenges in other areas, and is reluctant to engage in more of the same (see minimum unit pricing for alcohol).
Local policymakers could experiment on their own, but they won’t do it without proper authority from a central government.
This experience is part of a more general issue: people may describe multi-level policymaking as a source of venues for experimentation (‘laboratories of democracy’) to encourage policy learning and collaboration. However, this case, and cases like fracking, show that they can actually be sites of multiple veto points and multi-level reluctance.
If so, the remaining question for reflection is: what would it take to overcome these obstacles? The election of a Labour UK government? Scottish independence? Or, is there some other way to make it happen in the current context?
By James Nicholls and Paul Cairney, for the University of Stirling MPH and MPP programmes.
There are strong links between the study of public health and public policy. For example, public health scholars often draw on policy theories to help explain (often low amounts of) policy change to foster population health or reduce health inequalities. Studies include a general focus on public health strategies (such as HiAP) or specific policy instruments (such as a ban on smoking in public places). While public health scholars may seek to evaluate or influence policy, policy theories tend to focus on explaining processes and outcomes.
To demonstrate these links, we present:
A long-read blog post to (a) use an initial description of a key alcohol policy instrument (minimum unit pricing, adopted by the Scottish Government but not the UK Government) to (b) describe the application of policy concepts and theories and reflect on the empirical and practical implications. We then added some examples of further reading.
A 45 minute podcast to describe and explain these developments (click below or scroll to the end)
Minimum Unit Pricing in Scotland: background and development
Minimum Unit Pricing for alcohol was introduced in Scotland in 2018. In 2012, the UK Government had also announced plans to introduce MUP, but within a year dopped the policy following intense industry pressure. What do these two journeys tell us about policy processes?
When MUP was first proposed by Scottish Health Action on Alcohol Problems in 2007, it was a novel policy idea. Public health advocates had long argued that raising the price of alcohol could help tackle harmful consumption. However, conventional tax increases were not always passed onto consumers, so would not necessarily raise prices in the shops (and the Scottish Government did not have such taxation powers). MUP appeared to present a neat solution to this problem. It quickly became a prominent policy goal of public health advocates in Scotland and across the UK, while gaining increasing attention, and support, from the global alcohol policy community.
In 2008, the UK Minister for Health, Dawn Primarolo, had commissioned researchers at the University of Sheffield to look into links between alcohol pricing and harm. The Sheffield team developed economic models to analysis the predicted impact of different systems. MUP was included, and the ‘Sheffield Model’ would go on to play a decisive role in developing the case for the policy.
What problem would MUP help to solve?
Descriptions of the policy problem often differed in relation to each government. In the mid-2000s, alcohol harm had become a political problem for the UK government. Increasing consumption, alongside changes to the night-time economy, had started to gain widespread media attention. In 2004, just as a major liberalisation of the licensing system was underway in England, news stories began documenting the apparent horrors of ‘Binge Britain’: focusing on public drunkenness and disorder, but also growing rates of liver disease and alcohol-related hospital admissions.
In 2004, influential papers such as the Daily Mail began to target New Labour alcohol policy
Politicians began to respond, and the issue became especially useful for the Conservatives who were developing a narrative that Britain was ‘broken’ under New Labour. Labour’s liberalising reforms of alcohol licensing could conveniently be linked to this political framing. The newly formed Alcohol Health Alliance, a coalition set up under the leadership of Professor Sir Ian Gilmore, was also putting pressure on the UK Government to introduce stricter controls. In Scotland, while much of the debate on alcohol focused on crime and disorder, Scottish advocates were focused on framing the problem as one of public health. Emerging evidence showed that Scotland had dramatically higher rates of alcohol-related illness and death than the rest of Europe – a situation strikingly captured in a chart published in the Lancet.
Source: Leon, D. and McCambridge, J. (2006). Liver cirrhosis mortality rates in Britain from 1950 to 2002: an analysis of routine data. Lancet 367
The notion that Scotland faced an especially acute public health problem with alcohol was supported by key figures in the increasingly powerful Scottish National Party (in government since 2007), which, around this time, had developed working relationships with Alcohol Focus Scotland and other advocacy groups.
What happened next?
The SNP first announced that it would support MUP in 2008, but it did not implement this change until 2018. There are two key reasons for the delay:
Its minority government did not achieve enough parliamentary support to pass legislation. It then formed a majority government in 2011, and its legislation to bring MUP into law was passed in 2012.
Court action took years to resolve. The alcohol industry, which is historically powerful in Scotland, was vehemently opposed. A coalition of industry bodies, led by the Scotch Whisky Association, took the Scottish Government to court in an attempt to prove the policy was illegal. Ultimately, this process would take years, and conclude in rulings by the European Court of Justice (2016), Scottish Court of Session Inner House (2016), and UK Supreme Court (2017) which found in favour of the Scottish Government.
Once again, the alcohol industry swung into action, launching a campaign led by the Wine and Spirits Trade Association, asking ‘Why should moderate drinkers pay more?’
This public campaign was accompanied by intense behind-the-scenes lobbying, aided by the fact that the leadership of industry groups had close ties to Government and that the All-Party Parliamentary Group on Beer had the largest membership of any APPG in Westminster. The industry campaign made much of the fact there was little evidence to suggest MUP would reduce crime, but also argued strongly that the modelling produced by Sheffield University was not valid evidence in the first place. A year after the adopting the policy, the UK Government announced a U-turn and MUP was dropped.
How can we use policy theories and concepts to interpret these dynamics?
Here are some examples of using policy theories and concepts as a lens to interpret these developments.
1. What was the impact of evidence in the case for policy change?
First, many political actors (including policymakers) have many different ideas about what counts as good evidence.
The assessment, promotion, and use of evidence is highly contested, and never speaks for itself.
Second, policymakers have to ignore almost all evidence to make choices.
They address ‘bounded rationality’ by using two cognitive shortcuts: ‘rational’ measures set goals and identify trusted sources, while ‘irrational’ measures use gut instinct, emotions, and firmly held beliefs.
Third, policymakers do not control the policy process.
There is no centralised and orderly policy cycle. Rather, policymaking involves policymakers and influencers spread across many authoritative ‘venues’, with each venue having its own rules, networks, and ways of thinking.
In that context, policy theories identify the importance of contestation between policy actors, and describe the development of policy problems, and how evidence fits in. Approaches include:
The acceptability of a policy solution will often depend on how the problem is described. Policymakers use evidence to reduce uncertainty, or a lack of information around problems and how to solve them. However, politics is about exercising power to reduce ambiguity, or the ability to interpret the same problem in different ways.
By suggesting MUP would solve problems around crime, the UK Government made it easier for opponents to claim the policy wasn’t evidence-based. In Scotland, policymakers and advocates focused on health, where the evidence was stronger. In addition, the SNP’s approach fitted within a wider political independence frame, in which more autonomy meant more innovation.
Policy actors tell stories to appeal to the beliefs (or exploit the cognitive shortcuts) of their audiences. A narrative contains a setting (the policy problem), characters (such as the villain who caused it, or the victim of its effects), plot (e.g. a heroic journey to solve the problem), and moral (e.g. the solution to the problem).
Supporters of MUP tended to tell the story that there was an urgent public health crisis, caused largely by the alcohol industry, and with many victims, but that higher alcohol prices pointed to one way out of this hole. Meanwhile opponents told the story of an overbearing ‘nanny state’, whose victims – ordinary, moderate drinkers – should be left alone by government.
Policymakers make strategic and emotional choices, to identify ‘good’ populations deserving of government help, and ‘bad’ populations deserving punishment or little help. These judgements inform policy design (government policies and practices) and provide positive or dispiriting signals to citizens.
For example, opponents of MUP rejected the idea that alcohol harms existed throughout the population. They focused instead on dividing the majority of moderate drinkers from irresponsible minority of binge drinkers, suggesting that MUP would harm the former more than help the latter.
This competition to frame policy problems takes place in political systems that contain many ‘centres’, or venues for authoritative choice. Some diffusion of power is by choice, such as to share responsibilities with devolved and local governments. Some is by necessity, since policymakers can only pay attention to a small proportion of their responsibilities, and delegate the rest to unelected actors such as civil servants and public bodies (who often rely on interest groups to process policy).
For example, ‘alcohol policy’ is really a collection of instruments made or influenced by many bodies, including (until Brexit) European organisations deciding on the legality of MUP, UK and Scottish governments, as well as local governments responsible for alcohol licensing. In Scotland, this delegation of powers worked in favour of MUP, since Alcohol Focus Scotland were funded by the Scottish Government to help deliver some of their alcohol policy goals, and giving them more privileged access than would otherwise have been the case.
The role of evidence in MUP
In the case of MUP, similar evidence was available and communicated to policymakers, but used and interpreted differently, in different centres, by the politicians who favoured or opposed MUP.
In Scotland, the promotion, use of, and receptivity to research evidence – on the size of the problem and potential benefit of a new solution – played a key role in increasing political momentum. The forms of evidence were complimentary. The ‘hard’ science on a potentially effective solution seemed authoritative (although few understood the details), and was preceded by easily communicated and digested evidence on a concrete problem:
There was compelling evidence of a public health problem put forward by a well-organised ‘advocacy coalition’ (see below) which focused clearly on health harms. In government, there was strong attention to this evidence, such as the Lancet chart which one civil servant described as ‘look[ing] like the north face of the Eiger’. There were also influential ‘champions’ in Government willing to frame action as supporting the national wellbeing.
Reports from Sheffield University appeared to provide robust evidence that MUP could reduce harm, and advocacy was supported by research from Canada which suggested that similar policies there had been successful elsewhere.
Advocacy in England was also well-organised and influential, but was dealing with a larger – and less supportive – Government machine, and the dominant political frame for alcohol harms remained crime and disorder rather than health.
Debates on MUP modelling exemplify these differences in evidence communication and use. Those in favour appealed to econometric models, but sometimes simplifying their complexity and blurring the distinction between projected outcomes and proof of efficacy. Opponents went the other way and dismissed the modelling as mere speculation. What is striking is the extent to which an incredibly complex, and often poorly understand, set of econometric models – and the ’Sheffield Model’ in particular – came to occupy centre stage in a national policy debate. Katikireddi and colleagues talked about this as an example of evidence as rhetoric:
Support became less about engagement with the econometric modelling, and more an indicator of general concern about alcohol harm and the power of the industry.
Scepticism was often viewed as the ‘industry position’, and an indicator of scepticism towards public health policy more broadly.
2. Who influences policy change?
Advocacy plays a key role in alcohol policy, with industry and other actors competing with public health groups to define and solve alcohol policy problems. It prompts our attention to policy networks, or the actors who make and influence policy.
People engage in politics to turn their beliefs into policy. They form advocacy coalitions with people who share their beliefs, and compete with other coalitions. The action takes place within a subsystem devoted to a policy issue, and a wider policymaking process that provides constraints and opportunities to coalitions. Beliefs about how to interpret policy problems act as a glue to bind actors together within coalitions. If the policy issue is technical and humdrum, there may be room for routine cooperation. If the issue is highly charged, then people romanticise their own cause and demonise their opponents.
MUP became a highly charged focus of contestation between a coalition of public health advocates, who saw themselves as fighting for the wellbeing of the wider community (and who believed fundamentally that government had a duty to promote population health), and a coalition of industry actors who were defending their commercial interests, while depicting public health policies as illiberal and unfair.
3. Was there a ‘window of opportunity’ for MUP?
Policy theories – including Punctuated Equilibrium Theory – describe a tendency for policy change to be minor in most cases and major in few. Paradigmatic policy change is rare and may take place over decades, as in the case of UK tobacco control where many different policy instruments changed from the 1980s. Therefore, a major change in one instrument could represent a sea-change overall or a modest adjustment to the overall approach.
Multiple Streams Analysis is a popular way to describe the adoption of a new policy solution such as MUP. It describes disorderly policymaking, in which attention to a policy problem does not produce the inevitable development, implementation, and evaluation of solutions. Rather, these ‘stages’ should be seen as separate ‘streams’. A ‘window of opportunity’ for policy change occurs when the three ‘streams’ come together:
Problem stream. There is high attention to one way to define a policy problem.
Policy stream. A technically and politically feasible solution already exists (and is often pushed by a ‘policy entrepreneur’ with the resources and networks to exploit opportunities).
Politics stream. Policymakers have the motive and opportunity to choose that solution.
However, these windows open and close, often quickly, and often without producing policy change.
This approach can help to interpret different developments in relation to Scottish and UK governments:
Problem stream
The Scottish Government paid high attention to public health crises, including the role of high alcohol consumption.
The UK government paid often-high attention to alcohol’s role in crime and anti-social behaviour (‘Binge Britain’ and ‘Broken Britain’)
Policy stream
In Scotland, MUP connected strongly to the dominant framing, offering a technically feasible solution that became politically feasible in 2011.
The UK Prime Minister David Cameron’s made a surprising bid to adopt MUP in 2012, but ministers were divided on its technical feasibility (to address the problem they described) and its political feasibility seemed to be more about distracting from other crises than public health.
Politics stream
The Scottish Government was highly motivated to adopt MUP. MUP was a flagship policy for the SNP; an opportunity to prove its independent credentials, and to be seen to address a national public health problem. It had the opportunity from 2011, then faced interest group opposition that delayed implementation.
The Coalition Government was ideologically more committed to defending commercial interests, and to framing alcohol harms as one of individual (rather than corporate) responsibility. It took less than a year for the alcohol industry to successfully push for a UK government U-turn.
As a result, MUP became policy (eventually) in Scotland, but the window closed (without resolution) in England.
Paul Cairney and Donley Studlar (2014) ‘Public Health Policy in the United Kingdom: After the War on Tobacco, Is a War on Alcohol Brewing?’ World Medical and Health Policy, 6, 3, 308-323PDF
Niamh Fitzgerald and Paul Cairney (2022) ‘National objectives, local policymaking: public health efforts to translate national legislation into local policy in Scottish alcohol licensing’, Evidence and Policy, https://doi.org/10.1332/174426421X16397418342227, PDF
Podcast
You can listen directly here:
You can also listen on Spotify or iTunes via Anchor
By James Nicholls and Paul Cairney, for the University of Stirling MPH and MPP programmes.
There are strong links between the study of public health and public policy. For example, public health scholars often draw on policy theories to help explain (often low amounts of) policy change to foster population health or reduce health inequalities. Studies include a general focus on public health strategies (such as HiAP) or specific policy instruments (such as a ban on smoking in public places). While public health scholars may seek to evaluate or influence policy, policy theories tend to focus on explaining processes and outcomes,.
To demonstrate these links, we present this podcast and blog post to (1) use an initial description of a key alcohol policy instrument (minimum unit pricing in Scotland) to (2) describe the application of policy concepts and theories and reflect on the empirical and practical implications.
Using policy theories to interpret public health case studies: the example of a minimum unit price for alcohol | Paul Cairney: Politics & Public Policy (wordpress.com)
One take home message from the 750 Words series is to avoid seeing policy analysis simply as a technical (and ‘evidence-based’) exercise. Mainstream policy analysis texts break down the process into technical-looking steps, but also show how each step relates to a wider political context. Critical policy analysis texts focus more intensely on the role of politics in the everyday choices that we might otherwise take for granted or consider to be innocuous. The latter connect strongly to wider studies of the links between power and knowledge.
Power and ideas
Classic studies suggest that the most profound and worrying kinds of power are the hardest to observe. We often witness highly visible political battles and can use pluralist methods to identify who has material resources, how they use them, and who wins. However, key forms of power ensure that many such battles do not take place. Actors often use their resources to reinforce social attitudes and policymakers’ beliefs, to establish which issues are policy problems worthy of attention and which populations deserve government support or punishment. Key battles may not arise because not enough people think they are worthy of debate. Attention and support for debate may rise, only to be crowded out of a political agenda in which policymakers can only debate a small number of issues.
Studies of power relate these processes to the manipulation of ideas or shared beliefs under conditions of bounded rationality (see for example the NPF). Manipulation might describe some people getting other people to do things they would not otherwise do. They exploit the beliefs of people who do not know enough about the world, or themselves, to know how to identify and pursue their best interests. Or, they encourage social norms – in which we describe some behaviour as acceptable and some as deviant – which are enforced by (1) the state (for example, via criminal justice and mental health policy), (2) social groups, and (3) individuals who govern their own behaviour with reference to what they feel is expected of them (and the consequences of not living up to expectations).
Such beliefs, norms, and rules are profoundly important because they often remain unspoken and taken for granted. Indeed, some studies equate them with the social structures that appear to close off some action. If so, we may not need to identify manipulation to find unequal power relationships: strong and enduring social practices help some people win at the expense of others, by luck or design.
Relating power to policy analysis: whose knowledge matters?
The concept of‘epistemic violence’ is one way todescribe the act of dismissing an individual, social group, or population by undermining the value of their knowledge or claim to knowledge. Specific discussions include: (a) the colonial West’s subjugation of colonized populations, diminishing the voice of the subaltern; (b) privileging scientific knowledge and dismissing knowledge claims via personal or shared experience; and (c) erasing the voices of women of colour from the history of women’s activism and intellectual history.
It is in this context that we can understand ‘critical’ research designed to ‘produce social change that will empower, enlighten, and emancipate’ (p51). Powerlessness can relate to the visible lack of economic material resources and factors such as the lack of opportunity to mobilise and be heard.
750 Words posts examining this link between power and knowledge
Some posts focus on the role of power in research and/ or policy analysis:
These posts ask questions such as: who decides what evidence will be policy-relevant, whose knowledge matters, and who benefits from this selective use of evidence? They help to (1) identify the exercise of power to maintain evidential hierarchies (or prioritise scientific methods over other forms of knowledge gathering and sharing), and (2) situate this action within a wider context (such as when focusing on colonisation and minoritization). They reflect on how (and why) analysts should respect a wider range of knowledge sources, and how to produce more ethical research with an explicit emancipatory role. As such, they challenge the – naïve or cynical – argument that science and scientists are objective and that science-informed analysis is simply a technical exercise (see also Separating facts from values).
Many posts incorporate these discussions into many policy analysis themes.
Entrepreneurial policy analysis warns against a too-strong focus on the agency – rather than the unequal status and resources – of successful political actors.
A key argument in policy studies is that it is impossible to separate facts and values when making policy. We often treat our beliefs as facts, or describe certain facts as objective, but perhaps only to simplify our lives or support a political strategy (a ‘self-evident’ fact is very handy for an argument). People make empirical claims infused with their values and often fail to realise just how their values or assumptions underpin their claims.
This is not an easy argument to explain. One strategy is to use extreme examples to make the point. For example, Herbert Simon points to Hitler’s Mein Kampf as the ultimate example of value-based claims masquerading as facts. We can also identify historic academic research which asserts that men are more intelligent than women and some races are superior to others. In such cases, we would point out, for example, that the design of the research helped produce such conclusions: our values underpin our (a) assumptions about how to measure intelligence or other measures of superiority, and (b) interpretations of the results.
‘Wait a minute, though’ (you might say). “What about simple examples in which you can state facts with relative certainty – such as the statement ‘there are X number of words in this post’”. ‘Fair enough’, I’d say (you will have to speak with a philosopher to get a better debate about the meaning of your X words claim; I would simply say that it is trivially true). But this statement doesn’t take you far in policy terms. Instead, you’d want to say that there are too many or too few words, before you decided what to do about it.
In that sense, we have the most practical explanation of the unclear fact/ value distinction: the use of facts in policy is to underpin evaluations (assessments based on values). For example, we might point to the routine uses of data to argue that a public service is in ‘crisis’ or that there is a public health related epidemic (note: I wrote the post before COVID-19; it referred to crises of ‘non-communicable diseases’). We might argue that people only talk about ‘policy problems’ when they think we have a duty to solve them.
Or, facts and values often seem the hardest to separate when we evaluate the success and failure of policy solutions, since the measures used for evaluation are as political as any other part of the policy process. The gathering and presentation of facts is inherently a political exercise, and our use of facts to encourage a policy response is inseparable from our beliefs about how the world should work.
‘Modern science remains value-laden … even when so many people employ so many systematic methods to increase the replicability of research and reduce the reliance of evidence on individual scientists. The role of values is fundamental. Anyone engaging in research uses professional and personal values and beliefs to decide which research methods are the best; generate research questions, concepts and measures; evaluate the impact and policy relevance of the results; decide which issues are important problems; and assess the relative weight of ‘the evidence’ on policy effectiveness. We cannot simply focus on ‘what works’ to solve a problem without considering how we used our values to identify a problem in the first place. It is also impossible in practice to separate two choices: (1) how to gather the best evidence and (2) whether to centralize or localize policymaking. Most importantly, the assertion that ‘my knowledge claim is superior to yours’ symbolizes one of the most worrying exercises of power. We may decide to favour some forms of evidence over others, but the choice is value-laden and political rather than objective and innocuous’.
Implications for policy analysis
Many highly-intelligent and otherwise-sensible people seem to get very bothered with this kind of argument. For example, it gets in the way of (a) simplistic stories of heroic-objective-fact-based-scientists speaking truth to villainous-stupid-corrupt-emotional-politicians, (b) the ill-considered political slogan that you can’t argue with facts (or ‘science’), (c) the notion that some people draw on facts while others only follow their feelings, and (d) the idea that you can divide populations into super-facty versus post-truthy people.
A more sensible approach is to (1) recognise that all people combine cognition and emotion when assessing information, (2) treat politics and political systems as valuable and essential processes (rather than obstacles to technocratic policymaking), and (3) find ways to communicate evidence-informed analyses in that context. This article and 750 post explore how to reflect on this kind of communication.
This post forms one part of the Policy Analysis in 750 words series overview. The title comes from this article by Cairney and Kwiatkowski on ‘psychology based policy studies’.
One aim of this series is to combine insights from policy research (1000, 500) and policy analysis texts. How might we combine insights to think about effective communication?
1. Insights from policy analysis texts
Most texts in this series relate communication to understanding your audience (or client) and the political context. Your audience has limited attention or time to consider problems. They may have a good antennae for the political feasibility of any solution, but less knowledge of (or interest in) the technical details. In that context, your aim is to help them treat the problem as worthy of their energy (e.g. as urgent and important) and the solution as doable. Examples include:
Bardach: communicating with a client requires coherence, clarity, brevity, and minimal jargon.
Dunn: argumentation involves defining the size and urgency of a problem, assessing the claims made for each solution, synthesising information from many sources into a concise and coherent summary, and tailoring reports to your audience.
Smith: your audience makes a quick judgement on whether or not to read your analysis. Ask yourself questions including: how do I frame the problem to make it relevant, what should my audience learn, and how does each solution relate to what has been done before? Maximise interest by keeping communication concise, polite, and tailored to a policymaker’s values and interests.
2. Insights from studies of policymaker psychology
‘Rational’ shortcuts. Goal-oriented reasoning based on prioritizing trusted sources of information.
‘Irrational’ shortcuts. Emotional thinking, or thought fuelled by gut feelings, deeply held beliefs, or habits.
We can use such distinctions to examine the role of evidence-informed communication, to reduce:
Uncertainty, or a lack of policy-relevant knowledge. Focus on generating ‘good’ evidence and concise communication as you collate and synthesise information.
Ambiguity, or the ability to entertain more than one interpretation of a policy problem. Focus on argumentation and framing as you try to maximise attention to (a) one way of defining a problem, and (b) your preferred solution.
Policy process texts focus on policymaking reality: showing that ideal-types such as the policy cycle do not guide real-world action, and describing more accurate ways to guide policy analysts.
For example, they help us rethink the ‘know your audience’ mantra by:
Showing that many policymaking ‘centres’ create the instruments that produce policy change
Gone are the mythical days of a small number of analysts communicating to a single core executive (and of the heroic researcher changing the world by speaking truth to power). Instead, we have many analysts engaging with many centres, creating a need to not only (a) tailor arguments to different audiences, but also (b) develop wider analytical skills (such as to foster collaboration and the use of ‘design principles’).
How to communicate effectively with policymakers
In that context, we argue that effective communication requires analysts to:
1. Understand your audience and tailor your response (using insights from psychology)
2. Identify ‘windows of opportunity’ for influence (while noting that these windows are outside of anyone’s control)
3. Engage with real world policymaking rather than waiting for a ‘rational’ and orderly process to appear (using insights from policy studies).
This post summarises new research on the health equity strategy Health in All Policies. As our previous post suggests, it is common to hope that a major event will create a ‘window of opportunity’ for such strategies to flourish, but the current COVID-19 experience suggests otherwise. If so, what do HIAP studies tell us about how to respond, and do they offer any hope for future strategies? The full report is on Open Research Europe, accompanied by a brief interview on its contribution to the Horizon 2020 project – IMAJINE – on spatial justice.
COVID-19 should have prompted governments to treat health improvement as fundamental to public policy
Many had made strong rhetorical commitments to public health strategies focused on preventing a pandemic of non-communicable diseases (NCDs). To do so, they would address the ‘social determinants’ of health and health inequalities, defined by the WHO as ‘the unfair and avoidable differences in health status’ that are ‘shaped by the distribution of money, power and resources’ and ‘the conditions in which people are born, grow, live, work and age’.
COVID-19 reinforces the impact of the social determinants of health. Health inequalities result from factors such as income and social and environmental conditions, which influence people’s ability to protect and improve their health. COVID-19 had a visibly disproportionate impact on people with (a) underlying health conditions associated with NCDs, and (b) less ability to live and work safely.
Yet, the opposite happened. The COVID-19 response side-lined health improvement
This experience shows that the evidence does not speak for itself
The evidence on social determinants is clear to public health specialists, but the idea of social determinants is less well known or convincing to policymakers.
It also challenges the idea that the logic of health improvement is irresistible
Health in All Policies (HIAP) is the main vehicle for health improvement policymaking, underpinned by: a commitment to health equity by addressing the social determinants of health; the recognition that the most useful health policies are not controlled by health departments; the need for collaboration across (and outside) government; and, the search for high level political commitment to health improvement.
Its logic is undeniable to HIAP advocates, but not policymakers. A government’s public commitment to HIAP does not lead inevitably to the roll-out of a fully-formed HIAP model. There is a major gap between the idea of HIAP and its implementation. It is difficult to generate HIAP momentum, and it can be lost at any time.
Instead, we need to generate more realistic lessons from health improvement and promotion policy
However, most HIAP research does not provide these lessons. Most HIAP research combines:
functional logic (here is what we need)
programme logic (here is what we think we need to do to achieve it), and
hope.
Policy theory-informed empirical studies of policymaking could help produce a more realistic agenda, but very few HIAP studies seem to exploit their insights.
To that end, this review identifies lessons from studies of HIAP and policymaking
It summarises a systematic qualitative review of HIAP research. It includes 113 articles (2011-2020) that refer to policymaking theories or concepts while discussing HIAP.
We produced these conclusions from pre-COVID-19 studies of HIAP and policymaking, but our new policymaking context – and its ironic impact on HIAP – is impossible to ignore.
It suggests that HIAP advocates produced a 7-point playbook for the wrong game
The seven most common pieces of advice add up to a plausible but incomplete strategy:
adopt a HIAP model and toolkit
raise HIAP awareness and support in government
seek win-win solutions with partners
avoid the perception of ‘health imperialism’ when fostering intersectoral action
find HIAP policy champions and entrepreneurs
use HIAP to support the use of health impact assessments (HIAs)
challenge the traditional cost-benefit analysis approach to valuing HIAP.
Yet, two emerging pieces of advice highlight the limits to the current playbook and the search for its replacement:
treat HIAP as a continuous commitment to collaboration and health equity, not a uniform model; and,
address the contradictions between HIAP aims.
As a result, most country studies report a major, unexpected, and disappointing gap between HIAP commitment and actual outcomes
These general findings are apparent in almost all relevant studies. They stand out in the ‘best case’ examples where: (a) there is high political commitment and strategic action (such as South Australia), or (b) political and economic conditions are conducive to HIAP (such as Nordic countries).
These studies show that the HIAP playbook has unanticipated results, such as when the win-win strategy leads to HIAP advocates giving ground but receiving little in return.
HIAP strategies to challenge the status quo are also overshadowed by more important factors, including (a) a far higher commitment to existing healthcare policies and the core business of government, and (b) state retrenchment. Additional studies of decentralised HIAP models find major gaps between (a) national strategic commitment (backed by national legislation) and (b) municipal government progress.
Some studies acknowledge the need to use policymaking research to produce new ways to encourage and evaluate HIAP success
Studies of South Australia situate HIAP in a complex policymaking system in which the link between policy activity and outcomes is not linear.
Studies of Nordic HIAP show that a commitment to municipal responsibility and stakeholder collaboration rules out the adoption of a national uniform HIAP model.
However, most studies do not use policymaking research effectively or appropriately
Almost all HIAP studies only scratch the surface of policymaking research (while some try to synthesise its insights, but at the cost of clarity).
Most HIAP studies use policy theories to:
produce practical advice (such as to learn from ‘policy entrepreneurs’), or
supplement their programme logic (to describe what they think causes policy change and better health outcomes).
Most policy theories were not designed for this purpose.
Policymaking research helps primarily to explain the HIAP ‘implementation gap’
Its main lesson is that policy outcomes are beyond the control of policymakers and HIAP advocates. This explanation does not show how to close implementation gaps.
Its practical lessons come from critical reflection on dilemmas and politics, not the reinvention of a playbook
It prompts advocates to:
Treat HIAP as a political project, not a technical exercise or puzzle to be solved.
Re-examine the likely impact of a focus on intersectoral action and collaboration, to recognise the impact of imbalances of power and the logic of policy specialisation.
Revisit the meaning-in-practice of the vague aims that they take for granted without explaining, such as co-production, policy learning, and organisational learning.
Engage with key trade-offs, such as between a desire for uniform outcomes (to produce health equity) but acceptance of major variations in HIAP policy and policymaking.
Avoid reinventing phrases or strategies when facing obstacles to health improvement.
While policymakers often want to learn how other governments have responded to certain policies, policy learning is characterized by contestation. Policymakers compete to define the problem, set the parameters for learning, and determine which governments should take the lead. Emily St.Denny, Paul Cairney, and Sean Kippin discuss a framework that would encourage policy learning in multilevel systems.
Governments face similar policy problems and there is great potential for mutual learning and policy transfer. Yet, most policy research highlights the political obstacles to learning and the weak link between research and transfer. One solution may be to combine academic insights from policy research with practical insights from people with experience of learning in political environments. In that context, our role is to work with policy actors to produce pragmatic strategies to encourage realistic research-informed learning.
Pragmatic policy learning
Producing concepts, research questions, and methods that are interesting to both academics and practitioners is challenging. It requires balancing different approaches to gathering and considering ‘evidence’ when seeking to solve a policy problem. Practitioners need to gather evidence quickly, focusing on ‘what works’ or positive experiences from a small number of relevant countries. Policy scholars may seek more comprehensive research and warn against simple solutions. Further, they may do so without offering a feasible alternative to their audience.
To bridge these differences and facilitate policy learning, we encourage a pragmatic approach to policy learning that requires:
Seeing policy learning through the eyes of participants, to understand how they define and seek to solve this problem;
Incorporating insights from policy research to construct a feasible approach;
Reflecting on this experience to inform research.
Our aim is not ‘evidence-based policymaking’. Rather, it is to incorporate the fact that researchers and evidence form only one small component of a policymaking system characterized by complexity. Additionally, policy actors enjoy less control over these systems than we might like to admit. Learning is therefore best understood as a contested process in which actors combine evidence and beliefs to define policy problems, identify technically and politically feasible solutions, and negotiate who should be responsible for their adoption and delivery in multilevel policymaking systems. Taking seriously the contested, context-specific, and political nature of policymaking is crucial for producing effective advice from which to learn.
Our role is to facilitate policy learning and consider the transfer of policy solutions from successful experiences. Yet, we are confronted by the usual challenges. They include the need to: identify appropriate exemplars from where to draw lessons; help policy practitioners control for differences in context; and translate between academic and practitioner communities.
Additionally, we work on an issue – inequality – which is notoriously ambiguous and contested. It involves not only scientific information about the lives and experiences of people, but also political disagreement about the legitimate role of the state in intervening in people’s lives or redistributing of resources. Developing a policy learning framework that is able to generate practically useful insights for policy actors is difficult but key to ensuring policy effectiveness and coherence.
Drawing on work we carried out for the Scottish Government’s National Advisory Council on Women and Girlson approaches to reducing inequalities in relation to gender mainstreaming, we apply the IMAJINE framework to support policy learning. The IMAJINE framework guides such academic–practitioner analysis in four steps:
Step 1: Define the nature of policy learning in political systems.
Preparing for learning requires taking into account the interaction between:
Politics, in which actors contest the nature of problems and the feasibility of solutions;
Bounded rationality, which requires actors to use organizational and cognitive shortcuts to gather and use evidence;
These dynamics play out in different ways in each territory, which means that the importers and exporters of lessons are operating in different contexts and addressing inequalities in different ways. Therefore, we must ask how the importers and exporters of lessons: define the problem, decide what policies are feasible, establish which level of government should be responsible for policy and identify criteria to evaluate policy success.
Step 2: Map policymaking responsibilities for the selection of policy instruments.
The Council of Europe defines gender mainstreaming as ‘the (re)organisation, improvement, development and evaluation of policy processes, so that a gender equality perspective is incorporated in all policies at all levels and at all stages’.
Such definitions help explain why mainstreaming approaches often appear to be incoherent. To map the sheer weight of possible measures, and the spread of responsibility across many levels of government (such as local, Scottish, UK and EU), is to identify a potentially overwhelming scale of policymaking ambition. Further, governments tend to address this potential by breaking policymaking into manageable sectors. Each sector has its own rules and logics, producing coherent policymaking in each ‘silo’ but a sense of incoherence overall, particularly if the overarching aim is a low priority in government. Mapping these dynamics and responsibilities is necessary to ensure lessons learned can be effectively applied in similarly complex domestic systems.
Step 3: Learn from experience.
Policy actors want to draw lessons from the most relevant exemplars. Often, they will have implicit or explicit ideas concerning which countries they would like to learn more from. Negotiating which cases to explore, so that it takes into consideration both policy actors’ interests and the need to generate appropriate and useful lessons, is vital.
In the case of mainstreaming, we focused on three exemplar approaches, selected by members of our audience according to perceived levels of ambition: maximal (Sweden), medial (Canada) and minimal (the UK, which controls aspects of Scottish policy). These cases were also justified with reference to the academic literature which often uses these countries as exemplars of different approaches to policy design and implementation.
Step 4: Deliberate and reflect.
Work directly with policy participants to reflect on the implications for policy in their context. Research has many important insights on the challenges to and limitations of policy learning in complex systems. In particular, it suggests that learning cannot be comprehensive and does not lead to the importation of a well-defined package of measures. Bringing these sorts of insights to bear on policy actors’ practical discussions of how lessons can be drawn and applied from elsewhere is necessary, though ultimately insufficient. In our experience so far, step 4 is the biggest obstacle to our impact.
Paul Cairney (2020) ‘The UK Government’s COVID-19 policy: assessing evidence-informed policy analysis in real time’, British Politicshttps://rdcu.be/b9zAk (PDF)
The coronavirus feels like a new policy problem that requires new policy analysis. The analysis should be informed by (a) good evidence, translated into (b) good policy. However, don’t be fooled into thinking that either of those things are straightforward. There are simple-looking steps to go from defining a problem to making a recommendation, but this simplicity masks the profoundly political process that must take place. Each step in analysis involves political choices to prioritise some problems and solutions over others, and therefore prioritise some people’s lives at the expense of others.
My article in British Politics takes us through those steps in the UK, and situates them in a wider political and policymaking context. This post is shorter, and only scratches the surface of analysis.
5 steps to policy analysis
Define the problem.
Perhaps we can sum up the initial UK government approach as: (a) the impact of this virus and illness will be a level of death and illness that could overwhelm the population and exceed the capacity of public services, so (b) we need to contain the virus enough to make sure it spreads in the right way at the right time, so (c) we need to encourage and make people change their behaviour (primarily via hygiene and social distancing). However, there are many ways to frame this problem to emphasise the importance of some populations over others, and some impacts over others.
Identify technically and politically feasible solutions.
Solutions are not really solutions: they are policy instruments that address one aspect of the problem, including taxation and spending, delivering public services, funding research, giving advice to the population, and regulating or encouraging changes to social behaviour. Each new instrument contributes an existing mix, with unpredictable and unintended consequences. Some instruments seem technically feasible (they will work as intended if implemented), but will not be adopted unless politically feasible (enough people support their introduction). Or vice versa. From the UK government’s perspective, this dual requirement rules out a lot of responses.
Use values and goals to compare solutions.
Typical judgements combine: (a) broad descriptions of values such as efficiency, fairness, freedom, security, and human dignity, (b) instrumental goals, such as sustainable policymaking (can we do it, and for how long?), and political feasibility (will people agree to it, and will it make me more or less popular or trusted?), and (c) the process to make choices, such as the extent to which a policy process involves citizens or stakeholders (alongside experts) in deliberation. They combine to help policymakers come to high profile choices (such as the balance between individual freedom and state coercion), and low profile but profound choices (to influence the level of public service capacity, and level of state intervention, and therefore who and how many people will die).
Predict the outcome of each feasible solution.
It is difficult to envisage a way for the UK Government to publicise all of the thinking behind its choices (Step 3) and predictions (Step 4) in a way that would encourage effective public deliberation. People often call for the UK Government to publicise its expert advice and operational logic, but I am not sure how they would separate it from their normative logic about who should live or die, or provide a frank account without unintended consequences for public trust or anxiety. If so, one aspect of government policy is to keep some choices implicit and avoid a lot of debate on trade-offs. Another is to make choices continuously without knowing what their impact will be (the most likely scenario right now).
Make a choice, or recommendation to your client.
Your recommendation or choice would build on these four steps. Define the problem with one framing at the expense of the others. Romanticise some people and not others. Decide how to support some people, and coerce or punish others. Prioritise the lives of some people in the knowledge that others will suffer or die. Do it despite your lack of expertise and profoundly limited knowledge and information. Learn from experts, but don’t assume that only scientific experts have relevant knowledge (decolonise; coproduce). Recommend choices that, if damaging, could take decades to fix after you’ve gone. Consider if a policymaker is willing and able to act on your advice, and if your proposed action will work as intended. Consider if a government is willing and able to bear the economic and political costs. Protect your client’s popularity, and trust in your client, at the same time as protecting lives. Consider if your advice would change if the problem seemed to change. If you are writing your analysis, maybe keep it down to one sheet of paper (in other words, fewer words than in this post up to this point).
Policy analysis is not as simple as these steps suggest, and further analysis of the wider policymaking environment helps describe two profound limitations to simple analytical thought and action.
Policymakers must ignore almost all evidence
The amount of policy relevant information is infinite, and capacity is finite. So, individuals and governments need ways to filter out almost all of it. Individuals combine cognition and emotion to help them make choices efficiently, and governments have equivalent rules to prioritise only some information. They include: define a problem and a feasible response, seek information that is available, understandable, and actionable, and identify credible sources of information and advice. In that context, the vague idea of trusting or not trusting experts is nonsense, and the larger post highlights the many flawed ways in which all people decide whose expertise counts.
They do not control the policy process.
Policymakers engage in a messy and unpredictable world in which no single ‘centre’ has the power to turn a policy recommendation into an outcome.
There are many policymakers and influencers spread across a political system. For example, consider the extent to which each government department, devolved governments, and public and private organisations are making their own choices that help or hinder the UK government approach.
Most choices in government are made in ‘subsystems’, with their own rules and networks, over which ministers have limited knowledge and influence.
The social and economic context, and events, are largely out of their control.
The take home messages (if you accept this line of thinking)
The coronavirus is an extreme example of a general situation: policymakers will always have very limited knowledge of policy problems and control over their policymaking environment. They make choices to frame problems narrowly enough to seem solvable, rule out most solutions as not feasible, make value judgements to try help some more than others, try to predict the results, and respond when the results do not match their hopes or expectations.
This is not a message of doom and despair. Rather, it encourages us to think about how to influence government, and hold policymakers to account, in a thoughtful and systematic way that does not mislead the public or exacerbate the problem we are seeing. No one is helping their government solve the problem by saying stupid shit on the internet (OK, that last bit was a message of despair).
Further reading:
The article (PDF) sets out these arguments in much more detail, with some links to further thoughts and developments.
This series of ‘750 words’ posts summarises key texts in policy analysis and tries to situate policy analysis in a wider political and policymaking context. Note the focus on whose knowledge counts, which is not yet a big feature of this crisis.
These series of 500 words and 1000 words posts (with podcasts) summarise concepts and theories in policy studies.
This is the long version. It is long. Too long to call a blog post. Let’s call it a ‘living document’ that I update and amend as new developments arise (then start turning into a more organised paper). In most cases, I am adding tweets, so the date of the update is embedded. If I add a new section, I will add a date. If you seek specific topics (like ‘herd immunity’), it might be worth doing a search. The short version is shorter.
The coronavirus feels like a new policy problem. Governments already have policies for public health crises, but the level of uncertainty about the spread and impact of this virus seems to be taking it to a new level of policy, media, and public attention. The UK Government’s Prime Minister calls it ‘the worst public health crisis for a generation’.
As such, there is no shortage of opinions on what to do, but there is a shortage of well-considered opinions, producing little consensus. Many people are rushing to judgement and expressing remarkably firm opinions about the best solutions, but their contributions add up to contradictory evaluations, in which:
the government is doing precisely the right thing or the completely wrong thing,
we should listen to this expert saying one thing or another expert saying the opposite.
Lots of otherwise-sensible people are doing what they bemoan in politicians: rushing to judgement, largely accepting or sharing evidence only if it reinforces that judgement, and/or using their interpretation of any new development to settle scores with their opponents.
Yet, anyone who feels, without uncertainty, that they have the best definition of, and solution to, this problem is a fool. If people are also sharing bad information and advice, they are dangerous fools. Further, as Professor Madley puts it (in the video below), ‘anyone who tells you they know what’s going to happen over the next six months is lying’.
In that context, how can we make sense of public policy to address the coronavirus in a more systematic way?
Studies of policy analysis and policymaking do not solve a policy problem, but they at least give us a language to think it through.
In each step, note how quickly it is possible to be overwhelmed by uncertainty and ambiguity, even when the issue seems so simple at first.
Note how difficult it is to move from Step 1, and to separate Step 1 from the others. It is difficult to define the problem without relating it to the solution (or to the ways in which we will evaluate each solution).
Let’s relate that analysis to research on policymaking, to understand the wider context in which people pay attention to, and try to address, important problems that are largely out of their control.
Throughout, note that I am describing a thought process as simply as I can, not a full examination of relevant evidence. I am highlighting the problems that people face when ‘diagnosing’ policy problems, not trying to diagnose it myself. To do so, I draw initially on common advice from the key policy analysis texts (summaries of the texts that policy analysis students are most likely to read) that simplify the process a little too much. Still, the thought process that it encourages took me hours alone (spread over three days) to produce no real conclusion. Policymakers and advisers, in the thick of this problem, do not have that luxury of time or uncertainty.
In our latest guest blog, Jonny Pearson-Stuttard, RSPH Trustee and Public Health Doctor @imperialcollege sets out what we know about the spread of coronavirus to date, and why the Government has taken the measures it hashttps://t.co/XM7zKKjwtE
Provide a diagnosis of a policy problem, using rhetoric and eye-catching data to generate attention.
Identify its severity, urgency, cause, and our ability to solve it. Don’t define the wrong problem, such as by oversimplifying.
Problem definition is a political act of framing, as part of a narrative to evaluate the nature, cause, size, and urgency of an issue.
Define the nature of a policy problem, and the role of government in solving it, while engaging with many stakeholders.
‘Diagnose the undesirable condition’ and frame it as ‘a market or government failure (or maybe both)’.
Coronavirus as a physical problem is not the same as a coronavirus policy problem. To define the physical problem is to identify the nature, spread, and impact of a virus and illness on individuals and populations. To define a policy problem, we identify the physical problem and relate it (implicitly or explicitly) to what we think a government can, and should, do about it. Put more provocatively, it is only a policy problem if policymakers are willing and able to offer some kind of solution.
This point may seem semantic, but it raises a profound question about the capacity of any government to solve a problem like an epidemic, or for governments to cooperate to solve a pandemic. It is easy for an outsider to exhort a government to ‘do something!’ (or ‘ACT NOW!’) and express certainty about what would happen. However, policymakers inside government:
Do not enjoy the same confidence that they know what is happening, or that their actions will have their intended consequences, and
Will think twice about trying to regulate social behaviour under those circumstances, especially when they
Know that any action or inaction will benefit some and punish others.
For example, can a government make people wash their hands? Or, if it restricts gatherings at large events, can it stop people gathering somewhere else, with worse impact? If it closes a school, can it stop children from going to their grandparents to be looked after until it reopens? There are 101 similar questions and, in each case, I reckon the answer is no. Maybe government action has some of the desired impact; maybe not. If you agree, then the question might be: what would it really take to force people to change their behaviour?
The answer is: often too much for a government to consider (in a liberal democracy), particularly if policymakers are informed that it will not have the desired impact.
A couple of key takeaways from our analysis of early COVID-19 dynamics in Wuhan:
1. We estimated that the control measures introduced – unprecedented interventions that will have had a huge social and psychological toll – reduced transmission by around 55% in space of 2 weeks 1/
If so, the UK government’s definition of the policy problem will incorporate this implicit question: what can we do if we can influence, but not determine (or even predict well) how people behave?
Uncertainty about the coronavirus plus uncertainty about policy impact
Now, add that general uncertainty about the impact of government to this specific uncertainty about the likely nature and spread of the coronavirus:
The ideal spread involves all well people sharing the virus first, while all vulnerable people (e.g. older, and/or with existing health problems that affect their immune systems) protected in one isolated space, but it won’t happen like that; so, we are trying to minimise damage in the real world.
We mainly track the spread via deaths, with data showing a major spike appearing one month later, so the problem may only seem real to most people when it is too late to change behaviour
A lot of the spread will happen inside homes, where the role of government is minimal (compared to public places). So, for example, the impact of school closures could be good (isolation) or make things worse (children spreading the virus to vulnerable relatives) (see also ‘we don’t know [if the UKG decision not to close schools] was brilliant or catastrophic’). [Update 18.3.20: as it turned out, the First Minister’s argument for closing Scottish schools was that there were too few teachers available).
The choice in theory is between a rapid epidemic with a high peak, or a slowed-down epidemic over a longer period, but ‘anyone who tells you they know what’s going to happen over the next six months is lying’.
Maybe this epidemic will be so memorable as to shift social behaviour, but so much depends on trying to predict (badly) if individuals will actually change (see also Spiegelhalter on communicating risk).
None of this account tells policymakers what to do, but at least it helps them clarify three key aspects of their policy problem:
The impact of this virus and illness could overwhelm the population, to the extent that it causes mass deaths, causes a level of illness that exceeds the capacity of health services to treat, and contributes to an unpredictable amount of social and economic damage.
We need to contain the virus enough to make sure it (a) spreads at the right speed and/or (b) peaks at the right time. The right speed seems to be: a level that allows most people to recover alone, while the most vulnerable are treated well in healthcare settings that have enough capacity. The right time seems to be the part of the year with the lowest demand on health services (e.g. summer is better than winter). In other words, (a) reduce the size of the peak by ‘flattening the curve’, and/or (b) find the right time of year to address the peak, while (c) anticipating more than one peak.
My impression is that the most frequently-expressed aim is (a) …
Yesterday we entered the Delay phase of our #COVID_19uk Action Plan. @UKScienceChief explained why this is important.
It allows us to #FlattenTheCurve, which means reducing the impact in the short-term to ensure our health & care system can effectively protect vulnerable people pic.twitter.com/1I45C3v38V
— Department of Health and Social Care (@DHSCgovuk) March 13, 2020
… while the UK Government’s Deputy Chief Medical Officer also seems to be describing (b):
Dr Jenny Harries, Deputy Chief Medical Officer, came into Downing Street to answer some of the most commonly asked questions on coronavirus. pic.twitter.com/KCdeHsaz6a
We need to encourage or coerce people to change their behaviour, to look after themselves (e.g. by handwashing) and forsake their individual preferences for the sake of public health (e.g. by self-isolating or avoiding vulnerable people). Perhaps we can foster social trust and empathy to encourage responsible individual action. Perhaps people will only protect others if obliged to do so (compare Stone; Ostrom; game theory).
See also: From across the Ditch: How Australia has to decide on the least worst option for COVID-19 (Prof Tony Blakely on three bad options: (1) the likelihood of ‘elimination’ of the virus before vaccination is low; (2) an 18-month lock-down will help ‘flatten the curve’; (3) ‘to prepare meticulously for allowing the pandemic to wash through society over a period of six or so months. To tool up the production of masks and medical supplies. To learn as quickly as possible which treatments of people sick with COVID-19 saves lives. To work out our strategies for protection of the elderly and those with a chronic condition (for whom the mortality from COVID-19 is much higher’).
Why politicians fear being accused of over reaction. Which in turn might prevent them from reacting appropriately when a real crisis hits 👇🏽👇🏽 https://t.co/UrxHTAs2z5
If you are still with me, I reckon you would have worded those aims slightly differently, right? There is some ambiguity about these broad intentions, partly because there is some uncertainty, and partly because policymakers need to set rather vague intentions to generate the highest possible support for them. However, vagueness is not our friend during a crisis involving such high anxiety. Further, they are only delaying the inevitable choices that people need to make to turn a complex multi-faceted problem into something simple enough to describe and manage. The problem may be complex, but our attention focuses only on a small number of aspects, at the expense of the rest. Examples that have arisen, so far, include to accentuate:
The health of the whole population or people who would be affected disproportionately by the illness.
For example, the difference in emphasis affects the health advice for the relatively vulnerable (and the balance between exhortation and reassurance)
Inequalities in relation to health, socio-economic status (e.g. income, gender, race, ethnicity), or the wider economy.
For example, restrictive measures may reduce the risk of harm to some, but increase the burden on people with no savings or reliable sources of income.
For example, some people are hoarding large quantities of home and medical supplies that (a) other people cannot afford, and (b) some people cannot access, despite having higher need.
For example, social distancing will limit the spread of the virus (see the nascent evidence), but also produce highly unequal forms of social isolation that increase the risk of domestic abuse (possibly exacerbated by school closures) and undermine wellbeing. Or, there will be major policy changes, such as to the rules to detain people under mental health legislation, regarding abortion, or in relation to asylum (note: some of these tweets are from the US, partly because I’m seeing more attention to race – and the consequence of systematic racism on the socioeconomic inequalities so important to COVID-19 mortality – than in the UK).
COVID-19 has brought new focus to women’s continued inequality. Without a gendered response to both the health and economic crises, gender inequality will be further cemented. Read more on the blog: https://t.co/zYxSFpUTNE
“The epidemic has had a huge impact on domestic violence,” said Wan. “According to our statistics, 90% of the causes of violence are related to the COVID-19 epidemic.” https://t.co/xswemtf548
I just asked a DC cop what he’s noticed since the coronavirus sent people home. “More domestic violence,” he said, without missing a beat. https://t.co/kv9zH5VNj1
While black people make up about 12% of Michigan’s population, they make up about 40% of all COVID-19 deaths reported.
A social epidemiologist says the numbers don’t say everything, but there's something that can’t be ignored: inequality. @MichiganRadiohttps://t.co/bWsqFaCrUJ
Available evidence (though injuriously limited) shows that Black people are being infected & dying of #coronavirus at higher rates. Disproportionate Black suffering is what many of us have suspected and feared because it is consistent with the entirety of American history. https://t.co/qzmXvGCGvV
#Coronavirus is not the 'great equalizer'—race matters:
“I believe that the actions and omissions of world leaders in charge of fighting the #COVID19 pandemic will reveal historical and current impacts of colonial violence and continued health inequities” https://t.co/nUuBIKfrVL
— Dr. Malinda S. Smith (@MalindaSmith) April 6, 2020
BAME lives matter, so far they account for:
– 100% of Dr deaths – 50% nurse deaths – 35% of Patients in ICU
Yet account for only 14% of population and account for 44% of NHS staff. Who is asking the questions, why the disparity?https://t.co/VOL8FAmy45
BBC news reports on the disproportionate deaths of African Americans & minorities in the US from #COVID19, but silence on similar issues in the UK. Why? Where is the reporting? Where is the accountability? https://t.co/DkGPjfnWG1
What the coronavirus bill will do: https://t.co/qoBdKKr64H Mental Health Act – detention implemented using just one doctor’s opinion (not 2) & AMHP, & temporarily allow extension or removal of time limits to allow for greater flexibility where services are less able to respond
English obviously, but fascinating that have issued an explicitly ethical framework for COVID decisions re mental health and incapacity. Can Scotland do same? https://t.co/WccPntZOwf
WOW – government has relaxed restrictions on WHERE abortions can take place, temporary inclusion of 'the home' as a legal site for abortion: https://t.co/Vw714fWXEM
Abortion services for women from Northern Ireland remain available free of charge in England. This provision will continue until services are available to meet these needs in Northern Ireland. For more information, visit: https://t.co/YYjop5lSgUpic.twitter.com/M8k95aIisM
BREAKING NEWS!!!! The Home Office have confirmed that ALL evictions and terminations of asylum support have been paused for 3 months. Find out more and read the letter from Home Office Minister Chris Philp confirming this on our website at: https://t.co/KDlVr4PHyP
NEW Editorial: While responding to #COVID19, policy makers should consider the risk of deepening health inequalities. If vulnerable groups are not properly identified, the consequences of this pandemic will be even more devastating https://t.co/BrypuXH6vSpic.twitter.com/hka3nLzxdv
In relation to Prison Rule Changes – these would only ever be used as an absolute last resort, in order to protect staff & those in our care. I can confirm that emergency changes to showering have not been implemented in any establishment.
For example, governments cannot ignore the impact of their actions on the economy, however much they emphasise mortality, health, and wellbeing. Most high-profile emphasis was initially on the fate of large and small businesses, and people with mortgages, but a long period of crisis will a tip the balance from low income to unsustainable poverty (even prompting Iain Duncan Smith to propose policy change), and why favour people who can afford a mortgage over people scraping the money together for rent?
So…. Govt income protection package includes….. 1. 80% of wage costs up to £2500 2. Deferred VAT. 3. £7 billion uplift to Universal Credit and Woring Tax crdit. 4. £1 billion to cover 30% of house rental costs. 5. Self employed to get same as sickness benefit payments.
A need for more communication and exhortation, or for direct action to change behaviour.
The short term (do everything possible now) or long term (manage behaviour over many months).
The Imperial College COVID report is being discussed. But a major takeaway from it will likely survive discussion: the human cost of a pure mitigation strategy is inacceptable, whilst a pure suppression strategy is unsustainable over time (thread)
How to maintain trust in the UK government when (a) people are more or less inclined to trust a the current part of government and general trust may be quite low, and (b) so many other governments are acting differently from the UK.
For example, note the visible presence of the Prime Minister, but also his unusually high deference to unelected experts such as (a) UK Government senior scientists providing direct advice to ministers and the public, and (b) scientists drawing on limited information to model behaviour and produce realistic scenarios (we can return to the idea of ‘evidence-based policymaking’ later). This approach is not uncommon with epidemics/ pandemics (LD was then the UK Government’s Chief Medical Officer):
For example, note how often people are second guessing and criticising the UK Government position (and questioning the motives of Conservative ministers).
For example, people often try to lay blame for viruses on certain populations, based on their nationality, race, ethnicity, sexuality, or behaviour (e.g. with HIV).
For example, the (a) association between the coronavirus and China and Chinese people (e.g. restrict travel to/ from China; e.g. exacerbate racism), initially overshadowed (b) the general role of international travellers (e.g. place more general restrictions on behaviour), and (c) other ways to describe who might be responsible for exacerbating a crisis.
For social scientists wondering “what can I do now?” here’s a challenge:@cp_roth@LukasHenselEcon & others ran a survey with 2500 Italians yday & found that:
Under ‘normal’ policymaking circumstances, we would expect policymakers to resolve this ambiguity by exercising power to set the agenda and make choices that close off debate. Attention rises at first, a choice is made, and attention tends to move on to something else. With the coronavirus, attention to many different aspects of the problem has been lurching remarkably quickly. The definition of the policy problem often seems to be changing daily or hourly, and more quickly than the physical problem. It will also change many more times, particularly when attention to each personal story of illness or death prompts people to question government policy every hour. If the policy problem keeps changing in these ways, how could a government solve it?
@alexwickham doing fine work as a journalist again. Gets right into Government somehow and tells people what is going on.
10 Days That Changed Britain: "Heated" Debate Between Scientists Forced Boris Johnson To Act On Coronavirus https://t.co/hDLEAPT3Z0
Public expenditure (e.g. to boost spending for emergency care, crisis services, medical equipment)
Economic incentives and disincentives (e.g. to reduce the cost of business or borrowing, or tax unhealthy products)
Linking spending to entitlement or behaviour (e.g. social security benefits conditional on working or seeking work, perhaps with the rules modified during crises)
Formal regulations versus voluntary agreements (e.g. making organisations close, or encouraging them to close)
Public services: universal or targeted, free or with charges, delivered directly or via non-governmental organisations
As a result, what we call ‘policy’ is really a complex mix of instruments adopted by one or more governments. A truism in policy studies is that it is difficult to define or identify exactly what policy is because (a) each new instrument adds to a pile of existing measures (with often-unpredictable consequences), and (b) many instruments designed for individual sectors tend, in practice, to intersect in ways that we cannot always anticipate. When you think through any government response to the coronavirus, note how every measure is connected to many others.
Further, it is a truism in public policy that there is a gap between technical and political feasibility: the things that we think will be most likely to work as intended if implemented are often the things that would receive the least support or most opposition. For example:
Redistributing income and wealth to reduce socio-economic inequalities (e.g. to allay fears about the impact of current events on low-income and poverty) seems to be less politically feasible than distributing public services to deal with the consequences of health inequalities.
Providing information and exhortation seems more politically feasible than the direct regulation of behaviour. Indeed, compared to many other countries, the UK Government seems reluctant to introduce ‘quarantine’ style measures to restrict behaviour.
Under ‘normal’ circumstances, governments may be using these distinctions as simple heuristics to help them make modest policy changes while remaining sufficiently popular (or at least looking competent). If so, they are adding or modifying policy instruments during individual ‘windows of opportunity’ for specific action, or perhaps contributing to the sense of incremental change towards an ambitious goal.
Right now, we may be pushing the boundaries of what seems possible, since crises – and the need to address public anxiety – tend to change what seems politically feasible. However, many options that seem politically feasible may not be possible (e.g. to buy a lot of extra medical/ technology capacity quickly), or may not work as intended (e.g. to restrict the movement of people). Think of technical and political feasibility as necessary but insufficient on their own, which is a requirement that rules out a lot of responses.
Add in the UK legislation and we see that it is a major feat simply to account for all of the major moving parts (while noting that much policy change is not legislative)https://t.co/gKsIx7aHr2pic.twitter.com/Ms6fjaDbhF
A few 'somewhat overwritten' newspaper stories today using some of our quotes on PPE. Here is exactly what we are saying, in the tone in which we are saying it: https://t.co/j6PO420WSF
Typical value judgements relate to efficiency, equity and fairness, the trade-off between individual freedom and collective action, and the extent to which a policy process involves citizens in deliberation.
Normative assessments are based on values such as ‘equality, efficiency, security, democracy, enlightenment’ and beliefs about the preferable balance between state, communal, and market/ individual solutions
‘Specify the objectives to be attained in addressing the problem and the criteria to evaluate the attainment of these objectives as well as the satisfaction of other key considerations (e.g., equity, cost, equity, feasibility)’.
‘Effectiveness, efficiency, fairness, and administrative efficiency’ are common.
Identify (a) the values to prioritise, such as ‘efficiency’, ‘equity’, and ‘human dignity’, and (b) ‘instrumental goals’, such as ‘sustainable public finance or political feasibility’, to generate support for solutions.
Instrumental questions may include: Will this intervention produce the intended outcomes? Is it easy to get agreement and maintain support? Will it make me popular, or diminish trust in me even further?
How to weigh the many future health problems and deaths caused by the lockdown against those saved? How to account for the worse effects of the lockdown on the young and the poor? Near impossible ethical choices that government will have to make. https://t.co/DJgwE4b3rd
Step 3 is the most simple-looking but difficult task. Remember that it is a political, not technical, process. It is also a political process that most people would like to avoid doing (at least publicly) because it involves making explicit the ways in which we prioritise some people over others. Public policy is the choice to help some people and punish or refuse to help others (and includes the choice to do nothing).
Policy analysis texts describe a relatively simple procedure of identifying criteria and producing a table (with a solution in each row, and criteria in each column) to compare the trade-offs between each solution. However, these criteria are notoriously difficult to define, and people resolve that problem by exercising power to decide what each term means, and whose interests should be served when they resolve trade-offs. For example, see Stone on whose needs come first, who benefits from each definition of fairness, and how technical-looking processes such as ‘cost benefit analysis’ mask political choices.
Right now, the most obvious and visible trade-off, accentuated in the UK, is between individual freedom and collective action, or the balance between state, communal, and market/ individual solutions. In comparison with many countries (and China and Italy in particular), the UK Government seems to be favouring individual action over state quarantine measures. However, most trade-offs are difficult to categorise
What should be the balance between efforts to minimise the deaths of some (generally in older populations) and maximise the wellbeing of others? This is partly about human dignity during crisis, how we treat different people fairly, and the balance of freedom and coercion.
How much should a government spend to keep people alive using intensive case or expensive medicines, when the money could be spent improving the lives of far more people? This is partly about human dignity, the relative efficiency of policy measures, and fairness.
If you are like me, you don’t really want to answer such questions (indeed, even writing them looks callous). If so, one way to resolve them is to elect policymakers to make such choices on our behalf (perhaps aided by experts in moral philosophy, or with access to deliberative forums). To endure, this unusually high level of deference to elected ministers requires some kind of reciprocal act:
"I hope the UK government will be transparent about its decision-making; willing to listen to NHS staff concerns; humble in learning from other countries’ experiences; and pro-active in building relationships with them."https://t.co/CYUyvij2bK
I agree. There is a need to show that divergent opinons in the public health/virology expert sector have been heard, debates have been had and conclusions explained. This is what I need as a citizen. Also casting the public not a bog roll stowing mob is not helpful or kind. https://t.co/g61Nypcqlc
The Guardian calls this document a “secret” briefing from Public Health England. At a time of national crisis there is no place for secrecy from health experts. If you want public support, share your data, scenarios, and forecasts. Now. https://t.co/O8BpDlCJ7H
I am glad Johnson has listened, but we shouldn't have to drag him kicking and screaming to these decisions. A daily update is a basic step. Transparency, honesty, compassion are vital in this time of a global crisis! no more secret briefings PM.https://t.co/eMxZnMehUp
The CSA and CMO say they will publish the models underlying their strategy on Covid-19. Sharing the data and models is important for accountability, testing and learning. https://t.co/rOuJWwy93i
Dear Boris – Number 10 needs a professional communications operation, immediately. (Open letter to the Prime Minister. Britain has some great comms specialists. He needs to hire one of them urgently) https://t.co/8w6MBYHHbm
Still, I doubt that governments are making reportable daily choices with reference to a clear and explicit view of what the trade-offs and priorities should be, because their choices are about who will die, and their ability to predict outcomes is limited.
Focus on the outcomes that key actors care about (such as value for money), and quantify and visualise your predictions if possible. Compare the pros and cons of each solution, such as how much of a bad service policymakers will accept to cut costs.
‘Assess the outcomes of the policy options in light of the criteria and weigh trade-offs between the advantages and disadvantages of the options’.
Estimate the cost of a new policy, in comparison with current policy, and in relation to factors such as savings to society or benefits to certain populations. Use your criteria and projections to compare each alternative in relation to their likely costs and benefits.
Explain potential solutions in sufficient detail to predict the costs and benefits of each ‘alternative’ (including current policy).
Short deadlines dictate that you use ‘logic and theory, rather than systematic empirical evidence’ to make predictions efficiently.
Monitoring is crucial because it is difficult to predict policy success, and unintended consequences are inevitable. Try to measure the outcomes of your solution, while noting that evaluations are contested.
It is difficult to envisage a way for the UK Government to publicise the thinking behind its choices (Step 3) and predictions (Step 4) in a way that would encourage effective public deliberation, rather than a highly technical debate between a small number of academics:
Ferguson et al (link) simulate outbreak response. Positive: They show suppression (lockdown R0<1) is essential as mitigation (R0>1, “flattening the curve”) results in massive hospital overload and many dead. BUT 1/3 (review attached)https://t.co/srbBS7F1s5pic.twitter.com/qbEymBdOqm
I’m conscious that lots of people would like to see and run the pandemic simulation code we are using to model control measures against COVID-19. To explain the background – I wrote the code (thousands of lines of undocumented C) 13+ years ago to model flu pandemics…
Further, people often call for the UK Government to publicise its expert advice and operational logic, but I am not sure how they would separate it from their normative logic, or provide a frank account without unintended consequences for public trust or anxiety. If so, government policy involves (a) to keep some choices implicit to avoid a lot of debate on trade-offs, and (b) to make general statements about choices when they do not know what their impact will be.
Examine your case through the eyes of a policymaker. Keep it simple and concise.
Make a preliminary recommendation to inform an iterative process, drawing feedback from clients and stakeholder groups
Client-oriented advisors identify the beliefs of policymakers and tailor accordingly.
‘Unless your client asks you not to do so, you should explicitly recommend one policy’
I now invite you to make a recommendation (step 5) based on our discussion so far (steps 1-4). Define the problem with one framing at the expense of the others. Romanticise some people and not others. Decide how to support some people, and coerce or punish others. Prioritise the lives of some people in the knowledge that others will suffer or die. Do it despite your lack of expertise and profoundly limited knowledge and information. Learn from experts, but don’t assume that only scientific experts have relevant knowledge (decolonise; coproduce). Recommend choices that, if damaging, could take decades to fix after you’ve gone. Consider if a policymaker is willing and able to act on your advice, and if your proposed action will work as intended. Consider if a government is willing and able to bear the economic and political costs. Protect your client’s popularity, and trust in your client, at the same time as protecting lives. Consider if your advice would change if the problem would seem to change. If you are writing your analysis, maybe keep it down to one sheet of paper (and certainly far fewer words than in this post). Better you than me.
Please now watch this video before I suggest that things are not so simple.
Would that policy analysis were so simple
Imagine writing policy analysis in an imaginary world, in which there is a single powerful ‘rational’ policymaker at the heart of government, making policy via an orderly series of stages.
Your audience would be easy to identify at each stage, your analysis would be relatively simple, and you would not need to worry about what happens after you make a recommendation for policy change (since the selection of a solution would lead to implementation). You could adopt a simple 5 step policy analysis method, use widely-used tools such as cost-benefit analysis to compare solutions, and know where the results would feed into the policy process.
Studies of policy analysts describe how unrealistic this expectation tends to be (Radin, Brans, Thissen).
For example, there are many policymakers, analysts, influencers, and experts spread across political systems, and engaging with 101 policy problems simultaneously, which suggests that it is not even clear how everyone fits together and interacts in what we call (for the sake of simplicity) ‘the policy process’.
Instead, we can describe real world policymaking with reference to two factors.
The wider policymaking environment: 1. Limiting the use of evidence
First, policymakers face ‘bounded rationality’, in which they only have the ability to pay attention to a tiny proportion of available facts, are unable to separate those facts from their values (since we use our beliefs to evaluate the meaning of facts), struggle to make clear and consistent choices, and do not know what impact they will have. The consequences can include:
Limited attention, and lurches of attention. Policymakers can only pay attention to a tiny proportion of their responsibilities, and policymaking organizations struggle to process all policy-relevant information. They prioritize some issues and information and ignore the rest.
Power and ideas. Some ways of understanding and describing the world dominate policy debate, helping some actors and marginalizing others.
Beliefs and coalitions. Policymakers see the world through the lens of their beliefs. They engage in politics to turn their beliefs into policy, form coalitions with people who share them, and compete with coalitions who don’t.
Dealing with complexity. They engage in ‘trial-and-error strategies’ to deal with uncertain and dynamic environments (see the new section on trial-and-error- at the end).
Framing and narratives. Policy audiences are vulnerable to manipulation when they rely on other actors to help them understand the world. People tell simple stories to persuade their audience to see a policy problem and its solution in a particular way.
The social construction of populations. Policymakers draw on quick emotional judgements, and social stereotypes, to propose benefits to some target populations and punishments for others.
Rules and norms. Institutions are the formal rules and informal understandings that represent a way to narrow information searches efficiently to make choices quickly.
Learning. Policy learning is a political process in which actors engage selectively with information, not a rational search for truth.
Evidence-based or expert-informed policymaking
Don’t think science can or should make decisions Donna. In conditions of uncertainty, it must inform decision makers who must be transparent about the choices they make and be held to account for them https://t.co/Wj4s9IS6fO
Put simply, policymakers cannot oversee a simple process of ‘evidence-based policymaking’. Rather, to all intents and purposes:
They need to find ways to ignore most evidence so that they can focus disproportionately on some. Otherwise, they will be unable to focus well enough to make choices. The cognitive and organisational shortcuts, described above, help them do it almost instantly.
They also use their experience to help them decide – often very quickly – what evidence is policy-relevant under the circumstances. Relevance can include:
How it relates to the policy problem as they define it (Step 1).
If it relates to a feasible solution (Step 2).
If it is timely, available, understandable, and actionable.
If it seems credible, such as from groups representing wider populations, or from people they trust.
They use a specific shortcut: relying on expertise.
However, the vague idea of trusting or not trusting experts is a nonsense, largely because it is virtually impossible to set a clear boundary between relevant/irrelevant experts and find a huge consensus on (exactly) what is happening and what to do. Instead, in political systems, we define the policy problem or find other ways to identify the most relevant expertise and exclude other sources of knowledge.
In the UK Government’s case, it appears to be relying primarily on expertise from its own general scientific advisers, medical and public health advisers, and – perhaps more controversially – advisers on behavioural public policy.
Not a thread but an interesting exchange on #coronavirus & Behavioural Sciences including readings from @LiamDelaneyEcon https://t.co/7Yn89XwOk6
Here’s my article on why I wish my fellow psychologists and “behavioural scientists” would just stop talking about the coronavirus: https://t.co/ofjJWdIY9v
Right now, it is difficult to tell exactly how and why it relies on each expert (at least when the expert is not in a clearly defined role, in which case it would be irresponsible not to consider their advice). Further, there are regular calls on Twitter for ministers to be more open about their decisions.
Key point from @jameswilsdon 'It is problematic if political choices are being made and then the science advice system has to front them up. There needs to be a clearer sense of where science advice ends and political judgement begins.'https://t.co/TjLCJDZijO via @timeshighered
However, don’t underestimate the problems of identifying why we make choices, then justifying one expert or another (while avoiding pointless arguments), or prioritising one form of advice over another. Look, for example, at the kind of short-cuts that intelligent people use, which seem sensible enough, but would receive much more intense scrutiny if presented in this way by governments:
Sophisticated speculation by experts in a particular field, shared widely (look at the RTs), but questioned by other experts in another field:
2. This all assumes I'm correct in what I think the govt are doing and why. I could be wrong – and wouldn't be surprised. But it looks to me like. . .
— Professor Ian Donald 3.5% (@iandonald_psych) March 13, 2020
As many have said, it would be good to get an official version of this, with acknowledged uncertainties and sources of evidence https://t.co/jxgoysYb3L
But what happened is that they have as a group fallen into a logical error in their attempts to model what will bring this epidemic under control. They have not appreciated that the answer to this question is adaptive behavior change. 3/17
It would be really helpful to project risk of covid death with and without mitigation strategies? Possible to map with inside / outside projections (ie what we gain/ don’t gain with current measures ?)
Experts in one field trusting certain experts in another field based on personal or professional interaction:
Lots of concern about UK's approach to #COVID19. I'm not an epidemiologist or a virologist (=> can't judge the detail) but I knew Patrick Vallance before he was famous and I believe he is a man of integrity. Same for Chris Whitty. Science, not politics, is driving their thinking.
— Trisha Greenhalgh 😷 #BlackLivesMatter (@trishgreenhalgh) March 14, 2020
Experts in one field not trusting a government’s approach based on its use of one (of many) sources of advice:
Why is UK government listening to the ‘nudge unit’ on the pandemic, and not expert epidemiologists and the WHO? You would think the ‘anti-experts’ approach would have at least on this occasion, with so many lives at risk, given way to a scientific approach https://t.co/QZIicXYpsj
Experts representing a community of experts, criticising another expert (Prof John Ashton), for misrepresenting the amount of expert scepticism of government experts (yes, I am trying to confuse you):
The Chief Medical Officer @CMO_England and his team have the 100% support and backing of the Public Health community. Every DPH I know thinks he is doing an amazing job in difficult circumstances Sorry but JRA is just demonstrating he is out of touch on this https://t.co/ExmOjEgum0
Expert debate on how well policymakers are making policy based on expert advice
Disagree.
Not much audible consensus amongst scientists anywhere for UK approach. Science can only illuminate value judgements yet now used a shield for determining them. UK science advice has always been characterised by old boys, political operators. Blurring is concerning. https://t.co/iBt07QfvqH
Finding quite-sensible ways to trust certain experts over others, such as because they can be held to account in some way (and may be relatively worried about saying any old shit on the internet):
My current approach to making sense of conflicting expert opinion on #coronavirus: no expert is infallible, but some are accountable and others are not, and I will value the opinions of those who are accountable above the opinions of those who are not.
There are many more examples in which the shortcut to expertise is fine, but not particularly better than another shortcut (and likely to include a disproportionately high number of white men with STEM backgrounds).
Update: of course, they are better than the volume trumps expertise approach:
This meme is spreading (you could say, in a not very funny joke, that it has gone viral). The WHO Director-General did not say this (brief thread). https://t.co/3eMfy70tKZ
For what it’s worth, I tend to favour experts who:
(a) establish the boundaries of their knowledge, (b) admit to high uncertainty about the overall problem:
After having spent considerable time thinking how to mitigate and manage this pandemic, and analysing the available data. I failed to identify the best course of action. Even worse, I'm not sure there is such a thing as an acceptable solution to the problem we are facing. (2/12)
— Prof Francois Balloux (@BallouxFrancois) March 14, 2020
I would challenge anyone to provide an accurate estimate of prevalence. The difference between models & real life is that with models we can set the parameters as if they are known. In real life these parameters are as clear as mud. Extract 04/13/2020 https://t.co/Qg2OrCo8tR
(c) (in this case) make it clear that they are working on scenarios, not simple prediction
I am deeply uncomfortable with the message that UK is actively pursuing ‘herd immunity’ as the main COVID-19 strategy. Our group’s scenario modelling has focused on reducing two main things: peak healthcare demand and deaths… 1/
"Prediction models are just estimates of what might happen and a model is only as good as the data that goes into it." https://t.co/KXDILsbZgr via @ConversationUK
(d) examine critically the too-simple ideas that float around, such as the idea that the UK Government should emulate ‘what works’ somewhere else
It's easy to say 'let's just do what Wuhan did', but the measures there have involved a change to daily life that really has been unimaginable in scale and impact. And as we've seen, China cannot sustain them indefinitely. 3/
A lot of my colleagues in the @LSHTM modelling centre (@cmmid_lshtm) have been working extremely hard to help expand the COVID-19 evidence base over the past two months. I'd like to take a moment to highlight some of their work… 1/
8. There's no gotcha-ism. Updating your models and predictions in light of new evidence and new inferential methods and insightful counterpoints from colleagues isn't a sign of weakness, it's *doing science*.
I do not agree with this interpretation. Multiple papers that tested people at high risk found that asymptomatic infection is relatively uncommon, in the range of 6-32%. https://t.co/gv5e2upEwz
(e) situate their own position (in Prof Sridhar’s case, for mass testing) within a broader debate
Scientific community is well-intentioned but split in two camps: one argues why sacrifice short-term social/economic well-being if everyone will get virus regardless, & other which says we have to buy time in short-term & save lives now while figuring out exit plan.
How much effort does your govt want to put into suppressing this outbreak? There is no quick fix or easy solution. S.Korea & Germany show what huge govt effort, planning, strong leadership, & doing utmost to protect population look like. Do everything v. do minimum.
Been saying 3 objectives for weeks. Not to attack anyone, but to highlight what we have learned so far: 1. Testing, tracing, isolating 2. Protect health workers with PPE & testing 3. Buy time for NHS
Two weeks ago Boris Johnson said Britain was aiming to eventually test 250,000 people a day. The reality is still far off the aspiration https://t.co/2SHX40B9Ul
My new blog on whether Covid raises everyone’s relative risk of dying by a similar amount. https://t.co/76NSNuDJ3i Latest ONS data shows that, of recent death registrations, the proportion linked to Covid does not depend on age.
However, note that most of these experts are from a very narrow social background, and from very narrow scientific fields (first in modelling, then likely in testing), despite the policy problem being largely about (a) who, and how many people, a government should try to save, and (b) how far a government should go to change behaviour to do it (Update 2.4.20: I wrote that paragraph before adding so many people to the list). It is understandable to defer in this way during a crisis, but it also contributes to a form of ‘depoliticisation’ that masks profound choices that benefit some people and leave others vulnerable to harm.
— Louis M M Coiffait (@LouisMMCoiffait) April 6, 2020
See also: ‘What’s important is social distancing’ coronavirus testing ‘is a side issue’, says Deputy Chief Medical Officer [Professor Jonathan Van-Tam talks about the important distinction between a currently available test to see if someone has contracted the virus (an antigen test) and a forthcoming test to see if someone has had and recovered from COVID-19 (an antibody test)]. The full interview is here (please feel free to ignore the editorialising of the uploader):
We might need to change our criteria to decide on capacity and resources. COVID-19 showed that the standard CEO approach of doing more with less is no good. German planners have apparently safely ignored this holy managerial mantra. @Breconomicshttps://t.co/MKi3f1Pueq
Cross country comparisons of the efficacy of anti covid19 policies are going to be hard. There are so many likely inputs; and data on them is scarce and noisy.
The UK Govts chief medical officer has conceded that Germany “got ahead” in testing people for Covid-19 and said the UK needed to learn from that. Ministers have been challenged repeatedly during the pandemic over their failure to increase testing. https://t.co/V0bgcMR7l0
He says there is not as much scrutiny as we might normally wish and says concerns raised about human rights, the length of powers and need for safeguards should be heeded in Westminster. He also commits to legislate for reporting requirements for use of powers by SG 4/5
Glad Scottish Government recognise need for ethical guidance on Covid 19, and hope they can say more on human rights in next version https://t.co/GiyTd2Xksu
This is an excellent initiative from @policescotland – commissioning @johndscott to provide independent scrutiny of new Coronavirus Emergency Powers. Policing is by consent of the people, this step hopefully gives further public reassurance on the application of powers https://t.co/6MtrqdTqIm
Unprecedented restrictions are in force in order to limit social contact and slow the spread of the coronavirus. But the govt and police must make clear what is enforceable and what is guidance if they are to retain the trust and confidence of the public https://t.co/ieLcg2qVE5pic.twitter.com/mBOK2fppH2
— Institute for Gov (@instituteforgov) April 5, 2020
The wider policymaking environment: 2. Limited control
Second, policymakers engage in a messy and unpredictable world in which no single ‘centre’ has the power to turn a policy recommendation into an outcome. I normally use the following figure to think through the nature of a complex and unwieldy policymaking environment of which no ‘centre’ of government has full knowledge or control.
It helps us identify (further) the ways in which we can reject the idea that the UK Prime Minister and colleagues can fully understand and solve policy problems:
Actors. The environment contains many policymakers and influencers spread across many levels and types of government (‘venues’).
For example, consider how many key decisions that (a) have been made by organisations not in the UK central government, and (b) are more or less consistent with its advice, including:
Devolved governments announcing their own healthcare and public health responses (although the level of UK coordination seems more significant than the level of autonomy).
Public sector employers initiating or encouraging at-home working (and many Universities moving quickly from in-person to online teaching)
Private organisations cancelling cultural and sporting events.
There’s some coverage today suggesting Scotland proposing different policy to rest of UK on over 70s. This isn’t so. The policy of social distancing, not isolation, set out here by @jasonleitch is the policy all 4 nations have been discussing at COBR – and will do so again today. https://t.co/D89nwUDZTb
This is interesting, particularly the contrast with the approach to Brexit. The key difference is that Brexit blurred the boundaries between reserved and devolved competences in a way that health does not. https://t.co/4kSIcQFmJf
Context and events. Policy solutions relate to socioeconomic context and events which can be impossible to ignore and out of the control of policymakers. The coronavirus, and its impact on so many aspects on population health and wellbeing, is an extreme example of this problem.
Networks, Institutions, and Ideas. Policymakers and influencers operate in subsystems (specialist parts of political systems). They form networks or coalitions built on the exchange of resources or facilitated by trust underpinned by shared beliefs or previous cooperation. Many different parts of government have practices driven by their own formal and informal rules. Formal rules are often written down or known widely. Informal rules are the unwritten rules, norms and practices that are difficult to understand, and may not even be understood in the same way by participants. Political actors relate their analysis to shared understandings of the world – how it is, and how it should be – which are often so established as to be taken for granted. These dominant frames of reference establish the boundaries of the political feasibility of policy solutions. These kinds of insights suggest that most policy decisions are considered, made, and delivered in the name of – but not in the full knowledge of – government ministers.
Trial and error policymaking in complex policymaking systems (17.3.20)
One way of viewing the UK's COVID-19 policy is that it changed to reflect changing evidence. That is fair; it's both how science-guided policy *should* work, and how I think the govt's advisors are seeing it, as per the Imperial paper. But… 1/
There are many ways to conceptualise this policymaking environment, but few theories provide specific advice on what to do, or how to engage effectively in it. One notable exception is the general advice that comes from complexity theory, including:
Law-like behaviour is difficult to identify – so a policy that was successful in one context may not have the same effect in another.
Policymaking systems are difficult to control; policy makers should not be surprised when their policy interventions do not have the desired effect.
Policy makers in the UK have been too driven by the idea of order, maintaining rigid hierarchies and producing top-down, centrally driven policy strategies. An attachment to performance indicators, to monitor and control local actors, may simply result in policy failure and demoralised policymakers.
Policymaking systems or their environments change quickly. Therefore, organisations must adapt quickly and not rely on a single policy strategy.
On this basis, there is a tendency in the literature to encourage the delegation of decision-making to local actors:
Rely less on central government driven targets, in favour of giving local organisations more freedom to learn from their experience and adapt to their rapidly-changing environment.
To deal with uncertainty and change, encourage trial-and-error projects, or pilots, that can provide lessons, or be adopted or rejected, relatively quickly.
Encourage better ways to deal with alleged failure by treating ‘errors’ as sources of learning (rather than a means to punish organisations) or setting more realistic parameters for success/ failure (although see this example and this comment).
Encourage a greater understanding, within the public sector, of the implications of complex systems and terms such as ‘emergence’ or ‘feedback loops’.
In other words, this literature, when applied to policymaking, tends to encourage a movement from centrally driven targets and performance indicators towards a more flexible understanding of rules and targets by local actors who are more able to understand and adapt to rapidly-changing local circumstances.
Now, just imagine the UK Government taking that advice right now. I think it is fair to say that it would be condemned continuously (even more so than right now). Maybe that is because it is the wrong way to make policy in times of crisis. Maybe it is because too few people are willing and able to accept that the role of a small group of people at the centre of government is necessarily limited, and that effective policymaking requires trial-and-error rather than a single, fixed, grand strategy to be communicated to the public. The former highlights policy that changes with new information and perspective. The latter highlights errors of judgement, incompetence, and U-turns. In either case, the advice is changing as estimates of the coronavirus’ impact change:
I think this tension, in the way that we understand UK government, helps explain some of the criticism that it faces when changing its advice to reflect changes in its data or advice. This criticism becomes intense when people also question the competence or motives of ministers (and even people reporting the news) more generally, leading to criticism that ranges from mild to outrageous:
Incredible detail in this FT story: up until last week, the UK was basing its coronavirus control policy on a model based on hospitalisation rates for 😲a different disease😲 with lower rates of intensive care need than coronavirus pic.twitter.com/7rJYh9sqg2
Laura Kuenssberg says (BBC) that, “The science has changed.” This is not true. The science has been the same since January. What has changed is that govt advisors have at last understood what really took place in China and what is now taking place in Italy. It was there to see.
We can’t keep changing our #COVID19 control policies whenever the results of the “mathematical modelling” change. We need to implement standard WHO-approved epidemic control policies hard and fast, as well as providing more support to frontline NHS staff. https://t.co/HAM9OqbmqW
There may be perfectly valid or at least debatable reasons for each but obfuscation does not help public to understand uncertainty around decisions. In other words, not communicating rationale = incompetence (as in incompetent in terms of state craft, not nec individual decision)
One wonders if Brit leaders have decided that the ultimate way to cut national budgets is to cull the herd of the weak, those who require costly NHS care, and pray for "herd immunity" among the rest. Cruel, cost effective #COVID19 strategy?@richardhorton1
For me, this casual reference to a government policy to ‘cull the heard of the weak’ is outrageous, but you can find much worse on Twitter. It reflects wider debate on whether ‘herd immunity’ is or is not government policy. Much of it relates to interpretation of government statements, based on levels of trust/distrust in the UK Government, its Prime Minister and Secretaries of State, and the Prime Minister’s special adviser
I have enormous respect for the SAGE team and scientific advisors trying to understand the situation & inform the UK's response. If this article is accurate & partisan hacks were deliberately sacrificing lives based on their information, its scandalous. A week ago I was saying… https://t.co/WYsHbj6o0a
If you read the whole article you will see that Dominic Cummings has been, for the last 10 days, the most zealous advocate of a tough lockdown. Which is what his critics seem to want. The world is not black and white
1. Wilful misinterpretation (particularly on Twitter). For example, in the early development and communication of policy, Boris Johnson was accused (in an irresponsibly misleading way) of advocating for herd immunity rather than restrictive measures.
Below is one of the most misleading videos of its type. Look at how it cuts each segment into a narrative not provided by ministers or their advisors (see also this stinker):
The herd immunity strategy would’ve likely caused hundreds of thousands of deaths. They even told us so.
2. The accentuation of a message not being emphasised by government spokespeople.
See for example this interview, described by Sky News (13.3.20) as: The government’s chief scientific adviser Sir Patrick Vallance has told Sky News that about 60% of people will need to become infected with coronavirus in order for the UK to enjoy “herd immunity”. You might be forgiven for thinking that he was on Sky extolling the virtues of a strategy to that end (and expressing sincere concerns on that basis). This was certainly the write-up in respected papers like the FT (UK’s chief scientific adviser defends ‘herd immunity’ strategy for coronavirus). Yet, he was saying nothing of the sort. Rather, when prompted, he discussed herd immunity in relation to the belief that COVID-19 will endure long enough to become as common as seasonal flu.
The same goes for Vallance’s interview on the same day (13.3.20) during Radio 4’s Today programme (transcribed by the Spectator, which calls Vallance the author, and gives ittheheadline “How ‘herd immunity’ can help fight coronavirus” as if it is his main message). The Today Programme also tweeted only 30 seconds to single out that brief exchange:
Sir Patrick Vallance, the govt chief scientific adviser, says the thinking behind current approach to #coronavirus is to try and "reduce the peak" and to build up a "degree of herd immunity so that more people are immune to the disease". #R4Today
Yet, clearly his overall message – in this and other interviews – was that some interventions (e.g. staying at home; self-isolating with symptoms) would have bigger effects than others (e.g. school closures; prohibiting mass gatherings) during the ‘flattening of the peak’ strategy (‘What we don’t want is everybody to end up getting it in a short period of time so that we swamp and overwhelm NHS services’). Rather than describing ‘herd immunity’ as a strategy, he is really describing how to deal with its inevitability (‘Well, I think that we will end up with a number of people getting it’).
For anyone who thinks it was all obvious in January and February reading these minutes is a sobering experience. What comes over is the real uncertainty about what could be foretold from the Chinese experience and the ease with which the disease could be transmitted.4/n
Toby Young 'expert'. Nobody, including the Oxford team, believes this is true. Shame on The Sun for publishing this irresponsible rubbish. Shame on Toby Young for cynical misrepresentation of the science. pic.twitter.com/17hrOPW9b8
[OK, that proved to be a big departure from the trial-and-error discussion. Here we are, back again]
In some cases, maybe people are making the argument that trial-and-error is the best way to respond quickly, and adapt quickly, in a crisis but that the UK Government version is not what, say, the WHO thinks of as good kind of adaptive response. It is not possible to tell, at least from the general ways in which they justify acting quickly.
Dr Michael J Ryan, Executive Director at WHO. An off the cuff answer to a question at today's virtual press conference. Inspiring stuff! pic.twitter.com/Q4EUs8V1dG
The coronavirus is an extreme example of a general situation: policymakers will always have very limited knowledge of policy problems and control over their policymaking environment. They make choices to frame problems narrowly enough to seem solvable, rule out most solutions as not feasible, make value judgements to try help some more than others, try to predict the results, and respond when the results to not match their hopes or expectations.
This is not a message of doom and despair. Rather, it encourages us to think about how to influence government, and hold policymakers to account, in a thoughtful and systematic way that does not mislead the public or exacerbate the problem we are seeing.
Further reading, until I can think of a better conclusion:
This series of ‘750 words’ posts summarises key texts in policy analysis and tries to situate policy analysis in a wider political and policymaking context. Note the focus on whose knowledge counts, which is not yet a big feature of this crisis.
These series of 500 words and 1000 words posts (with podcasts) summarise concepts and theories in policy studies.
The scientific response to COVID-19 demands speed. But changing incentives and norms in academic science may be pushing the enterprise toward fast science at the expense of good science. Read Dan Sarewitz's editor's journal in the Spring 2020 ISSUES: https://t.co/JSSS45eTze
— Issues in Science and Technology (@ISSUESinST) April 7, 2020
#politvirus Public Health has always been #political because it’s actions impact on politics, economics, commercial interests, personal freedoms – this becomes most obvious in crisis – it will be key to analyse the political responses to #Covid_19 if we want to be better prepared https://t.co/JkUZrVeAxv
An assessment of the Government's response to date – written by Chair of Global Health at Edinburgh University..Prof Devi Sridhar https://t.co/N31QtFmQ2p
This is a really important paper. Partisanship is a huge influence on timing of state public health measures- Republican governors and Trump majorities slow adoption of measures. This might have big mortality effects in a few weeks. https://t.co/BEOAM69aSw
One reason Germany has so many ventilators (and intensive care beds) given in The Times: Not just more money in the system but design of hospital payment rates through the insurance system has driven up ICU investment be hospital managers pic.twitter.com/7R062IJI2k
This is worrying. Singapore was held up as one of the models for how to control #COVID19 through a sophisticated programme of testing and tracing without having to resort to the kinds of lockdowns many other countries are going through. https://t.co/6R0LY4IhuO
Today’s reflection- A number of Swedes are pretty shit at social distancing and probably need at least a modicum of discipline- the notion that we should be so very different here is ludicrous
WATCH: "Some countries initially talked about herd immunity as a strategy. In New Zealand we never, ever considered that. It would have meant tens of thousands of New Zealanders dying" — New Zealand Prime Minister @jacindaardernpic.twitter.com/W1ei6OUUyr
An online form to report lockdown breaches undermines the trust we have in each other – unhelpful in even the most benign of situations, and downright dangerous right now, writes Michael Macaulay. https://t.co/XCrnpfEVJt
Speechless every time someone says that this was totally unexpected & nobody saw this coming. See chapter 3: 'Preparing for the Worst: A Rapidly Spreading, Lethal Respiratory Pathogen' published by the @WHO Sept 2019. https://t.co/23qTrz7dN9
People are facing uncertainty for days, weeks & months. We need a manageable way forward to keep the health, social & economic costs at a minimum. My analysis on where COVID-19 response is heading & how it could end: https://t.co/qLDm8tv8a9
I wish the late great Mick Moran were still around – it feels like the next chapter of his analysis of the modern British state urgently needs to be written. https://t.co/ffxegGKVCu
I’m writing a book about @ExtinctionR. Here are some thoughts about today’s controversy. 1. This may or may not be a legit XR group. 2. That may matter because it may be done in order to smear XR & climate activism generally 1/n https://t.co/NyQhbv53a3
Cautionary words for anyone tempted to say "this must be good for the climate" or, worse, "this shows we can tackle climate change".
COVID19 is a re-framing of the climate issues – a dramatically changed context for the response – but those climate issues haven't gone away. https://t.co/gixVwnk6gq
We are concerned about regulation rollbacks which impact the food system slipping under the radar at the moment – we are going to be keeping an eye on things and use hashtag #Covid19Watchdoghttps://t.co/niinfSWv6f#TuesdayThoughts
A study in politics – when leadership fails. Would those that were ready to bash the @WHO take the time to read this? The critical issue for all countries is: what did they do after the PHEIC was declared? Why did USA and China not work together to fight #COVID19https://t.co/zK7hcEbU80
Not a single voice from the Global South – that’s not good enough if you are reporting on a global organisation – @who has 194 member states – it’s not the donors who should be running it #COVID19#geopoliticshttps://t.co/xqTaFEYLap
— Professor Paul Cairney (@CairneyPaul) April 9, 2020
The Australian #COVID19 modelling was published today. My thanks to James McCaw (@j_mccaw) for checking this thread. I’ll do two threads – one explaining the results and how we might interpret them; and another to try to explain how these models work. https://t.co/O6sGwggY9W
This was so predictable. Ireland was already closing pubs and restaurants. #COVIDー19 . Cheltenham Festival ‘spread coronavirus across country’ | News | The Times https://t.co/QVQnJblJiH
— Andrea Catherwood (@acatherwoodnews) April 3, 2020
expert comments about comparison between the COVID-19 situation in Ireland and the UKhttps://t.co/y4OBOhdbtT
The idea of a ‘policy entrepreneur’ is important to policy studies and policy analysis.
Let’s begin with its positive role in analysis, then use policy studies to help qualify its role within policymaking environments.
The take-home-messages are to
recognise the value of entrepreneurship, and invest in relevant skills and strategies, but
not overstate its spread or likely impact, and
note the unequal access to political resources associated with entrepreneurs.
Entrepreneurship and policy analysis
Mintrom identifies the intersection between policy entrepreneurship and policy analysis, to highlight the benefits of ‘positive thinking’, creativity, deliberation, and leadership.
“Policy entrepreneurs are energetic actors who engage in collaborative efforts in and around government to promote policy innovations. Given the enormous challenges now facing humanity, the need is great for such actors to step forward and catalyze change processes” (Mintrom, 2019: 307).
Although many entrepreneurs seem to be exceptional people, Mintrom (2019: 308-20) identifies:
Key attributes to compare
‘ambition’, to invest resources for future reward
‘social acuity’, to help anticipate how others are thinking
‘credibility’, based on authority and a good track record
‘sociability’, to empathise with others and form coalitions or networks
‘tenacity’, to persevere during adversity
The skills that can be learned
‘strategic thinking’, to choose a goal and determine how to reach it
‘team building’, to recognise that policy change is a collective effort, not the responsibility of heroic individuals (compare with Oxfam)
‘collecting evidence’, and using it ‘strategically’ to frame a problem and support a solution
‘making arguments’, using ‘tactical argumentation’ to ‘win others to their cause and build coalitions of supporters’ (2019: 313)
‘engaging multiple audiences’, by tailoring arguments and evidence to their beliefs and interests
‘negotiating’, such as by trading your support in this case for their support in another
The strategies built on these attributes and skills.
‘problem framing’, such as to tell a story of a crisis in need of urgent attention
‘using and expanding networks’, to generate attention and support
‘working with advocacy coalitions’, to mobilise a collection of actors who already share the same beliefs
‘leading by example’, to signal commitment and allay fears about risk
‘scaling up change processes’, using policy innovation in one area to inspire wider adoption.
Overall, entrepreneurship is ‘tough work’ requiring ‘courage’, but necessary for policy disruption, by: ‘those who desire to make a difference, who recognize the enormous challenges now facing humanity, and the need for individuals to step forward and catalyze change’ (2019: 320; compare with Luetjens).
Entrepreneurship and policy studies
Most policy actors fail
It is common to relate entrepreneurship to stories of exceptional individuals and invite people to learn from their success. However, the logical conclusion is that success is exceptional and most policy actors will fail.
A focus on key skills takes us away from this reliance on exceptional actors, and ties in with other policy studies-informed advice on how to navigate policymaking environments (see ‘Three habits of successful policy entrepreneurs’, these ANZSOG talks, and box 6.3 below)
However, note the final sentence, which reminds us that it is possible to invest a huge amount of time and effort in entrepreneurial skills without any of that investment paying off.
Even if entrepreneurs succeed, the explanation comes more from their environments than their individual skills
The other side of the entrepreneurship coin is the policymaking environment in which actors operate.
Policy studies of entrepreneurship (such as Kingdon on multiple streams) rely heavily on metaphors on evolution. Entrepreneurs are the actors most equipped to thrive within their environments (see Room).
However, Kingdon uses the additional metaphor of ‘surfers waiting for the big wave’, which suggests that their environments are far more important than them (at least when operating on a US federal scale – see Kingdon’s Multiple Streams Approach).
Entrepreneurs may be more influential at a more local scale, but the evidence of their success (independent of the conditions in which they operate) is not overwhelming. So, self-aware entrepreneurs know when to ‘surf the waves’ or try to move the sea.
The social background of influential actors
Many studies of entrepreneurs highlight the stories of tenacious individuals with limited resources but the burning desire to make a difference.
The alternative story is that political resources are distributed profoundly unequally. Few people have the resources to:
run for elected office
attend elite Universities, or find other ways to develop the kinds of personal networks that often relate to social background
develop the credibility built on a track record in a position of authority (such as in government or science).
be in the position to invest resources now, to secure future gains, or
be in an influential position to exploit windows of opportunity.
Therefore, when focusing on entrepreneurial policy analysis, we should encourage the development of a suite of useful skills, but not expect equal access to that development or the same payoff from entrepreneurial action.
See also:
Compare these skills with the ones we might associate with ‘systems thinking‘
If you want to see me say these depressing things with a big grin:
When describing ‘the policy sciences’, Lasswell distinguishes between:
‘knowledge of the policy process’, to foster policy studies (the analysis of policy)
‘knowledge in the process’, to foster policy analysis (analysis for policy)
The lines between each approach are blurry, and each element makes less sense without the other. However, the distinction is crucial to help us overcome the major confusion associated with this question:
Does policymaking proceed through a series of stages?
The short answer is no.
The longer answer is that you can find about 40 blog posts (of 500 and 1000 words) which compare (a) a stage-based model called the policy cycle, and (b) the many, many policy concepts and theories that describe a far messier collection of policy processes.
In a nutshell, most policy theorists reject this image because it oversimplifies a complex policymaking system. The image provides a great way to introduce policy studies, and serves a political purpose, but it does more harm than good:
Prescriptively, it gives you rotten advice about the nature of your policymaking task (for more on these points, see this chapter, article, article, and series).
Why does the stages/ policy cycle image persist? Two relevant explanations
It arose from a misunderstanding in policy studies
In another nutshell, Chris Weible and I argue (in a secret paper) that the stages approach represents a good idea gone wrong:
If you trace it back to its origins, you will find Lasswell’s description of decision functions: intelligence, recommendation, prescription, invocation, application, appraisal and termination.
These functions correspond reasonably well to a policy cycle’s stages: agenda setting, formulation, legitimation, implementation, evaluation, and maintenance, succession or termination.
However, Lasswell was imagining functional requirements, while the cycle seems to describe actual stages.
In other words, if you take Lasswell’s list of what policy analysts/ policymakers need to do, multiple it by the number of actors (spread across many organisations or venues) trying to do it, then you get the multi-centric policy processes described by modern theories. If, instead, you strip all that activity down into a single cycle, you get the wrong idea.
It is a functional requirement of policy analysis
This description should seem familiar, because the classic policy analysis texts appear to describe a similar series of required steps, such as:
define the problem
identify potential solutions
choose the criteria to compare them
evaluate them in relation to their predicted outcomes
In addition, studies of policy analysis in action suggest that:
an individual analyst’sneed for simple steps, to turn policymaking complexity into useful heuristics and pragmatic strategies,
should not be confused with
what actually happens when many policy analysts, influencers, and policymakers interact in policy processes (see Radin, and Brans, Geva-May, and Howlett).
What you need versus what you can expect
Overall, this discussion of policy studies and policy analysis reminds us of a major difference between:
Functional requirements. What you need from policymaking systems, to (a) manage your task (the 5-8 step policy analysis) and (b) understand and engage in policy processes (the simple policy cycle).
Actual processes and outcomes. What policy concepts and theories tell us about bounded rationality (which limit the comprehensiveness of your analysis) and policymaking complexity (which undermines your understanding and engagement in policy processes).
Of course, I am not about to provide you with a solution to these problems.
When describing ‘the policy sciences’, Lasswell distinguishes between:
‘knowledge of the policy process’, to foster policy studies (the analysis of policy)
‘knowledge in the process’, to foster policy analysis (analysis for policy)
The idea is that both elements are analytically separable but mutually informative: policy analysis is crucial to solving real policy problems, policy studies inform the feasibility of analysis, the study of policy analysts informs policy studies, and so on.
Both elements focus on similar questions – such as What is policy? – and explore their descriptive (what do policy actors do?) and prescriptive (what should they do?) implications.
Policy studies focus on the power to reduce ambiguity rather than simply the provision of information to reduce uncertainty. In other words, the power to decide whose interpretation of policy problems counts, and therefore to decide what information is policy-relevant.
This (unequal) competition takes place within a policy process over which no actor has full knowledge or control.
The classic 5-8 step policy analysis texts focus on how to define policy problems well, but they vary somewhat in their definition of doing it well (see also C.Smith):
Bardach recommends using rhetoric and eye-catching data to generate attention
Weimer and Vining and Mintrom recommend beginning with your client’s ‘diagnosis’, placing it in a wider perspective to help analyse it critically, and asking yourself how else you might define it (see also Bacchi, Stone)
Meltzer and Schwartz and Dunn identify additional ways to contextualise your client’s definition, such as by generating a timeline to help ‘map’ causation or using ‘problem-structuring methods’ to compare definitions and avoid making too many assumptions on a problem’s cause.
Thissen and Walker compare ‘rational’ and ‘argumentative’ approaches, treating problem definition as something to be measured scientifically or established rhetorically (see also Riker).
These approaches compare with more critical accounts that emphasise the role of power and politics to determine whose knowledge is relevant (L.T.Smith) and whose problem definition counts (Bacchi, Stone). Indeed, Bacchi andStone provide a crucial bridge between policy analysis and policy studies by reflecting on what policy analysts do and why.
What is the policy solution?
In policy studies, it is common to identify counterintuitive or confusing aspects of policy processes, including:
Few studies suggest that policy responses actually solve problems (and many highlight their potential to exacerbate them). Rather, ‘policy solutions’ is shorthand for proposed or alleged solutions.
Problem definition often sets the agenda for the production of ‘solutions’, but note the phrase solutions chasing problems (when actors have their ‘pet’ solutions ready, and they seek opportunities to promote them).
Policy studies: problem definition informs the feasibility and success of solutions
Generally speaking, to define the problem is to influence assessments of the feasibility of solutions:
Technical feasibility. Will they work as intended, given the alleged severity and cause of the problem?
Political feasibility. Will they receive sufficient support, given the ways in which key policy actors weigh up the costs and benefits of action?
Policy studies highlight the inextricable connection between technical and political feasibility. Put simply, (a) a ‘technocratic’ choice about the ‘optimality’ of a solution is useless without considering who will support its adoption, and (b) some types of solution will always be a hard sell, no matter their alleged effectiveness (Box 2.3 below).
In turn, problem definition informs: the ways in which actors will frame any evaluation of policy success, and the policy-relevance of the evidence to evaluate solutions. Simple examples include:
If you define tobacco in relation to: (a) its economic benefits, or (b) a global public health epidemic, evaluations relate to (a) export and taxation revenues, or (b) reductions in smoking in the population.
If you define ‘fracking’ in relation to: (a) seeking more benefits than costs, or (b) minimising environmental damage and climate change, evaluations relate to (a) factors such as revenue and effective regulation, or simply (b) how little it takes place.
Policyanalysis: recognising and pushing boundaries
Policy analysis texts tend to accommodate these insights when giving advice:
Bardach recommends identifying solutions that your audience might consider, perhaps providing a range of options on a notional spectrum of acceptability.
Smith highlights the value of ‘precedent’, or relating potential solutions to previous strategies.
Weimer and Vining identify the importance of ‘a professional mind-set’ that may be more important than perfecting ‘technical skills’
Mintrom notes that some solutions are easier to sell than others
Meltzer and Schwartz describe the benefits of making a preliminary recommendation to inform an iterative process, drawing feedback from clients and stakeholder groups
Dunn warns against too-narrow forms of ‘evidence based’ analysis which undermine a researcher’s ability to adapt well to the evidence-demands of policymakers
Thissen and Walker relate solution feasibility to a wide range of policy analysis ‘styles’
Still, note the difference in emphasis.
Policy analysis education/ training may be about developing the technical skills to widen definitions and apply many criteria to compare solutions.
Policy studies suggest that problem definition and a search for solutions takes place in an environment where many actors apply a much narrower lens and are not interested in debates on many possibilities (particularly if they begin with a solution).
I have exaggerated this distinction between each element, but it is worth considering the repeated interaction between them in practice: politics and policymaking provide boundaries for policy analysis, analysis could change those boundaries, and policy studies help us reflect on the impact of analysts.
I’ll take a quick break, then discuss how this conclusion relates to the idea of ‘entrepreneurial’ policy analysis.
One aim of this series is to combine insights from policy research (1000, 500) and policy analysis texts.
In this case, modern theories of the policy process help you identify your audience and their capacity to follow your advice. This simple insight may have a profound impact on the advice you give.
Policy analysis for an ideal-type world
For our purposes, an ideal-type is an abstract idea, which highlights hypothetical features of the world, to compare with ‘real world’ descriptions. It need not be an ideal to which we aspire. For example, comprehensive rationality describes the ideal type, and bounded rationality describes the ‘real world’ limitations to the ways in which humans and organisations process information.
Imagine writing policy analysis in the ideal-type world of a single powerful ‘comprehensively rational’ policymaker at the heart of government, making policy via an orderly policy cycle.
Your audience would be easy to identify, your analysis would be relatively simple, and you would not need to worry about what happens after you make a recommendation for policy change.
You could adopt a simple 5-8 step policy analysis method, use widely-used tools such as cost-benefit analysis to compare solutions, and know where the results would feed into the policy process.
I have perhaps over-egged this ideal-type pudding, but I think a lot of traditional policy analyses tapped into this basic idea and focused more on the science of analysis than the political and policymaking context in which it takes place (see Radin and Brans, Geva-May, and Howlett).
This image is a key feature of policy process theories, which describe:
Many policymakers and influencers spread across many levels and types of government (as the venues in which authoritative choice takes place). Consequently, it is not a straightforward task to identify and know your audience, particularly if the problem you seek to solve requires a combination of policy instruments controlled by different actors.
Each venue resembles an institution driven by formal and informal rules. Formal rules are written-down or widely-known. Informal rules are unwritten, difficult to understand, and may not even be understood in the same way by participants. Consequently, it is difficult to know if your solution will be a good fit with the standard operating procedures of organisations (and therefore if it is politically feasible or too challenging).
Policymakers and influencers operate in ‘subsystems’, forming networks built on resources such as trust or coalitions based on shared beliefs. Effective policy analysis may require you to engage with – or become part of – such networks, to allow you to understand the unwritten rules of the game and encourage your audience to trust the messenger. In some cases, the rules relate to your willingness to accept current losses for future gains, to accept the limited impact of your analysis now in the hope of acceptance at the next opportunity.
Actors relate their analysis to shared understandings of the world – how it is, and how it should be – which are often so well-established as to be taken for granted. Common terms include paradigms, hegemons, core beliefs, and monopolies of understandings. These dominant frames of reference give meaning to your policy solution. They prompt you to couch your solutions in terms of, for example, a strong attachment to evidence-based cases in public health, value for money in treasury departments, or with regard to core principles such as liberalism or socialism in different political systems.
Your solutions relate to socioeconomic context and the events that seem (a) impossible to ignore and (b) out of the control of policymakers. Such factors range from a political system’s geography, demography, social attitudes, and economy, while events can be routine elections or unexpected crises.
What would you recommend under these conditions? Rethinking 5-step analysis
There is a large gap between policymakers’ (a) formal responsibilities versus (b) actual control of policy processes and outcomes. Even the most sophisticated ‘evidence based’ analysis of a policy problem will fall flat if uninformed by such analyses of the policy process. Further, the terms of your cost-benefit analysis will be highly contested (at least until there is agreement on what the problem is, and how you would measure the success of a solution).
Modern policy analysis texts try to incorporate such insights from policy theories while maintaining a focus on 5-8 steps. For example:
Meltzer and Schwartz contrast their ‘flexible’ and ‘iterative’ approach with a too- rigid ‘rationalistic approach’.
Bardachand Dunn emphasise the value of political pragmatism and the ‘art and craft’ of policy analysis.
Weimer and Vininginvest 200 pages in economic analyses of markets and government, often highlighting a gap between (a) our ability to model and predict economic and social behaviour, and (b) what actually happens when governments intervene.
Mintrom invites you to see yourself as a policy entrepreneur, to highlight the value of of ‘positive thinking’, creativity, deliberation, and leadership, and perhaps seek ‘windows of opportunity’ to encourage new solutions. Alternatively, a general awareness of the unpredictability of events can prompt you to be modest in your claims, since the policymaking environment may be more important (than your solution) to outcomes.
Thissen and Walker focus more on a range of possible roles than a rigid 5-step process.
Without this wider perspective, we are focusing on policy analysis as a process rather than considering the political context in which analysts use it.
Additional posts on entrepreneurs and ‘systems thinking’ [to be added] encourage us to reflect on the limits to policy analysis in multi-centric policymaking systems.
One aim of this series is to combine insights from policy research (1000, 500) and policy analysis texts.
If we take key insights from policy theories seriously, we can use them to identify (a) the constraints to policy analytical capacity, and (b) the ways in which analysts might address them. I use the idea of policy analyst archetypes to compare a variety of possible responses.
Key constraints to policy analytical capacity
Terms like ‘bounded rationality’ highlight major limits on the ability of humans and organisations to process information.
Humans use heuristics or cognitive shortcuts to process enough information to make choices, and institutions are the rules used by organisations to limit information processing.
Policy actors need to find ways to act, with incomplete information about the problem they seek to solve and the likely impact of their ‘solution’.
They gather information to help reduce uncertainty, but problem definition is really about exercising power to reduce ambiguity: select one way to interpret a problem (at the expense of most others), and limit therefore limit the relevance and feasibility of solutions.
This context informs how actors might use the tools of policy analysis. Key texts in this series highlight the use of tools to establish technical feasibility (will it work as intended?), but policymakers also select tools for their political feasibility (who will support or oppose this measure?).
How might policy analysts address these constraints ethically?
Most policy analysis texts (in this series) consider the role of professional ethics and values during the production of policy analysis. However, they also point out that there is not a clearly defined profession and associated code of conduct (e.g. see Adachi). In that context, let us begin with some questions about the purpose of policy analysis and your potential role:
Is your primary role to serve individual clients or some notion of the ‘public good’?
Should you maximise your role as an individual or play your part in a wider profession?
What is the balance between the potential benefits of individual ‘entrepreneurship’ and collective ‘co-productive’ processes?
Which policy analysis techniques should you prioritise?
What forms of knowledge and evidence count in policy analysis?
What does it mean to communicate policy analysis responsibly?
Should you provide a clear recommendation or encourage reflection?
Policy analysis archetypes: pragmatists, entrepreneurs, manipulators, storytellers, and decolonisers
In that context, I have created a story of policy analysis archetypes to identify the elements that each text emphasises.
The pragmatic policy analyst
Bardach provides the classic simple, workable, 8-step system to present policy analysis to policymakers while subject to time and resource-pressed political conditions.
Dunn also uses Wildavsky’s famous phrase ‘art and craft’ to suggest that scientific and ‘rational’ methods can only take us so far.
The professional, client–oriented policy analyst
Weimer and Vining provide a similar 7-step client-focused system, but incorporating a greater focus on professional development and economic techniques (such as cost-benefit-analysis) to emphasise a particular form of professional analyst.
Meltzer and Schwartz also focus on advice to clients, but with a greater emphasis on a wide variety of methods or techniques (including service design) to encourage the co-design of policy analysis with clients.
The communicative policy analyst
C. Smith focuses on how to write and communicate policy analysis to clients in a political context.
Compare with Spiegelhalter and Gigerenzer on how to communicate responsibly when describing uncertainty, probability, and risk.
The manipulative policy analyst.
Riker helps us understand the relationship between two aspects of agenda setting: the rules/ procedures to make choice, and the framing of policy problems and solutions.
The entrepreneurial policy analyst
Mintrom shows how to combine insights from studies of policy entrepreneurship and policy analysis, to emphasise the benefits of collaboration and creativity.
The questioning policy analyst
Bacchi analyses the wider context in which people give and use such advice, to identify the emancipatory role of analysis and encourage policy analysts to challenge dominant social constructions of problems and populations.
The storytelling policy analyst
Stone identifies the ways in which people use storytelling and argumentation techniques to define problems and justify solutions. This process is about politics and power, not objectivity and optimal solutions.
The decolonizing policy analyst.
L.T. Smith does not describe policy analysis directly, but shows how the ‘decolonization of research methods’ can inform the generation and use of knowledge.
Compare with Hindess on the ways in which knowledge-based hierarchies rely on an untenable, circular logic.
Compare with Michener’s thread, discussing Doucet’s new essay on (a) the role of power and knowledge in limiting (b) the ways in which we gather evidence to analyse policy problems.
Incisive essay from @bailabomba on studying the use of research evidence through critical perspectives that center the marginalized. There is so too much good stuff in here to summarize via twitter (you should just read it). But let me point out a few things that resonated (1/n) https://t.co/nIahyIjwBo
Using archetypes to define the problem of policy analysis
Studies of the field (e.g. Radin plus Brans, Geva-May, and Howlett) suggest that there are many ways to do policy analysis. Further, as Thissen and Walker describe, such roles are notmutually exclusive, your views on their relative value could change throughout the process of analysis, and you could perform many of these roles.
Further, each text describes multiple roles, and some seem clustered together:
pragmatic, client-orientated, and communicative could sum-up the traditional 5-8 step approaches, while
questioning, storytelling, and decolonizing could sum up an important (‘critical’) challenge to narrow ways of thinking about policy analysis and the use of information.
Still, the emphasis matters.
Each text is setting an agenda or defining the problem of policy analysis more-or-less in relation to these roles. Put simply, the more you are reading about economic theory and method, the less you are reading about dominance and manipulation.
See also The new policy sciences for a discussion of how these issues inform Lasswell’s original vision for the policy sciences (combining the analysis of and for policy).
Please see the Policy Analysis in 750 words series overview before reading the summary. This post might well represent the largest breach of the ‘750 words’ limit, so please get comfortable. I have inserted a picture of a cat hanging in there baby after the main (*coughs*) 1400-word summary. The rest is bonus material, reflecting on the links between this book and the others in the series.
‘We define policy analysis as evidence-based advice giving, as the process by which one arrives at a policy recommendation to address a problem of public concern. Policy analysis almost always involves advice for a client’ (Meltzer and Schwartz, 2019: 15).
Meltzer and Schwartz (2019: 231-2) describe policy analysis as applied research, drawing on many sources of evidence, quickly, with limited time, access to scientific research, or funding to conduct a lot of new research (2019: 231-2). It requires:
careful analysis of a wide range of policy-relevant documents (including the ‘grey’ literature often produced by governments, NGOs, and think tanks) and available datasets
perhaps combined with expert interviews, focus groups, site visits, or an online survey (see 2019: 232-64 on methods).
Meltzer and Schwartz (2019: 21) outline a ‘five-step framework’ for client-oriented policy analysis. During each step, they contrast their ‘flexible’ and ‘iterative’ approach with a too- rigid ‘rationalistic approach’ (to reflect bounded, not comprehensive, rationality):
‘Define the problem’.
Problem definition is a political act of framing, not an exercise in objectivity (2019: 52-3). It is part of a narrative to evaluate the nature, cause, size, and urgency of an issue (see Stone), or perhaps to attach to an existing solution (2019: 38-40; compare with Mintrom).
In that context, ask yourself ‘Who is defining the problem? And for whom?’ and do enough research to be able to define it clearly and avoid misunderstanding among you and your client (2019: 37-8; 279-82):
Identify your client’s resources and motivation, such as how they seek to use your analysis, the format of analysis they favour, their deadline, and their ability to make or influence the policies you might suggest (2019: 49; compare with Weimer and Vining).
Tailor your narrative to your audience, albeit while recognising the need to learn from ‘multiple perspectives’ (2019: 40-5).
Make it ‘concise’ and ‘digestible’, not too narrowly defined, and not in a way that already closes off discussion by implying a clear cause and solution (2019: 51-2).
In doing so:
Ask yourself if you can generate a timeline, identify key stakeholders, and place a ‘boundary’ on the problem.
Establish if the problem is urgent, who cares about it, and who else might care (or not) (2019 : 46).
Focus on the ‘central’ problem that your solution will address, rather than the ‘related’ and ‘underlying’ problems that are ‘too large and endemic to be solved by the current analysis’ (2019: 47).
Avoid misdiagnosing a problem with reference to one cause. Instead, ‘map’ causation with reference to (say) individual and structural causes, intended and unintended consequences, simple and complex causation, market or government failure, and/ or the ability to blame an individual or organisation (2019: 48-9).
Combine quantitative and qualitative data to frame problems in relation to: severity, trends in severity, novelty, proximity to your audience, and urgency or crisis (2019: 53-4).
During this process, interrogate your own biases or assumptions and how they might affect your analysis (2019: 50).
2. ‘Identify potential policy options (alternatives) to address the problem’.
Common sources of ideas include incremental changes from current policy, ‘client suggestions’, comparable solutions (from another time, place, or policy area), reference to common policy instruments, and ‘brainstorming’ or ‘design thinking’ (2019: 67-9; see box 2.3 and 7.1, below, from Understanding Public Policy).
Identify a ‘wide range’ of possible solutions, then select the (usually 3-5) ‘most promising’ for further analysis (2019: 65). In doing so:
be careful not to frame alternatives negatively (e.g. ‘death tax’ – 2019: 66)
compare alternatives in ‘good faith’ rather than keeping some ‘off the table’ to ensure that your preferred solution looks good (2019: 66)
think about how to modify existing policies according to scale or geographical coverage, who to include (and based on what criteria), for how long, using voluntary versus mandatory provisions, and ensuring oversight (2019: 71-3)
consider combinations of common policy instruments, such as regulations and economic penalties/ subsidies (2019: 73-7)
consider established ways to ‘brainstorm’ ideas (2019: 77-8)
note the rise of instruments derived from the study of psychology and behavioural public policy (2019: 79-90)
learn from design principles, including ‘empathy’, ‘co-creating’ policy with service users or people affected, ‘prototyping’ (2019: 90-1)
3. ‘Specify the objectives to be attained in addressing the problem and the criteria to evaluate the attainment of these objectives as well as the satisfaction of other key considerations (e.g., equity, cost, equity, feasibility)’.
Your objectives relate to your problem definition and aims: what is the problem, what do you want to happen when you address it, and why?
For example, questions to your client may include: what is your organization’s ‘mission’, what is feasible (in terms of resources and politics), which stakeholders to you want to include, and how will you define success (2019: 105; 108-12)?
In that values-based context, your criteria relate to ways to evaluate each policy’s likely impact (2019: 106-7). They should ensure:
Comprehensiveness. E.g. how many people, and how much of their behaviour, can you influence while minimizing the ‘burden’ on people, businesses, or government? (2019: 113-4)
Mutual Exclusiveness. In other words, don’t have two objectives doing the same thing (2019: 114).
Common criteria include (2019: 116):
Effectiveness. The size of its intended impact on the problem (2019: 117).
Equity (fairness). The impact in terms of ‘vertical equity’ (e.g. the better off should pay more), ‘horizontal equity’ (e.g. you should not pay more if unmarried), fair process, fair outcomes, and ‘intergenerational’ equity (e.g. don’t impose higher costs on future populations) (2019: 118-19).
Feasibility (administrative, political, and technical). The likelihood of this policy being adopted and implemented well (2019: 119-21)
Cost (or financial feasibility). Who would bear the cost, and their willingness and ability to pay (2019: 122).
Efficiency. To maximise the benefit while minimizing costs (2019: 122-3).
4. ‘Assess the outcomes of the policy options in light of the criteria and weigh trade-offs between the advantages and disadvantages of the options’.
When explaining objectives and criteria,
‘label’ your criteria in relation to your policy objectives (e.g. to ‘maximize debt reduction’) rather than using generic terms (2019: 123-7)
produce a table – with alternatives in rows, and criteria in columns – to compare each option
quantify your policies’ likely outcomes, such as in relation to numbers of people affected and levels of income transfer, or a percentage drop in the size of the problem, but also
communicate the degree of uncertainty related to your estimates (2019: 128-32; see Spiegelhalter)
Consider using cost-benefit analysis to identify (a) the financial and opportunity cost of your plans (what would you achieve if you spent the money elsewhere?), compared to (b) the positive impact of your funded policy (2019: 141-55).
The principle of CBA may be intuitive, but a thorough CBA process is resource-intensive, vulnerable to bias and error, and no substitute for choice. It requires you to make a collection of assumptions about human behaviour and likely costs and benefits, decide whose costs and benefits should count, turn all costs and benefits into a single measure, and imagine how to maximise winners and compensate losers (2019: 155-81; compare Weimer and Vining with Stone).
One alternative is cost-effectiveness analysis, which quantifies costs and relates them to outputs (e.g. number of people affected, and how) without trying to translate them into a single measure of benefit (2019: 181-3).
These measures can be combined with other thought processes, such as with reference to ‘moral imperatives’, a ‘precautionary approach’, and ethical questions on power/ powerlessness (2019: 183-4).
5. ‘Arrive at a recommendation’.
Predict the most likely outcomes of each alternative, while recognising high uncertainty (2019: 189-92). If possible,
draw on existing, comparable, programmes to predict the effectiveness of yours (2019: 192-4)
combine such analysis with relevant theories to predict human behaviour (e.g. consider price ‘elasticity’ if you seek to raise the price of a good to discourage its use) (2019: 193-4)
apply statistical methods to calculate the probability of each outcome (2019: 195-6), and modify your assumptions to produce a range of possibilities, but
note Spiegelhalter’s cautionary tales and anticipate the inevitable ‘unintended consequences’ (when people do not respond to policy in the way you would like) (2019: 201-2)
use these estimates to inform a discussion on your criteria (equity, efficiency, feasibility) (2019: 196-200)
present the results visually – such as in a ‘matrix’ – to encourage debate on the trade-offs between options
simplify choices by omitting irrelevant criteria and options that do not compete well with others (2019: 203-10)
make sure that your recommendation (a) flows from the analysis, and (b) is in the form expected by your client (2019: 211-12)
consider making a preliminary recommendation to inform an iterative process, drawing feedback from clients and stakeholder groups (2019: 212).
Policy analysis in a wider context
Meltzer and Schwartz’s approach makes extra sense if you have already read some of the other texts in the series, including:
Weimer and Vining, which represents an exemplar of an X-step approach informed heavily by the study of economics and application of economic models such as cost-benefit-analysis (compare with Radin’s checklist).
Geva-May on the existence of a policy analysis profession with common skills, heuristics, and (perhaps) ethics (compare with Meltzer and Schwartz, 2019: 282-93)
the proliferation of analysts across multiple levels of government, NGOs, and the private sector (compare with Meltzer and Schwartz, 2019: 269-77)
the historic shift of analysis from formulation to all notional stages (contrast with Meltzer and Schwartz, 2019: 16-7 on policy analysis not including implementation or evaluation)
the difficulty in distinguishing between policy analysis and advocacy in practice (compare with Meltzer and Schwartz, 2019: 276-8, who suggest that actors can choose to perform these different roles)
the emerging sense that it is difficult to identify a single client in a multi-centric policymaking system. Put another way, we might be working for a specific client but accept that their individual influence is low.
a historic tendency for economics to dominate policy analysis,
the applicability of economic assumptions (focusing primarily on individualist behaviour and markets), and
the pervasiveness of ‘rationalist’ policy analysis built on X-steps.
Meltzer and Schwartz (2019: 1-3) agree that economic models are too dominant (identifying the value of insights from ‘other disciplines – including design, psychology, political science, and sociology’).
However, they argue that critiques of rational models exaggerate their limitations (2019: 23-6). For example:
these models need not rely solely on economic techniques or quantification, a narrow discussion or definition of the problem, or the sense that policy analysis should be comprehensive, and
it is not problematic for analyses to reflect their client’s values or for analysts to present ambiguous solutions to maintain wide support, partly because
we would expect the policy analysis to form only one part of a client’s information or strategy.
Further, they suggest that these critiques provide no useful alternative, to help guide new policy analysts. Yet, these guides are essential:
‘to be persuasive, and credible, analysts must situate the problem, defend their evaluative criteria, and be able to demonstrate that their policy recommendation is superior, on balance, to other alternative options in addressing the problem, as defined by the analyst. At a minimum, the analyst needs to present a clear and defensible ranking of options to guide the decisions of the policy makers’ (Meltzer and Schwartz, 2019: 4).
Meltzer and Schwartz (2019: 27-8) then explore ways to improve a 5-step model with insights from approaches such as ‘design thinking’, in which actors use a similar process – ‘empathize, define the problem, ideate, prototype, test and get feedback from others’ – to experiment with policy solutions without providing a narrow view on problem definition or how to evaluate responses.
Policy analysis and policy theory
One benefit to Meltzer and Schwartz’s approach is that it seeks to incorporate insights from policy theories and respond with pragmatism and hope. However, I think you also need to read the source material to get a better sense of those theories, key debates, and their implications. For example:
Meltzer and Schwartz (2019: 32) note correctly that ‘incremental’ does not sum up policy change well. Indeed, Punctuated Equilibrium Theory shows that policy change is characterised by a huge number of small and a small number of huge changes.
However, the direct implications of PET are not as clear as they suggest. Baumgartner and Jones have both noted that they can measure these outcomes and identify the same basic distribution across a political system, but not explain or predict why particular policies change dramatically.
It is useful to recommend to policy analysts that they invest some hope in major policy change, but also sensible to note that – in the vast majority of cases – it does not happen.
On his point, see Mintrom on policy analysis for the long term, Weiss on the ‘enlightenment’ function of research and analysis, and Box 6.3 (from Understanding Public Policy), on the sense that (a) we can give advice to ‘budding policy entrepreneurs’ on how to be effective analysts, but (b) should note that all their efforts could be for nothing.
Meltzer and Schwartz (2019: 32-3) tap briefly into the old debate on whether it is preferable to seek radical or incremental change. For more on that debate, see chapter 5 in the 1st ed of Understanding Public Policy in which Lindblom notes that proposals for radical/ incremental changes are not mutually exclusive.
Perhaps explore the possible tension between Meltzer and Schwartz’s (2019: 33-4) recommendation that (a) policy analysis should be ‘evidence-based advice giving’, and (b) ‘flexible and open-ended’.
I think that Stone’s response would be that phrases such as ‘evidence based’ are not ‘flexible and open-ended’. Rather, they tend to symbolise a narrow view of what counts as evidence (see also Smith, and Hindess).
Further, note that the phrase ‘evidence based policymaking’ is a remarkably vague term (see the EBPM page), perhaps better seen as a political slogan than a useful description or prescription of policymaking.
Finally, if you read enough of these policy analysis texts, you get a sense that many are bunched together even if they describe their approach as new or distinctive.
Indeed, Meltzer and Schwarz (2019: 22-3) provide a table (containing Bardach and Patashnik, Patton et al, Stokey and Zeckhauser, Hammond et al, and Weimer & Vining) of ‘quite similar’ X-step approaches.
Weimer and Vining also discuss the implications of policy theories and present the sense that X-step policy analysis should be flexible and adaptive.
Many texts – including Radin, and Smith (2016) – focus on the value of case studies to think through policy analysis in particular contexts, rather than suggesting that we can produce a universal blueprint.
However, as Geva-May might suggest, this is not a bad thing if our aim is to generate the sense that policy analysis is a profession with its own practices and heuristics.
Punctuated equilibrium theory (PET) tells a story of complex systems that are stable and dynamic:
Most policymaking exhibits long periods of stability, but with the ever-present potential for sudden instability.
Most policies stay the same for long periods. Some change very quickly and dramatically.
We can explain this dynamic with reference to bounded rationality: since policymakers cannot consider all issues at all times, they ignore most and promote relatively few to the top of their agenda.
This lack of attention to most issues helps explain why most policies may not change, while intense periods of attention to some issues prompts new ways to frame and solve policy problems.
Some explanation comes from the power of participants, to (a) minimize attention and maintain an established framing, or (b) expand attention in the hope of attracting new audiences more sympathetic to new ways of thinking.
Further explanation comes from policymaking complexity, in which the scale of conflict is too large to understand, let alone control.
The original PET story
The original PET story – described in more detail in the 1000 Words version – applies two approaches – policy communities and agenda setting – to demonstrate stable relationships between interest groups and policymakers:
They endure when participants have built up trust and agreement – about the nature of a policy problem and how to address it – and ensure that few other actors have a legitimate role or interest in the issue.
They come under pressure when issues attract high policymaker attention, such as following a ‘focusing event’ or a successful attempt by some groups to ‘venue shop’ (seek influential audiences in another policymaking venue). When an issue reaches the ‘top’ of this wider political agenda it is processed in a different way: more participants become involved, and they generate more ways to look at (and seek to solve) the policy.
The key focus is the competition to frame or define a policy problem (to exercise power to reduce ambiguity). The successful definition of a policy problem as technical or humdrum ensures that issues are monopolized and considered quietly in one venue. The reframing of that issue as crucial to other institutions, or the big political issues of the day, ensures that it will be considered by many audiences and processed in more than one venue (see also Schattschneider).
The modern PET story
The modern PET story is about complex systems and attention.
Its analysis of bounded rationality and policymaker psychology remains crucial, since PET measures the consequences of the limited attention of individuals and organisations.
However, note the much greater quantification of policy change across entire political systems (see the Comparative Agendas Project).
PET shows how policy actors and organisations contribute to ‘disproportionate information processing’, in which attention to information fluctuates out of proportion to (a) the size of policy problems and (b) the information on problems available to policymakers.
It also shows that the same basic distribution of policy change – ‘hyperincremental’ in most cases, but huge in some – is present in every political system studied by the CAP (summed up by the image below)
Just learned from @MikeMintrom 's book that a policy wonk is called a wonk because they know everything backwards. Now just sitting here wondering what else I don't know. pic.twitter.com/b1MMESsj83
Mintrom (2012: xxii; 17) describes policy analysis as ‘an enterprise primarily motivated by the desire to generate high quality information to support high-quality decisions’ and stop policymakers ‘from making ill-considered choices’ (2012: 17). It is about giving issues more ‘serious attention and deep thought’ than busy policymakers, rather than simply ‘an exercise in the application of techniques’ to serve clients (2012: 20; xxii).
It begins with six ‘Key Steps in Policy Analysis’ (2012: 3-5):
‘Engage in problem definition’
Problem definition influences the types of solutions that will be discussed (although, in some cases, solutions chase problems).
Define the nature and size of a policy problem, and the role of government in solving it (from maximal to minimal), while engaging with many stakeholders with different views (2012: 3; 58-60).
This task involves a juggling act. First, analysts should engage with their audience to work out what they need and when (2012 : 81). However, second, they should (a) develop ‘critical abilities’, (b) ask themselves ‘why they have been presented in specific ways, what their sources might be, and why they have arisen at this time’, and (c) present ‘alternative scenarios’ (2012: 22; 20; 27).
‘Propose alternative responses to the problem’
Governments use policy instruments – such as to influence markets, tax or subsidize activity, regulate behaviour, provide services (directly, or via commissioning or partnership), or provide information – as part of a coherent strategy or collection of uncoordinated measures (2012: 30-41). In that context, try to:
Generate knowledge about how governments have addressed comparable problems (including, the choice to not intervene if an industry self-regulates).
Identify the cause of a previous policy’s impact and if it would have the same effect now (2012: 21).
If minimal comparable information is available, consider wider issues from which to learn (2012: 76-7; e.g. alcohol policy based on tobacco).
Policymaking context, in which policies and institutions already exist to address most policy problems in some way.
‘Choose criteria for evaluating each alternative policy response’
There are no natural criteria, but ‘effectiveness, efficiency, fairness, and administrative efficiency’ are common (2012: 21). ‘Effective institutions’ have a marked impact on social and economic life and provide political stability (2012: 49). Governments can promote ‘efficient’ policies by (a) producing the largest number of winners and (b) compensating losers (2012: 51-2; see Weimer and Vining on Kaldor-Hicks). They can prioritise environmental ‘sustainability’ to mitigate climate change, the protection of human rights and ‘human flourishing’, and/or a fair allocation of resources (2012: 52-7).
‘Project the outcomes of pursuing each policy alternative’
Estimate the costs of a new policy, in comparison with current policy, and in relation to factors such as (a) overall savings to society, and/or (b) benefits to certain populations (any policy will benefit some social groups more than others). Mintrom (2012: 21) emphasises ‘prior knowledge and experience’ and ‘synthesizing’ work by others alongside techniques such as cost-benefit analyses.
‘Identify and analyse trade-offs among alternatives’
Use your criteria and projections to compare each alternative in relation to their likely costs and benefits.
‘Report findings and make an argument for the most appropriate response’
Mintrom (2012: 5) describes a range of advisory roles.
(a) Client-oriented advisors identify the beliefs of policymakers and anticipate the options worth researching (although they should not simply tell clients what they want to hear – 2012: 22). They may only have the time to answer a client’s question quickly and on their own. Or, they need to create and manage a team project (2012: 63-76).
(b) Other actors, ‘who want to change the world’, research options that are often not politically feasible in the short term but are too important to ignore (such as gender mainstreaming or action to address climate change).
In either case, the format of a written report – executive summary, contents, background, analytical strategy, analysis and findings (perhaps including a table comparing goals and trade-offs between alternatives), discussion, recommendation, conclusion, annex – may be similar (2012: 82-6).
Wider context: the changing role of policy analysts
Mintrom (2012: 5-7) describes a narrative – often attributed to Radin – of the changing nature of policy analysis, comparing:
(a) a small group of policy advisors, (b) with a privileged place in government, (c) giving allegedly technical advice, using economic techniques such as cost-benefit analysis.
(a) a much larger profession, (b) spread across – and outside of – government (including external consultants), and (c) engaging more explicitly in the politics of policy analysis and advice.
It reflects wider changes in government, (a) from the ‘clubby’ days to a much more competitive environment debating a larger number and wider range of policy issues, subject to (b) factors such as globalisation that change the task/ context of policy analysis.
If so, any advice on how to do policy analysis has to be flexible, to incorporate the greater diversity of actors and the sense that complex policymaking systems require flexibleskills and practices rather than standardised techniques and outputs.
The ethics of policy analysis
In that context, Mintrom (2012: 95-108) emphasises the enduring role for ethical policy analysis, which can relate to:
‘Universal’ principles such as fairness, compassion, and respect
Specific principles to project the analyst’s integrity, competence, responsibility, respectfulness, and concern for others
Professional practices, such as to
engage with many stakeholders in problem definition (to reflect a diversity of knowledge and views)
present a range of feasible solutions, making clear their distributional effects on target populations, opportunity costs (what policies/ outcomes would not be funded if this were), and impact on those who implement policy
be honest about (a) the method of calculation, and (b) uncertainty, when projecting outcomes
clarify the trade-offs between alternatives (don’t stack-up the evidence for one)
maximise effective information sharing, rather than exploiting the limited attention of your audience (compare with Riker).
Therefore, while Mintrom’s (2012: 3-5; 116) ‘Key Steps in Policy Analysis’ are comparable to Bardach and Weimer and Vining, his emphasis is often closer to Bacchi’s.
The entrepreneurial policy analyst
Mintrom (2012: 307-13) ends with a discussion of the intersection between policy entrepreneurship and analysis, highlighting the benefits of ‘positive thinking’, creativity, deliberation, and leadership. He expands on these ideas further in So you want to be a policy entrepreneur?
These posts introduce you to key concepts in the study of public policy. They are all designed to turn a complex policymaking world into something simple enough to understand. Some of them focus on small parts of the system. Others present ambitious ways to explain the system as a whole. The wide range of concepts should give you a sense of a variety of studies out there, but my aim is to show you that these studies have common themes.
Paul Cairney: Politics & Public Policy 