What have we learned so far from the UK government’s COVID-19 policy?

This post first appeared on LSE British Politics and Policy (27.11.20) and is based on this article in British Politics.

Paul Cairney assesses government policy in the first half of 2020. He identifies the intense criticism of its response so far, encouraging more systematic assessments grounded in policy research.

In March 2020, COVID-19 prompted policy change in the UK at a speed and scale only seen during wartime. According to the UK government, policy was informed heavily by science advice. Prime Minister Boris Johnson argued that, ‘At all stages, we have been guided by the science, and we will do the right thing at the right time’. Further, key scientific advisers such as Sir Patrick Vallance emphasised the need to gather evidence continuously to model the epidemic and identify key points at which to intervene, to reduce the size of the peak of population illness initially, then manage the spread of the virus over the longer term.

Both ministers and advisors emphasised the need for individual behavioural change, supplemented by government action, in a liberal democracy in which direct imposition is unusual and unsustainable. However, for its critics, the government experience has quickly become an exemplar of policy failure.

Initial criticisms include that ministers did not take COVID-19 seriously enough in relation to existing evidence, when its devastating effect was apparent in China in January and Italy from February; act as quickly as other countries to test for infection to limit its spread; or introduce swift-enough measures to close schools, businesses, and major social events. Subsequent criticisms highlight problems in securing personal protective equipment (PPE), testing capacity, and an effective test-trace-and-isolate system. Some suggest that the UK government was responding to the ‘wrong pandemic’, assuming that COVID-19 could be treated like influenza. Others blame ministers for not pursuing an elimination strategy to minimise its spread until a vaccine could be developed. Some criticise their over-reliance on models which underestimated the R (rate of transmission) and ‘doubling time’ of cases and contributed to a 2-week delay of lockdown. Many describe these problems and delays as the contributors to the UK’s internationally high number of excess deaths.

How can we hold ministers to account in a meaningful way?

I argue that these debates are often fruitless and too narrow because they do not involve systematic policy analysis, take into account what policymakers can actually do, or widen debate to consider whose lives matter to policymakers. Drawing on three policy analysis perspectives, I explore the questions that we should ask to hold ministers to account in a way that encourages meaningful learning from early experience.

These questions include:

Was the government’s definition of the problem appropriate?
Much analysis of UK government competence relates to specific deficiencies in preparation (such as shortages in PPE), immediate action (such as to discharge people from hospitals to care homes without testing them for COVID-19), and implementation (such as an imperfect test-trace-and-isolate system). The broader issue relates to its focus on intervening in late March to protect healthcare capacity during a peak of infection, rather than taking a quicker and more precautionary approach. This judgment relates largely to its definition of the policy problem which underpins every subsequent policy intervention.

Did the government select the right policy mix at the right time? Who benefits most from its choices?

Most debates focus on the ‘lock down or not?’ question without exploring fully the unequal impact of any action. The government initially relied on exhortation, based on voluntarism and an appeal to social responsibility. Initial policy inaction had unequal consequences on social groups, including people with underlying health conditions, black and ethnic minority populations more susceptible to mortality at work or discrimination by public services, care home residents, disabled people unable to receive services, non-UK citizens obliged to pay more to live and work while less able to access public funds, and populations (such as prisoners and drug users) that receive minimal public sympathy. Then, in March, its ‘stay at home’ requirement initiated a major new policy and different unequal impacts in relation to the income, employment, and wellbeing of different groups. These inequalities are list in more general discussions of impacts on the whole population.

Did the UK government make the right choices on the trade-offs between values, and what impacts could the government have reasonably predicted?

Initially, the most high-profile value judgment related to freedom from state coercion to reduce infection versus freedom from the harm of infection caused by others. Then, values underpinned choices on the equitable distribution of measures to mitigate the economic and wellbeing consequences of lockdown. A tendency for the UK government to project centralised and ‘guided by the science’ policymaking has undermined public deliberation on these trade-offs between policies. The latter will be crucial to ongoing debates on the trade-offs associated with national and regional lockdowns.

Did the UK government combine good policy with good policymaking?

A problem like COVID-19 requires trial-and-error policymaking on a scale that seems incomparable to previous experiences. It requires further reflection on how to foster transparent and adaptive policymaking and widespread public ownership for unprecedented policy measures, in a political system characterised by (a) accountability focused incorrectly on strong central government control and (b) adversarial politics that is not conducive to consensus seeking and cooperation.

These additional perspectives and questions show that too-narrow questions – such as was the UK government ‘following the science’ – do not help us understand the longer term development and wider consequences of UK COVID-19 policy. Indeed, such a narrow focus on science marginalises wider discussions of values and the populations that are most disadvantaged by government policy.

_____________________

Policy Analysis in 750 words: Classic 5-step advice

‘Policy analysis’ describes the identification of a policy problem and possible solutions.

Classic models of policy analysis are client-oriented. Most texts identify the steps that a policy analysis should follow, from identifying a problem and potential solutions, to finding ways to predict and evaluate the impact of each solution. Each text describes this process in different ways, as outlined in Boxes 1-5. However, for the most part, they follow the same five steps:

1. Define a policy problem identified by your client.
2. Identify technically and politically feasible solutions.
3. Use value-based criteria and political goals to compare solutions.
4. Predict the outcome of each feasible solution.
5. Make a recommendation to your client.

Further, they share the sense that analysts need to adapt pragmatically to a political environment. Assume that your audience is not an experienced policy analyst. Assume a political environment in which there is limited attention or time to consider problems, and some policy solutions will be politically infeasible. Describe the policy problem for your audience: to help them see it as something worthy of their energy. Discuss a small number of possible solutions, the differences between them, and their respective costs and benefits. Keep it short with the aid of visual techniques that sum up the issue concisely, to minimise cognitive load and make the problem seem solvable.

Box 1. Bardach (2012) A Practical Guide for Policy Analysis

1. ‘Define the problem’. Provide a diagnosis of a policy problem, using rhetoric and eye-catching data to generate attention.
2. ‘Assemble some evidence’. Gather relevant data efficiently.
3. ‘Construct the alternatives’. Identify the relevant and feasible policy solutions that your audience might consider.
4. ‘Select the criteria’. Typical value judgements relate to efficiency, equity and fairness, the trade-off between individual freedom and collective action, and the extent to which a policy process involves citizens in deliberation.
5. ‘Project the outcomes’. Focus on the outcomes that key actors care about (such as value for money), and quantify and visualise your predictions if possible.
6. ‘Confront the trade-offs’. Compare the pros and cons of each solution, such as how much of a bad service policymakers will accept to cut costs.
7. ‘Decide’. Examine your case through the eyes of a policymaker.
8. ‘Tell your story’. Identify your target audience and tailor your case. Weigh up the benefits of oral versus written presentation. Provide an executive summary. Focus on coherence and clarity. Keep it simple and concise. Avoid jargon.

Box 2. Dunn (2017) Public Policy Analysis

1. What is the policy problem to be solved? Identify its severity, urgency, cause, and our ability to solve it. Don’t define the wrong problem, such as by oversimplifying.
2. What effect will each potential policy solution have? ‘Forecasting’ methods can help provide ‘plausible’ predictions about the future effects of current/ alternative policies.
3. Which solutions should we choose, and why? Normative assessments are based on values such as ‘equality, efficiency, security, democracy, enlightenment’ and beliefs about the preferable balance between state, communal, and market/ individual solutions (2017: 6; 205).
4. What were the policy outcomes? ‘Monitoring is crucial because it is difficult to predict policy success, and unintended consequences are inevitable (2017: 250).
5. Did the policy solution work as intended? Did it improve policy outcomes? Try to measure the outcomes your solution, while noting that evaluations are contested (2017: 332-41).

Box 3. Meltzer and Schwartz (2019) Policy Analysis as Problem Solving

1. ‘Define the problem’. Problem definition is a political act of framing, as part of a narrative to evaluate the nature, cause, size, and urgency of an issue.
2. ‘Identify potential policy options (alternatives) to address the problem’. Identify many possible solutions, then select the ‘most promising’ for further analysis (2019: 65).
3. ‘Specify the objectives to be attained in addressing the problem and the criteria  to  evaluate  the  attainment  of  these  objectives  as  well as  the  satisfaction  of  other  key  considerations  (e.g.,  equity,  cost, equity, feasibility)’.
4. ‘Assess the outcomes of the policy options in light of the criteria and weigh trade-offs between the advantages and disadvantages of the options’.
5. ‘Arrive at a recommendation’. Make a preliminary recommendation to inform an iterative process, drawing feedback from clients and stakeholder groups (2019: 212).

Box 4. Mintrom (2012) Contemporary Policy Analysis

1. ‘Engage in problem definition’. Define the nature of a policy problem, and the role of government in solving it, while engaging with many stakeholders (2012: 3; 58-60).
2. ‘Propose alternative responses to the problem’. Identify how governments have addressed comparable problems, and a previous policy’s impact (2012: 21).
3. ‘Choose criteria for evaluating each alternative policy response’. ‘Effectiveness, efficiency, fairness, and administrative efficiency’ are common (2012: 21).
4. ‘Project the outcomes of pursuing each policy alternative’. Estimate the cost of a new policy, in comparison with current policy, and in relation to factors such as savings to society or benefits to certain populations.
5. ‘Identify and analyse trade-offs among alternatives’. Use your criteria and projections to compare each alternative in relation to their likely costs and benefits.
6. ‘Report findings and make an argument for the most appropriate response’. Client-oriented advisors identify the beliefs of policymakers and tailor accordingly (2012: 22).

Box 5 Weimer and Vining (2017) Policy Analysis: Concepts and Practice

1. ‘Write to Your Client’. Having a client such as an elected policymaker requires you to address the question they ask, by their deadline, in a clear and concise way that they can understand (and communicate to others) quickly (2017: 23; 370-4).
2. ‘Understand the Policy Problem’. First, ‘diagnose the undesirable condition’. Second, frame it as ‘a market or government failure (or maybe both)’.
3. ‘Be Explicit About Values’ (and goals). Identify (a) the values to prioritise, such as ‘efficiency’, ‘equity’, and ‘human dignity’, and (b) ‘instrumental goals’, such as ‘sustainable public finance or political feasibility’, to generate support for solutions.
4. ‘Specify Concrete Policy Alternatives’. Explain potential solutions in sufficient detail to predict the costs and benefits of each ‘alternative’ (including current policy).
5. ‘Predict and Value Impacts’. Short deadlines dictate that you use ‘logic and theory, rather than systematic empirical evidence’ to make predictions efficiently (2017: 27)
6. ‘Consider the Trade-Offs’. Each alternatives will fulfil certain goals more than others. Produce a summary table to make value-based choices about trade-offs (2017: 356-8).
7. ‘Make a Recommendation’. ‘Unless your client asks you not to do so, you should explicitly recommend one policy’ (2017: 28).

This is an excerpt from The Politics of Policy Analysis, found here: https://paulcairney.wordpress.com/policy-analysis-in-750-words/

Policy Analysis in 750 Words: how much impact can you expect from your analysis?

This post forms one part of the Policy Analysis in 750 words series overview.

Throughout this series you may notice three different conceptions about the scope of policy analysis:

1. ‘Ex ante’ (before the event) policy analysis. Focused primarily on defining a problem, and predicting the effect of solutions, to inform current choice (as described by Meltzer and Schwartz and Thissen and Walker).
2. ‘Ex post’ (after the event) policy analysis. Focused primarily on monitoring and evaluating that choice, perhaps to inform future choice (as described famously by Weiss).
3. Some combination of both, to treat policy analysis as a continuous (never-ending) process (as described by Dunn).

As usual, these are not hard-and-fast distinctions, but they help us clarify expectations in relation to different scenarios.

1. The impact of old-school ex ante policy analysis

Radin provides a valuable historical discussion of policymaking with the following elements:

• a small number of analysts, generally inside government (such as senior bureaucrats, scientific experts, and – in particular- economists),
• giving technical or factual advice,
• about policy formulation,
• to policymakers at the heart of government,
• on the assumption that policy problems would be solved via analysis and action.

This kind of image signals an expectation for high impact: policy analysts face low competition, enjoy a clearly defined and powerful audience, and their analysis is expected to feed directly into choice.

Radin goes on to describe a much different, modern policy environment: more competition, more analysts spread across and outside government, with a less obvious audience, and – even if there is a client – high uncertainty about where the analysis fits into the bigger picture.

Yet, the impetus to seek high and direct impact remains.

This combination of shifting conditions but unshifting hopes/ expectations helps explain a lot of the pragmatic forms of policy analysis you will see in this series, including:

• Keep it catchy, gather data efficiently, tailor your solutions to your audience, and tell a good story (Bardach)
• Speak with an audience in mind, highlight a well-defined problem and purpose, project authority, use the right form of communication, and focus on clarity, precision, conciseness, and credibility ( Smith)
• Address your client’s question, by their chosen deadline, in a clear and concise way that they can understand (and communicate to others) quickly (Weimer and Vining)
• Client-oriented advisors identify the beliefs of policymakers and anticipate the options worth researching (Mintrom)
• Identify your client’s resources and motivation, such as how they seek to use your analysis, the format of analysis they favour (make it ‘concise’ and ‘digestible’), their deadline, and their ability to make or influence the policies you might suggest (Meltzer and Schwartz).
• ‘Advise strategically’, to help a policymaker choose an effective solution within their political context (Thissen and Walker).
• Focus on producing ‘policy-relevant knowledge’ by adapting to the evidence-demands of policymakers and rejecting a naïve attachment to ‘facts speaking for themselves’ or ‘knowledge for its own sake’ (Dunn).

2. The impact of research and policy evaluation

Many of these recommendations are familiar to scientists and researchers, but generally in the context of far lower expectations about their likely impact, particularly if those expectations are informed by policy studies (compare Oliver & Cairney with Cairney & Oliver).

In that context, Weiss’ work is a key reference point. It gives us a menu of ways in which policymakers might use policy evaluation (and research evidence more widely):

• to inform solutions to a problem identified by policymakers
• as one of many sources of information used by policymakers, alongside ‘stakeholder’ advice and professional and service user experience
• as a resource used selectively by politicians, with entrenched positions, to bolster their case
• as a tool of government, to show it is acting (by setting up a scientific study), or to measure how well policy is working
• as a source of ‘enlightenment’, shaping how people think over the long term (compare with this discussion of ‘evidence based policy’ versus ‘policy based evidence’).

In other words, researchers may have a role, but they struggle (a) to navigate the politics of policy analysis, (b) find the right time to act, and (c) to secure attention, in competition with many other policy actors.

3. The potential for a form of continuous impact

Dunn suggests that the idea of ‘ex ante’ policy analysis is misleading, since policymaking is continuous, and evaluations of past choices inform current choices. Think of each policy analysis steps as ‘interdependent’, in which new knowledge to inform one step also informs the other four. For example, routine monitoring helps identify compliance with regulations, if resources and services reach ‘target groups’, if money is spent correctly, and if we can make a causal link between the policy solutions and outcomes. Its impact is often better seen as background information with intermittent impact.

Key conclusions to bear in mind

1. The demand for information from policy analysts may be disproportionately high when policymakers pay attention to a problem, and disproportionately low when they feel that they have addressed it.
2. Common advice for policy analysts and researchers often looks very similar: keep it concise, tailor it to your audience, make evidence ‘policy relevant’, and give advice (don’t sit on the fence). However, unless researchers are prepared to act quickly, to gather data efficiently (not comprehensively), to meet a tight brief for a client, they are not really in the impact business described by most policy analysis texts.
3. A lot of routine, continuous, impact tends to occur out of the public spotlight, based on rules and expectations that most policy actors take for granted.

Further reading

See the Policy Analysis in 750 words series overview to continue reading on policy analysis.

See the ‘evidence-based policymaking’ page to continue reading on research impact.

Bristol powerpoint: Paul Cairney Bristol EBPM January 2020

Policy Analysis in 750 Words: what you need as an analyst versus policymaking reality

This post forms one part of the Policy Analysis in 750 words series overview. Note for the eagle eyed: you are not about to experience déjà vu. I’m just using the same introduction.

When describing ‘the policy sciences’, Lasswell distinguishes between:

1. ‘knowledge of the policy process’, to foster policy studies (the analysis of policy)
2. ‘knowledge in the process’, to foster policy analysis (analysis for policy)

The lines between each approach are blurry, and each element makes less sense without the other. However, the distinction is crucial to help us overcome the major confusion associated with this question:

Does policymaking proceed through a series of stages?

The short answer is no.

The longer answer is that you can find about 40 blog posts (of 500 and 1000 words) which compare (a) a stage-based model called the policy cycle, and (b) the many, many policy concepts and theories that describe a far messier collection of policy processes.

In a nutshell, most policy theorists reject this image because it oversimplifies a complex policymaking system. The image provides a great way to introduce policy studies, and serves a political purpose, but it does more harm than good:

1. Descriptively, it is profoundly inaccurate (unless you imagine thousands of policy cycles interacting with each other to produce less orderly behaviour and less predictable outputs).
2. Prescriptively, it gives you rotten advice about the nature of your policymaking task (for more on these points, see this chapter, article, article, and series).

Why does the stages/ policy cycle image persist? Two relevant explanations

 

1. It arose from a misunderstanding in policy studies

In another nutshell, Chris Weible and I argue (in a secret paper) that the stages approach represents a good idea gone wrong:

• If you trace it back to its origins, you will find Lasswell’s description of decision functions: intelligence, recommendation, prescription, invocation, application, appraisal and termination.
• These functions correspond reasonably well to a policy cycle’s stages: agenda setting, formulation, legitimation, implementation, evaluation, and maintenance, succession or termination.
• However, Lasswell was imagining functional requirements, while the cycle seems to describe actual stages.

In other words, if you take Lasswell’s list of what policy analysts/ policymakers need to do, multiple it by the number of actors (spread across many organisations or venues) trying to do it, then you get the multi-centric policy processes described by modern theories. If, instead, you strip all that activity down into a single cycle, you get the wrong idea.

2. It is a functional requirement of policy analysis

This description should seem familiar, because the classic policy analysis texts appear to describe a similar series of required steps, such as:

1. define the problem
2. identify potential solutions
3. choose the criteria to compare them
4. evaluate them in relation to their predicted outcomes
5. recommend a solution
6. monitor its effects
7. evaluate past policy to inform current policy.

However, these texts also provide a heavy dose of caution about your ability to perform these steps (compare Bardach, Dunn, Meltzer and Schwartz, Mintrom, Thissen and Walker, Weimer and Vining)

In addition, studies of policy analysis in action suggest that:

• an individual analyst’s need for simple steps, to turn policymaking complexity into useful heuristics and pragmatic strategies,

should not be confused with

• what actually happens when many policy analysts, influencers, and policymakers interact in policy processes (see Radin, and Brans, Geva-May, and Howlett).

What you need versus what you can expect

Overall, this discussion of policy studies and policy analysis reminds us of a major difference between:

1. Functional requirements. What you need from policymaking systems, to (a) manage your task (the 5-8 step policy analysis) and (b) understand and engage in policy processes (the simple policy cycle).
2. Actual processes and outcomes. What policy concepts and theories tell us about bounded rationality (which limit the comprehensiveness of your analysis) and policymaking complexity (which undermines your understanding and engagement in policy processes).

Of course, I am not about to provide you with a solution to these problems.

Still, this discussion should help you worry a little bit less about the circular arguments you will find in key texts: here are some simple policy analysis steps, but policymaking is not as ‘rational’ as the steps suggest, but (unless you can think of an alternative) there is still value in the steps, and so on.

See also:

The New Policy Sciences

Policy Analysis in 750 Words: Defining policy problems and choosing solutions

This post forms one part of the Policy Analysis in 750 words series overview.

When describing ‘the policy sciences’, Lasswell distinguishes between:

1. ‘knowledge of the policy process’, to foster policy studies (the analysis of policy)
2. ‘knowledge in the process’, to foster policy analysis (analysis for policy)

The idea is that both elements are analytically separable but mutually informative: policy analysis is crucial to solving real policy problems, policy studies inform the feasibility of analysis, the study of policy analysts informs policy studies, and so on.

Both elements focus on similar questions – such as What is policy? – and explore their descriptive (what do policy actors do?) and prescriptive (what should they do?) implications.

1. What is the policy problem?

Policy studies tend to describe problem definition in relation to framing, narrative, social construction, power, and agenda setting.

Actors exercise power to generate attention for their preferred interpretation, and minimise attention to alternative frames (to help foster or undermine policy change, or translate their beliefs into policy).

Policy studies incorporate insights from psychology to understand (a) how policymakers might combine cognition and emotion to understand problems, and therefore (b) how to communicate effectively when presenting policy analysis.

Policy studies focus on the power to reduce ambiguity rather than simply the provision of information to reduce uncertainty. In other words, the power to decide whose interpretation of policy problems counts, and therefore to decide what information is policy-relevant.

This (unequal) competition takes place within a policy process over which no actor has full knowledge or control.

The classic 5-8 step policy analysis texts focus on how to define policy problems well, but they vary somewhat in their definition of doing it well (see also C.Smith):

• Bardach recommends using rhetoric and eye-catching data to generate attention
• Weimer and Vining and Mintrom recommend beginning with your client’s ‘diagnosis’, placing it in a wider perspective to help analyse it critically, and asking yourself how else you might define it (see also Bacchi, Stone)
• Meltzer and Schwartz and Dunn identify additional ways to contextualise your client’s definition, such as by generating a timeline to help ‘map’ causation or using ‘problem-structuring methods’ to compare definitions and avoid making too many assumptions on a problem’s cause.
• Thissen and Walker compare ‘rational’ and ‘argumentative’ approaches, treating problem definition as something to be measured scientifically or established rhetorically (see also Riker).

These approaches compare with more critical accounts that emphasise the role of power and politics to determine whose knowledge is relevant (L.T.Smith) and whose problem definition counts (Bacchi, Stone). Indeed, Bacchi and Stone provide a crucial bridge between policy analysis and policy studies by reflecting on what policy analysts do and why.

2. What is the policy solution?

In policy studies, it is common to identify counterintuitive or confusing aspects of policy processes, including:

• Few studies suggest that policy responses actually solve problems (and many highlight their potential to exacerbate them). Rather, ‘policy solutions’ is shorthand for proposed or alleged solutions.
• Problem definition often sets the agenda for the production of ‘solutions’, but note the phrase solutions chasing problems (when actors have their ‘pet’ solutions ready, and they seek opportunities to promote them).

Policy studies: problem definition informs the feasibility and success of solutions

Generally speaking, to define the problem is to influence assessments of the feasibility of solutions:

• Technical feasibility. Will they work as intended, given the alleged severity and cause of the problem?
• Political feasibility. Will they receive sufficient support, given the ways in which key policy actors weigh up the costs and benefits of action?

Policy studies highlight the inextricable connection between technical and political feasibility. Put simply, (a) a ‘technocratic’ choice about the ‘optimality’ of a solution is useless without considering who will support its adoption, and (b) some types of solution will always be a hard sell, no matter their alleged effectiveness (Box 2.3 below).

In that context, policy studies ask: what types of policy tools or instruments are actually used, and how does their use contribute to policy change? Measures include the size, substance, speed, and direction of policy change.

In turn, problem definition informs: the ways in which actors will frame any evaluation of policy success, and the policy-relevance of the evidence to evaluate solutions. Simple examples include:

• If you define tobacco in relation to: (a) its economic benefits, or (b) a global public health epidemic, evaluations relate to (a) export and taxation revenues, or (b) reductions in smoking in the population.
• If you define ‘fracking’ in relation to: (a) seeking more benefits than costs, or (b) minimising environmental damage and climate change, evaluations relate to (a) factors such as revenue and effective regulation, or simply (b) how little it takes place.

Policy analysis: recognising and pushing boundaries

Policy analysis texts tend to accommodate these insights when giving advice:

• Bardach recommends identifying solutions that your audience might consider, perhaps providing a range of options on a notional spectrum of acceptability.
• Smith highlights the value of ‘precedent’, or relating potential solutions to previous strategies.
• Weimer and Vining identify the importance of ‘a professional mind-set’ that may be more important than perfecting ‘technical skills’
• Mintrom notes that some solutions are easier to sell than others
• Meltzer and Schwartz describe the benefits of making a preliminary recommendation to inform an iterative process, drawing feedback from clients and stakeholder groups
• Dunn warns against too-narrow forms of ‘evidence based’ analysis which undermine a researcher’s ability to adapt well to the evidence-demands of policymakers
• Thissen and Walker relate solution feasibility to a wide range of policy analysis ‘styles’

Still, note the difference in emphasis.

Policy analysis education/ training may be about developing the technical skills to widen definitions and apply many criteria to compare solutions.

Policy studies suggest that problem definition and a search for solutions takes place in an environment where many actors apply a much narrower lens and are not interested in debates on many possibilities (particularly if they begin with a solution).

I have exaggerated this distinction between each element, but it is worth considering the repeated interaction between them in practice: politics and policymaking provide boundaries for policy analysis, analysis could change those boundaries, and policy studies help us reflect on the impact of analysts.

I’ll take a quick break, then discuss how this conclusion relates to the idea of ‘entrepreneurial’ policy analysis.

Further reading

Understanding Public Policy (2020: 28) describes the difference between governments paying for and actually using the ‘tools of policy formulation’. To explore this point, see ‘The use and non-use of policy appraisal tools in public policy making‘ and The Tools of Policy Formulation.

Policy Analysis in 750 Words: What can you realistically expect policymakers to do?

This post forms one part of the Policy Analysis in 750 words series overview.

One aim of this series is to combine insights from policy research (1000, 500) and policy analysis texts.

In this case, modern theories of the policy process help you identify your audience and their capacity to follow your advice. This simple insight may have a profound impact on the advice you give.

Policy analysis for an ideal-type world

For our purposes, an ideal-type is an abstract idea, which highlights hypothetical features of the world, to compare with ‘real world’ descriptions. It need not be an ideal to which we aspire. For example, comprehensive rationality describes the ideal type, and bounded rationality describes the ‘real world’ limitations to the ways in which humans and organisations process information.

 

Imagine writing policy analysis in the ideal-type world of a single powerful ‘comprehensively rational’ policymaker at the heart of government, making policy via an orderly policy cycle.

Your audience would be easy to identify, your analysis would be relatively simple, and you would not need to worry about what happens after you make a recommendation for policy change.

You could adopt a simple 5-8 step policy analysis method, use widely-used tools such as cost-benefit analysis to compare solutions, and know where the results would feed into the policy process.

I have perhaps over-egged this ideal-type pudding, but I think a lot of traditional policy analyses tapped into this basic idea and focused more on the science of analysis than the political and policymaking context in which it takes place (see Radin and Brans, Geva-May, and Howlett).

Policy analysis for the real world

Then imagine a far messier and less predictable world in which the nature of the policy issue is highly contested, responsibility for policy is unclear, and no single ‘centre’ has the power to turn a recommendation into an outcome.

This image is a key feature of policy process theories, which describe:

• Many policymakers and influencers spread across many levels and types of government (as the venues in which authoritative choice takes place). Consequently, it is not a straightforward task to identify and know your audience, particularly if the problem you seek to solve requires a combination of policy instruments controlled by different actors.
• Each venue resembles an institution driven by formal and informal rules. Formal rules are written-down or widely-known. Informal rules are unwritten, difficult to understand, and may not even be understood in the same way by participants. Consequently, it is difficult to know if your solution will be a good fit with the standard operating procedures of organisations (and therefore if it is politically feasible or too challenging).
• Policymakers and influencers operate in ‘subsystems’, forming networks built on resources such as trust or coalitions based on shared beliefs. Effective policy analysis may require you to engage with – or become part of – such networks, to allow you to understand the unwritten rules of the game and encourage your audience to trust the messenger. In some cases, the rules relate to your willingness to accept current losses for future gains, to accept the limited impact of your analysis now in the hope of acceptance at the next opportunity.
• Actors relate their analysis to shared understandings of the world – how it is, and how it should be – which are often so well-established as to be taken for granted. Common terms include paradigms, hegemons, core beliefs, and monopolies of understandings. These dominant frames of reference give meaning to your policy solution. They prompt you to couch your solutions in terms of, for example, a strong attachment to evidence-based cases in public health, value for money in treasury departments, or with regard to core principles such as liberalism or socialism in different political systems.
• Your solutions relate to socioeconomic context and the events that seem (a) impossible to ignore and (b) out of the control of policymakers. Such factors range from a political system’s geography, demography, social attitudes, and economy, while events can be routine elections or unexpected crises.

What would you recommend under these conditions? Rethinking 5-step analysis

There is a large gap between policymakers’ (a) formal responsibilities versus (b) actual control of policy processes and outcomes. Even the most sophisticated ‘evidence based’ analysis of a policy problem will fall flat if uninformed by such analyses of the policy process. Further, the terms of your cost-benefit analysis will be highly contested (at least until there is agreement on what the problem is, and how you would measure the success of a solution).

Modern policy analysis texts try to incorporate such insights from policy theories while maintaining a focus on 5-8 steps. For example:

• Meltzer and Schwartz contrast their ‘flexible’ and ‘iterative’ approach with a too- rigid ‘rationalistic approach’.
• Bardachand Dunn emphasise the value of political pragmatism and the ‘art and craft’ of policy analysis.
• Weimer and Vininginvest 200 pages in economic analyses of markets and government, often highlighting a gap between (a) our ability to model and predict economic and social behaviour, and (b) what actually happens when governments intervene.
• Mintrom invites you to see yourself as a policy entrepreneur, to highlight the value of of ‘positive thinking’, creativity, deliberation, and leadership, and perhaps seek ‘windows of opportunity’ to encourage new solutions. Alternatively, a general awareness of the unpredictability of events can prompt you to be modest in your claims, since the policymaking environment may be more important (than your solution) to outcomes.
• Thissen and Walker focus more on a range of possible roles than a rigid 5-step process.

Beyond 5-step policy analysis

1. Compare these pragmatic, client-orientated, and communicative models with the questioning, storytelling, and decolonizing approaches by Bacchi, Stone, and L.T. Smith.

• The latter encourage us to examine more closely the politics of policy processes, including the importance of framing, narrative, and the social construction of target populations to problem definition and policy design.
• Without this wider perspective, we are focusing on policy analysis as a process rather than considering the political context in which analysts use it.

2. Additional posts on entrepreneurs and ‘systems thinking’ [to be added] encourage us to reflect on the limits to policy analysis in multi-centric policymaking systems.

 

 

Policy Analysis in 750 Words: Reflecting on your role as a policy analyst

This post forms one part of the Policy Analysis in 750 words series overview.

One aim of this series is to combine insights from policy research (1000, 500) and policy analysis texts.

If we take key insights from policy theories seriously, we can use them to identify (a) the constraints to policy analytical capacity, and (b) the ways in which analysts might address them. I use the idea of policy analyst archetypes to compare a variety of possible responses.

Key constraints to policy analytical capacity

Terms like ‘bounded rationality’ highlight major limits on the ability of humans and organisations to process information.

• Humans use heuristics or cognitive shortcuts to process enough information to make choices, and institutions are the rules used by organisations to limit information processing.

Terms like policymaking ‘context’, ‘environments’, and multi-centric policymaking suggest that the policy process is beyond the limits of policymaker understanding and control.

• Policy actors need to find ways to act, with incomplete information about the problem they seek to solve and the likely impact of their ‘solution’.
• They gather information to help reduce uncertainty, but problem definition is really about exercising power to reduce ambiguity: select one way to interpret a problem (at the expense of most others), and limit therefore limit the relevance and feasibility of solutions.
• This context informs how actors might use the tools of policy analysis. Key texts in this series highlight the use of tools to establish technical feasibility (will it work as intended?), but policymakers also select tools for their political feasibility (who will support or oppose this measure?).

How might policy analysts address these constraints ethically?

Most policy analysis texts (in this series) consider the role of professional ethics and values during the production of policy analysis. However, they also point out that there is not a clearly defined profession and associated code of conduct (e.g. see Adachi). In that context, let us begin with some questions about the purpose of policy analysis and your potential role:

1. Is your primary role to serve individual clients or some notion of the ‘public good’?
2. Should you maximise your role as an individual or play your part in a wider profession?
3. What is the balance between the potential benefits of individual ‘entrepreneurship’ and collective ‘co-productive’ processes?
4. Which policy analysis techniques should you prioritise?
5. What forms of knowledge and evidence count in policy analysis?
6. What does it mean to communicate policy analysis responsibly?
7. Should you provide a clear recommendation or encourage reflection?

 

Policy analysis archetypes: pragmatists, entrepreneurs, manipulators, storytellers, and decolonisers

In that context, I have created a story of policy analysis archetypes to identify the elements that each text emphasises.

The pragmatic policy analyst

• Bardach provides the classic simple, workable, 8-step system to present policy analysis to policymakers while subject to time and resource-pressed political conditions.
• Dunn also uses Wildavsky’s famous phrase ‘art and craft’ to suggest that scientific and ‘rational’ methods can only take us so far.

The professional, client–oriented policy analyst

• Weimer and Vining provide a similar 7-step client-focused system, but incorporating a greater focus on professional development and economic techniques (such as cost-benefit-analysis) to emphasise a particular form of professional analyst.
• Meltzer and Schwartz also focus on advice to clients, but with a greater emphasis on a wide variety of methods or techniques (including service design) to encourage the co-design of policy analysis with clients.

The communicative policy analyst

•  C. Smith focuses on how to write and communicate policy analysis to clients in a political context.
• Compare with Spiegelhalter and Gigerenzer on how to communicate responsibly when describing uncertainty, probability, and risk.

The manipulative policy analyst.

• Riker helps us understand the relationship between two aspects of agenda setting: the rules/ procedures to make choice, and the framing of policy problems and solutions.

The entrepreneurial policy analyst

• Mintrom shows how to combine insights from studies of policy entrepreneurship and policy analysis, to emphasise the benefits of collaboration and creativity.

The questioning policy analyst

• Bacchi  analyses the wider context in which people give and use such advice, to identify the emancipatory role of analysis and encourage policy analysts to challenge dominant social constructions of problems and populations.

The storytelling policy analyst

• Stone identifies the ways in which people use storytelling and argumentation techniques to define problems and justify solutions. This process is about politics and power, not objectivity and optimal solutions.

The decolonizing policy analyst.

• L.T. Smith does not describe policy analysis directly, but shows how the ‘decolonization of research methods’ can inform the generation and use of knowledge.
• Compare with Hindess on the ways in which knowledge-based hierarchies rely on an untenable, circular logic.
• Compare with Michener’s thread, discussing Doucet’s new essay on (a) the role of power and knowledge in limiting (b) the ways in which we gather evidence to analyse policy problems.

https://twitter.com/povertyscholar/status/1207054211759910912

Using archetypes to define the problem of policy analysis

Studies of the field (e.g. Radin plus Brans, Geva-May, and Howlett) suggest that there are many ways to do policy analysis. Further, as Thissen and Walker describe, such roles are not mutually exclusive, your views on their relative value could change throughout the process of analysis, and you could perform many of these roles.

Further, each text describes multiple roles, and some seem clustered together:

• pragmatic, client-orientated, and communicative could sum-up the traditional 5-8 step approaches, while
• questioning, storytelling, and decolonizing could sum up an important (‘critical’) challenge to narrow ways of thinking about policy analysis and the use of information.

Still, the emphasis matters.

Each text is setting an agenda or defining the problem of policy analysis more-or-less in relation to these roles. Put simply, the more you are reading about economic theory and method, the less you are reading about dominance and manipulation.

How can you read further?

Michener’s ‘Policy Feedback in a Racialized Polity’ connects to studies of historical institutionalism, and reminds us to use insights from policy theories to identify the context for policy analysis.

I have co-authored a lot about uncertainty/ ambiguity in relation to ‘evidence based policymaking’, including:

• Fostering Evidence-informed Policy Making: Uncertainty Versus Ambiguity(PDF)
• To Bridge the Divide between Evidence and Policy: Reduce Ambiguity as Much as Uncertainty
• How to communicate effectively with policymakers: combine insights from psychology and policy studies

See also The new policy sciences for a discussion of how these issues inform Lasswell’s original vision for the policy sciences (combining the analysis of and for policy).

Policy Analysis in 750 words: Marleen Brans, Iris Geva-May, and Michael Howlett (2017) Routledge Handbook of Comparative Policy Analysis

Please see the Policy Analysis in 750 words series overview before reading the summary (and click here for the full list of authors). This post is a mere 500 words over budget (not including these words describing the number of words).

Marleen Brans, Iris Geva-May, and Michael Howlett (editors) (2017) Routledge Handbook of Comparative Policy Analysis (London: Routledge)

‘The Handbook … covers … the state of the art knowledge about the science, art and craft of policy analysis in different countries, at different levels of government and by all relevant actors in and outside government who contribute to the analysis of problems and the search for policy solutions’ (Brans et al, 2017: 1).

This book focuses on the interaction between (in Lasswell’s terms) ‘analysis for policy’ (policy analysis) and ‘analysis of policy’ (policy process research). In other words,

• what can the study of policy analysis tell us about policymaking, and
• what can studies of policymaking tell budding policy analysts about the nature of their task in relation to their policymaking environment?

Brans et al’s (2017: 1-6) opening discussion suggests that this task is rather unclear and complicated. They highlight the wide range of activity described by the term ‘policy analysis’:

1. The scope of policy analysis is wide, and its meaning unclear

Analysts can be found in many levels and types of government, in bodies holding governments to account, and outside of government, including interest groups, think tanks, and specialist firms (such as global accountancy or management consultancy firms – Saint-Martin, 2017).

Further, ‘what counts’ as policy analysis can relate to the people that do it, the rules they follow, the processes in which they engage, the form of outputs, and the expectations of clients (Veselý, 2017: 103; Vining and Boardman, 2017: 264).

2. The role of a policy analyst varies remarkably in relation to context

It varies over time, policy area, type of government (such as central, subnational, local), country, type of political system (e.g. majoritarian and consensus democracies), and ‘policy style’.

3. Analysis involves ‘science, art and craft’ and the rules are written and unwritten

The process of policy analysis – such as to gather and analyse information, define problems, design and compare solutions, and give policy advice – includes ‘applied social and scientific research as well as more implicit forms of practical knowledge’, and ‘both formal and informal professional practices’ (see also studies of institutions and networks).

4. The policy process is complex.

It is difficult to identify a straightforward process in which analysts are clearly engaged in multiple, well-defined ‘stages’ of policymaking.

5. Key principles and practices can be institutionalised, contested, or non-existent.

The idea of policy analysis principles – ‘of transparency, effectiveness, efficiency and accountability through systematic and evidence-based analysis’ – may be entrenched in places like the US but not globally.

In some political systems (particularly in the ‘Anglo-Saxon family of nations’), the most-described forms of policy analysis (in the 750 words series) may be taken for granted (2017: 4):

• Scholars encourage or criticise 5-step policy analysis techniques.
• Some methods (such as cost-benefit-analysis) are institutionalised and difficult to replace (or even supplement with more inclusive approaches).
• There is a high commitment to the (super-problematic) idea of ‘evidence based policymaking’ (Brans et al, 2017: 5).

Even so, the status of science and expertise is often contested, particularly in relation to salient and polarised issues, or more generally:

• During ‘attempts by elected politicians to restore the primacy of political judgement in the policymaking process, at the expense of technical or scientific evidence’ (2017: 5).
• When the ‘blending of expert policy analysis with public consultation and participation’ makes ‘advice more competitive and contested’ (2017: 5).
• When evidence based really means evidence informed, given that there are many legitimate claims to knowledge, and evidence forms one part of a larger process of policy design (van Nispen and de Jong, 2017: 153).

In many political systems, there may be less criticism of the idea of ‘systematic and evidence-based analysis’ because there less capacity to process information. It is difficult to worry about excessively technocratic approaches if they do not exist (a point that CW made to me just before I read this book).

Implications for policy analysis

1. It is difficult to think of policy analysis as a ‘profession’.

We may wonder if ‘policy analysis’ can ever be based on common skills and methods (such as described by Scott, 2017, and in Weimer and Vining), connected to ‘formal education and training’, a ‘a code of professional conduct’, and the ability of organisations to control membership (Adachi, 2017: 28; compare with Radin and Geva-May).

2. Policy analysis is a loosely-defined collection of practices that vary according to context.

Policy analysis may, instead, be considered a collection of ‘styles’ (Hassenteufel and Zittoun, 2017), influenced by:

• competing analytical approaches in different political systems (2017: 65)
• bureaucratic capacity for analysis (Mendez and Dussauge-Laguna, 2017: 82)
• a relative tendency to contract out analysis (Veselý, 2017: 113)
• the types and remits of advisory bodies (e.g. are they tasked simply with offering expert advice, or also to encourage wider participation to generate knowledge?) (Crowley and Head, 2017)
• the level of government in which analysts work, such as ‘subnational’ (Newman, 2017) or ‘local’ (Lundin and Öberg, 2017)
• the type of activity, such as when (‘performance’) budgeting analysis is influenced heavily by economic methods and ‘new public management’ reforms (albeit with limited success, followed by attempts at reform) (van Nispen and de Jong, 2017: 143-52)

Policy analysis can also describe a remarkably wide range of activity, including:

• Public inquiries (Marier, 2017)
• Advice to MPs, parliaments, and their committees (Wolfs and De Winter, 2017)
• The strategic analysis of public opinion or social media data (Rothmayr Allison, 2017; Kuo and Cheng, 2017)
• A diverse set of activities associated with ‘think tanks’ (Stone and Ladi, 2017) and ‘political party think tanks’ (Pattyn et al, 2017)
• Analysis for and by ‘business associations’ (Vining and Boardman, 2017), unions (Schulze and Schroeder, 2017), and voluntary/ non-profit organisations (Evans et al, 2017), all of whom juggle policy advice to government with keeping members on board.
• The more-or-less policy relevant work of academic researchers (Blum and Brans, 2017; compare with Dunn and see the EBPM page).

3. The analysis of and for policy is not so easy to separate in practice.

When defining policy analysis largely as a collection of highly-variable practices, in complex policymaking systems, we can see the symbiotic relationship between policy analysis and policy research. Studying policy analysis allows us to generate knowledge of policy processes. Policy process research demonstrates that the policymaking context influences how we think about policy analysis.

4. Policy analysis education and training is incomplete without policy process research

Put simply, we should not assume that graduates in ‘policy analysis’ will enter a central government with high capacity, coherent expectations, and a clear demand for the same basic skills. Yet, Fukuyama argues that US University programmes largely teach students:

‘a battery of quantitative methods … applied econometrics, cost-benefit analysis, decision analysis, and, most recently, use of randomized experiments for program evaluation’ that ‘will tell you what the optimal policy should be’, but not ‘how to achieve that outcome. The world is littered with optimal policies that don’t have a snowball’s chance in hell of being adopted’.

In that context, additional necessary skills include: stakeholder mapping, to identify who is crucial to policy success, defining policy problems in a way that stakeholders and policymakers can support, and including those actors continuously during a process of policy design and delivery. These skills are described at more length by Radin and Geva May, while Botha et al (2017) suggest that the policy analysis programmes (across North American and European Universities) offer a more diverse range of skills (and support for experiential learning) than Fukuyama describes.

Policy Analysis in 750 words: Deborah Stone (2012) Policy Paradox

Please see the Policy Analysis in 750 words series overview before reading the summary. This post is 750 words plus a bonus 750 words plus some further reading that doesn’t count in the word count even though it does.

Deborah Stone (2012) Policy Paradox: The Art of Political Decision Making 3^rd edition (Norton)

‘Whether you are a policy analyst, a policy researcher, a policy advocate, a policy maker, or an engaged citizen, my hope for Policy Paradox is that it helps you to go beyond your job description and the tasks you are given – to think hard about your own core values, to deliberate with others, and to make the world a better place’ (Stone, 2012: 15)

Stone (2012: 379-85) rejects the image of policy analysis as a ‘rationalist’ project, driven by scientific and technical rules, and separable from politics. Rather, every policy analyst’s choice is a political choice – to define a problem and solution, and in doing so choosing how to categorise people and behaviour – backed by strategic persuasion and storytelling.

The Policy Paradox: people entertain multiple, contradictory, beliefs and aims

Stone (2012: 2-3) describes the ways in which policy actors compete to define policy problems and public policy responses. The ‘paradox’ is that it is possible to define the same policies in contradictory ways.

‘Paradoxes are nothing but trouble. They violate the most elementary principle of logic: something can’t be two different things at once. Two contradictory interpretations can’t both be true. A paradox is just such an impossible situation, and political life is full of them’ (Stone, 2012: 2).

This paradox does not refer simply to a competition between different actors to define policy problems and the success or failure of solutions. Rather:

• The same actor can entertain very different ways to understand problems, and can juggle many criteria to decide that a policy outcome was a success and a failure (2012: 3).
• Surveys of the same population can report contradictory views – encouraging a specific policy response and its complete opposite – when asked different questions in the same poll (2012: 4; compare with Riker)

Policy analysts: you don’t solve the Policy Paradox with a ‘rationality project’

Like many posts in this series (Smith, Bacchi, Hindess), Stone (2010: 9-11) rejects the misguided notion of objective scientists using scientific methods to produce one correct answer (compare with Spiegelhalter and Weimer & Vining). A policy paradox cannot be solved by ‘rational, analytical, and scientific methods’ because:

• We can produce scientific evidence to reduce uncertainty, but not ambiguity.
• We can seek order and clarity in analysis and communication, but should not expect it in political systems (2012: 10).
• 5-step policy analysis (identify objectives, identify alternatives, predict their effects, evaluate alternatives, choose) is pervasive, but it ‘ignores our emotional feelings and moral intuitions’ (2012: 11).
• A policy cycle model does not explain why solutions often chase problems, or the continuous struggle over ideas even when a cycle is ostensibly complete (2012: 12-13)

Further, Stone (2012: 10-11) rejects the over-reliance, in policy analysis, on the misleading claim that:

• policymakers are engaging primarily with markets rather than communities (see 2012: 35 on the comparison between a ‘market model’ and ‘polis model’),
• economic models can sum up political life, and
• cost-benefit-analysis can reduce a complex problem into the sum of individual preferences using a single unambiguous measure.

Rather, many factors undermine such simplicity:

1. People do not simply act in their own individual interest. Nor can they rank-order their preferences in a straightforward manner according to their values and self-interest.

• Instead, they maintain a contradictory mix of objectives, which can change according to context and their way of thinking – combining cognition and emotion – when processing information (2012: 12; 30-4).

2. People are social actors. Politics is characterised by ‘a model of community where individuals live in a dense web of relationships, dependencies, and loyalties’ and exercise power with reference to ideas as much as material interests (2012: 10; 20-36; compare with Ostrom, more Ostrom, and Lubell).
3. Morals and emotions matter. If people juggle contradictory aims and measures of success, then a story infused with ‘metaphor and analogy’, and appealing to values and emotions, prompts people ‘to see a situation as one thing rather than another’ and therefore draw attention to one aim at the expense of the others (2012: 11; compare with Gigerenzer).

Policy analysis reconsidered: the ambiguity of values and policy goals

Stone (2012: 14) identifies the ambiguity of the criteria for success used in 5-step policy analyses. They do not form part of a solely technical or apolitical process to identify trade-offs between well-defined goals (compare Bardach, Weimer and Vining, and Mintrom). Rather, ‘behind every policy issue lurks a contest over conflicting, though equally plausible, conceptions of the same abstract goal or value’ (2012: 14). Examples of competing interpretations of valence issues include definitions of:

1. Equity, according to: (a) which groups should be included, how to assess merit, how to identify key social groups, if we should rank populations within social groups, how to define need and account for different people placing different values on a good or service, (b) which method of distribution to use (competition, lottery, election), and (c) how to balance individual, communal, and state-based interventions (2012: 39-62).
2. Efficiency, to use the least resources to produce the same objective, according to: (a) who determines the main goal and how to balance multiple objectives, (a) who benefits from such actions, and (c) how to define resources while balancing equity and efficiency – for example, does a public sector job and a social security payment represent a sunk cost to the state or a social investment in people? (2012: 63-84).
3. Welfare or Need, according to factors including (a) the material and symbolic value of goods, (b) short term support versus a long term investment in people, (c) measures of absolute poverty or relative inequality, and (d) debates on ‘moral hazard’ or the effect of social security on individual motivation (2012: 85-106)
4. Liberty, according to (a) a general balancing of freedom from coercion and freedom from the harm caused by others, (b) debates on individual and state responsibilities, and (c) decisions on whose behaviour to change to reduce harm to what populations (2012: 107-28)
5. Security, according to (a) our ability to measure risk scientifically (see Spiegelhalter and Gigerenzer), (b) perceptions of threat and experiences of harm, (c) debates on how much risk to safety to tolerate before intervening, (d) who to target and imprison, and (e) the effect of surveillance on perceptions of democracy (2012: 129-53).

Policy analysis as storytelling for collective action

Actors use policy-relevant stories to influence the ways in which their audience understands (a) the nature of policy problems and feasibility of solutions, within (b) a wider context of policymaking in which people contest the proper balance between state, community, and market action. Stories can influence key aspects of collective action, including:

1. Defining interests and mobilising actors, by drawing attention to – and framing – issues with reference to an imagined social group and its competition (e.g. the people versus the elite; the strivers versus the skivers) (2012: 229-47)
2. Making decisions, by framing problems and solutions (2012: 248-68). Stone (2012: 260) contrasts the ‘rational-analytic model’ with real-world processes in which actors deliberately frame issues ambiguously, shift goals, keep feasible solutions off the agenda, and manipulate analyses to make their preferred solution seem the most efficient and popular.
3. Defining the role and intended impact of policies, such as when balancing punishments versus incentives to change behaviour, or individual versus collective behaviour (2012: 271-88).
4. Setting and enforcing rules (see institutions), in a complex policymaking system where a multiplicity of rules interact to produce uncertain outcomes, and a powerful narrative can draw attention to the need to enforce some rules at the expense of others (2012: 289-310).
5. Persuasion, drawing on reason, facts, and indoctrination. Stone (2012: 311-30) highlights the context in which actors construct stories to persuade: people engage emotionally with information, people take certain situations for granted even though they produce unequal outcomes, facts are socially constructed, and there is unequal access to resources – held in particular by government and business – to gather and disseminate evidence.
6. Defining human and legal rights, when (a) there are multiple, ambiguous, and intersecting rights (in relation to their source, enforcement, and the populations they serve) (b) actors compete to make sure that theirs are enforced, (c) inevitably at the expense of others, because the enforcement of rights requires a disproportionate share of limited resources (such as policymaker attention and court time) (2012: 331-53)
7. Influencing debate on the powers of each potential policymaking venue – in relation to factors including (a) the legitimate role of the state in market, community, family, and individual life, (b) how to select leaders, (c) the distribution of power between levels and types of government – and who to hold to account for policy outcomes (2012: 354-77).

Key elements of storytelling include:

1. Symbols, which sum up an issue or an action in a single picture or word (2012:157-8)
2. Characters, such as heroes or villain, who symbolise the cause of a problem or source of solution (2012:159)
3. Narrative arcs, such as a battle by your hero to overcome adversity (2012:160-8)
4. Synecdoche, to highlight one example of an alleged problem to sum up its whole (2012: 168-71; compare the ‘welfare queen’ example with SCPD)
5. Metaphor, to create an association between a problem and something relatable, such as a virus or disease, a natural occurrence (e.g. earthquake), something broken, something about to burst if overburdened, or war (2012: 171-78; e.g. is crime a virus or a beast?)
6. Ambiguity, to give people different reasons to support the same thing (2012: 178-82)
7. Using numbers to tell a story, based on political choices about how to: categorise people and practices, select the measures to use, interpret the figures to evaluate or predict the results, project the sense that complex problems can be reduced to numbers, and assign authority to the counters (2012:183-205; compare with Speigelhalter)
8. Assigning Causation, in relation to categories including accidental or natural, ‘mechanical’ or automatic (or in relation to institutions or systems), and human-guided causes that have intended or unintended consequences (such as malicious intent versus recklessness)

• ‘Causal strategies’ include to: emphasise a natural versus human cause, relate it to ‘bad apples’ rather than systemic failure, and suggest that the problem was too complex to anticipate or influence
• Actors use these arguments to influence rules, assign blame, identify ‘fixers’, and generate alliances among victims or potential supporters of change (2012: 206-28).

Wider Context and Further Reading: 1. Policy analysis

This post connects to several other 750 Words posts, which suggest that facts don’t speak for themselves. Rather, effective analysis requires you to ‘tell your story’, in a concise way, tailored to your audience.

For example, consider two ways to establish cause and effect in policy analysis:

One is to conduct and review multiple randomised control trials.

Another is to use a story of a hero or a villain (perhaps to mobilise actors in an advocacy coalition).

2. Evidence-based policymaking

Stone (2012: 10) argues that analysts who try to impose one worldview on policymaking will find that ‘politics looks messy, foolish, erratic, and inexplicable’. For analysts, who are more open-minded, politics opens up possibilities for creativity and cooperation (2012: 10).

This point is directly applicable to the ‘politics of evidence based policymaking’. A common question to arise from this worldview is ‘why don’t policymakers listen to my evidence?’ and one answer is ‘you are asking the wrong question’.

3. Policy theories highlight the value of stories (to policy analysts and academics)

Policy problems and solutions necessarily involve ambiguity:

1. There are many ways to interpret problems, and we resolve such ambiguity by exercising power to attract attention to one way to frame a policy problem at the expense of others (in other words, not with reference to one superior way to establish knowledge).

• Punctuated equilibrium theory shows that policy change is primarily a function of such shifts in policymaker attention.

2. Policy is actually a collection of – often contradictory – policy instruments and institutions, interacting in complex systems or environments, to produce unclear messages and outcomes. As such, what we call ‘public policy’ (for the sake of simplicity) is subject to interpretation and manipulation as it is made and delivered, and we struggle to conceptualise and measure policy change. Indeed, it makes more sense to describe competing narratives of policy change.

4. Policy theories and storytelling

People communicate meaning via stories. Stories help us turn (a) a complex world, which provides a potentially overwhelming amount of information, into (b) something manageable, by identifying its most relevant elements and guiding action (compare with Gigerenzer on heuristics).

The Narrative Policy Framework identifies the storytelling strategies of actors seeking to exploit other actors’ cognitive shortcuts, using a particular format – containing the setting, characters, plot, and moral – to focus on some beliefs over others, and reinforce someone’s beliefs enough to encourage them to act.

Compare with Tuckett and Nicolic on the stories that people tell to themselves.

 

Policy Analysis in 750 words: Michael Mintrom (2012) Contemporary Policy Analysis

Please see the Policy Analysis in 750 words series overview before reading the summary. This summary is not 750 words. I can only apologise.

Michael Mintrom (2012) Contemporary Policy Analysis (Oxford University Press)

Just learned from @MikeMintrom 's book that a policy wonk is called a wonk because they know everything backwards. Now just sitting here wondering what else I don't know. pic.twitter.com/b1MMESsj83

— Professor Paul Cairney (@CairneyPaul) November 15, 2019

Mintrom (2012: xxii; 17) describes policy analysis as ‘an enterprise primarily motivated by the desire to generate high quality information to support high-quality decisions’ and stop policymakers ‘from making ill-considered choices’ (2012: 17). It is about giving issues more ‘serious attention and deep thought’ than busy policymakers, rather than simply ‘an exercise in the application of techniques’ to serve clients (2012: 20; xxii).

It begins with six ‘Key Steps in Policy Analysis’ (2012: 3-5):

1. ‘Engage in problem definition’

Problem definition influences the types of solutions that will be discussed (although, in some cases, solutions chase problems).

Define the nature and size of a policy problem, and the role of government in solving it (from maximal to minimal), while engaging with many stakeholders with different views (2012: 3; 58-60).

This task involves a juggling act. First, analysts should engage with their audience to work out what they need and when (2012 : 81). However, second, they should (a) develop ‘critical abilities’, (b) ask themselves ‘why they have been presented in specific ways, what their sources might be, and why they have arisen at this time’, and (c) present ‘alternative scenarios’ (2012: 22; 20; 27).

2. ‘Propose alternative responses to the problem’

Governments use policy instruments – such as to influence markets, tax or subsidize activity, regulate behaviour, provide services (directly, or via commissioning or partnership), or provide information – as part of a coherent strategy or collection of uncoordinated measures (2012: 30-41). In that context, try to:

• Generate knowledge about how governments have addressed comparable problems (including, the choice to not intervene if an industry self-regulates).
• Identify the cause of a previous policy’s impact and if it would have the same effect now (2012: 21).
• If minimal comparable information is available, consider wider issues from which to learn (2012: 76-7; e.g. alcohol policy based on tobacco).

Consider the wider:

• Political context, to anticipate how policymakers, implementers, target populations, and publics would react (2012: 20). Note that some policy measures are more difficult to ‘sell’ than others.
• Policymaking context, in which policies and institutions already exist to address most policy problems in some way.

 

3. ‘Choose criteria for evaluating each alternative policy response’

There are no natural criteria, but ‘effectiveness, efficiency, fairness, and administrative efficiency’ are common (2012: 21). ‘Effective institutions’ have a marked impact on social and economic life and provide political stability (2012: 49). Governments can promote ‘efficient’ policies by (a) producing the largest number of winners and (b) compensating losers (2012: 51-2; see Weimer and Vining on Kaldor-Hicks). They can prioritise environmental ‘sustainability’ to mitigate climate change, the protection of human rights and ‘human flourishing’, and/or a fair allocation of resources (2012: 52-7).

4. ‘Project the outcomes of pursuing each policy alternative’

Estimate the costs of a new policy, in comparison with current policy, and in relation to factors such as (a) overall savings to society, and/or (b) benefits to certain populations (any policy will benefit some social groups more than others). Mintrom (2012: 21) emphasises ‘prior knowledge and experience’ and ‘synthesizing’ work by others alongside techniques such as cost-benefit analyses.

5. ‘Identify and analyse trade-offs among alternatives’

Use your criteria and projections to compare each alternative in relation to their likely costs and benefits.

6. ‘Report findings and make an argument for the most appropriate response’

Mintrom (2012: 5) describes a range of advisory roles.

(a) Client-oriented advisors identify the beliefs of policymakers and anticipate the options worth researching (although they should not simply tell clients what they want to hear – 2012: 22). They may only have the time to answer a client’s question quickly and on their own. Or, they need to create and manage a team project (2012: 63-76).

(b) Other actors, ‘who want to change the world’, research options that are often not politically feasible in the short term but are too important to ignore (such as gender mainstreaming or action to address climate change).

In either case, the format of a written report – executive summary, contents, background, analytical strategy, analysis and findings (perhaps including a table comparing goals and trade-offs between alternatives), discussion, recommendation, conclusion, annex – may be similar (2012: 82-6).

Wider context: the changing role of policy analysts

Mintrom (2012: 5-7) describes a narrative – often attributed to Radin – of the changing nature of policy analysis, comparing:

1. (a) a small group of policy advisors, (b) with a privileged place in government, (c) giving allegedly technical advice, using economic techniques such as cost-benefit analysis.
2. (a) a much larger profession, (b) spread across – and outside of – government (including external consultants), and (c) engaging more explicitly in the politics of policy analysis and advice.

It reflects wider changes in government, (a) from the ‘clubby’ days to a much more competitive environment debating a larger number and wider range of policy issues, subject to (b) factors such as globalisation that change the task/ context of policy analysis.

If so, any advice on how to do policy analysis has to be flexible, to incorporate the greater diversity of actors and the sense that complex policymaking systems require flexible skills and practices rather than standardised techniques and outputs.

The ethics of policy analysis

In that context, Mintrom (2012: 95-108) emphasises the enduring role for ethical policy analysis, which can relate to:

1. ‘Universal’ principles such as fairness, compassion, and respect
2. Specific principles to project the analyst’s integrity, competence, responsibility, respectfulness, and concern for others
3. Professional practices, such as to

• engage with many stakeholders in problem definition (to reflect a diversity of knowledge and views)
• present a range of feasible solutions, making clear their distributional effects on target populations, opportunity costs (what policies/ outcomes would not be funded if this were), and impact on those who implement policy
• be honest about (a) the method of calculation, and (b) uncertainty, when projecting outcomes
• clarify the trade-offs between alternatives (don’t stack-up the evidence for one)
• maximise effective information sharing, rather than exploiting the limited attention of your audience (compare with Riker).

4. New analytical strategies (2012: 114-15; 246-84)

• Traditional analysis focuses on the successes and failures of markets and government (including serious consideration of non-governmental solutions), institutional analysis (including ‘inheritance’ and inertia), cost-benefit, and top-down implementation
• The analysis of factors such as gender and race helps to measure:

1. the extent to which social groups are already ‘systematically disadvantaged’,
2. the causes (such as racism and sexism) of – and potential solutions to – these outcomes, to make sure
3. that new policies reduce or do not perpetuate disadvantages, even when
4. politicians may gain electorally from scapegoating target populations and/ or
5. there are major obstacles to transformative policy change.

Therefore, while Mintrom’s (2012: 3-5; 116) ‘Key Steps in Policy Analysis’ are comparable to Bardach and Weimer and Vining, his emphasis is often closer to Bacchi’s.

The entrepreneurial policy analyst

Mintrom (2012: 307-13) ends with a discussion of the intersection between policy entrepreneurship and analysis, highlighting the benefits of ‘positive thinking’, creativity, deliberation, and leadership. He expands on these ideas further in So you want to be a policy entrepreneur?

Policy Analysis in 750 words: William Riker (1986) The Art of Political Manipulation

Please see the Policy Analysis in 750 words series overview before reading the summary.

William H. Riker (1986) The Art of Political Manipulation (New Haven: Yale University Press)

Most texts in this series describe the politics of policy analysis, in which your aim is to communicate with a client to help them get what they want, subject to professional standards and ethics (Smith, Bardach, and Weimer and Vining).

Such texts suggest that the evidence will not speak for itself, and that your framing of information could make a big difference between success and failure. However, they tend to dance around the question of how to exercise power to maximise your success.

The consequence may be some bland Aristotle-style advice, in which you should seek to be a persuasive narrator by combining:

• Pathos. The appeal to an audience’s emotions to maximise interest in a problem.
• Logos. The concise presentation of information and logic to make a persuasive case.
• Ethos. The credibility of the presenter, based on their experience and expertise.

Studies of narrative suggest that these techniques have some impact. Narrators tap into their audience’s emotions and beliefs, make a problem seem ‘concrete’ and urgent, and romanticise a heroic figure or cause. However, their success depends heavily on the context, and stories tend to be most influential of the audiences predisposed to accept them.

If so, a key option is to exploit a tendency for people to possess many contradictory beliefs, which suggests that (a) they could support many different goals or policy solutions, and (b) their support may relate strongly to the context and rules that determine the order and manner in which they make choices.

In other words, you may not be able to ‘change their minds’, but you can encourage them to pay more attention to, and place more value on, one belief (or one way to understand a policy problem) at the expense of another. This strategy could make the difference between belief and action.

Riker (1986: ix) uses the term ‘heresthetic’ to describe ‘structuring the world so you can win’. People ‘win politically because they have set up the situation in such a way that other people will want to join them’. Examples include:

1. Designing the order in which people make choices, because many policy preferences are ‘intransitive’: if A is preferred to B and B to C, A is not necessarily preferred to C.
2. Exploiting the ways in which people deal with ‘bounded rationality’ (the limits to their ability to process information to make choices).

For example, what if people are ‘cognitive misers’, seeking to process information efficiently rather than comprehensively? What if they combine cognition and emotion to make choices efficiently? Riker highlights the potential value of some combination of the following strategies:

1. Make your preferred problem framing or solution as easy to understand as possible.
2. Make other problems/ solutions difficult to process, such as by presenting them in the abstract and providing excessive detail.
3. Emphasize the high cognitive cost to the examination of all other options.
4. Experiment with choice-rule options that consolidate the vote for your preferred option while splitting the vote of others.
5. Design the comparison of a small number of options to make sure that yours is the most competitive.
6. Design the framing of choice (for example, is a vote primarily about the substantial issue or confidence in its proponents?).
7. Design the selection of criteria to evaluate options.
8. Design a series of votes, in sequence, to allow you to trade votes with others.
9. Conspire to make sure that the proponent of your preferred choice is seen as heroic (and the proponent of another choice as of flawed character and intellect).
10. Ensure that people make or vote for choices quickly, to ward off the possibility of further analysis and risk of losing control of the design of choice.
11. Make sure that you engage in these strategies without being detected or punished.

The point of this discussion is not to recommend that policy analysts become Machiavellian manipulators, fixing their eye on the prize, and doing anything to win.

Rather, it is to highlight the wider agenda setting context that you face when presenting evidence, values, and options.

It is a truism in policy studies that the evidence does not speak for itself. Instead, people engage in effective communication and persuasion to assign meaning to the evidence.

Similarly, it would be a mistake to expect success primarily from a well written and argued policy analysis document. Rather, much of its fate depends on who is exploiting the procedures and rules that influence how people make choices.

See also:

Evidence-based policymaking: political strategies for scientists living in the real world

Three habits of successful policy entrepreneurs

Evidence-informed policymaking: context is everything

Please note: some of this text comes from Box 4.3 in Understanding Public Policy 2^nd ed

 

Policy Analysis (usually) in 750 words: David Weimer and Adrian Vining (2017) Policy Analysis

Please see the Policy Analysis in 750 words series overview before reading the summary.

Please note that this book is the longest in the series (almost 500 pages), so a 750 word summary would have been too heroic.

David Weimer and Adrian Vining (2017) Policy Analysis: Concepts and Practice 6^th Edition (Routledge)

Weimer and Vining (2017: 23-8; 342-75) describe policy analysis in seven steps:

1. ‘Write to Your Client’

Having a client such as an elected policymaker (or governmental or nongovernmental organization) requires you to: address the question they ask, by their chosen deadline, in a clear and concise way that they can understand (and communicate to others) quickly (2017: 23; 370-4).

Their sample documents are 18 pages, including an executive summary and summary table.

2. ‘Understand the Policy Problem’

First, ‘diagnose the undesirable condition’, such as by

• placing your client’s initial ‘diagnosis’ in a wider perspective (e.g. what is the role of the state, and what is its capacity to intervene?), and
• providing relevant data (usually while recognising that you are not an expert in the policy problem).

Second, frame it as ‘a market or government failure (or maybe both)’, to

• show how individual or collective choices produce inefficient allocations of resources and poor outcomes (2017: 59-201 and 398-434 provides a primer on economics), and
• identify the ways in which people have addressed comparable problems in other policy areas (2017: 24).

3. ‘Be Explicit About Values’ (and goals)

Identify the values that you seek to prioritise, such as ‘efficiency’, ‘equity’, and ‘human dignity’.

Treat values as self-evident goals. They exist alongside the ‘instrumental goals’ – such as ‘sustainable public finance or political feasibility’ – necessary to generate support for policy solutions.

‘Operationalise’ those goals to help identify the likely consequences of different choices.

For example, define efficiency in relation to (a) the number of outputs per input and/or (b) a measurable or predictable gain in outcomes, such as ‘quality-adjusted life years’ in a population (2017: 25-6).

Weimer and Vining describe two analyses of efficiency at length:

• Cost Benefit Analysis (CBA) to (a) identify the most efficient outcomes by (b) translating all of the predicted impacts of an alternative into a single unit of analysis (such as a dollar amount), on the assumption (c) that we can produce winners from policy and compensate losers (see Kaldor-Hicks) (2017: 352-5 and 398-434).
• Public Agency Strategic Analysis (PASA) to identify ways in which public organisations can change to provide more benefits (such as ‘public value’) with the same resources (2017: 435-50).

4. ‘Specify Concrete Policy Alternatives’

Explain potential solutions in sufficient detail to predict the costs and benefits of each ‘alternative’ (including current policy).

Compare specific and well-worked alternatives, such as from ‘academic policy researchers’ or ‘advocacy organizations’.

Identify the potential to adopt and tailor more generic policy instruments (see 2017: 205-58 on the role of taxes, expenditure, regulation, staffing, and information-sharing; and compare with Hood and Margetts).

Engage in ‘borrowing’ proposals or models from credible sources, and ‘tinkering’ (using only the relevant elements of a proposal) to make sure they are relevant to your problem (2017: 26-7; 359).

5. ‘Predict and Value Impacts’

Ideally, you would have the time and resources to (a) produce new research and/or (b) ‘conduct a meta-analysis’ of relevant evaluations to (c) provide ‘confident assessments of impacts’ and ‘engage in highly touted evidence-based policy making’ (see EBPM).

However, ‘short deadlines’ and limited access to ‘directly relevant data’ prompt you to patch together existing research that does not answer your question directly (see 2017: 327-39; 409-11).

Consequently, ‘your predictions of the impacts of a unique policy alternative must necessarily be guided by logic and theory, rather than systematic empirical evidence’ (2017: 27) and ‘we must balance sometimes inconsistent evidence to reach conclusions about appropriate assertions’ (2017: 328).

6. ‘Consider the Trade-Offs’

It is almost inevitable that, if you compare multiple feasible alternatives, each one will fulfil certain goals more than others.

Producing, and discussing with your clients, a summary table allows you make value-based choices about trade-offs – such as between the most equitable or efficient choice – in the context of a need to manage costs and predict political feasibility (2017: 28; 356-8).

7. ‘Make a Recommendation’

‘Unless your client asks you not to do so, you should explicitly recommend one policy’ (2017: 28).

Even so, your analysis of alternatives is useful to (a) show your work (to emphasise the value of policy analysis), and (b) anticipate a change in circumstances (that affects the likely impact of each choice) or the choice by your client to draw different conclusions.

Policy analysis in a wider context: comparisons with other texts

1. Policy analysis requires flexibility and humility

As with Smith (and Bardach), note how flexible this advice must be, to reflect factors such as:

• the (unpredictable) effect that different clients and contexts have on your task
• the pressure on your limited time and resources
• the ambiguity of broad goals such as equity and human dignity
• a tendency of your clients to (a) not know, or (b) choose not to reveal their goals before you complete your analysis of possible policy solutions (2017: 347-9; compare with Lindblom)
• the need to balance many factors – (a) answering your client’s question with confidence, (b) describing levels of uncertainty and ambiguity, and (c) recognising the benefit of humility – to establish your reputation as a provider of credible and reliable analysis (2017: 341; 363; 373; 453).

2. Policy analysis as art and craft as well as science

While some proponents of EBPM may identify the need for highly specialist scientific research proficiency, Weimer and Vining (2017: 30; 34-40) describe:

• the need to supplement a ‘solid grounding’ in economics and statistics with political awareness (the ‘art and craft of policy analysis’), and
• the ‘development of a professional mind-set’ rather than perfecting ‘technical skills’ (see the policy analysis profession described by Radin).

This approach requires some knowledge of policy theories (see 1000 and 500) to appreciate the importance of factors such as networks, institutions, beliefs and motivation, framing, lurches of attention, and windows of opportunity to act (compare with ‘how far should you go?’).

Indeed, pp259-323 has useful discussions of (a) strategies including ‘co-optation’, ‘compromise’, ‘rhetoric’, Riker’s ‘heresthetics’, (b) the role of narrative in ‘writing implementation scenarios’, and (c) the complexity of mixing many policy interventions.

3. Normative and ethical requirements for policy analysis

Bacchi’s primary focus is to ask fundamental questions about what you are doing and why, and to challenge problem definitions that punish powerless populations.

In comparison, Weimer and Vining emphasise the client orientation which limits your time, freedom, and perhaps inclination to challenge so strongly.

Still, this normative role is part of an ethical duty to:

• balance a ‘responsibility to client’ with ‘analytical integrity’ and ‘adherence to one’s personal conception of the good society’, and challenge the client if they undermine professional values (2017: 43-50)
• reflect on the extent to which a policy analyst should seek to be an ‘Objective Technician’, ‘Client’s Advocate’ or ‘Issue Advocate’ (2017: 44; compare with Pielke and Jasanoff)
• recognise the highly political nature of seemingly technical processes such as cost-benefit-analysis (see 2017: 403-6 on ‘Whose Costs and Benefits Count’), and
• encourage politicians to put ‘aside their narrow personal and political interests for the greater good’ (2017: 454).