Policy Analysis in 750 words: the old page

I am redesigning the 750 words page. This is the old version.

The posts in this new series summarise key texts in policy analysis. They present the most common advice about how to ‘do’ policy analysis (to identify a policy problem and possible solutions) and situate this advice within the study of politics, power, and public policy.

This combination of ‘how to’ advice and ‘what actually happens’ research allows you to produce policy analyses and reflect on the political and pragmatic choices you need to make. Policy analysis is not a ‘rational’ or ‘technocratic’ process and we should not pretend otherwise. Rather, our aim in this series is to understand policy analysis through the lens of the policy theories that highlight:

• a competition to frame problems and identify the technical and political feasibility of solutions; in
• a policymaking environment over which no one has full understanding or control (even if elected policymakers need to project their control to boost their image of governing competence), during which
• governments add new policy solutions to an existing, complex, mix of solutions (rather than working from a blank canvas).

In that context, you may find that the summaries make more sense with reference to four main themes from the 1000 words and 500 word policy theories series (and this draft document):

1. How do you define a policy problem, and what types of solutions are available?
2. How does your chosen solution relate to existing policies?
3. Who is your audience, and what can you realistically expect them to do?
4. Who should be involved in the process of policy analysis?

I then add theme (5): Reflecting on your role as a policy analyst.

In each case, I prompt you to reflect on how (a) your knowledge of politics and policy processes, informs (b) the strategies you adopt when constructing policy analysis. The following description is a long read, but I think it provides essential context that could make the difference between effective and ineffective analysis. My MPP students can also note that this page is the same number of words as the policy analysis/ reflection exercise.

1. How do you define a policy problem, and what types of solutions are available?

The classic introduction to policy analysis is to ask what is policy? and explore the types of policy tools or instruments that may be used to produce policy change.

Modern discussions also incorporate insights from psychology to understand (a) how policymakers might combine cognition and emotion to reduce uncertainty and ambiguity, to understand problems, and therefore (b) how to communicate effectively when presenting policy analysis. This process is about the power to reduce ambiguity rather than simply the provision of information to reduce uncertainty.

In turn, problem definition influences assessments of the – technical and political -feasibility of solutions, and the ways in which actors will frame any evaluation of policy success.

      2. How does your chosen solution relate to existing policies?

Although you may propose the adoption of one (or more) policy instrument, it will likely add to many others. Governments already combine a large number of instruments to make policy, including legislation, expenditure, economic incentives and penalties, education, and various forms of service delivery. Those instruments combine to represent a complex policy mix whose overall effects are not simple to predict. This interaction provides essential context, particularly if you are asked to provide, say, a simple logic model or ‘theory of change’ to describe the likely impact of your new solution.

      3. Who is your audience, and what can you realistically expect them to do?

For example, imagine writing policy analysis in the ideal-type world of a single powerful ‘comprehensively rational’ policymaker at the heart of an orderly policy cycle. Your analysis would be relatively simple, and you would not need to worry about what happens after you make a recommendation for policy change. You could focus on widely-used tools such as cost-benefit analysis and know where the results would feed into the policy process. I have perhaps over-egged this ideal-type pudding, but I think a lot of traditional policy analysis buys into this basic idea and focuses primarily on the science of analysis rather than the political policymaking context in which it takes place.

Then imagine a far messier and less predictable world in which the nature of the policy issue is highly contested, responsibility for policy is unclear, and no single ‘centre’ has the power to turn a recommendation into an outcome. This image is a key feature of policy process theories, which describe:

• Many policymakers and influencers spread across many levels and types of government (as the venues in which authoritative choice takes place). Consequently, it is not a straightforward task to identify and know your audience, particularly if the problem you seek to solve requires a combination of policy instruments controlled by different actors.
• Each venue resembles an institution driven by formal and informal rules. Formal rules are written-down or widely-known. Informal rules are unwritten, difficult to understand, and may not even be understood in the same way by participants. Consequently, it is difficult to know if your solution will be a good fit with the standard operating procedures of organisations (and therefore if it is politically feasible or too challenging).
• Policymakers and influencers operate in ‘subsystems’, forming networks built on resources such as trust or coalitions based on shared beliefs. Effective policy analysis may require you to engage with – or become part of – such networks, to allow you to understand the unwritten rules of the game and encourage your audience to trust the messenger. In some cases, the rules relate to your willingness to accept current losses for future gains, to accept the limited impact of your analysis now in the hope of acceptance at the next opportunity.
• Actors relate their analysis to shared understandings of the world – how it is, and how it should be – which are often so well-established as to be taken for granted. Common terms include paradigms, hegemons, core beliefs, and monopolies of understandings. These dominant frames of reference give meaning to your policy solution. They prompt you to couch your solutions in terms of, for example, a strong attachment to evidence-based cases in public health, value for money in treasury departments, or with regard to core principles such as liberalism or socialism in different political systems.
• Your solutions relate to socioeconomic context and the events that seem (a) impossible to ignore and (b) out of the control of policymakers. Such factors range from a political system’s geography, demography, social attitudes, economy, while events can be routine elections or unexpected crises. To some extent, you could see yourself as a policy entrepreneur and treat events opportunistically, as ‘windows of opportunity’ to encourage new solutions. Alternatively, a general awareness of the unpredictability of events can prompt you to be modest in your claims, since the policymaking environment may be more important to outcomes than your favoured policy.

What would you recommend under these conditions?  The terms of your cost-benefit analysis would be contested (at least until there is agreement on what the problem is, and how you would measure the success of a solution). Further, even the most sophisticated ‘evidence based’ analysis of a policy problem will fall flat if uninformed by good analysis of the policy process.

Note that these problems amplify the limitations to policy analysis that are more likely to be described in this series. For example,  Weimer and Vining invest about 200 pages in economic analyses of markets and government, often highlighting a gap between (a) our ability to model and predict economic and social behaviour, and (b) what actually happens when governments intervene. To this, we should add the gap between policymakers’ formal responsibilities versus actual control of policy processes and outcomes.

      4. Who should be involved in the process of policy analysis?

Think of two visions for policy analysis. It should be primarily (1) ‘evidence based’ or (2) ‘coproduced’. While these choices are not mutually exclusive, there are key tensions between them that should not be ignored, such as when we ask:

• how many people should be involved in policy analysis?
• whose knowledge counts?
• who should control policy design?

Perhaps we can only produce a sensible combination of the two if we clarify their often very different implications for policy analysis. Let’s begin with one story for each (see also Approach 1 versus Approach 2) and see where they take us.

One story of ‘evidence based’ policy analysis is that it should be based on the best available evidence of ‘what works’. Often, the description of the ‘best’ evidence relates to the idea that there is a notional hierarchy of evidence (according to the research methods used). At the top would be the systematic review of randomised control trials, and nearer the bottom would be expertise, practitioner knowledge, and stakeholder feedback. This kind of hierarchy has major implications for policy learning and transfer, such as when importing policy interventions from abroad or ‘scaling up’ domestic projects. Put simply, the experimental method is designed to identify the causal effect of a very narrowly defined policy intervention. Its importation or scaling up would be akin to the description of medicine, in which the evidence suggests the causal effect of a specific active ingredient to be administered with the correct dosage. A very strong commitment to a uniform model precludes the processes we might associate with co-production, in which many voices contribute to a policy design to suit a specific context.

One story of ‘co-produced’ policy analysis is that it should be ‘reflexive’ and based on respectful conversations between a wide range of policymakers and citizens. Often, the description is of the diversity of valuable policy relevant information, with scientific evidence considered alongside community voices and normative values. This rejection of a hierarchy of evidence also has major implications for policy learning and transfer. Put simply, a co-production method is designed to identify the positive effect – widespread ‘ownership’ of the problem and commitment to a commonly-agreed solution – of a well-discussed intervention, often in the absence of central government control. Its use would be akin to a collaborative governance mechanism, in which the causal mechanism is perhaps the process used to foster agreement (including to produce the rules of collective action and the evaluation of success) rather than the intervention itself. A very strong commitment to this process precludes the adoption of a uniform model that we might associate with narrowly-defined stories of evidence based policymaking.

      5. Reflecting on your role as a policy analyst

If we take insights from policy theories seriously, we need to incorporate them into policy analysis, to consider policymaker psychology and policymaking context alongside the tools of policy analysis. We also need to consider the role of societal and professional values during the production of policy analysis. In other words, consider the limits to your influence and the ethics of your task.

In that context, I have begun to create a story of policy analysis archetypes to help explain this point in context:

• The pragmatic policy analyst. Bardach provides a simple, workable 8-step system to present policy analysis to policymakers while subject to time and resource-pressed political conditions (and Dunnis pragmatic for other reasons).
• The professional, client–oriented policy analyst. Weimer and Vining provide a similar 7-step client-focused system, but incorporating a greater focus on professional development and economic techniques (such as cost-benefit-analysis) to foster efficiency.

• The communicative policy analyst. Catherine Smith focuses on how to write and communicate policy analysis to clients in a political context.

• The entrepreneurial policy analyst. Mintrom shows how to incorporate insights from studies of policy entrepreneurship.
• The questioning policy analyst. Bacchi’s aim is to analyse the process in which people give and use such advice and to encourage policy analysts to question their role.
• The storytelling policy analyst. Stone’s aim is to identify the ways in which people use storytelling and argumentation techniques to define problems and justify solutions.
• The decolonizing policy analyst. Linda Tuhiwai Smith does not describe policy analysis directly, but shows how the ‘decolonization of research methods’ informs new approaches to policymaking.
• The manipulative policy analyst. A discussion of Riker helps us understand the relationship between two aspects of agenda setting: the rules/ procedures to make choice, and the framing of policy problems and solutions.

These descriptions allow you to reflect on your role, as part of a wider political or policymaking system:

1. Is your primary role to serve and communicate to clients? Is it to get what you want?
2. Are you primarily seeking to maximise your role as an individual or your part in a wider profession?
3. What is the balance between the potential benefits of individual ‘entrepreneurship’ and collective ‘co-productive’ processes?
4. Which policy analysis techniques and forms of knowledge do you prioritise?

The initial list of texts

My aim is to summarise the texts below (based initially on a module guide by Dr Raul Pacheco-Vega and any further suggestions you may have) and incorporate this analysis into the draft document How to write theory-driven policy analysis. See also Writing a Policy Paper. The reference has a weblink when a summary is available.

Eugene Bardach (2012) A Practical Guide for Policy Analysis 5^th ed. (CQ Press) (see also A step-by-step policy analysis using Bardach’s Eight Step Model)

Carol Bacchi (2009) Analysing Policy: What’s the problem represented to be? (NSW: Pearson Australia)

Eugene Bardach and Erik Patashnik (2015) A Practical Guide for Policy Analysis 5^th ed. (International Edition) (CQ Press)

Catherine Smith (2015) Writing Public Policy: A Practical Guide to Communicating in the Policy Making Process (Oxford: Oxford University Press)

David Weimer and Aidan Vining (2017) Policy Analysis: Concepts and Practice, 6^th Edition (London: Routledge)

Michael Mintrom (2012) Contemporary Policy Analysis (Oxford: Oxford University Press) (see also Contemporary Policy Analysis (Mintrom 2012))

Dunn, W. (2017) Public Policy Analysis 6^th Ed. (London: Routledge)

Geva-May, I. (2005) ‘Thinking Like a Policy Analyst. Policy Analysis as a Clinical Profession’, in Geva-May, I. (ed) Thinking Like a Policy Analyst. Policy Analysis as a Clinical Profession (Basingstoke: Palgrave) (scroll down after Radin) (see also Policy analysis as a clinical profession)

Meltzer, R. and Schwartz, A. (2019) Policy Analysis as Problem Solving (London: Routledge)

Radin, B. (2019) Policy Analysis in the Twenty-First Century (London: Routledge)

Marleen Brans, Iris Geva-May, and Michael Howlett (2017) Routledge Handbook of Comparative Policy Analysis

Thissen, W. and Walker, W. (Eds.). (2013) Public Policy Analysis (London: Springer)

Please let me know if you see some weird omissions from the list. They should give advice about policy analysis rather than describe the policy process (the latter are covered in the 1000 words and 500 word policy theories series). That said, these are some of the books that I use to widen-out the definition of policy analysis books:

William H. Riker (1986) The Art of Political Manipulation (New Haven: Yale University Press)

Linda Tuhiwai Smith (2012) Decolonizing Methodologies (London: Zed Books) (also discusses Lorde, Santos)

Barry Hindess (1977) Philosophy and Methodology in the Social Sciences

Policy Analysis in 750 words: Using Statistics and Explaining Risk (Sincerely)

Policy Analysis in 750 words: Deborah Stone (2012) Policy Paradox