Category Archives: Evidence Based Policymaking (EBPM)

Policy Analysis in 750 Words: entrepreneurial policy analysis

This post forms one part of the Policy Analysis in 750 words series overview and connects to ‘Three habits of successful policy entrepreneurs’.

The idea of a ‘policy entrepreneur’ is important to policy studies and policy analysis.

Let’s begin with its positive role in analysis, then use policy studies to help qualify its role within policymaking environments.

The take-home-messages are to

  1. recognise the value of entrepreneurship, and invest in relevant skills and strategies, but
  2. not overstate its spread or likely impact, and
  3. note the unequal access to political resources associated with entrepreneurs.

Box 11.3 UPP 2nd ed entrepreneurs

Entrepreneurship and policy analysis

Mintrom identifies the intersection between policy entrepreneurship and policy analysis, to highlight the benefits of ‘positive thinking’, creativity, deliberation, and leadership.

He expands on these ideas further in So you want to be a policy entrepreneur?:

Policy entrepreneurs are energetic actors who engage in collaborative efforts in and around government to promote policy innovations. Given the enormous challenges now facing humanity, the need is great for such actors to step forward and catalyze change processes” (Mintrom, 2019: 307).

Although many entrepreneurs seem to be exceptional people, Mintrom (2019: 308-20) identifies:

  1. Key attributes to compare
  • ‘ambition’, to invest resources for future reward
  • ‘social acuity’, to help anticipate how others are thinking
  • ‘credibility’, based on authority and a good track record
  • ‘sociability’, to empathise with others and form coalitions or networks
  • ‘tenacity’, to persevere during adversity
  1. The skills that can be learned
  • ‘strategic thinking’, to choose a goal and determine how to reach it
  • ‘team building’, to recognise that policy change is a collective effort, not the responsibility of heroic individuals (compare with Oxfam)
  • ‘collecting evidence’, and using it ‘strategically’ to frame a problem and support a solution
  • ‘making arguments’, using ‘tactical argumentation’ to ‘win others to their cause and build coalitions of supporters’ (2019: 313)
  • ‘engaging multiple audiences’, by tailoring arguments and evidence to their beliefs and interests
  • ‘negotiating’, such as by trading your support in this case for their support in another
  • ‘networking’, particularly when policymaking authority is spread across multiple venues.
  1. The strategies built on these attributes and skills.
  • ‘problem framing’, such as to tell a story of a crisis in need of urgent attention
  • ‘using and expanding networks’, to generate attention and support
  • ‘working with advocacy coalitions’, to mobilise a collection of actors who already share the same beliefs
  • ‘leading by example’, to signal commitment and allay fears about risk
  • ‘scaling up change processes’, using policy innovation in one area to inspire wider adoption.

p308 Mintrom for 750 words

Overall, entrepreneurship is ‘tough work’ requiring ‘courage’, but necessary for policy disruption, by: ‘those who desire to make a difference, who recognize the enormous challenges now facing humanity, and the need for individuals to step forward and catalyze change’ (2019: 320; compare with Luetjens).

Entrepreneurship and policy studies

  1. Most policy actors fail

It is common to relate entrepreneurship to stories of exceptional individuals and invite people to learn from their success. However, the logical conclusion is that success is exceptional and most policy actors will fail.

A focus on key skills takes us away from this reliance on exceptional actors, and ties in with other policy studies-informed advice on how to navigate policymaking environments (see ‘Three habits of successful policy entrepreneurs’, these ANZSOG talks, and box 6.3 below)

box 6.3

However, note the final sentence, which reminds us that it is possible to invest a huge amount of time and effort in entrepreneurial skills without any of that investment paying off.

  1. Even if entrepreneurs succeed, the explanation comes more from their environments than their individual skills

The other side of the entrepreneurship coin is the policymaking environment in which actors operate.

Policy studies of entrepreneurship (such as Kingdon on multiple streams) rely heavily on metaphors on evolution. Entrepreneurs are the actors most equipped to thrive within their environments (see Room).

However, Kingdon uses the additional metaphor of ‘surfers waiting for the big wave’, which suggests that their environments are far more important than them (at least when operating on a US federal scale – see Kingdon’s Multiple Streams Approach).

Entrepreneurs may be more influential at a more local scale, but the evidence of their success (independent of the conditions in which they operate) is not overwhelming. So, self-aware entrepreneurs know when to ‘surf the waves’ or try to move the sea.

  1. The social background of influential actors

Many studies of entrepreneurs highlight the stories of tenacious individuals with limited resources but the burning desire to make a difference.

The alternative story is that political resources are distributed profoundly unequally. Few people have the resources to:

  • run for elected office
  • attend elite Universities, or find other ways to develop the kinds of personal networks that often relate to social background
  • develop the credibility built on a track record in a position of authority (such as in government or science).
  • be in the position to invest resources now, to secure future gains, or
  • be in an influential position to exploit windows of opportunity.

Therefore, when focusing on entrepreneurial policy analysis, we should encourage the development of a suite of useful skills, but not expect equal access to that development or the same payoff from entrepreneurial action.

See also:

Compare these skills with the ones we might associate with ‘systems thinking

If you want to see me say these depressing things with a big grin:

2 Comments

Filed under 750 word policy analysis, agenda setting, Evidence Based Policymaking (EBPM), public policy, Uncategorized

Policy Analysis in 750 Words: complex systems and systems thinking

This post forms one part of the Policy Analysis in 750 words series overview and connects to previous posts on complexity. The first 750 words tick along nicely, then there is a picture of a cat hanging in there baby to signal where it can all go wrong. I updated it (22.6.20) to add category 11 then again (30.9.20) when I realised that the former category 11 was a lot like 6.

There are a million-and-one ways to describe systems and systems thinking. These terms are incredibly useful, but also at risk of meaning everything and therefore nothing (compare with planning and consultation).

Let’s explore how the distinction between policy studies and policy analysis can help us clarify the meaning of ‘complex systems’ and ‘systems thinking’ in policymaking.

For example, how might we close a potentially large gap between these two stories?

  1. Systems thinking in policy analysis.
  • Avoid the unintended consequences of too-narrow definitions of problems and processes (systems thinking, not simplistic thinking).
  • If we engage in systems thinking effectively, we can understand systems well enough to control, manage, or influence them.
  1. The study of complex policymaking systems.
  • Policy emerges from complex systems in the absence of: (a) central government control and often (b) policymaker awareness.
  • We need to acknowledge these limitations properly, to accept our limitations, and avoid the mechanistic language of ‘policy levers’ which exaggerate human or government control.

See also: Systems science and systems thinking for public health: a systematic review of the field

Six meanings of complex systems in policy and policymaking

Let’s begin by trying to clarify many meanings of complex system and relate them to systems thinking storylines.

For example, you will encounter three different meanings of complex system in this series alone, and each meaning presents different implications for systems thinking:

  1. A complex policymaking system

Policy outcomes seem to ‘emerge’ from policymaking systems in the absence of central government control. As such, we should rely less on central government driven targets (in favour of local discretion to adapt to environments), encourage trial-and-error learning, and rethink the ways in which we think about government ‘failure’ (see, for example, Hallsworth on ‘system stewardship’, the OECD on ‘Systemic Thinking for Policy Making‘, and this thread)

  • Systems thinking is about learning and adapting to the limits to policymaker control.

  1. Complex policy problems

Dunn (2017:  73) describes the interdependent nature of problems:

Subjectively experienced problems – crime, poverty, unemployment, inflation, energy, pollution, health, security – cannot be decomposed into independent subsets without running the risk of producing an approximately right solution to the wrong problem. A key characteristic of systems of problems is that the whole is greater – that is, qualitatively different – than the simple sum of its parts” (contrast with Meltzer and Schwartz on creating a ‘boundary’ to make problems seem solveable).

  • Systems thinking is about addressing policy problems holistically.
  1. Complex policy mixes

What we call ‘policy’ is actually a collection of policy instruments. Their overall effect is ‘non-linear’, difficult to predict, and subject to emergent outcomes, rather than cumulative (compare with Lindblom’s hopes for incrementalist change).

This point is crucial to policy analysis: does it involve a rethink of all instruments, or merely add a new instrument to the pile?

  • Systems thinking is about anticipating the disproportionate effect of a new policy instrument.

These three meanings are joined by at least three more (from Munro and Cairney on energy systems):

  1. Socio-technical systems (Geels)

Used to explain the transition from unsustainable to sustainable energy systems.

  • Systems thinking is about identifying the role of new technologies, protected initially in a ‘niche’, and fostered by a supportive ‘social and political environment’.
  1. Socio-ecological systems (Ostrom)

Used to explain how and why policy actors might cooperate to manage finite resources.

  • Systems thinking is about identifying the conditions under which actors develop layers of rules to foster trust and cooperation.
  1. Performing the metaphor of systems

Governments often use the language of complex systems – rather loosely – to indicate an awareness of the interconnectedness of things. They often perform systems thinking to give the impression that they are thinking and acting differently, but without backing up their words with tangible changes to policy instruments.

  • Systems thinking is about projecting the sense that (a) policy and policymaking is complicated, but (b) governments can still look like they are in control.

Four more meanings of systems thinking

Now, let’s compare these storylines with a small sample of wider conceptions of systems thinking:

  1. The old way of establishing order from chaos

Based on the (now-diminished) faith in science and rational management techniques to control the natural world for human benefit (compare Hughes and Hughes on energy with Checkland on ‘hard’ v ‘soft’ systems approaches, then see What you need as an analyst versus policymaking reality and Radin on the old faith in rationalist governing systems).

  • Systems thinking was about the human ability to turn potential chaos into well-managed systems (such as ‘large technical systems’ to distribute energy)
  1. The new way of accepting complexity but seeking to make an impact

Based on the idea that we can identify ‘leverage points’, or the places that help us ‘intervene in a system’ (see Meadows then compare with Arnold and Wade).

  • Systems thinking is about the human ability to use a small shift in a system to produce profound changes in that system.
  1. A way to rethink cause-and-effect

Based on the idea that current research methods are too narrowly focused on linearity rather than the emergent properties of systems of behaviour (for example, Rutter et al on how to analyse the cumulative effect of public health interventions, and Greenhalgh on responding more effectively to pandemics).

  • Systems thinking is about rethinking the ways in which governments, funders, or professions conduct policy-relevant research on social behaviour.

  1. A way of thinking about ourselves

Embrace the limits to human cognition, and accept that all understandings of complex systems are limited.

  • Systems thinking is about developing the ‘wisdom’ and ‘humility’ to accept our limited knowledge of the world.

hang-in-there-baby

How can we clarify systems thinking and use it effectively in policy analysis?

Now, imagine you are in a room of self-styled systems thinkers, and that no-one has yet suggested a brief conversation to establish what you all mean by systems thinking. I reckon you can make a quick visual distinction by seeing who looks optimistic.

I’ll be the morose-looking guy sitting in the corner, waiting to complain about ambiguity, so you would probably be better off sitting next to Luke Craven who still ‘believes in the power of systems thinking’.

If you can imagine some amalgam of these pessimistic/ optimistic positions, perhaps the conversation would go like this:

  1. Reasons to expect some useful collaboration.

Some of these 10 discussions seem to complement each other. For example:

  • We can use 3 and 9 to reject one narrow idea of ‘evidence-based policymaking’, in which the focus is on (a) using experimental methods to establish cause and effect in relation to one policy instrument, without showing (b) the overall impact on policy and outcomes (e.g. compare FNP with more general ‘families’ policy).
  • 1-3 and 10 might be about the need for policy analysts to show humility when seeking to understand and influence complex policy problems, solutions, and policymaking systems.

In other words, you could define systems thinking in relation to the need to rethink the ways in which we understand – and try to address – policy problems. If so, you can stop here and move on to the next post. There is no benefit to completing this post.

  1. Reasons to expect the same old frustrating discussions based on no-one defining terms well enough (collectively) to collaborate effectively (beyond using the same buzzwords).

Although all of these approaches use the language of complex systems and systems thinking, note some profound differences:

Holding on versus letting go.

  • Some are about intervening to take control of systems or, at least, make a disproportionate difference from a small change.
  • Some are about accepting our inability to understand, far less manage, these systems.

Talking about different systems.

  • Some are about managing policymaking systems, and others about social systems (or systems of policy problems), without making a clear connection between both endeavours.

For example, if you use approach 9 to rethink societal cause-and-effect, are you then going to pretend that you can use approach 7 to do something about it? Or, will our group have a difficult discussion about the greater likelihood of 6 (metaphorical policymaking) in the context of 1 (the inability of governments to control the policymaking systems we need to solve the problems raised by 9).

In that context, the reason that I am sitting in the corner, looking so morose, is that too much collective effort goes into (a) restating, over and over and over again, the potential benefits of systems thinking, leaving almost no time for (b) clarifying systems thinking well enough to move on to these profound differences in thinking. Systems thinking has not even helped us solve these problems with systems thinking.

See also:

Why systems thinkers and data scientists should work together to solve social challenges

5 Comments

Filed under 750 word policy analysis, Evidence Based Policymaking (EBPM), Prevention policy, public policy, UKERC

Policy Analysis in 750 Words: how much impact can you expect from your analysis?

This post forms one part of the Policy Analysis in 750 words series overview.

Throughout this series you may notice three different conceptions about the scope of policy analysis:

  1. ‘Ex ante’ (before the event) policy analysis. Focused primarily on defining a problem, and predicting the effect of solutions, to inform current choice (as described by Meltzer and Schwartz and Thissen and Walker).
  2. ‘Ex post’ (after the event) policy analysis. Focused primarily on monitoring and evaluating that choice, perhaps to inform future choice (as described famously by Weiss).
  3. Some combination of both, to treat policy analysis as a continuous (never-ending) process (as described by Dunn).

As usual, these are not hard-and-fast distinctions, but they help us clarify expectations in relation to different scenarios.

  1. The impact of old-school ex ante policy analysis

Radin provides a valuable historical discussion of policymaking with the following elements:

  • a small number of analysts, generally inside government (such as senior bureaucrats, scientific experts, and – in particular- economists),
  • giving technical or factual advice,
  • about policy formulation,
  • to policymakers at the heart of government,
  • on the assumption that policy problems would be solved via analysis and action.

This kind of image signals an expectation for high impact: policy analysts face low competition, enjoy a clearly defined and powerful audience, and their analysis is expected to feed directly into choice.

Radin goes on to describe a much different, modern policy environment: more competition, more analysts spread across and outside government, with a less obvious audience, and – even if there is a client – high uncertainty about where the analysis fits into the bigger picture.

Yet, the impetus to seek high and direct impact remains.

This combination of shifting conditions but unshifting hopes/ expectations helps explain a lot of the pragmatic forms of policy analysis you will see in this series, including:

  • Keep it catchy, gather data efficiently, tailor your solutions to your audience, and tell a good story (Bardach)
  • Speak with an audience in mind, highlight a well-defined problem and purpose, project authority, use the right form of communication, and focus on clarity, precision, conciseness, and credibility ( Smith)
  • Address your client’s question, by their chosen deadline, in a clear and concise way that they can understand (and communicate to others) quickly (Weimer and Vining)
  • Client-oriented advisors identify the beliefs of policymakers and anticipate the options worth researching (Mintrom)
  • Identify your client’s resources and motivation, such as how they seek to use your analysis, the format of analysis they favour (make it ‘concise’ and ‘digestible’), their deadline, and their ability to make or influence the policies you might suggest (Meltzer and Schwartz).
  • ‘Advise strategically’, to help a policymaker choose an effective solution within their political context (Thissen and Walker).
  • Focus on producing ‘policy-relevant knowledge’ by adapting to the evidence-demands of policymakers and rejecting a naïve attachment to ‘facts speaking for themselves’ or ‘knowledge for its own sake’ (Dunn).
  1. The impact of research and policy evaluation

Many of these recommendations are familiar to scientists and researchers, but generally in the context of far lower expectations about their likely impact, particularly if those expectations are informed by policy studies (compare Oliver & Cairney with Cairney & Oliver).

In that context, Weiss’ work is a key reference point. It gives us a menu of ways in which policymakers might use policy evaluation (and research evidence more widely):

  • to inform solutions to a problem identified by policymakers
  • as one of many sources of information used by policymakers, alongside ‘stakeholder’ advice and professional and service user experience
  • as a resource used selectively by politicians, with entrenched positions, to bolster their case
  • as a tool of government, to show it is acting (by setting up a scientific study), or to measure how well policy is working
  • as a source of ‘enlightenment’, shaping how people think over the long term (compare with this discussion of ‘evidence based policy’ versus ‘policy based evidence’).

In other words, researchers may have a role, but they struggle (a) to navigate the politics of policy analysis, (b) find the right time to act, and (c) to secure attention, in competition with many other policy actors.

  1. The potential for a form of continuous impact

Dunn suggests that the idea of ‘ex ante’ policy analysis is misleading, since policymaking is continuous, and evaluations of past choices inform current choices. Think of each policy analysis steps as ‘interdependent’, in which new knowledge to inform one step also informs the other four. For example, routine monitoring helps identify compliance with regulations, if resources and services reach ‘target groups’, if money is spent correctly, and if we can make a causal link between the policy solutions and outcomes. Its impact is often better seen as background information with intermittent impact.

Key conclusions to bear in mind

  1. The demand for information from policy analysts may be disproportionately high when policymakers pay attention to a problem, and disproportionately low when they feel that they have addressed it.
  2. Common advice for policy analysts and researchers often looks very similar: keep it concise, tailor it to your audience, make evidence ‘policy relevant’, and give advice (don’t sit on the fence). However, unless researchers are prepared to act quickly, to gather data efficiently (not comprehensively), to meet a tight brief for a client, they are not really in the impact business described by most policy analysis texts.
  3. A lot of routine, continuous, impact tends to occur out of the public spotlight, based on rules and expectations that most policy actors take for granted.

Further reading

See the Policy Analysis in 750 words series overview to continue reading on policy analysis.

See the ‘evidence-based policymaking’ page to continue reading on research impact.

ebpm pic

Bristol powerpoint: Paul Cairney Bristol EBPM January 2020

3 Comments

Filed under 750 word policy analysis, Evidence Based Policymaking (EBPM), Policy learning and transfer, public policy

Policy Analysis in 750 Words: Defining policy problems and choosing solutions

This post forms one part of the Policy Analysis in 750 words series overview.

When describing ‘the policy sciences’, Lasswell distinguishes between:

  1. ‘knowledge of the policy process’, to foster policy studies (the analysis of policy)
  2. ‘knowledge in the process’, to foster policy analysis (analysis for policy)

The idea is that both elements are analytically separable but mutually informative: policy analysis is crucial to solving real policy problems, policy studies inform the feasibility of analysis, the study of policy analysts informs policy studies, and so on.

Both elements focus on similar questions – such as What is policy? – and explore their descriptive (what do policy actors do?) and prescriptive (what should they do?) implications.

  1. What is the policy problem?

Policy studies tend to describe problem definition in relation to framing, narrative, social construction, power, and agenda setting.

Actors exercise power to generate attention for their preferred interpretation, and minimise attention to alternative frames (to help foster or undermine policy change, or translate their beliefs into policy).

Policy studies incorporate insights from psychology to understand (a) how policymakers might combine cognition and emotion to understand problems, and therefore (b) how to communicate effectively when presenting policy analysis.

Policy studies focus on the power to reduce ambiguity rather than simply the provision of information to reduce uncertainty. In other words, the power to decide whose interpretation of policy problems counts, and therefore to decide what information is policy-relevant.

This (unequal) competition takes place within a policy process over which no actor has full knowledge or control.

The classic 5-8 step policy analysis texts focus on how to define policy problems well, but they vary somewhat in their definition of doing it well (see also C.Smith):

  • Bardach recommends using rhetoric and eye-catching data to generate attention
  • Weimer and Vining and Mintrom recommend beginning with your client’s ‘diagnosis’, placing it in a wider perspective to help analyse it critically, and asking yourself how else you might define it (see also Bacchi, Stone)
  • Meltzer and Schwartz and Dunn identify additional ways to contextualise your client’s definition, such as by generating a timeline to help ‘map’ causation or using ‘problem-structuring methods’ to compare definitions and avoid making too many assumptions on a problem’s cause.
  • Thissen and Walker compare ‘rational’ and ‘argumentative’ approaches, treating problem definition as something to be measured scientifically or established rhetorically (see also Riker).

These approaches compare with more critical accounts that emphasise the role of power and politics to determine whose knowledge is relevant (L.T.Smith) and whose problem definition counts (Bacchi, Stone). Indeed, Bacchi and Stone provide a crucial bridge between policy analysis and policy studies by reflecting on what policy analysts do and why.

  1. What is the policy solution?

In policy studies, it is common to identify counterintuitive or confusing aspects of policy processes, including:

  • Few studies suggest that policy responses actually solve problems (and many highlight their potential to exacerbate them). Rather, ‘policy solutions’ is shorthand for proposed or alleged solutions.
  • Problem definition often sets the agenda for the production of ‘solutions’, but note the phrase solutions chasing problems (when actors have their ‘pet’ solutions ready, and they seek opportunities to promote them).

Policy studies: problem definition informs the feasibility and success of solutions

Generally speaking, to define the problem is to influence assessments of the feasibility of solutions:

  • Technical feasibility. Will they work as intended, given the alleged severity and cause of the problem?
  • Political feasibility. Will they receive sufficient support, given the ways in which key policy actors weigh up the costs and benefits of action?

Policy studies highlight the inextricable connection between technical and political feasibility. Put simply, (a) a ‘technocratic’ choice about the ‘optimality’ of a solution is useless without considering who will support its adoption, and (b) some types of solution will always be a hard sell, no matter their alleged effectiveness (Box 2.3 below).

In that context, policy studies ask: what types of policy tools or instruments are actually used, and how does their use contribute to policy change? Measures include the size, substance, speed, and direction of policy change.

box 2.3 2nd ed UPP

In turn, problem definition informs: the ways in which actors will frame any evaluation of policy success, and the policy-relevance of the evidence to evaluate solutions. Simple examples include:

  • If you define tobacco in relation to: (a) its economic benefits, or (b) a global public health epidemic, evaluations relate to (a) export and taxation revenues, or (b) reductions in smoking in the population.
  • If you define ‘fracking’ in relation to: (a) seeking more benefits than costs, or (b) minimising environmental damage and climate change, evaluations relate to (a) factors such as revenue and effective regulation, or simply (b) how little it takes place.

Policy analysis: recognising and pushing boundaries

Policy analysis texts tend to accommodate these insights when giving advice:

  • Bardach recommends identifying solutions that your audience might consider, perhaps providing a range of options on a notional spectrum of acceptability.
  • Smith highlights the value of ‘precedent’, or relating potential solutions to previous strategies.
  • Weimer and Vining identify the importance of ‘a professional mind-set’ that may be more important than perfecting ‘technical skills’
  • Mintrom notes that some solutions are easier to sell than others
  • Meltzer and Schwartz describe the benefits of making a preliminary recommendation to inform an iterative process, drawing feedback from clients and stakeholder groups
  • Dunn warns against too-narrow forms of ‘evidence based’ analysis which undermine a researcher’s ability to adapt well to the evidence-demands of policymakers
  • Thissen and Walker relate solution feasibility to a wide range of policy analysis ‘styles’

Still, note the difference in emphasis.

Policy analysis education/ training may be about developing the technical skills to widen definitions and apply many criteria to compare solutions.

Policy studies suggest that problem definition and a search for solutions takes place in an environment where many actors apply a much narrower lens and are not interested in debates on many possibilities (particularly if they begin with a solution).

I have exaggerated this distinction between each element, but it is worth considering the repeated interaction between them in practice: politics and policymaking provide boundaries for policy analysis, analysis could change those boundaries, and policy studies help us reflect on the impact of analysts.

I’ll take a quick break, then discuss how this conclusion relates to the idea of ‘entrepreneurial’ policy analysis.

Further reading

Understanding Public Policy (2020: 28) describes the difference between governments paying for and actually using the ‘tools of policy formulation’. To explore this point, see ‘The use and non-use of policy appraisal tools in public policy making‘ and The Tools of Policy Formulation.

p28 upp 2nd ed policy tools

3 Comments

Filed under 750 word policy analysis, agenda setting, Evidence Based Policymaking (EBPM), public policy

Policy Analysis in 750 Words: What can you realistically expect policymakers to do?

This post forms one part of the Policy Analysis in 750 words series overview.

One aim of this series is to combine insights from policy research (1000, 500) and policy analysis texts.

In this case, modern theories of the policy process help you identify your audience and their capacity to follow your advice. This simple insight may have a profound impact on the advice you give.

Policy analysis for an ideal-type world

For our purposes, an ideal-type is an abstract idea, which highlights hypothetical features of the world, to compare with ‘real world’ descriptions. It need not be an ideal to which we aspire. For example, comprehensive rationality describes the ideal type, and bounded rationality describes the ‘real world’ limitations to the ways in which humans and organisations process information.

 

Imagine writing policy analysis in the ideal-type world of a single powerful ‘comprehensively rational’ policymaker at the heart of government, making policy via an orderly policy cycle.

Your audience would be easy to identify, your analysis would be relatively simple, and you would not need to worry about what happens after you make a recommendation for policy change.

You could adopt a simple 5-8 step policy analysis method, use widely-used tools such as cost-benefit analysis to compare solutions, and know where the results would feed into the policy process.

I have perhaps over-egged this ideal-type pudding, but I think a lot of traditional policy analyses tapped into this basic idea and focused more on the science of analysis than the political and policymaking context in which it takes place (see Radin and Brans, Geva-May, and Howlett).

Policy analysis for the real world

Then imagine a far messier and less predictable world in which the nature of the policy issue is highly contestedresponsibility for policy is unclear, and no single ‘centre’ has the power to turn a recommendation into an outcome.

This image is a key feature of policy process theories, which describe:

  • Many policymakers and influencers spread across many levels and types of government (as the venues in which authoritative choice takes place). Consequently, it is not a straightforward task to identify and know your audience, particularly if the problem you seek to solve requires a combination of policy instruments controlled by different actors.
  • Each venue resembles an institution driven by formal and informal rules. Formal rules are written-down or widely-known. Informal rules are unwritten, difficult to understand, and may not even be understood in the same way by participants. Consequently, it is difficult to know if your solution will be a good fit with the standard operating procedures of organisations (and therefore if it is politically feasible or too challenging).
  • Policymakers and influencers operate in ‘subsystems’, forming networks built on resources such as trust or coalitions based on shared beliefs. Effective policy analysis may require you to engage with – or become part of – such networks, to allow you to understand the unwritten rules of the game and encourage your audience to trust the messenger. In some cases, the rules relate to your willingness to accept current losses for future gains, to accept the limited impact of your analysis now in the hope of acceptance at the next opportunity.
  • Actors relate their analysis to shared understandings of the world – how it is, and how it should be – which are often so well-established as to be taken for granted. Common terms include paradigms, hegemons, core beliefs, and monopolies of understandings. These dominant frames of reference give meaning to your policy solution. They prompt you to couch your solutions in terms of, for example, a strong attachment to evidence-based cases in public health, value for money in treasury departments, or with regard to core principles such as liberalism or socialism in different political systems.
  • Your solutions relate to socioeconomic context and the events that seem (a) impossible to ignore and (b) out of the control of policymakers. Such factors range from a political system’s geography, demography, social attitudes, and economy, while events can be routine elections or unexpected crises.

What would you recommend under these conditions? Rethinking 5-step analysis

There is a large gap between policymakers’ (a) formal responsibilities versus (b) actual control of policy processes and outcomes. Even the most sophisticated ‘evidence based’ analysis of a policy problem will fall flat if uninformed by such analyses of the policy process. Further, the terms of your cost-benefit analysis will be highly contested (at least until there is agreement on what the problem is, and how you would measure the success of a solution).

Modern policy analysis texts try to incorporate such insights from policy theories while maintaining a focus on 5-8 steps. For example:

  • Meltzer and Schwartz contrast their ‘flexible’ and ‘iterative’ approach with a too- rigid ‘rationalistic approach’.
  • Bardachand Dunn emphasise the value of political pragmatism and the ‘art and craft’ of policy analysis.
  • Weimer and Vininginvest 200 pages in economic analyses of markets and government, often highlighting a gap between (a) our ability to model and predict economic and social behaviour, and (b) what actually happens when governments intervene.
  • Mintrom invites you to see yourself as a policy entrepreneur, to highlight the value of of ‘positive thinking’, creativity, deliberation, and leadership, and perhaps seek ‘windows of opportunity’ to encourage new solutions. Alternatively, a general awareness of the unpredictability of events can prompt you to be modest in your claims, since the policymaking environment may be more important (than your solution) to outcomes.
  • Thissen and Walker focus more on a range of possible roles than a rigid 5-step process.

Beyond 5-step policy analysis

  1. Compare these pragmatic, client-orientated, and communicative models with the questioning, storytelling, and decolonizing approaches by Bacchi, Stone, and L.T. Smith.
  • The latter encourage us to examine more closely the politics of policy processes, including the importance of framing, narrative, and the social construction of target populations to problem definition and policy design.
  • Without this wider perspective, we are focusing on policy analysis as a process rather than considering the political context in which analysts use it.
  1. Additional posts on entrepreneurs and ‘systems thinking’ [to be added] encourage us to reflect on the limits to policy analysis in multi-centric policymaking systems.

 

 

2 Comments

Filed under 750 word policy analysis, agenda setting, Evidence Based Policymaking (EBPM), public policy

Policy Analysis in 750 Words: Reflecting on your role as a policy analyst

This post forms one part of the Policy Analysis in 750 words series overview.

One aim of this series is to combine insights from policy research (1000, 500) and policy analysis texts.

If we take key insights from policy theories seriously, we can use them to identify (a) the constraints to policy analytical capacity, and (b) the ways in which analysts might address them. I use the idea of policy analyst archetypes to compare a variety of possible responses.

Key constraints to policy analytical capacity

Terms like ‘bounded rationality’ highlight major limits on the ability of humans and organisations to process information.

Terms like policymaking ‘context’, ‘environments’, and multi-centric policymaking suggest that the policy process is beyond the limits of policymaker understanding and control.

  • Policy actors need to find ways to act, with incomplete information about the problem they seek to solve and the likely impact of their ‘solution’.
  • They gather information to help reduce uncertainty, but problem definition is really about exercising power to reduce ambiguity: select one way to interpret a problem (at the expense of most others), and limit therefore limit the relevance and feasibility of solutions.
  • This context informs how actors might use the tools of policy analysis. Key texts in this series highlight the use of tools to establish technical feasibility (will it work as intended?), but policymakers also select tools for their political feasibility (who will support or oppose this measure?).

box 2.3 2nd ed UPP

How might policy analysts address these constraints ethically?

Most policy analysis texts (in this series) consider the role of professional ethics and values during the production of policy analysis. However, they also point out that there is not a clearly defined profession and associated code of conduct (e.g. see Adachi). In that context, let us begin with some questions about the purpose of policy analysis and your potential role:

  1. Is your primary role to serve individual clients or some notion of the ‘public good’?
  2. Should you maximise your role as an individual or play your part in a wider profession?
  3. What is the balance between the potential benefits of individual ‘entrepreneurship’ and collective ‘co-productive’ processes?
  4. Which policy analysis techniques should you prioritise?
  5. What forms of knowledge and evidence count in policy analysis?
  6. What does it mean to communicate policy analysis responsibly?
  7. Should you provide a clear recommendation or encourage reflection?

 

Policy analysis archetypes: pragmatists, entrepreneurs, manipulators, storytellers, and decolonisers

In that context, I have created a story of policy analysis archetypes to identify the elements that each text emphasises.

The pragmatic policy analyst

  • Bardach provides the classic simple, workable, 8-step system to present policy analysis to policymakers while subject to time and resource-pressed political conditions.
  • Dunn also uses Wildavsky’s famous phrase ‘art and craft’ to suggest that scientific and ‘rational’ methods can only take us so far.

The professional, clientoriented policy analyst

  • Weimer and Vining provide a similar 7-step client-focused system, but incorporating a greater focus on professional development and economic techniques (such as cost-benefit-analysis) to emphasise a particular form of professional analyst.
  • Meltzer and Schwartz also focus on advice to clients, but with a greater emphasis on a wide variety of methods or techniques (including service design) to encourage the co-design of policy analysis with clients.

The communicative policy analyst

  •  C. Smith focuses on how to write and communicate policy analysis to clients in a political context.
  • Compare with Spiegelhalter and Gigerenzer on how to communicate responsibly when describing uncertainty, probability, and risk.

The manipulative policy analyst.

  • Riker helps us understand the relationship between two aspects of agenda setting: the rules/ procedures to make choice, and the framing of policy problems and solutions.

The entrepreneurial policy analyst

  • Mintrom shows how to combine insights from studies of policy entrepreneurship and policy analysis, to emphasise the benefits of collaboration and creativity.

The questioning policy analyst

  • Bacchi  analyses the wider context in which people give and use such advice, to identify the emancipatory role of analysis and encourage policy analysts to challenge dominant social constructions of problems and populations.

The storytelling policy analyst

  • Stone identifies the ways in which people use storytelling and argumentation techniques to define problems and justify solutions. This process is about politics and power, not objectivity and optimal solutions.

The decolonizing policy analyst.

  • L.T. Smith does not describe policy analysis directly, but shows how the ‘decolonization of research methods’ can inform the generation and use of knowledge.
  • Compare with Hindess on the ways in which knowledge-based hierarchies rely on an untenable, circular logic.
  • Compare with Michener’s thread, discussing Doucet’s new essay on (a) the role of power and knowledge in limiting (b) the ways in which we gather evidence to analyse policy problems.

Using archetypes to define the problem of policy analysis

Studies of the field (e.g. Radin plus Brans, Geva-May, and Howlett) suggest that there are many ways to do policy analysis. Further, as Thissen and Walker describe, such roles are not mutually exclusive, your views on their relative value could change throughout the process of analysis, and you could perform many of these roles.

Further, each text describes multiple roles, and some seem clustered together:

  • pragmatic, client-orientated, and communicative could sum-up the traditional 5-8 step approaches, while
  • questioning, storytelling, and decolonizing could sum up an important (‘critical’) challenge to narrow ways of thinking about policy analysis and the use of information.

Still, the emphasis matters.

Each text is setting an agenda or defining the problem of policy analysis more-or-less in relation to these roles. Put simply, the more you are reading about economic theory and method, the less you are reading about dominance and manipulation.

How can you read further?

Michener’s ‘Policy Feedback in a Racialized Polity’ connects to studies of historical institutionalism, and reminds us to use insights from policy theories to identify the context for policy analysis.

I have co-authored a lot about uncertainty/ ambiguity in relation to ‘evidence based policymaking’, including:

See also The new policy sciences for a discussion of how these issues inform Lasswell’s original vision for the policy sciences (combining the analysis of and for policy).

3 Comments

Filed under 750 word policy analysis, agenda setting, Evidence Based Policymaking (EBPM), feminism, public policy, Storytelling

Policy Analysis in 750 Words: Who should be involved in the process of policy analysis?

This post forms one part of the Policy Analysis in 750 words series overview.

Think of two visions for policy analysis. It should be primarily:

These choices are not mutually exclusive, but there are key tensions between them that should not be ignored, such as when we ask:

  • how many people should be involved in policy analysis?
  • whose knowledge counts?
  • who should control policy design?

Perhaps we can only produce a sensible combination of the two if we clarify their often very different implications for policy analysis. Let’s begin with one story for each and see where they take us.

A story of ‘evidence-based policymaking’

One story of ‘evidence based’ policy analysis is that it should be based on the best available evidence of ‘what works’.

Often, the description of the ‘best’ evidence relates to the idea that there is a notional hierarchy of evidence according to the research methods used.

At the top would be the systematic review of randomised control trials, and nearer the bottom would be expertise, practitioner knowledge, and stakeholder feedback.

This kind of hierarchy has major implications for policy learning and transfer, such as when importing policy interventions from abroad or ‘scaling up’ domestic projects.

Put simply, the experimental method is designed to identify the causal effect of a very narrowly defined policy intervention. Its importation or scaling up would be akin to the description of medicine, in which the evidence suggests the causal effect of a specific active ingredient to be administered with the correct dosage. A very strong commitment to a uniform model precludes the processes we might associate with co-production, in which many voices contribute to a policy design to suit a specific context (see also: the intersection between evidence and policy transfer).

A story of co-production in policymaking

One story of ‘co-produced’ policy analysis is that it should be ‘reflexive’ and based on respectful conversations between a wide range of policymakers and citizens.

Often, the description is of the diversity of valuable policy relevant information, with scientific evidence considered alongside community voices and normative values.

This rejection of a hierarchy of evidence also has major implications for policy learning and transfer. Put simply, a co-production method is designed to identify the positive effect – widespread ‘ownership’ of the problem and commitment to a commonly-agreed solution – of a well-discussed intervention, often in the absence of central government control.

Its use would be akin to a collaborative governance mechanism, in which the causal mechanism is perhaps the process used to foster agreement (including to produce the rules of collective action and the evaluation of success) rather than the intervention itself. A very strong commitment to this process precludes the adoption of a uniform model that we might associate with narrowly-defined stories of evidence based policymaking.

Where can you find these stories in the 750-words series?

  1. Texts focusing on policy analysis as evidence-based/ informed practice (albeit subject to limits) include: Weimer and Vining, Meltzer and Schwartz, Brans, Geva-May, and Howlett (compare with Mintrom, Dunn)
  2. Texts on being careful while gathering and analysing evidence include: Spiegelhalter
  3. Texts that challenge the ‘evidence based’ story include: Bacchi, T. Smith, Hindess, Stone

 

How can you read further?

See the EBPM page and special series ‘The politics of evidence-based policymaking: maximising the use of evidence in policy

There are 101 approaches to co-production, but let’s see if we can get away with two categories:

  1. Co-producing policy (policymakers, analysts, stakeholders). Some key principles can be found in Ostrom’s work and studies of collaborative governance.
  2. Co-producing research to help make it more policy-relevant (academics, stakeholders). See the Social Policy and Administration special issue ‘Inside Co-production’ and Oliver et al’s ‘The dark side of coproduction’ to get started.

To compare ‘epistemic’ and ‘reflexive’ forms of learning, see Dunlop and Radaelli’s ‘The lessons of policy learning: types, triggers, hindrances and pathologies

My interest has been to understand how governments juggle competing demands, such as to (a) centralise and localise policymaking, (b) encourage uniform and tailored solutions, and (c) embrace and reject a hierarchy of evidence. What could possibly go wrong when they entertain contradictory objectives? For example:

  • Paul Cairney (2019) “The myth of ‘evidence based policymaking’ in a decentred state”, forthcoming in Public Policy and Administration(Special Issue, The Decentred State) (accepted version)
  • Paul Cairney (2019) ‘The UK government’s imaginative use of evidence to make policy’, British Politics, 14, 1, 1-22 Open AccessPDF
  • Paul Cairney and Kathryn Oliver (2017) ‘Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy?’ Health Research Policy and Systems (HARPS), DOI: 10.1186/s12961-017-0192-x PDF
  • Paul Cairney (2017) “Evidence-based best practice is more political than it looks: a case study of the ‘Scottish Approach’”, Evidence and Policy, 13, 3, 499-515 PDF

 

4 Comments

Filed under 750 word policy analysis, Evidence Based Policymaking (EBPM), public policy

Policy Analysis in 750 words: Marleen Brans, Iris Geva-May, and Michael Howlett (2017) Routledge Handbook of Comparative Policy Analysis

Please see the Policy Analysis in 750 words series overview before reading the summary (and click here for the full list of authors). This post is a mere 500 words over budget (not including these words describing the number of words).

Brans et al 2017 cover

Marleen Brans, Iris Geva-May, and Michael Howlett (editors) (2017) Routledge Handbook of Comparative Policy Analysis (London: Routledge)

The Handbook … covers … the state of the art knowledge about the science, art and craft of policy analysis in different countries, at different levels of government and by all relevant actors in and outside government who contribute to the analysis of problems and the search for policy solutions’ (Brans et al, 2017: 1).

This book focuses on the interaction between (in Lasswell’s terms) ‘analysis for policy’ (policy analysis) and ‘analysis of policy’ (policy process research). In other words,

  • what can the study of policy analysis tell us about policymaking, and
  • what can studies of policymaking tell budding policy analysts about the nature of their task in relation to their policymaking environment?

Brans et al’s (2017: 1-6) opening discussion suggests that this task is rather unclear and complicated. They highlight the wide range of activity described by the term ‘policy analysis’:

  1. The scope of policy analysis is wide, and its meaning unclear

Analysts can be found in many levels and types of government, in bodies holding governments to account, and outside of government, including interest groups, think tanks, and specialist firms (such as global accountancy or management consultancy firms – Saint-Martin, 2017).

Further, ‘what counts’ as policy analysis can relate to the people that do it, the rules they follow, the processes in which they engage, the form of outputs, and the expectations of clients (Veselý, 2017: 103; Vining and Boardman, 2017: 264).

  1. The role of a policy analyst varies remarkably in relation to context

It varies over time, policy area, type of government (such as central, subnational, local), country, type of political system (e.g. majoritarian and consensus democracies), and ‘policy style’.

  1. Analysis involves ‘science, art and craft’ and the rules are written and unwritten

The process of policy analysis – such as to gather and analyse information, define problems, design and compare solutions, and give policy advice – includes ‘applied social and scientific research as well as more implicit forms of practical knowledge’, and ‘both formal and informal professional practices’ (see also studies of institutions and networks).

  1. The policy process is complex.

It is difficult to identify a straightforward process in which analysts are clearly engaged in multiple, well-defined ‘stages’ of policymaking.

  1. Key principles and practices can be institutionalised, contested, or non-existent.

The idea of policy analysis principles – ‘of transparency, effectiveness, efficiency and accountability through systematic and evidence-based analysis’ – may be entrenched in places like the US but not globally.

In some political systems (particularly in the ‘Anglo-Saxon family of nations’), the most-described forms of policy analysis (in the 750 words series) may be taken for granted (2017: 4):

Even so, the status of science and expertise is often contested, particularly in relation to salient and polarised issues, or more generally:

  • During ‘attempts by elected politicians to restore the primacy of political judgement in the policymaking process, at the expense of technical or scientific evidence’ (2017: 5).
  • When the ‘blending of expert policy analysis with public consultation and participation’ makes ‘advice more competitive and contested’ (2017: 5).
  • When evidence based really means evidence informed, given that there are many legitimate claims to knowledge, and evidence forms one part of a larger process of policy design (van Nispen and de Jong, 2017: 153).

In many political systems, there may be less criticism of the idea of ‘systematic and evidence-based analysis’ because there less capacity to process information. It is difficult to worry about excessively technocratic approaches if they do not exist (a point that CW made to me just before I read this book).

Implications for policy analysis

  1. It is difficult to think of policy analysis as a ‘profession’.

We may wonder if ‘policy analysis’ can ever be based on common skills and methods (such as described by Scott, 2017, and in Weimer and Vining), connected to ‘formal education and training’, a ‘a code of professional conduct’, and the ability of organisations to control membership (Adachi, 2017: 28; compare with Radin and Geva-May).

  1. Policy analysis is a loosely-defined collection of practices that vary according to context.

Policy analysis may, instead, be considered a collection of ‘styles’ (Hassenteufel and Zittoun, 2017), influenced by:

  • competing analytical approaches in different political systems (2017: 65)
  • bureaucratic capacity for analysis (Mendez and Dussauge-Laguna, 2017: 82)
  • a relative tendency to contract out analysis (Veselý, 2017: 113)
  • the types and remits of advisory bodies (e.g. are they tasked simply with offering expert advice, or also to encourage wider participation to generate knowledge?) (Crowley and Head, 2017)
  • the level of government in which analysts work, such as ‘subnational’ (Newman, 2017) or ‘local’ (Lundin and Öberg, 2017)
  • the type of activity, such as when (‘performance’) budgeting analysis is influenced heavily by economic methods and ‘new public management’ reforms (albeit with limited success, followed by attempts at reform) (van Nispen and de Jong, 2017: 143-52)

Policy analysis can also describe a remarkably wide range of activity, including:

  • Public inquiries (Marier, 2017)
  • Advice to MPs, parliaments, and their committees (Wolfs and De Winter, 2017)
  • The strategic analysis of public opinion or social media data (Rothmayr Allison, 2017; Kuo and Cheng, 2017)
  • A diverse set of activities associated with ‘think tanks’ (Stone and Ladi, 2017) and ‘political party think tanks’ (Pattyn et al, 2017)
  • Analysis for and by ‘business associations’ (Vining and Boardman, 2017), unions (Schulze and Schroeder, 2017), and voluntary/ non-profit organisations (Evans et al, 2017), all of whom juggle policy advice to government with keeping members on board.
  • The more-or-less policy relevant work of academic researchers (Blum and Brans, 2017; compare with Dunn and see the EBPM page).
  1. The analysis of and for policy is not so easy to separate in practice.

When defining policy analysis largely as a collection of highly-variable practices, in complex policymaking systems, we can see the symbiotic relationship between policy analysis and policy research. Studying policy analysis allows us to generate knowledge of policy processes. Policy process research demonstrates that the policymaking context influences how we think about policy analysis.

  1. Policy analysis education and training is incomplete without policy process research

Put simply, we should not assume that graduates in ‘policy analysis’ will enter a central government with high capacity, coherent expectations, and a clear demand for the same basic skills. Yet, Fukuyama argues that US University programmes largely teach students:

a battery of quantitative methods … applied econometrics, cost-benefit analysis, decision analysis, and, most recently, use of randomized experiments for program evaluation’ that ‘will tell you what the optimal policy should be’, but not ‘how to achieve that outcome. The world is littered with optimal policies that don’t have a snowball’s chance in hell of being adopted’.

In that context, additional necessary skills include: stakeholder mapping, to identify who is crucial to policy success, defining policy problems in a way that stakeholders and policymakers can support, and including those actors continuously during a process of policy design and delivery. These skills are described at more length by Radin and Geva May, while Botha et al (2017) suggest that the policy analysis programmes (across North American and European Universities) offer a more diverse range of skills (and support for experiential learning) than Fukuyama describes.

6 Comments

Filed under 750 word policy analysis, Evidence Based Policymaking (EBPM), public policy

Policy Analysis in 750 words: Wil Thissen and Warren Walker (2013) Public Policy Analysis

Thissen Walker 2013 cover

Please see the Policy Analysis in 750 words series overview before reading the summary. Please note that this is an edited book and the full list of authors (PDF) is here. I’m using the previous sentence as today’s excuse for not sticking to 750 words.

Wil Thissen and Warren Walker (editors) (2013) Public Policy Analysis: New Developments (Springer)

Our premise is that there is no single, let alone ‘one best’, way of conducting policy analyses (Thissen and Walker, 2013: 2)

Thissen and Walker (2013: 2) begin by identifying the proliferation of (a) policy analysts inside and outside government, (b) the many approaches and methods that could count as policy analysis (see Radin), and therefore (c) a proliferation of concepts to describe it.

Like Vining and Weimar, they distinguish between:

  1. Policy analysis, as the advice given to clients before they make a choice. Thissen and Walker (2013: 4) describe analysts working with a potential range of clients, when employed directly by governments or organisations, or acting more as entrepreneurs with multiple audiences in mind (compare with Bardach, Weimer & Vining, Mintrom).
  2. Policy process research, as the study of such actors within policymaking systems (see 500 and 1000).

Policy theory: implications for policy analysis

Policy process research informs our understanding of policy analysis, identifying what analysts and their clients (a) can and cannot do, which informs (b) what they should do.

As Enserink et al (2012: 12-3) describe, policy analysis (analysis for policy) will differ profoundly if the policy process is ‘chaotic and messy’ rather than ‘neat and rational’.

The range of policy concepts and theories (analysis of policy) at our disposal helps add meaning to policy analysis as a practice. Like Radin, Enserink et al trace historic attempts to seek ‘rational’ policy analysis then conclude that modern theories – describing policymaking complexity – are ‘more in line with political reality’ (2012: 13-6).

As such, policy analysis shifts from:

(a) A centralised process with few actors inside government, to (b) a messy process including many policymakers and influencers, inside and outside government

(a) Translating science into policy, to (b) a competition to frame issues and assess policy-relevant knowledge

(a) An ‘optimal’ solution from one perspective, to (b) a negotiated solution based on many perspectives (in which optimality is contested)

(a) Analysing a policy problem/ solution with a common metric (such as cost benefit analysis), to (b) developing skills relating to: stakeholder analysis, network management, collaboration, mediation or conflict resolution based on sensitivity to the role of different beliefs, and the analysis of policymaking institutions to help resolve fragmentation (2013: 17-34).

Their Table 2.1 (2012: 35) outlines these potential differences (pop your reading glasses on …. now!):

Enserink et al 2012 page 35

In many cases, the role of an analyst remains uncertain. If we follow the ACF story, does an analyst appeal to one coalition or seek to mediate between them? If we follow MSA, do they wait for a ‘window of opportunity’ or seek to influence problem definition and motivation to adopt certain solutions?

Policy Analysis: implications for policy theory

In that context, rather than identify a 5-step plan for policy analysis, Mayer et al (2013: 43-50) suggest that policy analysts tend to perform one or more of six activities:

  1. ‘Research and analyze’, to collect information relevant to policy problems.
  2. ‘Design and recommend’, to produce a range of potential solutions.
  3. ‘Clarify values and arguments’, to identify potential conflicts and facilitate high quality debate.
  4. ‘Advise strategically’, to help a policymaker choose an effective solution within their political context.
  5. ‘Democratize’, to pursue a ‘normative and ethical objective: it should further equal access to, and influence on, the policy process for all stakeholders’ (2013: 47)
  6. ‘Mediate’, to foster many forms of cooperation between governments, stakeholders (including business), researchers, and/ or citizens.

Styles of policy analysis

Policy analysts do not perform these functions sequentially or with equal weight.

Rather, Mayer et al (2013: 50-5) describe ‘six styles of policy analysis’ that vary according to the analyst’s ‘assumptions about science (epistemology), democracy, learning, and change’ (and these assumptions may change during the process):

  1. Rational, based on the idea that we can conduct research in a straightforward way within a well-ordered policy process (or modify the analysis to reflect limits to research and order).
  2. Argumentative, based on a competition to define policy problems and solutions (see Stone).
  3. Client advice, based on the assumption that analysis is part of a ‘political game’, and analysts bring knowledge of political strategy and policymaking complexity.
  4. Participatory, to facilitate a more equal access to information and debate among citizens.
  5. Process, based on the idea that the faithful adherence to good procedures aids high quality analysis (and perhaps mitigates an ‘erratic and volatile’ policy process)
  6. Interactive, based on the idea that the rehearsal of many competing perspectives is useful to policymaker deliberations (compare with reflexive learning).

In turn, these styles prompt different questions to evaluate the activities associated with analysis (2013: 56):

p56 Mayer et al

In relation to the six policy analysis activities,

  • the criteria for good policy analysis include: the quality of knowledge, usefulness of advice to clients and stakeholders, quality of argumentation, pragmatism of advice, transparency of processes, and ability to secure a mediated settlement (2013: 58).
  • The positive role for analysts includes ‘independent scientist’ or expert, ‘ethicist’, ‘narrator’, ‘counsellor’, ‘entrepreneur’,’ democratic advocate’, or ‘facilitator’ (2013: 59).

Further, their – rather complicated – visualisations of these roles (e.g. p60; compare with the Appendix) project the (useful) sense that (a) individuals face a trade-off between roles (even if they seek to combine some), and (b) many people making many trade-offs adds up to a complex picture of activity.

Therefore, we should bear in mind that

(a) there exist some useful 5-step guides for budding analyst, but

(b) even if they adopt a simple strategy, analysts will also need to find ways to understand and engage with a complex policymaking systems containing a huge number of analysts, policymakers, and influencers.

Policy Analysis styles: implications for problem definition and policy design

Thissen (2013: 66-9) extends the focus on policymaking context and policy analysis styles to problem definition, including:

  1. A rational approach relies on research knowledge to diagnose problems (the world is knowable, use the best scientific methods to produce knowledge, and subject the results to scientific scrutiny).
  2. A ‘political game model’ emphasises key actors and their perspectives, value conflicts, trust, and interdependence (assess the potential to make deals and use skills of mediation and persuasion to secure them).

These different starting points influence they ways in which analysts might take steps to identify: how people perceive policy problems, if other definitions are more useful, how to identify a problem’s cause and effect, and the likely effect of a proposed solution, communicate uncertainty, and relate the results to a ‘policy arena’ with its own rules on resolving conflict and producing policy instruments (2013: 70-84; 93-4).

Similarly, Bots (2013: 114) suggests that these styles inform a process of policy design, constructed to change people’s minds during repeated interactions with clients (such as by appealing to scientific evidence or argumentation).

Bruijn et al (2013: 134-5) situate such activities in modern discussions of policy analysis:

  1. In multi-centric systems, with analysts focused less on ‘unilateral decisions using command and control’ and more on ‘consultation and negotiation among stakeholders’ in networks.
  • The latter are necessary because there will always be contestation about what the available information tells us about the problem, often without a simple way to negotiate choices on solutions.
  1. In relation to categories of policy problems, including
  • ‘tamed’ (high knowledge/ technically solvable, with no political conflict)
  • ‘untamed ethical/political’ (technically solvable, with high moral and political conflict)
  • ‘untamed scientific’ (high consensus but low scientific knowledge)
  • ‘untamed’ problems (low consensus, low knowledge).

Put simply, ‘rational’ approaches may help address low knowledge, while other skills are required to manage processes such as conflict resolution and stakeholder engagement (2013: 136-40)

Policy Analysis styles: implications for models

Part 2 of the book relates such styles (and assumptions about how ‘rational’ and comprehensive our analyses can be) to models of policy analysis. For example,

  1. Walker and van Daalen (2013: 157-84) explore models designed to compare the status quo with a future state, often based on the (shaky) assumption that the world is knowable and we can predict with sufficient accuracy the impact of policy solutions.
  2. Hermans and Cunningham (2013: 185-213) describe models to trace agent behaviour in networks and systems, and create multiple possible scenarios, which could help explore the ‘implementability’ of policies.
  3. Walker et al (2013: 215-61) relate policy analysis styles to how analysts might deal with uncertainty.
  • Some models may serve primarily to reduce ‘epistemic’ uncertainty associated with insufficient knowledge about the future (perhaps with a focus on methods and statistical analysis).
  • Others may focus on resolving ambiguity, in which many actors may define/ interpret problems and feasible solutions in different ways.

Overall, this book contains one of the most extensive discussions of 101 different technical models for policy analysis, but the authors emphasise their lack of value without initial clarity about (a) our beliefs regarding the nature of policymaking and (b) the styles of analysis we should use to resolve policy problems. Few of these initial choices can be resolved simply with reference to scientific analysis or evidence.

9 Comments

Filed under 750 word policy analysis, Evidence Based Policymaking (EBPM), public policy, Research design

Policy Analysis in 750 words: Rachel Meltzer and Alex Schwartz (2019) Policy Analysis as Problem Solving

Please see the Policy Analysis in 750 words series overview before reading the summary. This post might well represent the largest breach of the ‘750 words’ limit, so please get comfortable. I have inserted a picture of a cat hanging in there baby after the main (*coughs*) 1400-word summary. The rest is bonus material, reflecting on the links between this book and the others in the series.

Meltzer Schwartz 2019 cover

Rachel Meltzer and Alex Schwartz (2019) Policy Analysis as Problem Solving (Routledge)

We define policy analysis as evidence-based advice giving, as the process by which one arrives at a policy recommendation to address a problem of public concern. Policy analysis almost always involves advice for a client’ (Meltzer and Schwartz, 2019: 15).

Meltzer and Schwartz (2019: 231-2) describe policy analysis as applied research, drawing on many sources of evidence, quickly, with limited time, access to scientific research, or funding to conduct a lot of new research (2019: 231-2). It requires:

  • careful analysis of a wide range of policy-relevant documents (including the ‘grey’ literature often produced by governments, NGOs, and think tanks) and available datasets
  • perhaps combined with expert interviews, focus groups, site visits, or an online survey (see 2019: 232-64 on methods).

Meltzer and Schwartz (2019: 21) outline a ‘five-step framework’ for client-oriented policy analysis. During each step, they contrast their ‘flexible’ and ‘iterative’ approach with a too- rigid ‘rationalistic approach’ (to reflect bounded, not comprehensive, rationality):

  1. ‘Define the problem’.

Problem definition is a political act of framing, not an exercise in objectivity (2019: 52-3). It is part of a narrative to evaluate the nature, cause, size, and urgency of an issue (see Stone), or perhaps to attach to an existing solution (2019: 38-40; compare with Mintrom).

In that context, ask yourself ‘Who is defining the problem? And for whom?’ and do enough research to be able to define it clearly and avoid misunderstanding among you and your client (2019: 37-8; 279-82):

  • Identify your client’s resources and motivation, such as how they seek to use your analysis, the format of analysis they favour, their deadline, and their ability to make or influence the policies you might suggest (2019: 49; compare with Weimer and Vining).
  • Tailor your narrative to your audience, albeit while recognising the need to learn from ‘multiple perspectives’ (2019: 40-5).
  • Make it ‘concise’ and ‘digestible’, not too narrowly defined, and not in a way that already closes off discussion by implying a clear cause and solution (2019: 51-2).

In doing so:

  • Ask yourself if you can generate a timeline, identify key stakeholders, and place a ‘boundary’ on the problem.
  • Establish if the problem is urgent, who cares about it, and who else might care (or not) (2019 : 46).
  • Focus on the ‘central’ problem that your solution will address, rather than the ‘related’ and ‘underlying’ problems that are ‘too large and endemic to be solved by the current analysis’ (2019: 47).
  • Avoid misdiagnosing a problem with reference to one cause. Instead, ‘map’ causation with reference to (say) individual and structural causes, intended and unintended consequences, simple and complex causation, market or government failure, and/ or the ability to blame an individual or organisation (2019: 48-9).
  • Combine quantitative and qualitative data to frame problems in relation to: severity, trends in severity, novelty, proximity to your audience, and urgency or crisis (2019: 53-4).

During this process, interrogate your own biases or assumptions and how they might affect your analysis (2019: 50).

2. ‘Identify potential policy options (alternatives) to address the problem’.

Common sources of ideas include incremental changes from current policy, ‘client suggestions’, comparable solutions (from another time, place, or policy area), reference to common policy instruments, and ‘brainstorming’ or ‘design thinking’ (2019: 67-9; see box 2.3 and 7.1, below, from Understanding Public Policy).

box 2.3 2nd ed UPP

Identify a ‘wide range’ of possible solutions, then select the (usually 3-5) ‘most promising’ for further analysis (2019: 65). In doing so:

  • be careful not to frame alternatives negatively (e.g. ‘death tax’ – 2019: 66)
  • compare alternatives in ‘good faith’ rather than keeping some ‘off the table’ to ensure that your preferred solution looks good (2019: 66)
  • beware ‘ best practice’ ideas that are limited in terms of (a) applicability (if made at a smaller scale, or in a very different jurisdiction), and (b) evidence of success (2019: 70; see studies of policy learning and transfer)
  • think about how to modify existing policies according to scale or geographical coverage, who to include (and based on what criteria), for how long, using voluntary versus mandatory provisions, and ensuring oversight (2019: 71-3)
  • consider combinations of common policy instruments, such as regulations and economic penalties/ subsidies (2019: 73-7)
  • consider established ways to ‘brainstorm’ ideas (2019: 77-8)
  • note the rise of instruments derived from the study of psychology and behavioural public policy (2019: 79-90)
  • learn from design principles, including ‘empathy’, ‘co-creating’ policy with service users or people affected, ‘prototyping’ (2019: 90-1)

box 7.1

3. ‘Specify the objectives to be attained in addressing the problem and the criteria to  evaluate  the  attainment  of  these  objectives  as  well as  the  satisfaction  of  other  key  considerations  (e.g.,  equity,  cost, equity, feasibility)’.

Your objectives relate to your problem definition and aims: what is the problem, what do you want to happen when you address it, and why?

  • For example, questions to your client may include: what is your organization’s ‘mission’, what is feasible (in terms of resources and politics), which stakeholders to you want to include, and how will you define success (2019: 105; 108-12)?

In that values-based context, your criteria relate to ways to evaluate each policy’s likely impact (2019: 106-7). They should ensure:

  • Comprehensiveness. E.g. how many people, and how much of their behaviour, can you influence while minimizing the ‘burden’ on people, businesses, or government? (2019: 113-4)
  • Mutual Exclusiveness. In other words, don’t have two objectives doing the same thing (2019: 114).

Common criteria include (2019: 116):

  1. Effectiveness. The size of its intended impact on the problem (2019: 117).
  2. Equity (fairness). The impact in terms of ‘vertical equity’ (e.g. the better off should pay more), ‘horizontal equity’ (e.g. you should not pay more if unmarried), fair process, fair outcomes, and ‘intergenerational’ equity (e.g. don’t impose higher costs on future populations) (2019: 118-19).
  3. Feasibility (administrative, political, and technical). The likelihood of this policy being adopted and implemented well (2019: 119-21)
  4. Cost (or financial feasibility). Who would bear the cost, and their willingness and ability to pay (2019: 122).
  5. Efficiency. To maximise the benefit while minimizing costs (2019: 122-3).

 

4. ‘Assess the outcomes of the policy options in light of the criteria and weigh trade-offs between the advantages and disadvantages of the options’.

When explaining objectives and criteria,

  • ‘label’ your criteria in relation to your policy objectives (e.g. to ‘maximize debt reduction’) rather than using generic terms (2019: 123-7)
  • produce a table – with alternatives in rows, and criteria in columns – to compare each option
  • quantify your policies’ likely outcomes, such as in relation to numbers of people affected and levels of income transfer, or a percentage drop in the size of the problem, but also
  • communicate the degree of uncertainty related to your estimates (2019: 128-32; see Spiegelhalter)

Consider using cost-benefit analysis to identify (a) the financial and opportunity cost of your plans (what would you achieve if you spent the money elsewhere?), compared to (b) the positive impact of your funded policy (2019: 141-55).

  • The principle of CBA may be intuitive, but a thorough CBA process is resource-intensive, vulnerable to bias and error, and no substitute for choice. It requires you to make a collection of assumptions about human behaviour and likely costs and benefits, decide whose costs and benefits should count, turn all costs and benefits into a single measure, and imagine how to maximise winners and compensate losers (2019: 155-81; compare Weimer and Vining with Stone).
  • One alternative is cost-effectiveness analysis, which quantifies costs and relates them to outputs (e.g. number of people affected, and how) without trying to translate them into a single measure of benefit (2019: 181-3).
  • These measures can be combined with other thought processes, such as with reference to ‘moral imperatives’, a ‘precautionary approach’, and ethical questions on power/ powerlessness (2019: 183-4).

 

5. ‘Arrive at a recommendation’.

Predict the most likely outcomes of each alternative, while recognising high uncertainty (2019: 189-92). If possible,

  • draw on existing, comparable, programmes to predict the effectiveness of yours (2019: 192-4)
  • combine such analysis with relevant theories to predict human behaviour (e.g. consider price ‘elasticity’ if you seek to raise the price of a good to discourage its use) (2019: 193-4)
  • apply statistical methods to calculate the probability of each outcome (2019: 195-6), and modify your assumptions to produce a range of possibilities, but
  • note Spiegelhalter’s cautionary tales and anticipate the inevitable ‘unintended consequences’ (when people do not respond to policy in the way you would like) (2019: 201-2)
  • use these estimates to inform a discussion on your criteria (equity, efficiency, feasibility) (2019: 196-200)
  • present the results visually – such as in a ‘matrix’ – to encourage debate on the trade-offs between options
  • simplify choices by omitting irrelevant criteria and options that do not compete well with others (2019: 203-10)
  • make sure that your recommendation (a) flows from the analysis, and (b) is in the form expected by your client (2019: 211-12)
  • consider making a preliminary recommendation to inform an iterative process, drawing feedback from clients and stakeholder groups (2019: 212).

 

hang-in-there-baby

 

Policy analysis in a wider context

Meltzer and Schwartz’s approach makes extra sense if you have already read some of the other texts in the series, including:

  1. Weimer and Vining, which represents an exemplar of an X-step approach informed heavily by the study of economics and application of economic models such as cost-benefit-analysis (compare with Radin’s checklist).
  2. Geva-May on the existence of a policy analysis profession with common skills, heuristics, and (perhaps) ethics (compare with Meltzer and Schwartz, 2019: 282-93)
  3. Radin, on:
  • the proliferation of analysts across multiple levels of government, NGOs, and the private sector (compare with Meltzer and Schwartz, 2019: 269-77)
  • the historic shift of analysis from formulation to all notional stages (contrast with Meltzer and Schwartz, 2019: 16-7 on policy analysis not including implementation or evaluation)
  • the difficulty in distinguishing between policy analysis and advocacy in practice (compare with Meltzer and Schwartz, 2019: 276-8, who suggest that actors can choose to perform these different roles)
  • the emerging sense that it is difficult to identify a single client in a multi-centric policymaking system. Put another way, we might be working for a specific client but accept that their individual influence is low.
  1. Stone’s challenge to
  • a historic tendency for economics to dominate policy analysis,
  • the applicability of economic assumptions (focusing primarily on individualist behaviour and markets), and
  • the pervasiveness of ‘rationalist’ policy analysis built on X-steps.

Meltzer and Schwartz (2019: 1-3) agree that economic models are too dominant (identifying the value of insights from ‘other disciplines – including design, psychology, political science, and sociology’).

However, they argue that critiques of rational models exaggerate their limitations (2019: 23-6). For example:

  • these models need not rely solely on economic techniques or quantification, a narrow discussion or definition of the problem, or the sense that policy analysis should be comprehensive, and
  • it is not problematic for analyses to reflect their client’s values or for analysts to present ambiguous solutions to maintain wide support, partly because
  • we would expect the policy analysis to form only one part of a client’s information or strategy.

Further, they suggest that these critiques provide no useful alternative, to help guide new policy analysts. Yet, these guides are essential:

to be persuasive, and credible, analysts must situate the problem, defend their evaluative criteria, and be able to demonstrate that their policy recommendation is superior, on balance, to other alternative options in addressing the problem, as defined by the analyst. At a minimum, the analyst needs to present a clear and defensible ranking of options to guide the decisions of the policy makers’ (Meltzer and Schwartz, 2019: 4).

Meltzer and Schwartz (2019: 27-8) then explore ways to improve a 5-step model with insights from approaches such as ‘design thinking’, in which actors use a similar process – ‘empathize, define the problem, ideate, prototype, test and get feedback from others’ – to experiment with policy solutions without providing a narrow view on problem definition or how to evaluate responses.

Policy analysis and policy theory

One benefit to Meltzer and Schwartz’s approach is that it seeks to incorporate insights from policy theories and respond with pragmatism and hope. However, I think you also need to read the source material to get a better sense of those theories, key debates, and their implications. For example:

  1. Meltzer and Schwartz (2019: 32) note correctly that ‘incremental’ does not sum up policy change well. Indeed, Punctuated Equilibrium Theory shows that policy change is characterised by a huge number of small and a small number of huge changes.
  • However, the direct implications of PET are not as clear as they suggest. Baumgartner and Jones have both noted that they can measure these outcomes and identify the same basic distribution across a political system, but not explain or predict why particular policies change dramatically.
  • It is useful to recommend to policy analysts that they invest some hope in major policy change, but also sensible to note that – in the vast majority of cases – it does not happen.
  • On his point, see Mintrom on policy analysis for the long term, Weiss on the ‘enlightenment’ function of research and analysis, and Box 6.3 (from Understanding Public Policy), on the sense that (a) we can give advice to ‘budding policy entrepreneurs’ on how to be effective analysts, but (b) should note that all their efforts could be for nothing.

box 6.3

  1. Meltzer and Schwartz (2019: 32-3) tap briefly into the old debate on whether it is preferable to seek radical or incremental change. For more on that debate, see chapter 5 in the 1st ed of Understanding Public Policy in which Lindblom notes that proposals for radical/ incremental changes are not mutually exclusive.
  2. Perhaps explore the possible tension between Meltzer and Schwartz’s (2019: 33-4) recommendation that (a) policy analysis should be ‘evidence-based advice giving’, and (b) ‘flexible and open-ended’.
  • I think that Stone’s response would be that phrases such as ‘evidence based’ are not ‘flexible and open-ended’. Rather, they tend to symbolise a narrow view of what counts as evidence (see also Smith, and Hindess).
  • Further, note that the phrase ‘evidence based policymaking’ is a remarkably vague term (see the EBPM page), perhaps better seen as a political slogan than a useful description or prescription of policymaking.

 

Finally, if you read enough of these policy analysis texts, you get a sense that many are bunched together even if they describe their approach as new or distinctive.

  • Indeed, Meltzer and Schwarz (2019: 22-3) provide a table (containing Bardach and Patashnik, Patton et al, Stokey and Zeckhauser, Hammond et al, and Weimer & Vining) of ‘quite similar’ X-step approaches.
  • Weimer and Vining also discuss the implications of policy theories and present the sense that X-step policy analysis should be flexible and adaptive.
  • Many texts – including Radin, and Smith (2016) – focus on the value of case studies to think through policy analysis in particular contexts, rather than suggesting that we can produce a universal blueprint.

However, as Geva-May might suggest, this is not a bad thing if our aim is to generate the sense that policy analysis is a profession with its own practices and heuristics.

 

 

15 Comments

Filed under 750 word policy analysis, agenda setting, Evidence Based Policymaking (EBPM), public policy

Policy Analysis in 750 words: Deborah Stone (2012) Policy Paradox

Please see the Policy Analysis in 750 words series overview before reading the summary. This post is 750 words plus a bonus 750 words plus some further reading that doesn’t count in the word count even though it does.

Stone policy paradox 3rd ed cover

Deborah Stone (2012) Policy Paradox: The Art of Political Decision Making 3rd edition (Norton)

‘Whether you are a policy analyst, a policy researcher, a policy advocate, a policy maker, or an engaged citizen, my hope for Policy Paradox is that it helps you to go beyond your job description and the tasks you are given – to think hard about your own core values, to deliberate with others, and to make the world a better place’ (Stone, 2012: 15)

Stone (2012: 379-85) rejects the image of policy analysis as a ‘rationalist’ project, driven by scientific and technical rules, and separable from politics. Rather, every policy analyst’s choice is a political choice – to define a problem and solution, and in doing so choosing how to categorise people and behaviour – backed by strategic persuasion and storytelling.

The Policy Paradox: people entertain multiple, contradictory, beliefs and aims

Stone (2012: 2-3) describes the ways in which policy actors compete to define policy problems and public policy responses. The ‘paradox’ is that it is possible to define the same policies in contradictory ways.

‘Paradoxes are nothing but trouble. They violate the most elementary principle of logic: something can’t be two different things at once. Two contradictory interpretations can’t both be true. A paradox is just such an impossible situation, and political life is full of them’ (Stone, 2012: 2).

This paradox does not refer simply to a competition between different actors to define policy problems and the success or failure of solutions. Rather:

  • The same actor can entertain very different ways to understand problems, and can juggle many criteria to decide that a policy outcome was a success and a failure (2012: 3).
  • Surveys of the same population can report contradictory views – encouraging a specific policy response and its complete opposite – when asked different questions in the same poll (2012: 4; compare with Riker)

Policy analysts: you don’t solve the Policy Paradox with a ‘rationality project’

Like many posts in this series (Smith, Bacchi, Hindess), Stone (2010: 9-11) rejects the misguided notion of objective scientists using scientific methods to produce one correct answer (compare with Spiegelhalter and Weimer & Vining). A policy paradox cannot be solved by ‘rational, analytical, and scientific methods’ because:

Further, Stone (2012: 10-11) rejects the over-reliance, in policy analysis, on the misleading claim that:

  • policymakers are engaging primarily with markets rather than communities (see 2012: 35 on the comparison between a ‘market model’ and ‘polis model’),
  • economic models can sum up political life, and
  • cost-benefit-analysis can reduce a complex problem into the sum of individual preferences using a single unambiguous measure.

Rather, many factors undermine such simplicity:

  1. People do not simply act in their own individual interest. Nor can they rank-order their preferences in a straightforward manner according to their values and self-interest.
  • Instead, they maintain a contradictory mix of objectives, which can change according to context and their way of thinking – combining cognition and emotion – when processing information (2012: 12; 30-4).
  1. People are social actors. Politics is characterised by ‘a model of community where individuals live in a dense web of relationships, dependencies, and loyalties’ and exercise power with reference to ideas as much as material interests (2012: 10; 20-36; compare with Ostrom, more Ostrom, and Lubell; and see Sousa on contestation).
  2. Morals and emotions matter. If people juggle contradictory aims and measures of success, then a story infused with ‘metaphor and analogy’, and appealing to values and emotions, prompts people ‘to see a situation as one thing rather than another’ and therefore draw attention to one aim at the expense of the others (2012: 11; compare with Gigerenzer).

Policy analysis reconsidered: the ambiguity of values and policy goals

Stone (2012: 14) identifies the ambiguity of the criteria for success used in 5-step policy analyses. They do not form part of a solely technical or apolitical process to identify trade-offs between well-defined goals (compare Bardach, Weimer and Vining, and Mintrom). Rather, ‘behind every policy issue lurks a contest over conflicting, though equally plausible, conceptions of the same abstract goal or value’ (2012: 14). Examples of competing interpretations of valence issues include definitions of:

  1. Equity, according to: (a) which groups should be included, how to assess merit, how to identify key social groups, if we should rank populations within social groups, how to define need and account for different people placing different values on a good or service, (b) which method of distribution to use (competition, lottery, election), and (c) how to balance individual, communal, and state-based interventions (2012: 39-62).
  2. Efficiency, to use the least resources to produce the same objective, according to: (a) who determines the main goal and how to balance multiple objectives, (a) who benefits from such actions, and (c) how to define resources while balancing equity and efficiency – for example, does a public sector job and a social security payment represent a sunk cost to the state or a social investment in people? (2012: 63-84).
  3. Welfare or Need, according to factors including (a) the material and symbolic value of goods, (b) short term support versus a long term investment in people, (c) measures of absolute poverty or relative inequality, and (d) debates on ‘moral hazard’ or the effect of social security on individual motivation (2012: 85-106)
  4. Liberty, according to (a) a general balancing of freedom from coercion and freedom from the harm caused by others, (b) debates on individual and state responsibilities, and (c) decisions on whose behaviour to change to reduce harm to what populations (2012: 107-28)
  5. Security, according to (a) our ability to measure risk scientifically (see Spiegelhalter and Gigerenzer), (b) perceptions of threat and experiences of harm, (c) debates on how much risk to safety to tolerate before intervening, (d) who to target and imprison, and (e) the effect of surveillance on perceptions of democracy (2012: 129-53).

Policy analysis as storytelling for collective action

Actors use policy-relevant stories to influence the ways in which their audience understands (a) the nature of policy problems and feasibility of solutions, within (b) a wider context of policymaking in which people contest the proper balance between state, community, and market action. Stories can influence key aspects of collective action, including:

  1. Defining interests and mobilising actors, by drawing attention to – and framing – issues with reference to an imagined social group and its competition (e.g. the people versus the elite; the strivers versus the skivers) (2012: 229-47)
  2. Making decisions, by framing problems and solutions (2012: 248-68). Stone (2012: 260) contrasts the ‘rational-analytic model’ with real-world processes in which actors deliberately frame issues ambiguously, shift goals, keep feasible solutions off the agenda, and manipulate analyses to make their preferred solution seem the most efficient and popular.
  3. Defining the role and intended impact of policies, such as when balancing punishments versus incentives to change behaviour, or individual versus collective behaviour (2012: 271-88).
  4. Setting and enforcing rules (see institutions), in a complex policymaking system where a multiplicity of rules interact to produce uncertain outcomes, and a powerful narrative can draw attention to the need to enforce some rules at the expense of others (2012: 289-310).
  5. Persuasion, drawing on reason, facts, and indoctrination. Stone (2012: 311-30) highlights the context in which actors construct stories to persuade: people engage emotionally with information, people take certain situations for granted even though they produce unequal outcomes, facts are socially constructed, and there is unequal access to resources – held in particular by government and business – to gather and disseminate evidence.
  6. Defining human and legal rights, when (a) there are multiple, ambiguous, and intersecting rights (in relation to their source, enforcement, and the populations they serve) (b) actors compete to make sure that theirs are enforced, (c) inevitably at the expense of others, because the enforcement of rights requires a disproportionate share of limited resources (such as policymaker attention and court time) (2012: 331-53)
  7. Influencing debate on the powers of each potential policymaking venue – in relation to factors including (a) the legitimate role of the state in market, community, family, and individual life, (b) how to select leaders, (c) the distribution of power between levels and types of government – and who to hold to account for policy outcomes (2012: 354-77).

Key elements of storytelling include:

  1. Symbols, which sum up an issue or an action in a single picture or word (2012:157-8)
  2. Characters, such as heroes or villain, who symbolise the cause of a problem or source of solution (2012:159)
  3. Narrative arcs, such as a battle by your hero to overcome adversity (2012:160-8)
  4. Synecdoche, to highlight one example of an alleged problem to sum up its whole (2012: 168-71; compare the ‘welfare queen’ example with SCPD)
  5. Metaphor, to create an association between a problem and something relatable, such as a virus or disease, a natural occurrence (e.g. earthquake), something broken, something about to burst if overburdened, or war (2012: 171-78; e.g. is crime a virus or a beast?)
  6. Ambiguity, to give people different reasons to support the same thing (2012: 178-82)
  7. Using numbers to tell a story, based on political choices about how to: categorise people and practices, select the measures to use, interpret the figures to evaluate or predict the results, project the sense that complex problems can be reduced to numbers, and assign authority to the counters (2012:183-205; compare with Speigelhalter)
  8. Assigning Causation, in relation to categories including accidental or natural, ‘mechanical’ or automatic (or in relation to institutions or systems), and human-guided causes that have intended or unintended consequences (such as malicious intent versus recklessness)
  • ‘Causal strategies’ include to: emphasise a natural versus human cause, relate it to ‘bad apples’ rather than systemic failure, and suggest that the problem was too complex to anticipate or influence
  • Actors use these arguments to influence rules, assign blame, identify ‘fixers’, and generate alliances among victims or potential supporters of change (2012: 206-28).

Wider Context and Further Reading: 1. Policy analysis

This post connects to several other 750 Words posts, which suggest that facts don’t speak for themselves. Rather, effective analysis requires you to ‘tell your story’, in a concise way, tailored to your audience.

For example, consider two ways to establish cause and effect in policy analysis:

One is to conduct and review multiple randomised control trials.

Another is to use a story of a hero or a villain (perhaps to mobilise actors in an advocacy coalition).

  1. Evidence-based policymaking

Stone (2012: 10) argues that analysts who try to impose one worldview on policymaking will find that ‘politics looks messy, foolish, erratic, and inexplicable’. For analysts, who are more open-minded, politics opens up possibilities for creativity and cooperation (2012: 10).

This point is directly applicable to the ‘politics of evidence based policymaking’. A common question to arise from this worldview is ‘why don’t policymakers listen to my evidence?’ and one answer is ‘you are asking the wrong question’.

  1. Policy theories highlight the value of stories (to policy analysts and academics)

Policy problems and solutions necessarily involve ambiguity:

  1. There are many ways to interpret problems, and we resolve such ambiguity by exercising power to attract attention to one way to frame a policy problem at the expense of others (in other words, not with reference to one superior way to establish knowledge).
  1. Policy is actually a collection of – often contradictory – policy instruments and institutions, interacting in complex systems or environments, to produce unclear messages and outcomes. As such, what we call ‘public policy’ (for the sake of simplicity) is subject to interpretation and manipulation as it is made and delivered, and we struggle to conceptualise and measure policy change. Indeed, it makes more sense to describe competing narratives of policy change.

box 13.1 2nd ed UPP

  1. Policy theories and storytelling

People communicate meaning via stories. Stories help us turn (a) a complex world, which provides a potentially overwhelming amount of information, into (b) something manageable, by identifying its most relevant elements and guiding action (compare with Gigerenzer on heuristics).

The Narrative Policy Framework identifies the storytelling strategies of actors seeking to exploit other actors’ cognitive shortcuts, using a particular format – containing the setting, characters, plot, and moral – to focus on some beliefs over others, and reinforce someone’s beliefs enough to encourage them to act.

Compare with Tuckett and Nicolic on the stories that people tell to themselves.

 

 

14 Comments

Filed under 750 word policy analysis, Evidence Based Policymaking (EBPM), Psychology Based Policy Studies, public policy, Storytelling

Policy Analysis in 750 words: Linda Tuhiwai Smith (2012) Decolonizing Methodologies

Please see the  Policy Analysis in 750 words series overview before reading the summary. The reference to 750 words is increasingly misleading.

Linda Tuhiwai Smith (2012) Decolonizing Methodologies 2nd edition (London: Zed Books)

 ‘Whose research is it? Who owns it? Whose interests does it serve? Who will benefit from it? Who has designed its questions and framed its cope? Who will carry it out? Who will write it up? How will its results be disseminated?’ (Smith, 2012: 10; see also 174-7)

Many texts in this series highlight the politics of policy analysis, but few (such as Bacchi) identify the politics of the research that underpins policy analysis.

You can find some discussion of these issues in the brief section on ‘co-production’, in wider studies of co-produced research and policy, and ‘evidence based policymaking’, and in posts on power and knowledge and feminist institutionalism. However, the implications rarely feed into standard policy analysis texts. This omission is important, because the production of knowledge – and the exercise of power to define whose knowledge counts – is as political as it gets.

Smith (2012) demonstrates this point initially by identifying multiple, often hidden, aspects of politics and power that relate to ‘research’ and ‘indigenous peoples’:

 

  1. The term ‘indigenous peoples’ is contested, and its meaning-in-use can range from
  • positive self-identification, to highlight common international experiences and struggles for self-determination but distinctive traditions; other terms include ‘First Nations’ in Canada or, in New Zealand, ‘Maori’ as opposed to ‘Pakeha’ (the colonizing population) (2012: 6)
  • negative external-identification, including – in some cases – equating ‘indigenous’ (or similar terms) with ‘dirtiness, savagery, rebellion and, since 9/11, terrorism’ (2012: xi-xii).

 

  1. From the perspective of ‘the colonized’, “the term ‘research’ is inextricably linked to European imperialism and colonialism” (2012: 1; 21-6). Western research practices (and the European ‘Enlightenment’) reflect and reinforce political practices associated with colonial rule (2012: 2; 23).

To the colonized, the ways in which academic research has been implicated in the throes of imperialism remains a painful memory’ (2012: back cover).

“The word itself, ‘research’, is probably one of the dirtiest words in the indigenous world’s vocabulary” (2012: xi).

 

  1. People in indigenous communities describe researchers who exploit ‘their culture, their knowledge, their resources’ (and, in some cases, their bodies) to bolster their own income, career or profession (2012: xi; 91-4; 102-7), in the context of a long history of subjugation and slavery that makes such practices possible (2012: 21-6; 28-9; 176-7), and “justified as being for ‘the good of mankind’” (2012: 26).

 

 

  1. Western researchers think – hubristically – that they can produce a general understanding of the practices and cultures of indigenous peoples (e.g. using anthropological methods). Instead, they produce – irresponsibly or maliciously – negative and often dehumanizing images that feed into policies ‘employed to deny the validity of indigenous peoples’ claim to existence’ and solve the ‘indigenous problem’ (2012: 1; 8-9; 26-9; 62-5; 71-2; 81-91; 94-6).

For example, research contributes to a tendency for governments to

  • highlight, within indigenous communities, indicators of inequality (in relation to factors such as health, education, crime, and family life), and relate it to
  • indigenous cultures and low intelligence, rather than
  • the ways in which colonial legacy and current policy contributes to poverty and marginalisation (2012: 4; 12; compare with Social Construction and Policy Design).

 

  1. Western researchers’ views on how to produce high-quality scientific evidence lead them to ‘see indigenous peoples, their values and practices as political hindrances that get in the way of good research’ (2012: xi; 66-71; compare with ‘hierarchy of evidence’). Similarly, the combination of a state’s formal laws and unwritten rules and assumptions can serve to dismiss indigenous community knowledge as not meeting evidential standards (2012: 44-9).

 

  1. Many indigenous researchers need to negotiate the practices and expectations of different groups, such as if they are portrayed as:
  • ‘insiders’ in relation to an indigenous community (and, for example, expected by that community to recognise the problems with Western research traditions)
  • ‘outsiders’, by (a) an indigenous community in relation to their ‘Western education’ (2012: 5), or (b) by a colonizing state commissioning insider research
  • less technically proficient or less likely to maintain confidentiality than a ‘non-indigenous researcher’ (2012: 12)

Can policy analysis be informed by a new research agenda?

In that context, Smith (2012: xiii; 111-25) outlines a new agenda built on the recognition that research is political and connected explicitly to political and policy aims (2012: xiii; compare with Feminism, Postcolonialism, and Critical Policy Studies)

At its heart is a commitment to indigenous community ‘self-determination’, ‘survival’, ‘recovery’, and ‘development’, aided by processes such as social movement mobilization and decolonization (2012: 121). This agenda informs the meaning of ethical conduct, signalling that research:

  • serves explicit political goals and requires researchers to reflect on their role as activists in an emancipatory project, in contrast to the disingenuous argument that science or scientists are objective (2012: 138-42; 166-77; 187-8; 193-5; 198-215; 217-26)
  • is not ‘something done only by white researchers to indigenous peoples’ (2012: 122),
  • is not framed so narrowly, in relation to specific methods or training, that it excludes (by definition) most indigenous researchers, community involvement in research design, and methods such as storytelling (2012: 127-38; 141; for examples of methods, see 144-63; 170-1)
  • requires distinctive methods and practices to produce knowledge, reinforced by mutual support during the nurturing of such practices
  • requires a code of respectful conduct that extends ‘beyond issues of individual consent and confidentiality’) (2012: 124; 179-81).

Wider context: informing the ‘steps’ to policy analysis

This project informs directly the ‘steps’ to policy analysis described in Bardach, Weimer and Vining, and Mintrom, including:

Problem definition

Mintrom describes the moral and practical value of engaging with stakeholders to help frame policy problems and design solutions (as part of a similarly-worded aim to transform and improve the world).

However, Smith (2012: 228-32; 13) describes such a profound gulf, in the framing of problems, that cannot be bridged simply via consultation or half-hearted ‘co-production’ exercises.

For example, if a government policy analyst relates poor health to individual and cultural factors in indigenous communities, and people in those communities relate it to colonization, land confiscation, minimal self-determination, and an excessive focus on individuals, what could we realistically expect from set-piece government-led stakeholder analyses built on research that has already set the policy agenda (compare with Bacchi)?

Rather, Smith (2012: 15-16) describes the need, within research practices, for continuous awareness of, and respect for, a community’s ‘cultural protocols, values and behaviours’ as part of ‘an ethical and respectful approach’. Indeed, the latter could have mutual benefits which underpin the long-term development of trust: a community may feel less marginalised by the analysis-to-policy process, and future analysts may be viewed with less suspicion.

Even so, a more respectful policy process is not the same as accepting that some communities may benefit more from writing about their own experiences than contributing to someone else’s story. Writing about the past, present, and future is an exercise of power to provide a dominant perspective with which to represent people and problems (2012: 29-41; 52-9)

Analysing and comparing solutions

Imagine a cost-benefit analysis designed to identify the most efficient outcomes by translating all of the predicted impacts on people into a single unit of analysis (such as a dollar amount, or quality-adjusted-life-years). Assumptions include that we can: (a) assign the same value to a notionally similar experience, and (b) produce winners from policy and compensate losers.

Yet, this calculation hinges on the power to decide how we should understand such experiences and place relative values on outcomes, and to take a calculation of their value to one population and generalise it to others. Smith’s analysis suggests that such processes will not produce outcomes that we can describe honestly as societal improvements. Rather, they feed into a choice to produce winners from policy and fail to compensate losers in an adequate or appropriate manner.

See also:

  1. In relation to policy theories

This post – Policy Concepts in 1000 Words: Feminism, Postcolonialism, and Critical Policy Studies – provides a tentative introduction to the ways in which many important approaches can inform policy theories, such as by

The 2nd edition of Understanding Public Policy summarises these themes as follows:

p49 2nd ed UPPp50 2nd ed UPP

  1. In relation to policy analysis

If you look back to the Policy Analysis in 750 words series overview, you will see that a popular way to address policy issues is through the ‘coproduction’ of research and policy, perhaps based on a sincere commitment to widen a definition of useful knowledge/ ways of thinking and avoid simply making policy from the ‘centre’ or ‘top down’.

Yet, the post you are now reading, summarising Decolonizing Methodologies, should prompt us to question the extent to which a process could be described sincerely as ‘coproduction’ if there is such an imbalance of power and incongruence of ideas between participants.

Although many key texts do not discuss ‘policy analysis’ directly, they provide ways to reflect imaginatively on this problem. I hope that I am not distorting their original messages, but please note that the following are my stylized interpretations of key texts.

Audre Lorde (2018*) The Master’s Tools Will Never Dismantle the Master’s House (Penguin) (*written from 1978-82)

Lorde Masters Tools

One issue with very quick client-oriented policy analysis is that it encourages analysts to (a) work with an already-chosen definition of the policy problem, and (b) use well-worn methods to collect information, including (c) engaging with ideas and people with whom they are already familiar.

Some forms of research and policy analysis may be more conducive to challenging existing frames and encouraging wider stakeholder engagement. Still, compare this mild shift from the status quo with a series of issues and possibilities identified by Lorde (2018):

  • Some people are so marginalised and dismissed that they struggle to communicate – about the ways in which they are oppressed, and how they might contribute to imagining a better world – in ways that would be valued (or even noticed) during stakeholder consultation (2018: 1-5 ‘Poetry is not a luxury’).
  • The ‘european-american male tradition’ only allows for narrowly defined (‘rational’) means of communication (2018: 6-15 ‘Uses of the Erotic’)

A forum can be designed ostensibly to foster communication and inclusivity, only to actually produce the opposite, by signalling to some participants that

  • they are a token afterthought, whose views and experiences are – at best – only relevant to a very limited aspect of a wide discussion, and
  • their differences will be feared, not celebrated, becoming a source of conflict, not mutual nurture or cooperation.

It puts marginalised people in the position of having to work hard simply to be heard. They learn that powerful people are only willing to listen if others do the work for them, because (a) they are ignorant of experiences other than their own, and/or (b) they profess ignorance strategically to suck the energy from people whose views they fear and do not understand. No one should feel immune from such criticism even if they profess to be acting with good intentions (2018: 16-21 ‘The Master’s Tools Will Never Dismantle the Master’s House’).

  • The correct response to racism is anger. Therefore, do not prioritise (a) narrow rules of civility, or the sensibilities of the privileged, if (b) your aim is to encourage conversations with people who are trying to express the ways in which they deal with overwhelming and continuous hatred, violence, and oppression (2018: 22-35, ‘Uses of Anger: Women Responding to Racism’)

Boaventura de Sousa Santos (2014) Epistemologies of the South: Justice Against Epistemicide (Routledge)

Sousa cover

Imagine global policy processes and policy analysis, in which some countries and international organisations negotiate agreements, influenced (or not) by critical social movements in pursuit of social justice. Santos (2014) identifies a series of obstacles including:

  • A tendency for Western (as part of the Global North) ways of thinking to dominate analysis, at the expense of insights from the Global South (2014: viii), producing
  • A tendency for ‘Western centric’ ideas to inform the sense that some concepts and collective aims – such as human dignity and human rights – can be understood universally, rather than through the lens of struggles that are specific to some regions (2014: 21; 38)
  • A lack of imagination or willingness to imagine different futures and conceptions of social justice (2014: 24)

Consequently, actors may come together to discuss major policy change on ostensibly the same terms, only for some groups to – intentionally and unintentionally – dominate thought and action and reinforce the global inequalities they propose to reduce.

Sarah Ahmed (2017) Living a Feminist Life (Duke University Press)

Ahmed cover.jpg

Why might your potential allies in ‘coproduction’ be suspicious of your motives, or sceptical about the likely outcomes of such an exchange? One theme throughout Smith’s (2012) book is that people often co-opt key terms (such as ‘decolonizing’) to perform the sense that they care about social change, to try to look like they are doing something important, while actually designing ineffective or bad faith processes to protect the status of themselves or their own institution or profession.

Ahmed (2017: 103) describes comparable initiatives – such as to foster ‘equality and diversity’ – as a public relations exercise for organisations, rather than a sincere desire to do the work. Consequently, there is a gap ‘between a symbolic commitment and a lived reality’ (2017: 90). Indeed, the aim may be to project a sense of transformation to hinder that transformation (2017: 90), coupled with a tendency to use a ‘safe’ and non-confrontational language (‘diversity’) to project the sense that we can only push people so far, at the expense of terms such as ‘racism’ that would signal challenge, confrontation, and a commitment to high impact (2017: chapter 4).

..

Putting these insights together suggests that a stated commitment to co-produced research and policy might begin with good intentions. Even so, a commitment to sincere engagement does not guarantee an audience or prevent you from exacerbating the very problems you profess to solve.

12 Comments

Filed under 750 word policy analysis, Evidence Based Policymaking (EBPM), public policy, Research design, Storytelling

Policy Analysis in 750 words: Michael Mintrom (2012) Contemporary Policy Analysis

Please see the Policy Analysis in 750 words series overview before reading the summary. This summary is not 750 words. I can only apologise.

Michael Mintrom (2012) Contemporary Policy Analysis (Oxford University Press)

Mintrom (2012: xxii; 17) describes policy analysis as ‘an enterprise primarily motivated by the desire to generate high quality information to support high-quality decisions’ and stop policymakers ‘from making ill-considered choices’ (2012: 17). It is about giving issues more ‘serious attention and deep thought’ than busy policymakers, rather than simply ‘an exercise in the application of techniques’ to serve clients (2012: 20; xxii).

It begins with six ‘Key Steps in Policy Analysis’ (2012: 3-5):

  1. ‘Engage in problem definition’

Problem definition influences the types of solutions that will be discussed (although, in some cases, solutions chase problems).

Define the nature and size of a policy problem, and the role of government in solving it (from maximal to minimal), while engaging with many stakeholders with different views (2012: 3; 58-60).

This task involves a juggling act. First, analysts should engage with their audience to work out what they need and when (2012 : 81). However, second, they should (a) develop ‘critical abilities’, (b) ask themselves ‘why they have been presented in specific ways, what their sources might be, and why they have arisen at this time’, and (c) present ‘alternative scenarios’ (2012: 22; 20; 27).

  1. ‘Propose alternative responses to the problem’

Governments use policy instruments – such as to influence markets, tax or subsidize activity, regulate behaviour, provide services (directly, or via commissioning or partnership), or provide information – as part of a coherent strategy or collection of uncoordinated measures (2012: 30-41). In that context, try to:

  • Generate knowledge about how governments have addressed comparable problems (including, the choice to not intervene if an industry self-regulates).
  • Identify the cause of a previous policy’s impact and if it would have the same effect now (2012: 21).
  • If minimal comparable information is available, consider wider issues from which to learn (2012: 76-7; e.g. alcohol policy based on tobacco).

Consider the wider:

 

  1. ‘Choose criteria for evaluating each alternative policy response’

There are no natural criteria, but ‘effectiveness, efficiency, fairness, and administrative efficiency’ are common (2012: 21). ‘Effective institutions’ have a marked impact on social and economic life and provide political stability (2012: 49). Governments can promote ‘efficient’ policies by (a) producing the largest number of winners and (b) compensating losers (2012: 51-2; see Weimer and Vining on Kaldor-Hicks). They can prioritise environmental ‘sustainability’ to mitigate climate change, the protection of human rights and ‘human flourishing’, and/or a fair allocation of resources (2012: 52-7).

  1. ‘Project the outcomes of pursuing each policy alternative’

Estimate the costs of a new policy, in comparison with current policy, and in relation to factors such as (a) overall savings to society, and/or (b) benefits to certain populations (any policy will benefit some social groups more than others). Mintrom (2012: 21) emphasises ‘prior knowledge and experience’ and ‘synthesizing’ work by others alongside techniques such as cost-benefit analyses.

  1. ‘Identify and analyse trade-offs among alternatives’

Use your criteria and projections to compare each alternative in relation to their likely costs and benefits.

  1. ‘Report findings and make an argument for the most appropriate response’

Mintrom (2012: 5) describes a range of advisory roles.

(a) Client-oriented advisors identify the beliefs of policymakers and anticipate the options worth researching (although they should not simply tell clients what they want to hear – 2012: 22). They may only have the time to answer a client’s question quickly and on their own. Or, they need to create and manage a team project (2012: 63-76).

(b) Other actors, ‘who want to change the world’, research options that are often not politically feasible in the short term but are too important to ignore (such as gender mainstreaming or action to address climate change).

In either case, the format of a written report – executive summary, contents, background, analytical strategy, analysis and findings (perhaps including a table comparing goals and trade-offs between alternatives), discussion, recommendation, conclusion, annex – may be similar (2012: 82-6).

Wider context: the changing role of policy analysts

Mintrom (2012: 5-7) describes a narrative – often attributed to Radin – of the changing nature of policy analysis, comparing:

  1. (a) a small group of policy advisors, (b) with a privileged place in government, (c) giving allegedly technical advice, using economic techniques such as cost-benefit analysis.
  2. (a) a much larger profession, (b) spread across – and outside of – government (including external consultants), and (c) engaging more explicitly in the politics of policy analysis and advice.

It reflects wider changes in government, (a) from the ‘clubby’ days to a much more competitive environment debating a larger number and wider range of policy issues, subject to (b) factors such as globalisation that change the task/ context of policy analysis.

If so, any advice on how to do policy analysis has to be flexible, to incorporate the greater diversity of actors and the sense that complex policymaking systems require flexible skills and practices rather than standardised techniques and outputs.

The ethics of policy analysis

In that context, Mintrom (2012: 95-108) emphasises the enduring role for ethical policy analysis, which can relate to:

  1. ‘Universal’ principles such as fairness, compassion, and respect
  2. Specific principles to project the analyst’s integrity, competence, responsibility, respectfulness, and concern for others
  3. Professional practices, such as to
  • engage with many stakeholders in problem definition (to reflect a diversity of knowledge and views)
  • present a range of feasible solutions, making clear their distributional effects on target populations, opportunity costs (what policies/ outcomes would not be funded if this were), and impact on those who implement policy
  • be honest about (a) the method of calculation, and (b) uncertainty, when projecting outcomes
  • clarify the trade-offs between alternatives (don’t stack-up the evidence for one)
  • maximise effective information sharing, rather than exploiting the limited attention of your audience (compare with Riker).
  1. New analytical strategies (2012: 114-15; 246-84)
  1. the extent to which social groups are already ‘systematically disadvantaged’,
  2. the causes (such as racism and sexism) of – and potential solutions to – these outcomes, to make sure
  3. that new policies reduce or do not perpetuate disadvantages, even when
  4. politicians may gain electorally from scapegoating target populations and/ or
  5. there are major obstacles to transformative policy change.

Therefore, while Mintrom’s (2012: 3-5; 116) ‘Key Steps in Policy Analysis’ are comparable to Bardach and Weimer and Vining, his emphasis is often closer to Bacchi’s.

The entrepreneurial policy analyst

Mintrom (2012: 307-13) ends with a discussion of the intersection between policy entrepreneurship and analysis, highlighting the benefits of ‘positive thinking’, creativity, deliberation, and leadership. He expands on these ideas further in So you want to be a policy entrepreneur?

15 Comments

Filed under 750 word policy analysis, agenda setting, Evidence Based Policymaking (EBPM), public policy

Policy Analysis in 750 words: William Riker (1986) The Art of Political Manipulation

Please see the Policy Analysis in 750 words series overview before reading the summary.

William H. Riker (1986) The Art of Political Manipulation (New Haven: Yale University Press)

Most texts in this series describe the politics of policy analysis, in which your aim is to communicate with a client to help them get what they want, subject to professional standards and ethics (Smith, Bardach, and Weimer and Vining).

Such texts suggest that the evidence will not speak for itself, and that your framing of information could make a big difference between success and failure. However, they tend to dance around the question of how to exercise power to maximise your success.

The consequence may be some bland Aristotle-style advice, in which you should seek to be a persuasive narrator by combining:

  • Pathos. The appeal to an audience’s emotions to maximise interest in a problem.
  • Logos. The concise presentation of information and logic to make a persuasive case.
  • Ethos. The credibility of the presenter, based on their experience and expertise.

Studies of narrative suggest that these techniques have some impact. Narrators tap into their audience’s emotions and beliefs, make a problem seem ‘concrete’ and urgent, and romanticise a heroic figure or cause. However, their success depends heavily on the context, and stories tend to be most influential of the audiences predisposed to accept them.

If so, a key option is to exploit a tendency for people to possess many contradictory beliefs, which suggests that (a) they could support many different goals or policy solutions, and (b) their support may relate strongly to the context and rules that determine the order and manner in which they make choices.

In other words, you may not be able to ‘change their minds’, but you can encourage them to pay more attention to, and place more value on, one belief (or one way to understand a policy problem) at the expense of another. This strategy could make the difference between belief and action.

Riker (1986: ix) uses the term ‘heresthetic’ to describe ‘structuring the world so you can win’. People ‘win politically because they have set up the situation in such a way that other people will want to join them’. Examples include:

  1. Designing the order in which people make choices, because many policy preferences are ‘intransitive’: if A is preferred to B and B to C, A is not necessarily preferred to C.
  2. Exploiting the ways in which people deal with ‘bounded rationality’ (the limits to their ability to process information to make choices).

For example, what if people are ‘cognitive misers’, seeking to process information efficiently rather than comprehensively? What if they combine cognition and emotion to make choices efficiently? Riker highlights the potential value of some combination of the following strategies:

  1. Make your preferred problem framing or solution as easy to understand as possible.
  2. Make other problems/ solutions difficult to process, such as by presenting them in the abstract and providing excessive detail.
  3. Emphasize the high cognitive cost to the examination of all other options.
  4. Experiment with choice-rule options that consolidate the vote for your preferred option while splitting the vote of others.
  5. Design the comparison of a small number of options to make sure that yours is the most competitive.
  6. Design the framing of choice (for example, is a vote primarily about the substantial issue or confidence in its proponents?).
  7. Design the selection of criteria to evaluate options.
  8. Design a series of votes, in sequence, to allow you to trade votes with others.
  9. Conspire to make sure that the proponent of your preferred choice is seen as heroic (and the proponent of another choice as of flawed character and intellect).
  10. Ensure that people make or vote for choices quickly, to ward off the possibility of further analysis and risk of losing control of the design of choice.
  11. Make sure that you engage in these strategies without being detected or punished.

The point of this discussion is not to recommend that policy analysts become Machiavellian manipulators, fixing their eye on the prize, and doing anything to win.

Rather, it is to highlight the wider agenda setting context that you face when presenting evidence, values, and options.

It is a truism in policy studies that the evidence does not speak for itself. Instead, people engage in effective communication and persuasion to assign meaning to the evidence.

Similarly, it would be a mistake to expect success primarily from a well written and argued policy analysis document. Rather, much of its fate depends on who is exploiting the procedures and rules that influence how people make choices.

See also:

Evidence-based policymaking: political strategies for scientists living in the real world

Three habits of successful policy entrepreneurs

Evidence-informed policymaking: context is everything

Please note: some of this text comes from Box 4.3 in Understanding Public Policy 2nd ed

box 4.3 Riker topbox 4.3 Riker bottom

 

6 Comments

Filed under 750 word policy analysis, agenda setting, Evidence Based Policymaking (EBPM), public policy

Policy Analysis (usually) in 750 words: David Weimer and Adrian Vining (2017) Policy Analysis

Please see the Policy Analysis in 750 words series overview before reading the summary.

Please note that this book is the longest in the series (almost 500 pages), so a 750 word summary would have been too heroic.

David Weimer and Adrian Vining (2017) Policy Analysis: Concepts and Practice 6th Edition (Routledge)

Weimer and Vining (2017: 23-8; 342-75) describe policy analysis in seven steps:

  1. ‘Write to Your Client’

Having a client such as an elected policymaker (or governmental or nongovernmental organization) requires you to: address the question they ask, by their chosen deadline, in a clear and concise way that they can understand (and communicate to others) quickly (2017: 23; 370-4).

Their sample documents are 18 pages, including an executive summary and summary table.

  1. ‘Understand the Policy Problem’

First, ‘diagnose the undesirable condition’, such as by

  • placing your client’s initial ‘diagnosis’ in a wider perspective (e.g. what is the role of the state, and what is its capacity to intervene?), and
  • providing relevant data (usually while recognising that you are not an expert in the policy problem).

Second, frame it as ‘a market or government failure (or maybe both)’, to

  • show how individual or collective choices produce inefficient allocations of resources and poor outcomes (2017: 59-201 and 398-434 provides a primer on economics), and
  • identify the ways in which people have addressed comparable problems in other policy areas (2017: 24).
  1. ‘Be Explicit About Values’ (and goals)

Identify the values that you seek to prioritise, such as ‘efficiency’, ‘equity’, and ‘human dignity’.

Treat values as self-evident goals. They exist alongside the ‘instrumental goals’ – such as ‘sustainable public finance or political feasibility’ – necessary to generate support for policy solutions.

‘Operationalise’ those goals to help identify the likely consequences of different choices.

For example, define efficiency in relation to (a) the number of outputs per input and/or (b) a measurable or predictable gain in outcomes, such as ‘quality-adjusted life years’ in a population (2017: 25-6).

Weimer and Vining describe two analyses of efficiency at length:

  • Cost Benefit Analysis (CBA) to (a) identify the most efficient outcomes by (b) translating all of the predicted impacts of an alternative into a single unit of analysis (such as a dollar amount), on the assumption (c) that we can produce winners from policy and compensate losers (see Kaldor-Hicks) (2017: 352-5 and 398-434).
  • Public Agency Strategic Analysis (PASA) to identify ways in which public organisations can change to provide more benefits (such as ‘public value’) with the same resources (2017: 435-50).
  1. ‘Specify Concrete Policy Alternatives’

Explain potential solutions in sufficient detail to predict the costs and benefits of each ‘alternative’ (including current policy).

Compare specific and well-worked alternatives, such as from ‘academic policy researchers’ or ‘advocacy organizations’.

Identify the potential to adopt and tailor more generic policy instruments (see 2017: 205-58 on the role of taxes, expenditure, regulation, staffing, and information-sharing; and compare with Hood and Margetts).

Engage in ‘borrowing’ proposals or models from credible sources, and ‘tinkering’ (using only the relevant elements of a proposal) to make sure they are relevant to your problem (2017: 26-7; 359).

  1. ‘Predict and Value Impacts’

Ideally, you would have the time and resources to (a) produce new research and/or (b) ‘conduct a meta-analysis’ of relevant evaluations to (c) provide ‘confident assessments of impacts’ and ‘engage in highly touted evidence-based policy making’ (see EBPM).

However, ‘short deadlines’ and limited access to ‘directly relevant data’ prompt you to patch together existing research that does not answer your question directly (see 2017: 327-39; 409-11).

Consequently, ‘your predictions of the impacts of a unique policy alternative must necessarily be guided by logic and theory, rather than systematic empirical evidence’ (2017: 27) and ‘we must balance sometimes inconsistent evidence to reach conclusions about appropriate assertions’ (2017: 328).

  1. ‘Consider the Trade-Offs’

It is almost inevitable that, if you compare multiple feasible alternatives, each one will fulfil certain goals more than others.

Producing, and discussing with your clients, a summary table allows you make value-based choices about trade-offs – such as between the most equitable or efficient choice – in the context of a need to manage costs and predict political feasibility (2017: 28; 356-8).

  1. ‘Make a Recommendation’

‘Unless your client asks you not to do so, you should explicitly recommend one policy’ (2017: 28).

Even so, your analysis of alternatives is useful to (a) show your work (to emphasise the value of policy analysis), and (b) anticipate a change in circumstances (that affects the likely impact of each choice) or the choice by your client to draw different conclusions.

Policy analysis in a wider context: comparisons with other texts

  1. Policy analysis requires flexibility and humility

As with Smith (and Bardach), note how flexible this advice must be, to reflect factors such as:

  • the (unpredictable) effect that different clients and contexts have on your task
  • the pressure on your limited time and resources
  • the ambiguity of broad goals such as equity and human dignity
  • a tendency of your clients to (a) not know, or (b) choose not to reveal their goals before you complete your analysis of possible policy solutions (2017: 347-9; compare with Lindblom)
  • the need to balance many factors – (a) answering your client’s question with confidence, (b) describing levels of uncertainty and ambiguity, and (c) recognising the benefit of humility – to establish your reputation as a provider of credible and reliable analysis (2017: 341; 363; 373; 453).
  1. Policy analysis as art and craft as well as science

While some proponents of EBPM may identify the need for highly specialist scientific research proficiency, Weimer and Vining (2017: 30; 34-40) describe:

  • the need to supplement a ‘solid grounding’ in economics and statistics with political awareness (the ‘art and craft of policy analysis’), and
  • the ‘development of a professional mind-set’ rather than perfecting ‘technical skills’ (see the policy analysis profession described by Radin).

This approach requires some knowledge of policy theories (see 1000 and 500) to appreciate the importance of factors such as networks, institutions, beliefs and motivation, framing, lurches of attention, and windows of opportunity to act (compare with ‘how far should you go?’).

Indeed, pp259-323 has useful discussions of (a) strategies including ‘co-optation’, ‘compromise’, ‘rhetoric’, Riker’s ‘heresthetics’, (b) the role of narrative in ‘writing implementation scenarios’, and (c) the complexity of mixing many policy interventions.

  1. Normative and ethical requirements for policy analysis

Bacchi’s primary focus is to ask fundamental questions about what you are doing and why, and to challenge problem definitions that punish powerless populations.

In comparison, Weimer and Vining emphasise the client orientation which limits your time, freedom, and perhaps inclination to challenge so strongly.

Still, this normative role is part of an ethical duty to:

  • balance a ‘responsibility to client’ with ‘analytical integrity’ and ‘adherence to one’s personal conception of the good society’, and challenge the client if they undermine professional values (2017: 43-50)
  • reflect on the extent to which a policy analyst should seek to be an ‘Objective Technician’, ‘Client’s Advocate’ or ‘Issue Advocate’ (2017: 44; compare with Pielke and Jasanoff)
  • recognise the highly political nature of seemingly technical processes such as cost-benefit-analysis (see 2017: 403-6 on ‘Whose Costs and Benefits Count’), and
  • encourage politicians to put ‘aside their narrow personal and political interests for the greater good’ (2017: 454).

 

20 Comments

Filed under 750 word policy analysis, Evidence Based Policymaking (EBPM), public policy

Prevenir es mejor que curar, entonces, ¿por qué no hacemos más?

Serie: El proceso de las políticas públicas.

Paul Cairney, Profesor de Política y Políticas Públicas en la Universidad de Stirling, Escocia. Enlace a texto original en inglés.

Esta publicación proporciona una amplia cantidad de antecedentes de mi plática en la Escuela de Gobierno de Australia y Nueva Zelanda (ANZSOG, por sus siglas en inglés), la cual se titula “Prevenir es mejor que curar, entonces, ¿por qué no hacemos más?” [en inglés] Si lo lees todo, es una lectura larga. Si no, es una lectura corta antes de la lectura larga. Aquí está la descripción de la plática:

“¿Te suena familiar? Comienza una nueva administración en el gobierno, la cual promete cambiar el equilibrio en las políticas sociales y de salud, – de costosos remedios y atención de alta dependencia o complejidad, a prevención e intervención temprana-. Se comprometen a una mejor formulación de políticas públicas; y dicen que la entrega de políticas y programas se hará de forma coordinada, delegando responsabilidades a nivel local y enfocándose en resultados a largo plazo en lugar de soluciones a corto plazo; y que garantizarán que la política se base en evidencia. Y luego todo se vuelve demasiado difícil y el ciclo comienza nuevamente, dejando a su paso algunos especialistas exhaustos y desilusionados. ¿Por qué sucede esto repetidamente, en diferentes países y con gobiernos de diferentes doctrinas, incluso con la mejor voluntad del mundo?

  • De acuerdo con la pregunta verás que no estoy sugiriendo que todas las políticas públicas de prevención o intervención temprana fallen. Por el contrario, utilizo teorías de políticas públicas para proporcionar una explicación general de la brecha significativa entre las expectativas (realistas) expresadas en las estrategias de prevención y los resultados reales. Luego se puede discutir sobre cómo disminuir esa brecha.
  • También verás la frase “incluso con la mejor voluntad del mundo”, que considero clave para esta plática. Nadie necesita que yo ensaye las formas comunes y generalmente vagas de explicar las políticas de prevención fallidas, incluida la “intratabilidad” [en inglés] de los problemas de políticas públicas o la “patología” [en inglés] de las mismas. Más bien, demuestro que tales políticas públicas pueden “fracasar” incluso cuando existe un acuerdo franco y amplio entre las partes sobre la necesidad de pasar del diseño de políticas reactivas a más preventivas. También sugiero que la explicación general del fracaso (baja “voluntad política”) a menudo es perjudicial para posibilidades de éxito en el futuro.
  • Comencemos por definir la política pública de prevención y la formulación de políticas públicas.

Cuando los gobiernos se involucran en la “prevención”, buscan:

  1. Reformar las políticas públicas

La política pública de prevención es realmente un conjunto de políticas diseñadas para intervenir lo antes posible en la vida de las personas para mejorar su bienestar y reducir las desigualdades o la demanda de servicios agudos. El objetivo es pasar de los servicios públicos reactivos a los preventivos, interviniendo de manera temprana en la vida de las personas para abordar una amplia gama de problemas de largo alcance, incluidos el crimen y el comportamiento antisocial, la mala salud y los comportamientos no saludables, el bajo nivel educativo, el desempleo y la baja empleabilidad, antes de que se vuelvan demasiado severos.

  1. Reformar la formulación de la política pública

La formulación de políticas públicas preventivas describe las formas en que los gobiernos reforman sus prácticas para apoyar las políticas de prevención, incluido el compromiso de:

  • “Unir” a departamentos y servicios gubernamentales para resolver “problemas intratables” que trascienden áreas.
  • Producir objetivos a largo plazo para obtener mayores resultados a través de otorgar mayor responsabilidad en el diseño del servicio a los organismos públicos locales, las partes interesadas, las “comunidades” y los usuarios del servicio
  • Reducir los objetivos a corto plazo en favor de resultados a largo plazo.
  1. Asegurar que la política pública es “basada en evidencia”

Tres razones generales por las cuales las políticas públicas de “prevención” nunca parecen tener éxito.

  1. Los formuladores de política pública no saben el significado de la prevención

Expresan un compromiso con la prevención antes de definirla completamente. Cuando comienzan a dar sentido a la prevención, descubren lo difícil que es perseguirla y las elecciones controvertidas que esto implica (ver también incertidumbre versus ambigüedad)

  1. Se involucran en un sistema de formulación de políticas públicas que es demasiado complejo para controlarse

Intentan compartir la responsabilidad entre varios actores y coordinan acciones para direccionar los resultados de las políticas públicas. Sin embargo, no poseen la capacidad de diseñar dichas relaciones y controlar los resultados de las políticas públicas.

Sin embargo, también deben demostrarle al electorado que tienen el control y descubrir lo difícil que es localizar y centralizar las políticas públicas.

  1. No pueden y no quieren producir la “formulación de la política pública basada en evidencia”

Los formuladores buscan atajos cognitivos (y sus equivalentes organizacionales) para recopilar suficiente información para tomar decisiones “suficientemente buenas”. Cuando buscan evidencia sobre la prevención, descubren que es irregular, poco concluyente y a menudo contraria a sus creencias, y no una “bala mágica” para ayudar a justificar las elecciones.

A lo largo de este proceso, su compromiso con la política pública de prevención puede ser sincero, pero no se materializa. No articulan completamente lo que significa prevención ni aprecian la dimensión de dicha tarea. Cuando intentan ofrecer estrategias de prevención, se enfrentan a varios problemas que por sí solos parecerían desalentadores. Muchos de los problemas que tratan de “prevenir” son “intratables” o difíciles de definir y aparentemente imposibles de resolver, como la pobreza, el desempleo, las viviendas de baja calidad y la falta de ellas, el crimen y las desigualdades en salud y educación. Se enfrentan a elecciones difíciles sobre cuán lejos deberían llegar para cambiar el equilibrio entre el Estado y el mercado, redistribuir la riqueza y los ingresos, distribuir recursos públicos e intervenir en la vida de las personas para cambiar su comportamiento y sus formas de pensar. Su enfoque en el largo plazo se enfrenta a una gran competencia por problemas de políticas públicas cortoplacistas más destacados que los impulsan a mantener servicios públicos “reactivos”. Su deseo puro de “localizar” la formulación de políticas, a menudo cede el paso a la política electoral nacional, en la que los gobiernos centrales se enfrentan a la presión para formular políticas públicas desde “arriba” y ser decisivos. Su búsqueda de políticas “basadas en evidencia” a menudo revela una falta de evidencia sobre qué intervenciones políticas funcionan y la medida en que se pueden “expandir” con éxito.

Un mal diagnostico por parte de los encargados de la formulación de la política pública y actores influyentes hará que los problemas no se resuelvan

  • Si los actores con poder en las políticas públicas hacen la suposición simplista de que un problema es causado por cuestiones que no son vitales para el Estado, darán malos consejos.
  • Si los nuevos formuladores realmente piensan que el problema fue la falta de compromiso y la competencia de sus predecesores, comenzarán con las mismas esperanzas sobre el impacto que pueden tener, solo para desencantarse cuando vean la diferencia entre sus objetivos abstractos y los resultados del mundo real.
  • La mala explicación del éxito limitado contribuye en gran medida a observar (a) un período inicial de entusiasmo y actividad, reemplazado por (b) desencanto e inactividad, y (c) la repetición de este ciclo.

Agreguemos más detalles a estas explicaciones generales:

  1. ¿Qué hace que la prevención sea tan difícil de definir?

Cuando se ve como un eslogan simple, “prevención” parece un objetivo intuitivamente atractivo. Puede generar un consenso entre los partidos políticos, reuniendo grupos de la “izquierda”, buscando reducir las desigualdades, y de la “derecha”, buscando reducir la inactividad económica y el costo de servicios.

Tal consenso es superficial e ilusorio. Al hacer una estrategia detallada, la prevención está abierta a muchas interpretaciones por parte de muchos formuladores de políticas públicas. Imagina los muchos tipos de políticas de prevención y formulación de políticas que podríamos producir:

 

     1. ¿Qué problema tratamos de resolver?

La formulación de políticas públicas de prevención representa una solución heroica a varias crisis: grandes desigualdades, servicios públicos con recursos insuficientes y un gobierno disfuncional.

 

     2. ¿En qué medidas debemos centrarnos?

¿En qué desigualdades debemos concentrarnos principalmente? Riqueza, ocupación y empleo, ingresos, raza, etnia, género, sexualidad, discapacidad, salud mental.

¿De acuerdo a cuál medida de desigualdad? Económica, salud, comportamiento saludable, educación, bienestar, castigo.

     3. ¿En qué solución deberíamos centrarnos?

Para reducir la pobreza y las desigualdades socioeconómicas, mejorar la calidad de vida nacional, reducir los costos de los servicios públicos o aumentar la relación precio-calidad.

     4. ¿Qué “herramientas” o instrumentos de política debemos utilizar?

¿Políticas redistributivas para abordar las causas “estructurales” de pobreza y desigualdad?

O tal vez políticas centradas en el individuo para: (a) aumentar la “resistencia” mental de los usuarios de servicios públicos, (b) obligar o (c) exhortar a las personas a cambiar su comportamiento. 

     5. ¿Cómo se interviene lo antes posible en la vida de las personas?

Prevención primaria. Concentrándose en toda la población para evitar que ocurra un problema invirtiendo de forma temprana o modificando el entorno social o físico. Similar a la vacunación del total de la población.

Prevención secundaria. Enfocándose en los grupos en riesgo para identificar un problema en una etapa temprana con el objetivo de minimizar el daño.

Prevención terciaria. Concentrándose en los grupos afectados para evitar que un problema empeore.

 

     6. ¿Cómo se alcanza la “formulación de políticas públicas basada en evidencia”? 3 modelos ideales (en preparación).

¿Usando ensayos controlados aleatorios y revisión sistemática para identificar las mejores intervenciones?

¿Narrativas para compartir las mejores prácticas de gobernanza?

¿Métodos de “mejora” para experimentar a menor escala y compartir las mejores prácticas?

 

     7. ¿Cómo se relaciona la recopilación de evidencia con la formulación de políticas públicas a largo plazo? 

¿Una estrategia nacional impulsa resultados a largo plazo?

¿El gobierno central produce acuerdos u objetivos para las autoridades locales?

 

  1. ¿La formulación de políticas públicas preventivas es una filosofía o un profundo proceso de reforma?

¿Qué tan serios son los gobiernos nacionales (sobre el localismo, los servicios públicos impulsados por los usuarios del servicio y la formulación de políticas integrales u holísticas), cuando los responsables del resultado son políticos electos?

 

  1. ¿Cuál es la naturaleza de la intervención del Estado?

Puede ser punitivo o de apoyo. Ver: ¿Cómo harían Lisa Simpson y Monty Burns una política social progresista? [en inglés]

 

  1. Tomar “decisiones difíciles”: ¿Qué problemas surgen cuando la política se enfrenta a la formulación de políticas públicas?

 

Cuando los formuladores de políticas se mueven desde un amplia filosofía y lenguaje hacia políticas y prácticas específicas, encuentran una serie de obstáculos, que incluyen:

La escala de la tarea se vuelve abrumadora y no se adapta a los ciclos electorales.

Desarrollar políticas públicas y reformar su formulación lleva tiempo, su efecto puede tardar una generación en verse.

 

Existe competencia por los recursos para la formulación de las políticas públicas, tales como la atención y el dinero.

La prevención es general, a largo plazo y de poca importancia. Compite contra los principales problemas a corto plazo que los políticos se sienten obligados a resolver primero.

La prevención es similar a la inversión de capital sin garantía de retorno sobre la inversión. Las reducciones en los fondos de “lucha contra incendios”, “servicios de primera línea” para solventar las iniciativas de prevención, son difíciles de vender. Los gobiernos invierten en pequeñas acciones, y la inversión es vulnerable cuando se necesita dinero rápidamente para financiar crisis en el servicio público.

 

Los beneficios son difíciles de ver y medir.

Los impactos a corto plazo son difíciles de medir, los impactos a largo plazo son difíciles de atribuir a una sola intervención, y la prevención no necesariamente implica ahorrar dinero (ni proporciona ahorros “canjeables”).

Las políticas reactivas tienen un impacto más visible, como reducir los tiempos de espera en el hospital o aumentar el número de maestros u oficiales de policía.

 

Los problemas son “intratables”.

Llegar a la “causa raíz” de los problemas no es sencillo; los formuladores de políticas públicas a menudo no tienen certeza de la causa de los problemas o el efecto de sus soluciones. Pocos aspectos de la prevención en la política social se asemejan a la prevención de enfermedades, en la que se conocen las causas de muchas enfermedades, así como sus formas de detección y prevención.

 

La gestión del desempeño no conduce a la prevención.

Los sistemas de gestión del desempeño alientan a los administradores del sector público a considerar servicios cuyos objetivos sean medibles a corto plazo, sobre aquellos compartidos con socios de prestación de servicios públicos o referentes al bienestar de sus pobladores.

La gestión del desempeño consiste en establecer prioridades cuando los gobiernos tienen demasiados objetivos que cumplir. Cuando los gobiernos centrales alientan a los órganos de gobierno locales a formar asociaciones a largo plazo para abordar las desigualdades y cumplir los objetivos a corto plazo, lo último es lo primero.

 

Los gobiernos enfrentan grandes dilemas éticos.

Las elecciones políticas coexisten con juicios normativos sobre el papel del Estado y la responsabilidad personal, a menudo socavando acuerdos entre partidos políticos.

 

Un aspecto de la prevención puede debilitar al otro.

Una visión cínica de las iniciativas de prevención es que representan una solución política rápida en lugar de una solución significativa a largo plazo:

  • Los gobiernos centrales describen la prevención como la solución a los costos del sector público. A la vez, delegan la responsabilidad de la formulación de políticas públicas y reducen los presupuestos de los organismos públicos subnacionales.
  • Luego los organismos públicos de acuerdo a la urgencia priorizan sus responsabilidades legales.

 

Alguien debe rendir cuentas.

Si todos están involucrados en la formulación y elaboración de políticas públicas, no queda claro quién puede será responsable de los resultados. Esto es incompatible con la responsabilidad democrática al estilo de “Westminster” en donde se sabe quién es responsable y, por lo tanto, a quién culpar o reconocerle el buen desempeño.

 

     3. La evidencia no es una “bala mágica”

 

En una serie de pláticas [en inglés], identifico las razones por las cuales la “formulación de políticas públicas basada en evidencia” (EBPM) [en inglés] no describe bien el proceso de la política pública.

En otras publicaciones también sugiero que es más difícil para la evidencia “ganar la batalla” [en inglés] en las extensas áreas de la política de prevención en comparación con campos más específicos, por ejemplo el control del tabaco.

En general, una regla simple sobre EBPM es que nunca hay una panacea que sustituya al juicio. La política se trata de tomar decisiones que beneficien a algunos mientras que otros pierden. Puedes usar la evidencia para ayudar a comprender esas opciones, pero no para producir una solución “técnica”.

Una regla adicional con los problemas “intratables” es que la evidencia no es lo suficientemente buena como para generar claridad sobre la causa del problema. O simplemente encuentras cosas que no quieres saber.

La intervención temprana en las “políticas públicas familiares” parece ser un buen candidato para este último, por tres razones principales:

 

  1. Muy pocas intervenciones cumplen con los más altos estándares de evidencia

Hay dos tipos principales de intervenciones relevantes “basadas en evidencia” en este campo [en inglés].

Los primeros son “proyectos de intervención familiar” (FIPs, por sus siglas en inglés). En general, se centran en familias de bajos ingresos a menudo de padres solteros, en riesgo de desalojo y vinculados a comportamientos antisociales. Dichos proyectos proporcionan dos formas de intervención:

  • Apoyo intensivo las 24 horas del día, los 7 días de la semana. Los programas incluyen grupos y actividades después de la escuela (para niños) y clases de habilidades (para padres). En algunos casos también consideran tratamiento para las adicciones o la depresión. Dicho tratamiento se lleva a cabo en alojamientos destinados para este fin con reglas estrictas sobre acceso y comportamiento.
  • Un modelo de apoyo y capacitación.

 

La evidencia del éxito proviene de la evaluación más un contrafáctico: esta intervención es costosa, pero se cree que habría costado mucho más dinero y esfuerzo si no se hubiese intervenido. En general, no existe un ensayo controlado aleatorio (RCT, por sus siglas en inglés) para establecer la causa de los mejores resultados, o demostrar que esos resultados no habrían sucedido sin esta intervención.

El segundo son proyectos transferidos de otros países (principalmente los Estados Unidos de América. y Australia) en función de su exitosa reputación que se basa en la evidencia de los RCTs. Hay más evidencia cuantitativa de éxito, pero aún es difícil saber si el proyecto puede transferirse de manera efectiva y si su éxito puede replicarse en otro país con impulsores, problemas y servicios políticos muy diferentes.

 

  1. La evidencia sobre la “expansión” de la prevención primaria es relativamente débil

 Kenneth Dodge [en inglés] (2009) resume un problema general:

  • Hay pocos ejemplos de proyectos efectivos que especialistas llevan a cabo a “a escala”.
  • Existen problemas importantes en torno a la “fidelidad” al proyecto original cuando se amplía (incluida la necesidad de supervisar una expansión de profesionales bien capacitados)
  • Es difícil predecir el efecto de un programa, que se mostró prometedor cuando se aplicó a una determinada población, a una nueva y diferente.

 

  1. La evidencia sobre la intervención temprana secundaria también es débil

 Este punto sobre diferentes poblaciones con diferentes motivaciones se demuestra en un estudio (publicado en 2014) por Stephen Scott y otros [en inglés], acerca de dos intervenciones de Incredible Years para abordar los “síntomas de trastorno de oposición desafiante y los rasgos de personalidad antisocial” en niños de 3 a 7 años (para una discusión más amplia de tales programas, ver Fundamentos para la vida: ¿qué funciona para apoyar la interacción entre padres e hijos en los primeros años? [en inglés], publicado por la Early Intervention Foundation (Fundación de Intervención Temprana)).

Destacan un dilema clásico en la intervención temprana: la evidencia de efectividad solo es clara cuando los niños han sido remitidos clínicamente (“enfoque indicado”), pero no está claro cuando los niños han sido identificados como de alto riesgo utilizando predictores socioeconómicos (“enfoque selectivo”):

 

Un enfoque indicado es más sencillo de administrar, ya que hay menos niños con problemas graves, son más fáciles de identificar y sus padres generalmente están preparados para participar en el tratamiento; sin embargo, los problemas podrían ya estar demasiado arraigados para tratarlos. Por el contrario, un enfoque selectivo se centra en casos menos severos, pero debido a que los problemas están menos establecidos se debe evaluar a poblaciones enteras y algunos casos desarrollarán problemas graves.

 

Para nuestros propósitos, esto podría representar la forma más inconveniente de evidencia sobre intervención temprana: se podría intervenir temprano con respaldo limitado de evidencia que resulte probablemente exitoso o se podría tener una probabilidad mucho mayor de éxito cuando se interviene más tarde, en otras palabras, cuando se está acabando de tiempo para llamarlo ‘intervención temprana’.

Conclusión: Un vago consenso no sustituye la elección política.

Los gobiernos comienzan con la sensación de que han encontrado la solución a muchos problemas, solo para descubrir que tienen que tomar y defender elecciones altamente “políticas”.

Por ejemplo, considera el uso “creativo” de evidencia del gobierno del Reino Unido para hacer una política familiar [en inglés]. En pocas palabras, el gobierno eligió actuar rápido y a la ligera con la evidencia, demonizando a 117,000 familias para proporcionarle cobertura política a una redistribución de recursos hacia proyectos de intervención familiar.

Con justa razón, se podría objetar este estilo de política. Sin embargo, también se tendría que producir una alternativa factible.

Por ejemplo, el Gobierno escocés ha adoptado un enfoque diferente (quizás más cercano a lo que se esperaría en Nueva Zelanda), pero aún necesita producir y defender una narrativa acerca de sus elecciones. El gobierno de Escocia enfrenta casi las mismas limitaciones que el Reino Unido, su auto descripción hacia un “cambio decisivo” hacia la prevención [en inglés], no lo es.

Después de todo, la prevención no es diferente de cualquier otra área de política pública, excepto que ha demostrado ser mucho más complicada y difícil de mantener que la mayoría de las demás. La prevención es parte de un lenguaje excelente pero no una panacea para los problemas de política pública.

 

Otras lecturas:

Prevención [en inglés]

 

Vea también:

¿Qué haces cuando el 20% de la población causa el 80% de sus problemas? Posiblemente nada [en inglés].

Política de intervención temprana, desde “familias con problemas” hasta “personas nombradas”: problemas con la evidencia y encuadre de problemas [en inglés]

 

Traductores

Anette Bonifant Cisneros anette.bonifant@york.ac.uk

Juan Guillermo Vieira jgvieiras@unal.edu.co

Leave a comment

Filed under Evidence Based Policymaking (EBPM), Políticas Públicas, Prevention policy

La Ciencia Política mejora la comprensión de la formulación de políticas públicas basadas en evidencia. Sin embargo, ¿produce mejores recomendaciones?

Serie: El proceso de las políticas públicas.

Paul Cairney, Profesor de Política y Políticas Públicas en la Universidad de Stirling, Escocia. Enlace a texto original en inglés.

Estoy seguro de que las teorías de las políticas públicas provenientes de la ciencia política mejoran la comprensión de la formulación de políticas públicas basadas en evidencia (EBPM, por sus siglas en inglés). En particular cuando se les compara con teorías provenientes de otras disciplinas tales como las ciencias de la salud o ambientales. En los siguientes párrafos, presento dos mensajes clave: atienda tanto a la ambigüedad como a la incertidumbre y enfóquese en la complejidad, no en la linealidad.

No estoy tan seguro de poder brindar consejos más realistas sobre la forma en que las personas con jornadas laborales de tiempo completo -exceptuando el cabildeo- pueden participar en la formulación de políticas públicas.

Por lo tanto, en esta publicación describo de manera amplia algunas implicaciones generales, pero considere que mi consejo tiene importantes implicaciones conductuales, éticas y de recursos que no siempre podrían ser factibles o atractivas para los científicos que se dedican principalmente a la ciencia.

Dicho lo anterior, contaré una historia interesante acerca de las limitaciones de los estudios EBPM cuando se basan principalmente en las perspectivas de los científicos de la salud y el medio ambiente [en inglés]. Primero describo las expresiones de frustración de algunos científicos ante los políticos:

“Conocemos la evidencia, entonces, ¿por qué los políticos no harán nada al respecto?”

“¿Por qué los políticos solamente seleccionan la “evidencia” que se ajusta a sus agendas personales?”

“¿Política basada en evidencia? Más bien evidencia basada en políticas públicas, ¿estoy en lo cierto amigos? “.

En segundo lugar, señalo algunos problemas y posibles soluciones:

  • Un enfoque en mejores suministros de evidencia sólo ayuda a reducir la incertidumbre, no la ambigüedad. No se convencerá a los formuladores de política pública para que actúen simplemente proporcionando más evidencia o reduciendo un informe de 100,000 a 1,000 palabras.
  • En vez de quejarse sobre formuladores cínicos o poco científicos, se podría reconocer lo poco realista que es esperar un momento mágico en el cual el formulador de políticas públicas a cargo capte la evidencia y luego ponga en marcha una política radicalmente nueva. Tales esperanzas se basan en el ideal de un proceso de políticas públicas “lineal” con etapas ordenadas para la toma de decisiones.

En pocas palabras, las teorías de las políticas públicas ayudan a mejorar tales discusiones. Dichas teorías identifican las formas de pensar en las políticas públicas [en inglés], tratando de aclarar el modo en que piensan los formuladores y proporcionando información acerca de la forma en que funcionan los procesos de políticas públicas, en lugar del modo en que nos gustaría que funcionasen (en algunas ocasiones refiriéndose a la “medicina basada en evidencia” [en inglés y  PDF]). Este es el primer paso hacia mejores recomendaciones acerca de la manera de adaptarse e influir en ese proceso con evidencia.

Es difícil contar una buena historia sobre lo que se hace con estas ideas nuevas, particularmente al considerar sus implicaciones en la profesión científica.

Comenzaré con dos recomendaciones basadas en ideas provenientes de estudios de políticas públicas:

1. Enfóquese tanto en la ambigüedad como en la incertidumbre

Los formuladores usan dos atajos para convertir un sinfín de información en una decisión manejable.

  • “Racionales”: limitando sus opciones (en preparación) y restringiendo la búsqueda de información a fuentes de confianza, de modo tal que dicha tarea se vuelva manejable.
  • “Irracionales”: Tomando decisiones rápidas basadas en instintos, intuiciones, emociones, creencias, ideologías y hábitos.

Por lo tanto, la estrategia de producir más evidencia accesible para reducir la incertidumbre científica, solo aborda un atajo. Además, con frecuencia resulta ineficaz, ya que es más probable que los formuladores acepten “evidencia” proveniente de múltiples de fuentes, no únicamente de científicos. Se sabe que no todos leen, entienden, priorizan o aprecian el atractivo de un riguroso artículo académico publicado en alguna revista científica. Por lo tanto, sería sensato el buscar nuevas formas de presentar la información, usando informes breves y utilizando “divulgadores del conocimiento”. Sin embargo, también se deben reconocer los límites de tales procesos cuando la formulación de políticas públicas sigue siendo tan competitiva, sabiendo que los políticos recurren al conocimiento en el que confían (no en ti).

Los promotores de las políticas públicas también necesitan soluciones basadas en la ambigüedad, para reflejar la tendencia de los formuladores a aceptar narrativas simples que refuercen sus prejuicios. Muchas teorías de políticas públicas pueden adaptarse para proporcionar recomendaciones partiendo de:

  • Combinar hechos con argumentos emocionales, para provocar sacudidas en la atención de los formuladores de políticas de una imagen de la política hacia otra (Teoría del Equilibrio Interrumpido, en preparación).
  • Contar historias que manipulan los sesgos de las personas, reparten elogios y culpas y resaltan el valor moral y político de las soluciones (Marco de las Narrativas de Políticas Públicas, en preparación).
  • Producir una solución que sea factible y aproveche el momento cuando los formuladores de políticas tengan la oportunidad de adoptarla (Análisis de Corrientes Múltiples, en preparación).
  • Interpretar nueva evidencia a través de los lentes de las creencias preexistentes de los actores dentro de las coaliciones, algunas de las cuales dominan las redes de políticas públicas (Marco de las Coaliciones Promotoras).

2. Enfóquese en la complejidad, no en la linealidad

 Demasiados estudios -por ejemplo- en ciencias de la salud capturan la formulación de políticas públicas refiriéndose a un ciclo simple de etapas bien ordenadas, donde ocurre un solo evento que modifica el destino de la política pública. En este evento, “la evidencia” da forma a una decisión tomada por una persona con autoridad, quien es fácilmente identificable. Por el contrario, los estudios de políticas públicas identifican una formulación desordenada de políticas, la cual ocurre en un entorno volátil que exhibe:

  • Una amplia gama de actores (individuales y organizaciones) quienes influencian la política pública en distintos niveles de gobierno.
  • Diversas reglas y normas que son acatadas por diferentes niveles y tipos de gobierno.
  • Relaciones estrechas entre formuladores y actores influyentes (“redes”).
  • Una tendencia hacia ciertas creencias o “paradigmas” en el dominio de la discusión.
  • Condiciones cambiantes y eventos que pueden desviar repentinamente la atención de los formuladores de políticas públicas.

Esta visión más amplia cambia el análisis y brinda formas más realistas de adaptarse y trabajar: observando dónde está la acción; qué actores están tomando las decisiones más importantes; las reglas que deben cumplirse ante esos actores; la mejor manera de presentar un argumento adaptándolo a sus creencias específicas; el lenguaje que usan en el establecimiento de criterios relacionados con el significado de una política pública viable; cómo identificar y trabajar con aliados en posiciones privilegiadas con respecto a los formuladores; y cómo utilizar las crisis o los eventos repentinos para atraer la atención de los formuladores.

Hay tres problemas principales con estas recomendaciones:

  1. Manipulación es una palabra impopular

Las opciones A y B requieren que seas manipulador. No del tipo “maquiavélico”, sino que se debe estar preparado para proponer mensajes simples diseñados para influir en el debate, aparentando una mayor certeza científica que la que se tiene o estar dispuesto a participar en debates cargados de emociones poco ligados a la evidencia.

Es habitual que los científicos expresen incertidumbre y un deseo de no anticiparse a la evidencia. Sin embargo, se está compitiendo con personas que no tienen esa sensibilidad. No cumplen o inclusive desconocen las reglas de los científicos, además ganarán, aunque sean menos expertos que tú. Mientras retrocedes a producir y verificar “la evidencia”, diversos actores reconocerán que debes tener un impacto inmediato con la información disponible. Mientras el problema gana relevancia, los formuladores sienten que tienen que actuar a pesar de una alta incertidumbre.

Por otro lado, si te conviertes en un defensor de la causa, puedes perder un recurso clave: algunas personas piensan que eres un científico objetivo, dedicado a la verdad. Es una estrategia legítima optar por mantenerse alejado de la formulación de políticas, para conservar una imagen personal y profesional. Es justo si se reconoce que es una elección con probables consecuencias.

Esta fue una elección que enfrentaron los defensores del control del tabaco [en inglés], muchos de los cuales sintieron que tenían que ir más allá de la evidencia para competir con poderosas compañías tabacaleras. Es una elección que enfrentaron organizaciones como Public Health England ante la creencia de muchas personas quienes piensan que los cigarros y los cigarrillos electrónicos son igualmente dañinos. Se enfrentaron ante la elección de manifestar que “se necesita más evidencia” (lo que conlleva a abandonar el debate y quizás refuerza los efectos de un conocimiento público deficiente) o que los cigarrillos electrónicos son un 95% menos dañinos [en inglés] (para influir en el comportamiento mientras reúnen más evidencia). También es una elección que enfrentan los científicos de alimentos que compiten para influir en las políticas sobre alimentos genéticamente modificados [en inglés] con (a) ciertas compañías que protegen sus negocios y (b) grupos que advierten sobre los alimentos Frankenstein.

  1. Pareciera un empleo de tiempo completo

 Las opciones C y D requieren una participación en la promoción y defensa de políticas públicas durante años e inclusive décadas, para desarrollar un conocimiento suficiente de las personas involucradas (¿Quién vale la pena conocer? ¿Quiénes son sus aliados? ¿Qué argumentos funcionan con ciertas personas?) y saber cuándo impulsar determinada política pública. No existe un incentivo profesional claro para participar en dicha actividad. Los incentivos académicos están cambiando en países como el Reino Unido, sin embargo, todavía dudo en aconsejar a un colega más joven que busque lograr “impacto” en lugar de publicar otro artículo en una prestigiosa revista científica.

  1. Los formuladores de las políticas públicas no siempre actúan de acuerdo a esta recomendación

Los formuladores de políticas reconocerán que toman decisiones dentro de un proceso de políticas públicas impredecible y desordenado y no “lineal”. Muchos podrían incluso aceptar las implicaciones de las teorías de políticas públicas, como la teoría de la complejidad [en inglés], la cual sugiere que los formuladores deberían buscar nuevas formas de actuar cuando reconocen sus limitaciones: usar prueba y error; seguir cambiando las políticas para adaptarse a las nuevas condiciones; delegar y compartir el poder con actores locales capaces de responder en sus jurisdicciones, entre otros.

Sin embargo, este consejo pragmático va en contra de la idea de la responsabilidad democrática al estilo de Westminster, en la que los ministros (secretarios de dependencias de gobierno) siguen siendo responsables ante el Parlamento y ante el público. Bajo este esquema se conoce quién está a cargo, por lo tanto, se sabe a quién culpar.

Frecuentemente los formuladores mantienen simultáneamente dos facetas: la cara pública para competir en las elecciones y mostrar una imagen de control, y la cara menos pública, para negociar con muchos actores y tomar decisiones pragmáticas. Entonces, por ejemplo, tienen un alto potencial para que produzcan “buena política y malas políticas públicas” y no se les debe reprochar automáticamente cuando rechazan la opción de producir “mala política y buenas políticas públicas”. Ya que es probable que se irriten contigo y se vuelvan reacios a seguir tu consejo la próxima vez.

Estas tres consideraciones producen un dilema importante sobre la forma de participar

Imagina una reacción a este consejo bien intencionado: necesitas simplificar la evidencia y manipular a las personas o a los debates cuando participas en discusiones de alto nivel. Sabiendo que las grandes decisiones se llevan a cabo en otro lugar tendrás que influir en diferentes personas con diferentes argumentos más adelante. Averiguar quién influye mejor puede llevarte años y para entonces podría ser demasiado tarde.

De repente, el consejo original: producir informes breves, emplear divulgadores del conocimiento, participar en talleres académicos y profesionales, parece bastante atractivo.

Así que puede tomar más tiempo el producir consejos viables basados en las implicaciones de las teorías de las políticas públicas. Mientras tanto, al menos esta discusión debería ayudar a aclarar por qué hay una brecha entre la evidencia científica y la formulación de políticas. Así como a generar algunos consejos pragmáticos: hazlo bien o no lo hagas; si te involucras poco a poco en el proceso de las políticas públicas espera poca recompensa. Además de que la influencia en la política requiere una inversión que muchos científicos pueden no estar dispuestos o no ser capaces de financiar (y muchas inversiones no valdrán la pena).

Vea también:

Este post es uno de muchos acerca de EBPM. La lista completa (en inglés) se encuentra aquí: https://paulcairney.wordpress.com/ebpm/

Traductores

Anette Bonifant Cisneros anette.bonifant@york.ac.uk

Enrique García Tejeda enriquegarciatejeda@gmail.com

 

Leave a comment

Filed under Evidence Based Policymaking (EBPM), Políticas Públicas

Formulación de Políticas Públicas Basadas en Evidencia

Serie: El proceso de las políticas públicas.

Paul Cairney, Profesor de Política y Políticas Públicas en la Universidad de Stirling, Escocia. Enlace a texto original en inglés.

 

El término ‘Formulación de Políticas Públicas Basadas en Evidencia’ (en inglés Evidence Based Policymaking) es ampliamente usado y aceptado en los medios de comunicación y en las redes sociales. A menudo, este concepto representa un ideal que los gobiernos no logran alcanzar. Un alegato frecuente es que los formuladores de políticas públicas ignoran, no entienden o no actúan de acuerdo con la evidencia correcta.

Sin embargo, si observas los estudios de políticas públicas, tiendes a encontrar discusiones altamente críticas acerca del concepto, además de comentarios acerca de la ingenuidad de las personas si éstas creen que la formulación de políticas basadas en evidencia es una posibilidad (EBPM, por sus siglas en inglés). Algo de ello simplemente tiene que ver con la falta de claridad sobre lo que EBPM significa. Otro tanto con el argumento de que en los estudios de políticas públicas la gente no entiende el proceso de las políticas cuando demandan el uso de EBPM. De lo anterior se desprenden 2 argumentos comunes en los estudios de políticas públicas:

  1. EBPM es un modelo ideal, útil únicamente para describir lo que no sucede y lo que no puede suceder

EBPM debería tratarse de la misma manera que el modelo ideal del “formulador racional de políticas públicas”. Al identificar los límites de la racionalidad comprensiva (también conocida como absoluta o sinóptica), exploramos las implicaciones de la “racionalidad limitada”. Por ejemplo, al afirmar que los formuladores de políticas públicas no tienen la capacidad de recopilar y analizar toda la información, identificamos las heurísticas y los atajos que ellos utilizan para recopilar la información que pueden. Esto puede revelar sesgos hacia ciertas fuentes de información (que pueden ser más importantes que la naturaleza de la evidencia misma). Al afirmar que los formuladores de política sólo pueden prestar atención a una pequeña fracción de los problemas de los que son responsables, identificamos qué asuntos son colocados como prioridad en la agenda pública y cuáles son ignorados. Nuevamente, hay mucho más en este proceso que la naturaleza de la evidencia: se trata de la manera en que los defensores “encuadran” los problemas y cómo estos problemas son entendidos por los formuladores de políticas, quiénes son considerados responsables de resolverlos.

  1. Los científicos usan evidencia para destacar problemas públicos, pero no para promover cambios en las políticas

La literatura sobre política públicas contiene teorías y estudios que usan la ciencia de la formulación de las políticas públicas para explicar cómo dicha formulación funciona. Por ejemplo, los estudios de “equilibrio interrumpido” usan la racionalidad limitada para identificar largos periodos de estabilidad y continuidad en la formulación de las políticas, las cuales son interrumpidas por profundas e importantes ráfagas de inestabilidad y cambio. En algunos casos, los formuladores ignoran algunas pruebas durante años, luego, abruptamente, prestan una atención desproporcionada a la misma evidencia. Esto puede derivar de la sustitución de algunos formuladores de políticas por otros (por ejemplo, después de las elecciones) o de un suceso que capte la atención el cual los impulsa a desviar su atención desde otros lugares. Además, los estudios de difusión de políticas públicas utilizan la racionalidad limitada para identificar la emulación de políticas en ausencia de aprendizaje; el traslado de una política pública por parte de un gobierno que podría no saber mucho sobre por qué dicha política tuvo éxito en otro lugar. En tales casos, una política puede introducirse tanto por su reputación como por la evidencia de su éxito transferible. En otros estudios, como en el “marco de las coaliciones promotoras”, identificamos una batalla de ideas, en la cual diferentes grupos buscan reunir e interpretar evidencia de maneras muy diferentes. EBPM trata sobre la interpretación dominante del mundo, los principales eventos y las consecuencias de políticas públicas hasta ahora.

En cada caso, el primer punto en general es que los responsables de las políticas públicas tienen que tomar decisiones importantes en momentos de incertidumbre (falta de información), ambigüedad (incertidumbre sobre la manera de entender un problema y su solución), y conflicto (sobre la forma de interpretar la información y extraer conclusiones). Sin embargo, lo hacen recurriendo a atajos. Por ejemplo, usan la información de fuentes en las que confían y adaptan esa información a las creencias que ya tienen. El segundo punto, es que, incluso en los sistemas “Westminster”, hay muchos responsables involucrados en las políticas públicas. Podemos comenzar con la identificación simple de un único formulador completamente racional en el centro del proceso de políticas públicas. Sin embargo, terminaríamos identificando una imagen complicada en la cual muchos actores (en distintos niveles o tipos de gobierno), influyen en la forma en que se presentan las pruebas y se formulan las políticas.

En este contexto, una simple petición para que el gobierno haga algo con “la evidencia” puede parecer algo ingenuo. Tal apelación a la evidencia relacionada con un problema en particular está incompleta sin una apelación previa a la evidencia en el proceso de las políticas públicas. En lugar de lamentar la falta de EBPM, necesitamos una mejor comprensión de los límites de la EBPM para informar la forma en que conceptualizamos la relación entre la información y las políticas públicas. Esto es tan importante para el científico que busca influir en la elaboración de políticas públicas como lo es para el científico que formula las políticas. El primero debería identificar la manera en que funciona el proceso de política y tratar de influir en el sobre esa base (no en la manera que nos gustaría que fuera). La comprensión de solo un aspecto de EBPM es el rechazo de EBPM.

Vea también:

Este post es uno de muchos acerca de EBPM. La lista completa (en inglés) se encuentra aquí:

https://paulcairney.wordpress.com/ebpm/

Formulación de Políticas Basadas en Evidencia” y el Estudio de las Políticas Públicas

Un “cambio decisivo hacia la prevención”: ¿Cómo convertimos una idea en una política basada en evidencia?  (en preparación)

Traductores

Anette Bonifant Cisneros anette.bonifant@york.ac.uk

Enrique García Tejeda enriquegarciatejeda@gmail.com

Leave a comment

Filed under Evidence Based Policymaking (EBPM), Políticas Públicas

“Formulación de Políticas Públicas Basada en Evidencia” y el Estudio de las Políticas Públicas

Serie: El proceso de las políticas públicas.

Paul Cairney, Profesor de Política y Políticas Públicas en la Universidad de Stirling, Escocia. Enlace a texto original en inglés.

Esta publicación acompaña una clase de 40 minutos (disponible para descarga en inglés), la cual considera “la formulación de políticas públicas basada en evidencia” (EBPM, por sus siglas en inglés) a través de la lente de las teorías de las políticas públicas. La teoría es importante porque provee un lenguaje con el cual entender la EBPM como parte de una discusión más amplia en torno al proceso de las políticas públicas, mientras que la lente de EBPM nos permite pensar en la aplicación de conceptos y teorías en el “mundo real”.

Para dicho fin, haré hincapié en tres puntos importantes:

  1. Las definiciones y la claridad son importantes. La “Formulación de políticas públicas basada en evidencia”, “políticas basadas en evidencia” y frases relacionadas como “evidencia basada en políticas” son usadas increíblemente a la ligera en los debates públicos. Un enfoque centrado en preguntas básicas de los estudios de políticas públicas, tales como ¿Qué es una política y cómo podemos medir el cambio de políticas públicas?, nos ayuda a aclarar los problemas, a rechazar debates superficiales sobre “políticas públicas basadas en evidencia frente evidencia basada en políticas públicas” y, en algunos casos, a identificar los supuestos muy diferentes que las personas hacen sobre la forma en que funciona y debería funcionar la formulación de políticas públicas.
  2. Los modelos realistas son importantes. Discutir acerca de EBPM nos ayuda a identificar las principales fallas en los modelos simples de formulación de políticas públicas, como es el caso del “ciclo de la política pública”. Analizaré las ideas que se obtienen al considerar cómo los expertos en políticas describen las implicaciones de la “racionalidad limitada” y la complejidad de la formulación de políticas.
  3. Las estrategias realistas son importantes. Existe una amplia discusión académica sobre la necesidad de superar las “barreras” entre la evidencia y la política pública. A menudo esto es sin teoría y produce recomendaciones ingenuas sobre las formas de mejorar el suministro de información y de capacitar a los encargados de formulación de políticas públicas para comprender la evidencia. Identifico dos estrategias más útiles (pero potencialmente polémicas): ser manipulador y aprender dónde está la “acción”.

Las definiciones y la claridad son importantes, así que ¿Qué es la “formulación de políticas públicas basada en evidencia”?

¿Qué es la política pública? Es muy difícil decir que es la política pública y  medir cuánto ha cambiado (en preparación). Yo utilizo la definición práctica (en preparación) , “la suma total de la acción del gobierno, desde señales de intención hasta los resultados finales”, para resaltar cualidades importantes: (a) es difícil combinar lo que la gente dice que hará y lo que realmente hace: (b) el resultado de la política pública puede ser muy diferente de la intención original: (c) la política pública se realiza de manera rutinaria a través de la cooperación entre formuladores de política pública electos y no electos, así como con actores que no tienen un papel formal en el proceso; (d) la formulación de la política pública también se trata del poder de no hacer algo.

Además es importante identificar los diversos componentes o instrumentos de política que constituyen las políticas públicas, incluyendo: el nivel de gasto; el uso de incentivos económicos o sanciones; leyes y regulaciones; el uso de acuerdos voluntarios y códigos de conducta; la provisión de servicios públicos; campañas educativas; financiamiento para estudios científicos o para causas especificas; cambio organizacional; y el nivel de recursos o métodos dedicados a la implementación de políticas públicas (Cairney 2012, p. 26).

En este contexto, estamos tratando de capturar un proceso en el cual los actores elaboran y proporcionan “políticas públicas” de forma continua, no precisamente identificar un evento fragmentado, el cual brinda una oportunidad única de utilizar una pieza de evidencia científica para que los responsables de la formulación de las políticas generen una respuesta puntual.

¿Quiénes son los formuladores de políticas públicas? La definición intuitiva es “personas quienes hacen políticas públicas”, sin embargo, hay dos distinciones importantes: (1) entre participantes electos y no electos, ya que personas tales como los funcionarios públicos también toman decisiones importantes; (2) entre personas y organizaciones, donde estas últimas se utilizan como una abreviatura para referirse a un grupo de personas tomando decisiones en conjunto. Hay líneas divisorias difusas entre las personas que hacen e influyen en las políticas públicas. Términos como “comunidad de política pública” sugieren que las decisiones de política son tomadas por varias personas con responsabilidad formal, pero influencia informal. Por lo tanto, debemos clarificar lo que queremos decir con “formuladores de política pública” cuando identificamos la manera en que usan la evidencia.

¿Qué es evidencia? Podemos definir evidencia como un argumento respaldado por información. La evidencia científica hace referencia a información producida de una manera particular. Algunos describen “científico” en términos generales, para referirse a la información recopilada sistemáticamente utilizando métodos reconocidos, mientras que otros se refieren a una jerarquía especifica entre métodos científicos, donde en la parte superior se encuentran los Ensayos Controlados Aleatorios (RCTs, por sus siglas en inglés) y la revisión sistemática de los RCTs. Este es un tema crucial:

Los encargados de la formulación de políticas públicas buscarán muchos tipos de información que muchos científicos no considerarían como “evidencia”.

Esta discusión ayuda a identificar dos temas clave potencialmente confusos cuando la gente discute acerca de EBPM:

  1. Cuando describes “política pública basada en evidencia” y la EBPM necesitas aclarar cuál es la política y quien la está elaborando. No se trata únicamente de algunos políticos haciendo declaraciones.
  2. Cuando describes “evidencia” tienes que aclarar que constituye dicha evidencia y como se vería una política reactiva “basada en evidencia”. Este punto está frecuentemente en el centro de discusiones infructuosas sobre “evidencia basada en políticas públicas”, el cual parece describir casi una docena de presuntos errores por parte de los formuladores (en preparación) de política pública ((refiriendose a ignorar la evidencia, usar tipos incorrectos o producir una respuesta desproporcionada).

Los modelos realistas son importantes, entonces, ¿Qué tiene de malo el ciclo de la política pública?

Una forma tradicional de entender la formulación de la política pública en el “mundo real” es compararla con un modelo ideal: ¿Qué sucede cuando no se cumplen las condiciones ideales? Hacemos esto en particular con el “ciclo de la política pública” y la “racionalidad comprensiva” (más información en inglés).

Entonces, considera este modelo ideal modificado de EBPM:

  • Hay un grupo clave de formuladores de políticas públicas en el “centro”, que elabora políticas de “arriba hacia abajo”, dividiendo su tarea en etapas claramente definidas y ordenadas;
  • Los científicos están en una posición privilegiada para ayudar a los responsables de la política pública a tomar buenas decisiones al acercarlos lo más posible al ideal de “racionalidad comprensiva”, en el cual tengan la mejor información disponible para informar todas las opciones y consecuencias.

Hasta ahora, todo parece bien (aunque podrías detenerte a considerar quién está mejor posicionado para proporcionar evidencia y quién, -o qué métodos de recopilación de evidencia-, deberían ser privilegiados o excluidos (en preparación), pero ¿qué sucede cuando nos alejamos del modelo ideal? Aquí hay dos ideas de un documento publicado por Cairney, Oliver, and Wellstead (2016)

Lecciones de la teoría de políticas públicas: 1. Identificar entornos de formulación de políticas con múltiples niveles.

Primero, la formulación de políticas tiene lugar en un entorno de políticas menos ordenado y predecible, que exhibe:

  • Una amplia gama de actores (individuos y organizaciones) que influyen en la política pública en diversos niveles de gobierno.
  • Una proliferación de reglas y normas que son seguidas por diferentes niveles o tipos de gobierno.
  • Relaciones estrechas (“redes”) entre formuladores de políticas y actores poderosos.
  • Una tendencia hacia ciertas creencias o “paradigmas” que dominan una discusión.
  • Condiciones cambiantes en la política pública y eventos que pueden atraer la atención de los formuladores de políticas en el corto plazo.

Un enfoque en este panorama más amplio desvía nuestra atención del uso de evidencia científica por un grupo electo de élite de formuladores de políticas en lo alto, hacia el uso de evidencia por un amplio rango de actores con influencia en el proceso multinivel de políticas públicas.

También muestra a los científicos y profesionales que están compitiendo con muchos otros actores para presentar evidencia de una manera particular para asegurarse la atención de los formuladores de política. El apoyo a soluciones particulares varía según la organización que tome la iniciativa y el modo en que ésta comprenda el problema.

Algunas redes son estrechamente unidas y son de difícil acceso porque las burocracias tienen procedimientos operativos que favorecen determinadas fuentes de evidencia y a algunos participantes sobre otros. Además, hay un lenguaje que indica qué formas de pensar se encuentran ampliamente usadas y aceptadas lo cual toma tiempo aprender. Las creencias bien establecidas proporcionan el contexto para la formulación de políticas públicas: la nueva evidencia sobre la efectividad de una solución debe ir acompañada de un cambio de atención y una persuasión exitosa. En algunos casos, las “crisis” sociales o económicas pueden provocar un cambio abrupto de atención de un tema a otro, y algunas formas de evidencia se pueden utilizar para alentar ese cambio. En este contexto, diversos estudios de practicantes analizan, por ejemplo, la decisión del gobierno central en un determinado momento en el tiempo, en vez de un proceso a largo plazo. Superar las barreras para influir en esa pequeña parte del proceso no proporcionará una solución general.

Lecciones de la teoría de políticas públicas: 2. Los formuladores de políticas usan dos “atajos” para tomar decisiones

¿Cómo manejan los encargados de formular políticas su “racionalidad limitada”? Emplean dos tipos de atajos: “racionales”, persiguiendo objetivos claros y priorizando ciertas clases y fuentes de información, e “irracionales”, recurriendo a las emociones, sentimientos viscerales, creencias profundamente establecidas, hábitos y lo familiar para tomar decisiones rápidamente. En consecuencia, las teorías de políticas públicas están centradas en los vínculos entre evidencia, persuasión y “encuadre” (en el contexto amplio de que una discusión tiende a ser dominada por ciertas creencias).

El “encuadre” (o Framing, en inglés) se refiere a las formas en que entendemos, visualizamos y categorizamos los problemas. Los problemas son multifacéticos, pero la racionalidad limitada acota la atención de los formuladores de políticas públicas, y los actores compiten para destacar una imagen a expensas de otras. El resultado de este proceso determina quién está involucrado (por ejemplo, definir un tema como técnico limita la participación de actores considerando mayormente a aquellos que son expertos), quién es responsable de la política, cuánta atención prestan y qué tipo de solución favorecen. Por ejemplo, es más probable el control del tabaco cuando los formuladores de políticas lo ven principalmente como una epidemia de salud pública, en lugar de un bien económico, mientras que la política de “fracking” depende de su imagen principal como auge del petróleo o como un desastre ambiental (en inglés analizo aquí ambos ejemplos).

La evidencia científica juega un papel en este proceso, pero no debemos exagerar la capacidad de los científicos para salirse con la suya respecto a la evidencia. Más bien, las teorías de políticas públicas señalan las estrategias que los practicantes de políticas tendrían que adoptar para aumentar su demanda de evidencia:

  • Combinar hechos con argumentos emocionales, para provocar sacudidas en la atención de los formuladores de políticas de una imagen de la política hacia otra (Teoría del Equilibrio Interrumpido, en preparación).
  • Crear narrativas simples fáciles de entender, ayuda a manipular los sesgos de las personas, reparten elogios y culpas y resaltan el valor moral y político de las soluciones (Marco de las Narrativas de Políticas Públicas, en preparación).
  • Interpretar nueva evidencia a través de los lentes de las creencias preexistentes de los actores dentro de las coaliciones, algunas de las cuales dominan las redes de políticas públicas (Marco de las Coaliciones Promotoras).
  • Producir una solución que sea factible y aprovechar el momento cuando los formuladores de políticas tengan la oportunidad de adoptarla (Análisis de Corrientes Múltiples, en preparación).

Además, el impacto de una estrategia de encuadre puede no ser inmediato, incluso si pareciera tener éxito. La evidencia científica puede provocar una sacudida de atención a un problema de política pública, lo que provocará un cambio de opinión en un lugar o la participación de nuevos actores de otros lugares. Sin embargo, por ejemplo, puede tomar años producir apoyo para una solución de política pública “basada en evidencia”, de acuerdo con su viabilidad técnica y política (¿funcionará según lo previsto y los formuladores de políticas tendrán la motivación y la oportunidad de seleccionarla?).

Esta discusión ayuda a identificar dos puntos clave de posible confusión cuando las personas discuten el ciclo de la política pública y la racionalidad comprensiva:

  1. Estos conceptos están ahí para ayudarnos a entender lo que no sucede. ¿Cuáles son las implicaciones de los límites de estos modelos en el mundo real?
  2. No te ayudan a dar buenos consejos a las personas que intentan influir en el proceso de las políticas públicas. Siempre es relevante enfocarse en las “etapas” de formulación de políticas públicas y mejorar la “racionalidad” cuando se asesora a los encargados de la formulación. No obstante, por muy poco realistas que sean estos modelos, aún querrías recopilar la máxima información y pasar por un proceso de etapas. Lo anterior es muy diferente de (a) brindar asesoría sobre cómo influir en el proceso, o (b) evaluar los pros y los contras de un sistema político con referencia a los modelos ideales.

Las estrategias realistas son importantes, entonces, ¿hasta dónde se debería llegar para superar las “barreras” entre la evidencia y la política pública?

No puedes descartar en EBPM la política. Aun cuando la selección de la evidencia es política (¿la evidencia debería ser científica? y ¿qué cuenta cómo evidencia científica?).

Además, los proveedores de evidencia científica enfrentan grandes dilemas cuando buscan maximizar el “impacto” de su investigación. Equipado con este conocimiento del proceso de políticas públicas, ¿cómo deberías buscar comprometerte e influir en las decisiones tomadas dentro del proceso?

Si estas interesado en la discusión final, mira este breve video y la siguiente publicación del blog: La ciencia política mejora nuestro entendimiento de la formulación de políticas públicas basada en evidencia, ¿pero produce mejores consejos? (en preparación).

Vea también:

Este post es uno de muchos acerca de EBPM. La lista completa (en inglés) se encuentra aquí:

https://paulcairney.wordpress.com/ebpm/

Para cerrar la brecha entre evidencia y política pública: reducir tanto la ambigüedad como la incertidumbre (en preparación)

Traductores

Anette Bonifant Cisneros  anette.bonifant@york.ac.uk

Enrique García Tejeda enriquegarciatejeda@gmail.com

1 Comment

Filed under Evidence Based Policymaking (EBPM), Políticas Públicas

Cuatro obstáculos para la Formulación de Políticas Públicas Basada en Evidencia (EBPM)

Serie: El proceso de las políticas públicas.

Paul Cairney, Profesor de Política y Políticas Públicas en la Universidad de Stirling, Escocia. Enlace a texto original en inglés.

1. Incluso si existe “la evidencia”, ésta no te dice qué hacer.

  • A veces hay evidencia clara ante la existencia de un problema, pero no de su solución.
  • La evidencia puede advertirnos que algo es efectivo, pero no indica si es adecuado.
  • Los científicos pueden exagerar el consenso científico cuando se convierten en defensores.
  • Los científicos a menudo discrepan sobre lo que están haciendo, cómo deberían hacerlo y cómo la ciencia debería contribuir a la política pública.
  • Estos problemas se exacerban cuando: los problemas traspasan áreas de políticas públicas tradicionales y límites disciplinarios, la base de evidencia es irregular y la evidencia proviene de otras áreas, de una manera poco familiar o poco organizada.

 

2. La demanda de evidencia no coincide con la oferta

  • Los gobiernos podrían financiar investigaciones para buscar una “panacea” o información definitiva para eliminar la necesidad de la elección política.
  • Las investigaciones a menudo se centran en pocos aspectos medibles de las intervenciones, sin embargo, los formuladores de políticas públicas consideran problemas complejos.
  • En contraste con los científicos y especialistas, los formuladores de políticas públicas prestan atención o entienden la evidencia de distinta manera.
  • La demanda de información puede ser impredecible.
  • Buscan muchas fuentes de información: científicas, prácticas, de opinión.
  • A menudo tienen que tomar decisiones rápidamente y bajo incertidumbre.
  • Utilizan la investigación selectivamente: para reforzar su opinión, legitimar sus acciones y demostrar que están actuando.
  • Las personas que proporcionan evidencia desean un impacto instantáneo, pero el efecto puede ser más sutil, inclusive podría llevar años o décadas.

3. Las personas toman decisiones en un complejo sistema de formulación de políticas públicas, en el que el papel de la evidencia suele no ser claro

  • El proceso de la política pública contiene muchos actores y toma tiempo entender cómo funciona el sistema.
  • Los científicos compiten con diferentes actores (mejor informados del proceso de política pública) para asegurar ser escuchados por los formuladores de políticas públicas y presentar evidencia de una manera particular.
  • El apoyo para brindar soluciones basadas en evidencia varía según el departamento o la unidad que toma la iniciativa y la manera que entiende el problema.
  • Las burocracias y los organismos públicos tienen procedimientos operativos que favorecen algunas fuentes de evidencia en particular y a algunos participantes por encima otros.
  • Las creencias establecidas proporcionan el contexto para considerar una nueva evidencia.
  • La atención a la evidencia puede fluctuar de manera impredecible después de los cambios en el entorno de la política pública.

4. La formulación de políticas públicas basada en evidencia no es lo mismo que una buena formulación de políticas públicas

  • Reducir la brecha entre la evidencia y la política pública significa centralizar el poder en manos de un pequeño número de formuladores y garantizar que la evidencia científica sea su única fuente de conocimiento.
  • Los gobiernos pueden buscar legítimamente formas alternativas de “buena” formulación de políticas públicas basándose en consultas y generando un cierto consenso con la sociedad, practicantes y usuarios.

Dotados de este conocimiento, como científicos podemos elegir cómo adaptarnos a dichas circunstancias, por ejemplo: identificando el lugar de la acción; aprendiendo sobre las propiedades de los sistemas de formulación de políticas públicas, las reglas del juego y cómo encuadrar la evidencia para que se ajuste a las agendas de políticas públicas; formar coaliciones con otros actores influyentes; y, participar en el proceso de políticas públicas el tiempo suficiente para explotar las ventanas de oportunidad.

Vea también:

Este post es uno de muchos acerca de EBPM. La lista completa (en inglés) se encuentra aquí:

https://paulcairney.wordpress.com/ebpm/

 

¿Cómo la teoría de las políticas públicas podría tener un impacto en la formulación de políticas públicas? (en preparación)

Un “cambio decisivo hacia la prevención”: ¿Cómo convertimos una idea en una política basada en evidencia? (en inglés)

Para más publicaciones acerca de la teoría de políticas públicas discutidas con sus argumentos originales, ver

https://paulcairney.wordpress.com/espanol/

o en inglés

https://paulcairney.wordpress.com/1000-words/

 

Traductores

Anette Bonifant Cisneros  anette.bonifant@york.ac.uk

Enrique García Tejeda enriquegarciatejeda@gmail.com

Leave a comment

Filed under Evidence Based Policymaking (EBPM), Políticas Públicas