Category Archives: 750 word policy analysis

Policy Analysis in 750 Words: policy analysis for marginalized groups in racialized political systems

Note: this post forms one part of the Policy Analysis in 750 words series overview.

For me, this story begins with a tweet by Professor Jamila Michener, about a new essay by Dr Fabienne Doucet, ‘Centering the Margins: (Re)defining Useful Research Evidence Through Critical Perspectives’:

Research and policy analysis for marginalized groups

For Doucet (2019: 1), it begins by describing the William T. Grant Foundation’s focus on improving the ‘use of research evidence’ (URE), and the key questions that we should ask when improving URE:

  1. For what purposes do policymakers find evidence useful?

Examples include to: inform a definition of problems and solutions, foster practitioner learning, support an existing political position, or impose programmes backed by evidence (compare with How much impact can you expect from your analysis?).

  1.   Who decides what to use, and what is useful?

For example, usefulness could be defined by the researchers providing evidence, the policymakers using it, the stakeholders involved in coproduction, or the people affected by research and policy (compare with Bacchi, Stone and Who should be involved in the process of policy analysis?).

  1. How do critical theories inform these questions? (compare with T. Smith)

First, they remind us that so-called ‘rational’ policy processes have incorporated research evidence to help:

‘maintain power hierarchies and accept social inequity as a given. Indeed, research has been historically and contemporaneously (mis)used to justify a range of social harms from enslavement, colonial conquest, and genocide, to high-stakes testing, disproportionality in child welfare services, and “broken windows” policing’ (Doucet, 2019: 2)

Second, they help us redefine usefulness in relation to:

‘how well research evidence communicates the lived experiences of marginalized groups so that the understanding of the problem and its response is more likely to be impactful to the community in the ways the community itself would want’ (Doucet, 2019: 3)

In that context, potential responses include to:

  1. Recognise the ways in which research and policy combine to reproduce the subordination of social groups.
  • General mechanisms include: the reproduction of the assumptions, norms, and rules that produce a disproportionate impact on social groups (compare with Social Construction and Policy Design).
  • Specific mechanism include: judging marginalised groups harshly according to ‘Western, educated, industrialized, rich and democratic’ norms (‘WEIRD’)
  1. Reject the idea that scientific research can be seen as objective or neutral (and that researchers are beyond reproach for their role in subordination).
  2. Give proper recognition to ‘experiential knowledge’ and ‘transdiciplinary approaches’ to knowledge production, rather than privileging scientific knowledge.
  3. Commit to social justice, to help ‘eliminate oppressions and to emancipate and empower marginalized groups’, such as by disrupting ‘the policies and practices that disproportionately harm marginalized groups’ (2019: 5-7)
  4. Develop strategies to ‘center race’, ‘democratize’ research production, and ‘leverage’ transdisciplinary methods (including poetry, oral history and narrative, art, and discourse analysis – compare with Lorde) (2019: 10-22)

Policy analysis in a ‘racialized polity’

A key way to understand these processes is to use, and improve, policy theories to explain the dynamics and impacts of a racialized political system. For example, ‘policy feedback theory’ (PFT) draws on elements from historical institutionalism and SCPD to identify the rules, norms, and practices that reinforce subordination.

In particular, Michener’s (2019: 424) ‘Policy Feedback in a Racialized Polity’ develops a ‘racialized feedback framework (RFF)’ to help explain the ‘unrelenting force with which racism and White supremacy have pervaded social, economic, and political institutions in the United States’. Key mechanisms include (2019: 424-6):

  1. Channelling resources’, in which the rules, to distribute government resources, benefit some social groups and punish others.
  • Examples include: privileging White populations in social security schemes and the design/ provision of education, and punishing Black populations disproportionately in prisons (2019: 428-32).
  • These rules also influence the motivation of social groups to engage in politics to influence policy (some citizens are emboldened, others alienated).
  1. Generating interests’, in which ‘racial stratification’ is a key factor in the power of interest groups (and balance of power in them).
  2. Shaping interpretive schema’, in which race is a lens through which actors understand, interpret, and seek to solve policy problems.
  3. The ways in which centralization (making policy at the federal level) or decentralization influence policy design.
  • For example, the ‘historical record’ suggests that decentralization is more likely to ‘be a force of inequality than an incubator of power for people of color’ (2019: 433).

Insufficient attention to race and racism: what are the implications for policy analysis?

One potential consequence of this lack of attention to race, and the inequalities caused by racism in policy, is that we place too much faith in the vague idea of ‘pragmatic’ policy analysis.

Throughout the 750 words series, you will see me refer generally to the benefits of pragmatism:

In that context, pragmatism relates to the idea that policy analysis consists of ‘art and craft’, in which analysts assess what is politically feasible if taking a low-risk client-oriented approach.

In this context, pragmatism may be read as a euphemism for conservatism and status quo protection.

In other words, other posts in the series warn against too-high expectations for entrepreneurial and systems thinking approaches to major policy change, but they should not be read as an excuse to reject ambitious plans for much-needed changes to policy and policy analysis (compare with Meltzer and Schwartz, who engage with this dilemma in client-oriented advice).

Connections to blog themes

This post connects well to:

 

 

Leave a comment

Filed under 750 word policy analysis, Evidence Based Policymaking (EBPM), public policy, Storytelling

Policy Analysis in 750 Words: entrepreneurial policy analysis

This post forms one part of the Policy Analysis in 750 words series overview and connects to ‘Three habits of successful policy entrepreneurs’.

The idea of a ‘policy entrepreneur’ is important to policy studies and policy analysis.

Let’s begin with its positive role in analysis, then use policy studies to help qualify its role within policymaking environments.

The take-home-messages are to

  1. recognise the value of entrepreneurship, and invest in relevant skills and strategies, but
  2. not overstate its spread or likely impact, and
  3. note the unequal access to political resources associated with entrepreneurs.

Box 11.3 UPP 2nd ed entrepreneurs

Entrepreneurship and policy analysis

Mintrom identifies the intersection between policy entrepreneurship and policy analysis, to highlighting the benefits of ‘positive thinking’, creativity, deliberation, and leadership.

He expands on these ideas further in So you want to be a policy entrepreneur?:

Policy entrepreneurs are energetic actors who engage in collaborative efforts in and around government to promote policy innovations. Given the enormous challenges now facing humanity, the need is great for such actors to step forward and catalyze change processes” (Mintrom, 2019: 307).

Although many entrepreneurs seem to be exceptional people, Mintrom (2019: 308-20) identifies:

  1. Key attributes to compare
  • ‘ambition’, to invest resources for future reward
  • ‘social acuity’, to help anticipate how others are thinking
  • ‘credibility’, based on authority and a good track record
  • ‘sociability’, to empathise with others and form coalitions or networks
  • ‘tenacity’, to persevere during adversity
  1. The skills that can be learned
  • ‘strategic thinking’, to choose a goal and determine how to reach it
  • ‘team building’, to recognise that policy change is a collective effort, not the responsibility of heroic individuals (compare with Oxfam)
  • ‘collecting evidence’, and using it ‘strategically’ to frame a problem and support a solution
  • ‘making arguments’, using ‘tactical argumentation’ to ‘win others to their cause and build coalitions of supporters’ (2019: 313)
  • ‘engaging multiple audiences’, by tailoring arguments and evidence to their beliefs and interests
  • ‘negotiating’, such as by trading your support in this case for their support in another
  • ‘networking’, particularly when policymaking authority is spread across multiple venues.
  1. The strategies built on these attributes and skills.
  • ‘problem framing’, such as to tell a story of a crisis in need of urgent attention
  • ‘using and expanding networks’, to generate attention and support
  • ‘working with advocacy coalitions’, to mobilise a collection of actors who already share the same beliefs
  • ‘leading by example’, to signal commitment and allay fears about risk
  • ‘scaling up change processes’, using policy innovation in one area to inspire wider adoption.

p308 Mintrom for 750 words

Overall, entrepreneurship is ‘tough work’ requiring ‘courage’, but necessary for policy disruption, by: ‘those who desire to make a difference, who recognize the enormous challenges now facing humanity, and the need for individuals to step forward and catalyze change’ (2019: 320; compare with Luetjens).

Entrepreneurship and policy studies

  1. Most policy actors fail

It is common to relate entrepreneurship to stories of exceptional individuals and invite people to learn from their success. However, the logical conclusion is that success is exceptional and most policy actors will fail.

A focus on key skills takes us away from this reliance on exceptional actors, and ties in with other policy studies-informed advice on how to navigate policymaking environments (see ‘Three habits of successful policy entrepreneurs’, these ANZSOG talks, and box 6.3 below)

box 6.3

However, note the final sentence, which reminds us that it is possible to invest a huge amount of time and effort in entrepreneurial skills without any of that investment paying off.

  1. Even if entrepreneurs succeed, the explanation comes more from their environments than their individual skills

The other side of the entrepreneurship coin is the policymaking environment in which actors operate.

Policy studies of entrepreneurship (such as Kingdon on multiple streams) rely heavily on metaphors on evolution. Entrepreneurs are the actors most equipped to thrive within their environments (see Room).

However, Kingdon uses the additional metaphor of ‘surfers waiting for the big wave’, which suggests that their environments are far more important than them (at least when operating on a US federal scale – see Kingdon’s Multiple Streams Approach).

Entrepreneurs may be more influential at a more local scale, but the evidence of their success (independent of the conditions in which they operate) is not overwhelming. So, self-aware entrepreneurs know when to ‘surf the waves’ or try to move the sea.

  1. The social background of influential actors

Many studies of entrepreneurs highlight the stories of tenacious individuals with limited resources but the burning desire to make a difference.

The alternative story is that political resources are distributed profoundly unequally. Few people have the resources to:

  • run for elected office
  • attend elite Universities, or find other ways to develop the kinds of personal networks that often relate to social background
  • develop the credibility built on a track record in a position of authority (such as in government or science).
  • be in the position to invest resources now, to secure future gains, or
  • be in an influential position to exploit windows of opportunity.

Therefore, when focusing on entrepreneurial policy analysis, we should encourage the development of a suite of useful skills, but not expect equal access to that development or the same payoff from entrepreneurial action.

See also:

Compare these skills with the ones we might associate with ‘systems thinking

If you want to see me say these depressing things with a big grin:

 

 

2 Comments

Filed under 750 word policy analysis, agenda setting, Evidence Based Policymaking (EBPM), public policy, Uncategorized

Policy Analysis in 750 Words: complex systems and systems thinking

This post forms one part of the Policy Analysis in 750 words series overview and connects to previous posts on complexity. The first 750 words tick along nicely, then there is a picture of a cat hanging in there baby to signal where it can all go wrong.

There are a million-and-one ways to describe systems and systems thinking. These terms are incredibly useful, but also at risk of meaning everything and therefore nothing (compare with planning and consultation).

Let’s explore how the distinction between policy studies and policy analysis can help us clarify the meaning of ‘complex systems’ and ‘systems thinking’ in policymaking.

For example, how might we close a potentially large gap between these two stories?

  1. Systems thinking in policy analysis.
  • Avoid the unintended consequences of too-narrow definitions of problems and processes (systems thinking, not simplistic thinking).
  • If we engage in systems thinking effectively, we can understand systems well enough to control, manage, or influence them.
  1. The study of complex policymaking systems.
  • Policy emerges from complex systems in the absence of: (a) central government control and often (b) policymaker awareness.
  • We need to acknowledge these limitations properly, to accept our limitations, and avoid the mechanistic language of ‘policy levers’ which exaggerate human or government control.

Six meanings of complex systems in policy and policymaking

Let’s begin by trying to clarify many meanings of complex system and relate them to systems thinking storylines.

For example, you will encounter three different meanings of complex system in this series alone, and each meaning presents different implications for systems thinking:

  1. A complex policymaking system

Policy outcomes seem to ‘emerge’ from policymaking systems in the absence of central government control. As such, we should rely less on central government driven targets (in favour of local discretion to adapt to environments), encourage trial-and-error learning, and rethink the ways in which we think about government ‘failure’ (see, for example, Hallsworth on ‘system stewardship’, and this thread)

  • Systems thinking is about learning and adapting to the limits to policymaker control.

  1. Complex policy problems

Dunn (2017:  73) describes the interdependent nature of problems:

Subjectively experienced problems – crime, poverty, unemployment, inflation, energy, pollution, health, security – cannot be decomposed into independent subsets without running the risk of producing an approximately right solution to the wrong problem. A key characteristic of systems of problems is that the whole is greater – that is, qualitatively different – than the simple sum of its parts” (contrast with Meltzer and Schwartz on creating a ‘boundary’ to make problems seem solveable).

  • Systems thinking is about addressing policy problems holistically.
  1. Complex policy mixes

What we call ‘policy’ is actually a collection of policy instruments. Their overall effect is ‘non-linear’, difficult to predict, and subject to emergent outcomes, rather than cumulative (compare with Lindblom’s hopes for incrementalist change).

This point is crucial to policy analysis: does it involve a rethink of all instruments, or merely add a new instrument to the pile?

  • Systems thinking is about anticipating the disproportionate effect of a new policy instrument.

These three meanings are joined by at least three more (from Munro and Cairney on energy systems):

  1. Socio-technical systems (Geels)

Used to explain the transition from unsustainable to sustainable energy systems.

  • Systems thinking is about identifying the role of new technologies, protected initially in a ‘niche’, and fostered by a supportive ‘social and political environment’.
  1. Socio-ecological systems (Ostrom)

Used to explain how and why policy actors might cooperate to manage finite resources.

  • Systems thinking is about identifying the conditions under which actors develop layers of rules to foster trust and cooperation.
  1. The metaphor of systems

Used by governments – rather loosely – to indicate an awareness of the interconnectedness of things.

  • Systems thinking is about projecting the sense that (a) policy and policymaking is complicated, but (b) governments can still look like they are in control.

Four more meanings of systems thinking

Now, let’s compare these storylines with a small sample of wider conceptions of systems thinking:

  1. The old way of establishing order from chaos

Based on the (now-diminished) faith in science and rational management techniques to control the natural world for human benefit (compare Hughes and Hughes on energy with Checkland on ‘hard’ v ‘soft’ systems approaches, then see What you need as an analyst versus policymaking reality and Radin on the old faith in rationalist governing systems).

  • Systems thinking was about the human ability to turn potential chaos into well-managed systems (such as ‘large technical systems’ to distribute energy)
  1. The new way of accepting complexity but seeking to make an impact

Based on the idea that we can identify ‘leverage points’, or the places that help us ‘intervene in a system’ (see Meadows then compare with Arnold and Wade).

  • Systems thinking is about the human ability to use a small shift in a system to produce profound changes in that system.
  1. A way to rethink cause-and-effect

Based on the idea that current research methods are too narrowly focused on linearity rather than the emergent properties of systems of behaviour (for example, Rutter et al on how to analyse the cumulative effect of public health interventions).

  • Systems thinking is about rethinking the ways in which governments, funders, or professions conduct policy-relevant research on social behaviour.
  1. A way of thinking about ourselves

Embrace the limits to human cognition, and accept that all understandings of complex systems are limited.

  • Systems thinking is about developing the ‘wisdom’ and ‘humility’ to accept our limited knowledge of the world.

hang-in-there-baby

 

How can we clarify systems thinking and use it effectively in policy analysis?

Now, imagine you are in a room of self-styled systems thinkers, and that no-one has yet suggested a brief conversation to establish what you all mean by systems thinking. I reckon you can make a quick visual distinction by seeing who looks optimistic.

I’ll be the morose-looking guy sitting in the corner, waiting to complain about ambiguity, so you would probably be better off sitting next to Luke Craven who still ‘believes in the power of systems thinking’.

If you can imagine some amalgam of these pessimistic/ optimistic positions, perhaps the conversation would go like this:

  1. Reasons to expect some useful collaboration.

Some of these 10 discussions seem to complement each other. For example:

  • We can use 3 and 9 to reject one narrow idea of ‘evidence-based policymaking’, in which the focus is on (a) using experimental methods to establish cause and effect in relation to one policy instrument, without showing (b) the overall impact on policy and outcomes (e.g. compare FNP with more general ‘families’ policy).
  • 1-3 and 10 might be about the need for policy analysts to show humility when seeking to understand and influence complex policy problems, solutions, and policymaking systems.

In other words, you could define systems thinking in relation to the need to rethink the ways in which we understand – and try to address – policy problems. If so, you can stop here and move on to the next post. There is no benefit to completing this post.

  1. Reasons to expect the same old frustrating discussions based on no-one defining terms well enough (collectively) to collaborate effectively (beyond using the same buzzwords).

Although all of these approaches use the language of complex systems and systems thinking, note some profound differences:

Holding on versus letting go.

  • Some are about intervening to take control of systems or, at least, make a disproportionate difference from a small change.
  • Some are about accepting our inability to understand, far less manage, these systems.

Talking about different systems.

  • Some are about managing policymaking systems, and others about social systems (or systems of policy problems), without making a clear connection between both endeavours.

For example, if you use approach 9 to rethink societal cause-and-effect, are you then going to pretend that you can use approach 7 to do something about it? Or, will our group have a difficult discussion about the greater likelihood of 6 (metaphorical policymaking) in the context of 1 (the inability of governments to control the policymaking systems we need to solve the problems raised by 9).

In that context, the reason that I am sitting in the corner, looking so morose, is that too much collective effort goes into (a) restating, over and over and over again, the potential benefits of systems thinking, leaving almost no time for (b) clarifying systems thinking well enough to move on to these profound differences in thinking. Systems thinking has not even helped us solve these problems with systems thinking.

See also:

Why systems thinkers and data scientists should work together to solve social challenges

3 Comments

Filed under 750 word policy analysis, Evidence Based Policymaking (EBPM), Prevention policy, public policy, UKERC

Policy Analysis in 750 Words: how much impact can you expect from your analysis?

This post forms one part of the Policy Analysis in 750 words series overview.

Throughout this series you may notice three different conceptions about the scope of policy analysis:

  1. ‘Ex ante’ (before the event) policy analysis. Focused primarily on defining a problem, and predicting the effect of solutions, to inform current choice (as described by Meltzer and Schwartz and Thissen and Walker).
  2. ‘Ex post’ (after the event) policy analysis. Focused primarily on monitoring and evaluating that choice, perhaps to inform future choice (as described famously by Weiss).
  3. Some combination of both, to treat policy analysis as a continuous (never-ending) process (as described by Dunn).

As usual, these are not hard-and-fast distinctions, but they help us clarify expectations in relation to different scenarios.

  1. The impact of old-school ex ante policy analysis

Radin provides a valuable historical discussion of policymaking with the following elements:

  • a small number of analysts, generally inside government (such as senior bureaucrats, scientific experts, and – in particular- economists),
  • giving technical or factual advice,
  • about policy formulation,
  • to policymakers at the heart of government,
  • on the assumption that policy problems would be solved via analysis and action.

This kind of image signals an expectation for high impact: policy analysts face low competition, enjoy a clearly defined and powerful audience, and their analysis is expected to feed directly into choice.

Radin goes on to describe a much different, modern policy environment: more competition, more analysts spread across and outside government, with a less obvious audience, and – even if there is a client – high uncertainty about where the analysis fits into the bigger picture.

Yet, the impetus to seek high and direct impact remains.

This combination of shifting conditions but unshifting hopes/ expectations helps explain a lot of the pragmatic forms of policy analysis you will see in this series, including:

  • Keep it catchy, gather data efficiently, tailor your solutions to your audience, and tell a good story (Bardach)
  • Speak with an audience in mind, highlight a well-defined problem and purpose, project authority, use the right form of communication, and focus on clarity, precision, conciseness, and credibility ( Smith)
  • Address your client’s question, by their chosen deadline, in a clear and concise way that they can understand (and communicate to others) quickly (Weimer and Vining)
  • Client-oriented advisors identify the beliefs of policymakers and anticipate the options worth researching (Mintrom)
  • Identify your client’s resources and motivation, such as how they seek to use your analysis, the format of analysis they favour (make it ‘concise’ and ‘digestible’), their deadline, and their ability to make or influence the policies you might suggest (Meltzer and Schwartz).
  • ‘Advise strategically’, to help a policymaker choose an effective solution within their political context (Thissen and Walker).
  • Focus on producing ‘policy-relevant knowledge’ by adapting to the evidence-demands of policymakers and rejecting a naïve attachment to ‘facts speaking for themselves’ or ‘knowledge for its own sake’ (Dunn).
  1. The impact of research and policy evaluation

Many of these recommendations are familiar to scientists and researchers, but generally in the context of far lower expectations about their likely impact, particularly if those expectations are informed by policy studies (compare Oliver & Cairney with Cairney & Oliver).

In that context, Weiss’ work is a key reference point. It gives us a menu of ways in which policymakers might use policy evaluation (and research evidence more widely):

  • to inform solutions to a problem identified by policymakers
  • as one of many sources of information used by policymakers, alongside ‘stakeholder’ advice and professional and service user experience
  • as a resource used selectively by politicians, with entrenched positions, to bolster their case
  • as a tool of government, to show it is acting (by setting up a scientific study), or to measure how well policy is working
  • as a source of ‘enlightenment’, shaping how people think over the long term (compare with this discussion of ‘evidence based policy’ versus ‘policy based evidence’).

In other words, researchers may have a role, but they struggle (a) to navigate the politics of policy analysis, (b) find the right time to act, and (c) to secure attention, in competition with many other policy actors.

  1. The potential for a form of continuous impact

Dunn suggests that the idea of ‘ex ante’ policy analysis is misleading, since policymaking is continuous, and evaluations of past choices inform current choices. Think of each policy analysis steps as ‘interdependent’, in which new knowledge to inform one step also informs the other four. For example, routine monitoring helps identify compliance with regulations, if resources and services reach ‘target groups’, if money is spent correctly, and if we can make a causal link between the policy solutions and outcomes. Its impact is often better seen as background information with intermittent impact.

Key conclusions to bear in mind

  1. The demand for information from policy analysts may be disproportionately high when policymakers pay attention to a problem, and disproportionately low when they feel that they have addressed it.
  2. Common advice for policy analysts and researchers often looks very similar: keep it concise, tailor it to your audience, make evidence ‘policy relevant’, and give advice (don’t sit on the fence). However, unless researchers are prepared to act quickly, to gather data efficiently (not comprehensively), to meet a tight brief for a client, they are not really in the impact business described by most policy analysis texts.
  3. A lot of routine, continuous, impact tends to occur out of the public spotlight, based on rules and expectations that most policy actors take for granted.

Further reading

See the Policy Analysis in 750 words series overview to continue reading on policy analysis.

See the ‘evidence-based policymaking’ page to continue reading on research impact.

ebpm pic

Bristol powerpoint: Paul Cairney Bristol EBPM January 2020

2 Comments

Filed under 750 word policy analysis, Evidence Based Policymaking (EBPM), Policy learning and transfer, public policy

Policy Analysis in 750 Words: what you need as an analyst versus policymaking reality

This post forms one part of the Policy Analysis in 750 words series overview. Note for the eagle eyed: you are not about to experience déjà vu. I’m just using the same introduction.

When describing ‘the policy sciences’, Lasswell distinguishes between:

  1. ‘knowledge of the policy process’, to foster policy studies (the analysis of policy)
  2. ‘knowledge in the process’, to foster policy analysis (analysis for policy)

The lines between each approach are blurry, and each element makes less sense without the other. However, the distinction is crucial to help us overcome the major confusion associated with this question:

Does policymaking proceed through a series of stages?

The short answer is no.

The longer answer is that you can find about 40 blog posts (of 500 and 1000 words) which compare (a) a stage-based model called the policy cycle, and (b) the many, many policy concepts and theories that describe a far messier collection of policy processes.

cycle

In a nutshell, most policy theorists reject this image because it oversimplifies a complex policymaking system. The image provides a great way to introduce policy studies, and serves a political purpose, but it does more harm than good:

  1. Descriptively, it is profoundly inaccurate (unless you imagine thousands of policy cycles interacting with each other to produce less orderly behaviour and less predictable outputs).
  2. Prescriptively, it gives you rotten advice about the nature of your policymaking task (for more on these points, see this chapter, article, article, and series).

Why does the stages/ policy cycle image persist? Two relevant explanations

 

  1. It arose from a misunderstanding in policy studies

In another nutshell, Chris Weible and I argue (in a secret paper) that the stages approach represents a good idea gone wrong:

  • If you trace it back to its origins, you will find Lasswell’s description of decision functions: intelligence, recommendation, prescription, invocation, application, appraisal and termination.
  • These functions correspond reasonably well to a policy cycle’s stages: agenda setting, formulation, legitimation, implementation, evaluation, and maintenance, succession or termination.
  • However, Lasswell was imagining functional requirements, while the cycle seems to describe actual stages.

In other words, if you take Lasswell’s list of what policy analysts/ policymakers need to do, multiple it by the number of actors (spread across many organisations or venues) trying to do it, then you get the multi-centric policy processes described by modern theories. If, instead, you strip all that activity down into a single cycle, you get the wrong idea.

  1. It is a functional requirement of policy analysis

This description should seem familiar, because the classic policy analysis texts appear to describe a similar series of required steps, such as:

  1. define the problem
  2. identify potential solutions
  3. choose the criteria to compare them
  4. evaluate them in relation to their predicted outcomes
  5. recommend a solution
  6. monitor its effects
  7. evaluate past policy to inform current policy.

However, these texts also provide a heavy dose of caution about your ability to perform these steps (compare Bardach, Dunn, Meltzer and Schwartz, Mintrom, Thissen and Walker, Weimer and Vining)

In addition, studies of policy analysis in action suggest that:

  • an individual analyst’s need for simple steps, to turn policymaking complexity into useful heuristics and pragmatic strategies,

should not be confused with

What you need versus what you can expect

Overall, this discussion of policy studies and policy analysis reminds us of a major difference between:

  1. Functional requirements. What you need from policymaking systems, to (a) manage your task (the 5-8 step policy analysis) and (b) understand and engage in policy processes (the simple policy cycle).
  2. Actual processes and outcomes. What policy concepts and theories tell us about bounded rationality (which limit the comprehensiveness of your analysis) and policymaking complexity (which undermines your understanding and engagement in policy processes).

Of course, I am not about to provide you with a solution to these problems.

Still, this discussion should help you worry a little bit less about the circular arguments you will find in key texts: here are some simple policy analysis steps, but policymaking is not as ‘rational’ as the steps suggest, but (unless you can think of an alternative) there is still value in the steps, and so on.

See also:

The New Policy Sciences

2 Comments

Filed under 750 word policy analysis, agenda setting, public policy

Policy Analysis in 750 Words: Defining policy problems and choosing solutions

This post forms one part of the Policy Analysis in 750 words series overview.

When describing ‘the policy sciences’, Lasswell distinguishes between:

  1. ‘knowledge of the policy process’, to foster policy studies (the analysis of policy)
  2. ‘knowledge in the process’, to foster policy analysis (analysis for policy)

The idea is that both elements are analytically separable but mutually informative: policy analysis is crucial to solving real policy problems, policy studies inform the feasibility of analysis, the study of policy analysts informs policy studies, and so on.

Both elements focus on similar questions – such as What is policy? – and explore their descriptive (what do policy actors do?) and prescriptive (what should they do?) implications.

  1. What is the policy problem?

Policy studies tend to describe problem definition in relation to framing, narrative, social construction, power, and agenda setting.

Actors exercise power to generate attention for their preferred interpretation, and minimise attention to alternative frames (to help foster or undermine policy change, or translate their beliefs into policy).

Policy studies incorporate insights from psychology to understand (a) how policymakers might combine cognition and emotion to understand problems, and therefore (b) how to communicate effectively when presenting policy analysis.

Policy studies focus on the power to reduce ambiguity rather than simply the provision of information to reduce uncertainty. In other words, the power to decide whose interpretation of policy problems counts, and therefore to decide what information is policy-relevant.

This (unequal) competition takes place within a policy process over which no actor has full knowledge or control.

The classic 5-8 step policy analysis texts focus on how to define policy problems well, but they vary somewhat in their definition of doing it well (see also C.Smith):

  • Bardach recommends using rhetoric and eye-catching data to generate attention
  • Weimer and Vining and Mintrom recommend beginning with your client’s ‘diagnosis’, placing it in a wider perspective to help analyse it critically, and asking yourself how else you might define it (see also Bacchi, Stone)
  • Meltzer and Schwartz and Dunn identify additional ways to contextualise your client’s definition, such as by generating a timeline to help ‘map’ causation or using ‘problem-structuring methods’ to compare definitions and avoid making too many assumptions on a problem’s cause.
  • Thissen and Walker compare ‘rational’ and ‘argumentative’ approaches, treating problem definition as something to be measured scientifically or established rhetorically (see also Riker).

These approaches compare with more critical accounts that emphasise the role of power and politics to determine whose knowledge is relevant (L.T.Smith) and whose problem definition counts (Bacchi, Stone). Indeed, Bacchi and Stone provide a crucial bridge between policy analysis and policy studies by reflecting on what policy analysts do and why.

  1. What is the policy solution?

In policy studies, it is common to identify counterintuitive or confusing aspects of policy processes, including:

  • Few studies suggest that policy responses actually solve problems (and many highlight their potential to exacerbate them). Rather, ‘policy solutions’ is shorthand for proposed or alleged solutions.
  • Problem definition often sets the agenda for the production of ‘solutions’, but note the phrase solutions chasing problems (when actors have their ‘pet’ solutions ready, and they seek opportunities to promote them).

Policy studies: problem definition informs the feasibility and success of solutions

Generally speaking, to define the problem is to influence assessments of the feasibility of solutions:

  • Technical feasibility. Will they work as intended, given the alleged severity and cause of the problem?
  • Political feasibility. Will they receive sufficient support, given the ways in which key policy actors weigh up the costs and benefits of action?

Policy studies highlight the inextricable connection between technical and political feasibility. Put simply, (a) a ‘technocratic’ choice about the ‘optimality’ of a solution is useless without considering who will support its adoption, and (b) some types of solution will always be a hard sell, no matter their alleged effectiveness (Box 2.3 below).

In that context, policy studies ask: what types of policy tools or instruments are actually used, and how does their use contribute to policy change? Measures include the size, substance, speed, and direction of policy change.

box 2.3 2nd ed UPP

In turn, problem definition informs: the ways in which actors will frame any evaluation of policy success, and the policy-relevance of the evidence to evaluate solutions. Simple examples include:

  • If you define tobacco in relation to: (a) its economic benefits, or (b) a global public health epidemic, evaluations relate to (a) export and taxation revenues, or (b) reductions in smoking in the population.
  • If you define ‘fracking’ in relation to: (a) seeking more benefits than costs, or (b) minimising environmental damage and climate change, evaluations relate to (a) factors such as revenue and effective regulation, or simply (b) how little it takes place.

Policy analysis: recognising and pushing boundaries

Policy analysis texts tend to accommodate these insights when giving advice:

  • Bardach recommends identifying solutions that your audience might consider, perhaps providing a range of options on a notional spectrum of acceptability.
  • Smith highlights the value of ‘precedent’, or relating potential solutions to previous strategies.
  • Weimer and Vining identify the importance of ‘a professional mind-set’ that may be more important than perfecting ‘technical skills’
  • Mintrom notes that some solutions are easier to sell than others
  • Meltzer and Schwartz describe the benefits of making a preliminary recommendation to inform an iterative process, drawing feedback from clients and stakeholder groups
  • Dunn warns against too-narrow forms of ‘evidence based’ analysis which undermine a researcher’s ability to adapt well to the evidence-demands of policymakers
  • Thissen and Walker relate solution feasibility to a wide range of policy analysis ‘styles’

Still, note the difference in emphasis.

Policy analysis education/ training may be about developing the technical skills to widen definitions and apply many criteria to compare solutions.

Policy studies suggest that problem definition and a search for solutions takes place in an environment where many actors apply a much narrower lens and are not interested in debates on many possibilities (particularly if they begin with a solution).

I have exaggerated this distinction between each element, but it is worth considering the repeated interaction between them in practice: politics and policymaking provide boundaries for policy analysis, analysis could change those boundaries, and policy studies help us reflect on the impact of analysts.

I’ll take a quick break, then discuss how this conclusion relates to the idea of ‘entrepreneurial’ policy analysis.

Further reading

Understanding Public Policy (2020: 28) describes the difference between governments paying for and actually using the ‘tools of policy formulation’. To explore this point, see ‘The use and non-use of policy appraisal tools in public policy making‘ and The Tools of Policy Formulation.

p28 upp 2nd ed policy tools

2 Comments

Filed under 750 word policy analysis, agenda setting, Evidence Based Policymaking (EBPM), public policy

Policy Analysis in 750 Words: What can you realistically expect policymakers to do?

This post forms one part of the Policy Analysis in 750 words series overview.

One aim of this series is to combine insights from policy research (1000, 500) and policy analysis texts.

In this case, modern theories of the policy process help you identify your audience and their capacity to follow your advice. This simple insight may have a profound impact on the advice you give.

Policy analysis for an ideal-type world

For our purposes, an ideal-type is an abstract idea, which highlights hypothetical features of the world, to compare with ‘real world’ descriptions. It need not be an ideal to which we aspire. For example, comprehensive rationality describes the ideal type, and bounded rationality describes the ‘real world’ limitations to the ways in which humans and organisations process information.

 

Imagine writing policy analysis in the ideal-type world of a single powerful ‘comprehensively rational’ policymaker at the heart of government, making policy via an orderly policy cycle.

Your audience would be easy to identify, your analysis would be relatively simple, and you would not need to worry about what happens after you make a recommendation for policy change.

You could adopt a simple 5-8 step policy analysis method, use widely-used tools such as cost-benefit analysis to compare solutions, and know where the results would feed into the policy process.

I have perhaps over-egged this ideal-type pudding, but I think a lot of traditional policy analyses tapped into this basic idea and focused more on the science of analysis than the political and policymaking context in which it takes place (see Radin and Brans, Geva-May, and Howlett).

Policy analysis for the real world

Then imagine a far messier and less predictable world in which the nature of the policy issue is highly contestedresponsibility for policy is unclear, and no single ‘centre’ has the power to turn a recommendation into an outcome.

This image is a key feature of policy process theories, which describe:

  • Many policymakers and influencers spread across many levels and types of government (as the venues in which authoritative choice takes place). Consequently, it is not a straightforward task to identify and know your audience, particularly if the problem you seek to solve requires a combination of policy instruments controlled by different actors.
  • Each venue resembles an institution driven by formal and informal rules. Formal rules are written-down or widely-known. Informal rules are unwritten, difficult to understand, and may not even be understood in the same way by participants. Consequently, it is difficult to know if your solution will be a good fit with the standard operating procedures of organisations (and therefore if it is politically feasible or too challenging).
  • Policymakers and influencers operate in ‘subsystems’, forming networks built on resources such as trust or coalitions based on shared beliefs. Effective policy analysis may require you to engage with – or become part of – such networks, to allow you to understand the unwritten rules of the game and encourage your audience to trust the messenger. In some cases, the rules relate to your willingness to accept current losses for future gains, to accept the limited impact of your analysis now in the hope of acceptance at the next opportunity.
  • Actors relate their analysis to shared understandings of the world – how it is, and how it should be – which are often so well-established as to be taken for granted. Common terms include paradigms, hegemons, core beliefs, and monopolies of understandings. These dominant frames of reference give meaning to your policy solution. They prompt you to couch your solutions in terms of, for example, a strong attachment to evidence-based cases in public health, value for money in treasury departments, or with regard to core principles such as liberalism or socialism in different political systems.
  • Your solutions relate to socioeconomic context and the events that seem (a) impossible to ignore and (b) out of the control of policymakers. Such factors range from a political system’s geography, demography, social attitudes, and economy, while events can be routine elections or unexpected crises.

What would you recommend under these conditions? Rethinking 5-step analysis

There is a large gap between policymakers’ (a) formal responsibilities versus (b) actual control of policy processes and outcomes. Even the most sophisticated ‘evidence based’ analysis of a policy problem will fall flat if uninformed by such analyses of the policy process. Further, the terms of your cost-benefit analysis will be highly contested (at least until there is agreement on what the problem is, and how you would measure the success of a solution).

Modern policy analysis texts try to incorporate such insights from policy theories while maintaining a focus on 5-8 steps. For example:

  • Meltzer and Schwartz contrast their ‘flexible’ and ‘iterative’ approach with a too- rigid ‘rationalistic approach’.
  • Bardachand Dunn emphasise the value of political pragmatism and the ‘art and craft’ of policy analysis.
  • Weimer and Vininginvest 200 pages in economic analyses of markets and government, often highlighting a gap between (a) our ability to model and predict economic and social behaviour, and (b) what actually happens when governments intervene.
  • Mintrom invites you to see yourself as a policy entrepreneur, to highlight the value of of ‘positive thinking’, creativity, deliberation, and leadership, and perhaps seek ‘windows of opportunity’ to encourage new solutions. Alternatively, a general awareness of the unpredictability of events can prompt you to be modest in your claims, since the policymaking environment may be more important (than your solution) to outcomes.
  • Thissen and Walker focus more on a range of possible roles than a rigid 5-step process.

Beyond 5-step policy analysis

  1. Compare these pragmatic, client-orientated, and communicative models with the questioning, storytelling, and decolonizing approaches by Bacchi, Stone, and L.T. Smith.
  • The latter encourage us to examine more closely the politics of policy processes, including the importance of framing, narrative, and the social construction of target populations to problem definition and policy design.
  • Without this wider perspective, we are focusing on policy analysis as a process rather than considering the political context in which analysts use it.
  1. Additional posts on entrepreneurs and ‘systems thinking’ [to be added] encourage us to reflect on the limits to policy analysis in multi-centric policymaking systems.

 

 

1 Comment

Filed under 750 word policy analysis, agenda setting, Evidence Based Policymaking (EBPM), public policy