Policy Analysis in 750 Words: How to communicate effectively with policymakers

This post forms one part of the Policy Analysis in 750 words series overview. The title comes from this article by Cairney and Kwiatkowski on ‘psychology based policy studies’.

One aim of this series is to combine insights from policy research (1000, 500) and policy analysis texts. How might we combine insights to think about effective communication?

1. Insights from policy analysis texts

Most texts in this series relate communication to understanding your audience (or client) and the political context. Your audience has limited attention or time to consider problems. They may have a good antennae for the political feasibility of any solution, but less knowledge of (or interest in) the technical details. In that context, your aim is to help them treat the problem as worthy of their energy (e.g. as urgent and important) and the solution as doable. Examples include:

• Bardach: communicating with a client requires coherence, clarity, brevity, and minimal jargon.
• Dunn: argumentation involves defining the size and urgency of a problem, assessing the claims made for each solution, synthesising information from many sources into a concise and coherent summary, and tailoring reports to your audience.
• Smith: your audience makes a quick judgement on whether or not to read your analysis. Ask yourself questions including: how do I frame the problem to make it relevant, what should my audience learn, and how does each solution relate to what has been done before? Maximise interest by keeping communication concise, polite, and tailored to a policymaker’s values and interests.

2. Insights from studies of policymaker psychology

These insights emerged from the study of bounded rationality: policymakers do not have the time, resources, or cognitive ability to consider all information, possibilities, solutions, or consequences of their actions. They use two types of informational shortcut associated with concepts such as cognition and emotion, thinking ‘fast and slow’, ‘fast and frugal heuristics’, or, if you like more provocative terms:

• ‘Rational’ shortcuts. Goal-oriented reasoning based on prioritizing trusted sources of information.
• ‘Irrational’ shortcuts. Emotional thinking, or thought fuelled by gut feelings, deeply held beliefs, or habits.

We can use such distinctions to examine the role of evidence-informed communication, to reduce:

• Uncertainty, or a lack of policy-relevant knowledge. Focus on generating ‘good’ evidence and concise communication as you collate and synthesise information.
• Ambiguity, or the ability to entertain more than one interpretation of a policy problem. Focus on argumentation and framing as you try to maximise attention to (a) one way of defining a problem, and (b) your preferred solution.

Many policy theories describe the latter, in which actors: combine facts with emotional appeals, appeal to people who share their beliefs, tell stories to appeal to the biases of their audience, and exploit dominant ways of thinking or social stereotypes to generate attention and support. These possibilities produce ethical dilemmas for policy analysts.

3. Insights from studies of complex policymaking environments

None of this advice matters if it is untethered from reality.

Policy analysis texts focus on political reality to note that even a perfectly communicated solution is worthless if technically feasible but politically unfeasible.

Policy process texts focus on policymaking reality: showing that ideal-types such as the policy cycle do not guide real-world action, and describing more accurate ways to guide policy analysts.

For example, they help us rethink the ‘know your audience’ mantra by:

Identifying a tendency for most policy to be processed in policy communities or subsystems:

• Elected policymakers prioritise some issues and ignore the rest, and pay attention to some information and ignore the rest.
• Note the importance of ‘serial’ and ‘parallel’ processing: policymakers may process issues one at a time, but political systems process many issues in many different venues (or multiple ‘centres’).
• This dynamic produces many different audiences spread across policymaking systems: blurring boundaries between those who make and influence policy; following their own rules, norms, or dominant ways to understand problems (and assumptions about whose framing and whose knowledge matters); and, responding to different policy conditions or events (which might create different windows of opportunity for action).

Showing that many policymaking ‘centres’ create the instruments that produce policy change

Gone are the mythical days of a small number of analysts communicating to a single core executive (and of the heroic researcher changing the world by speaking truth to power). Instead, we have many analysts engaging with many centres, creating a need to not only (a) tailor arguments to different audiences, but also (b) develop wider analytical skills (such as to foster collaboration and the use of ‘design principles’).

How to communicate effectively with policymakers

In that context, we argue that effective communication requires analysts to:

1. Understand your audience and tailor your response (using insights from psychology)

2. Identify ‘windows of opportunity’ for influence (while noting that these windows are outside of anyone’s control)

3. Engage with real world policymaking rather than waiting for a ‘rational’ and orderly process to appear (using insights from policy studies).

See also:

Why don’t policymakers listen to your evidence?

3. How to combine principles on ‘good evidence’, ‘good governance’, and ‘good practice’

Entrepreneurial policy analysis

Policy Analysis in 750 Words: policy analysis for marginalized groups in racialized political systems

Note: this post forms one part of the Policy Analysis in 750 words series overview.

For me, this story begins with a tweet by Professor Jamila Michener, about a new essay by Dr Fabienne Doucet, ‘Centering the Margins: (Re)defining Useful Research Evidence Through Critical Perspectives’:

Incisive essay from @bailabomba on studying the use of research evidence through critical perspectives that center the marginalized. There is so too much good stuff in here to summarize via twitter (you should just read it). But let me point out a few things that resonated (1/n) https://t.co/nIahyIjwBo

— Jamila Michener (@povertyscholar) December 17, 2019

Research and policy analysis for marginalized groups

For Doucet (2019: 1), it begins by describing the William T. Grant Foundation’s focus on improving the ‘use of research evidence’ (URE), and the key questions that we should ask when improving URE:

1. For what purposes do policymakers find evidence useful?

Examples include to: inform a definition of problems and solutions, foster practitioner learning, support an existing political position, or impose programmes backed by evidence (compare with How much impact can you expect from your analysis?).

2.   Who decides what to use, and what is useful?

For example, usefulness could be defined by the researchers providing evidence, the policymakers using it, the stakeholders involved in coproduction, or the people affected by research and policy (compare with Bacchi, Stone and Who should be involved in the process of policy analysis?).

3. How do critical theories inform these questions? (compare with T. Smith)

First, they remind us that so-called ‘rational’ policy processes have incorporated research evidence to help:

‘maintain power hierarchies and accept social inequity as a given. Indeed, research has been historically and contemporaneously (mis)used to justify a range of social harms from enslavement, colonial conquest, and genocide, to high-stakes testing, disproportionality in child welfare services, and “broken windows” policing’ (Doucet, 2019: 2)

Second, they help us redefine usefulness in relation to:

‘how well research evidence communicates the lived experiences of marginalized groups so that the understanding of the problem and its response is more likely to be impactful to the community in the ways the community itself would want’ (Doucet, 2019: 3)

In that context, potential responses include to:

1. Recognise the ways in which research and policy combine to reproduce the subordination of social groups.

• General mechanisms include: the reproduction of the assumptions, norms, and rules that produce a disproportionate impact on social groups (compare with Social Construction and Policy Design).
• Specific mechanism include: judging marginalised groups harshly according to ‘Western, educated, industrialized, rich and democratic’ norms (‘WEIRD’)

2. Reject the idea that scientific research can be seen as objective or neutral (and that researchers are beyond reproach for their role in subordination).
3. Give proper recognition to ‘experiential knowledge’ and ‘transdiciplinary approaches’ to knowledge production, rather than privileging scientific knowledge.
4. Commit to social justice, to help ‘eliminate oppressions and to emancipate and empower marginalized groups’, such as by disrupting ‘the policies and practices that disproportionately harm marginalized groups’ (2019: 5-7)
5. Develop strategies to ‘center race’, ‘democratize’ research production, and ‘leverage’ transdisciplinary methods (including poetry, oral history and narrative, art, and discourse analysis – compare with Lorde) (2019: 10-22)

See also Doucet, F. (2021) ‘Identifying and Testing Strategies to Improve the Use of Antiracist Research Evidence through Critical Race Lenses‘

Policy analysis in a ‘racialized polity’

A key way to understand these processes is to use, and improve, policy theories to explain the dynamics and impacts of a racialized political system. For example, ‘policy feedback theory’ (PFT) draws on elements from historical institutionalism and SCPD to identify the rules, norms, and practices that reinforce subordination.

In particular, Michener’s (2019: 424) ‘Policy Feedback in a Racialized Polity’ develops a ‘racialized feedback framework (RFF)’ to help explain the ‘unrelenting force with which racism and White supremacy have pervaded social, economic, and political institutions in the United States’. Key mechanisms include (2019: 424-6):

1. ‘Channelling resources’, in which the rules, to distribute government resources, benefit some social groups and punish others.

• Examples include: privileging White populations in social security schemes and the design/ provision of education, and punishing Black populations disproportionately in prisons (2019: 428-32).
• These rules also influence the motivation of social groups to engage in politics to influence policy (some citizens are emboldened, others alienated).

2. ‘Generating interests’, in which ‘racial stratification’ is a key factor in the power of interest groups (and balance of power in them).
3. ‘Shaping interpretive schema’, in which race is a lens through which actors understand, interpret, and seek to solve policy problems.
4. The ways in which centralization (making policy at the federal level) or decentralization influence policy design.

• For example, the ‘historical record’ suggests that decentralization is more likely to ‘be a force of inequality than an incubator of power for people of color’ (2019: 433).

Insufficient attention to race and racism: what are the implications for policy analysis?

One potential consequence of this lack of attention to race, and the inequalities caused by racism in policy, is that we place too much faith in the vague idea of ‘pragmatic’ policy analysis.

Throughout the 750 words series, you will see me refer generally to the benefits of pragmatism:

• How much impact can you expect from your analysis? focuses on making sure that your client-oriented advice builds on your client’s timetable and definition of the problem.
• What you need as an analyst versus policymaking reality focuses on managing your expectations about your ability to change policy (and influence policymaking).
• What can you realistically expect policymakers to do? describes how the pragmatic policy analysis texts (e.g. Bardach and Dunn) emphasise the value of this approach (albeit while inviting you to compare their accounts with the questioning, storytelling, and decolonizing approaches by Bacchi, Stone, and T. Smith).

In that context, pragmatism relates to the idea that policy analysis consists of ‘art and craft’, in which analysts assess what is politically feasible if taking a low-risk client-oriented approach.

In this context, pragmatism may be read as a euphemism for conservatism and status quo protection.

In other words, other posts in the series warn against too-high expectations for entrepreneurial and systems thinking approaches to major policy change, but they should not be read as an excuse to reject ambitious plans for much-needed changes to policy and policy analysis (compare with Meltzer and Schwartz, who engage with this dilemma in client-oriented advice).

Connections to blog themes

This post connects well to:

• policy analysis posts on T. Smith, Bacchi, Stone, and Who should be involved in the process of policy analysis?
• 500/1000 Words posts on Power and Knowledge, Feminist Institutionalism, Social Construction and Policy Design, and Feminism, Postcolonialism, and Critical Policy Studies, and
• the ‘politics of evidence based policymaking’ series.

Policy Analysis in 750 Words: Reflecting on your role as a policy analyst

This post forms one part of the Policy Analysis in 750 words series overview.

One aim of this series is to combine insights from policy research (1000, 500) and policy analysis texts.

If we take key insights from policy theories seriously, we can use them to identify (a) the constraints to policy analytical capacity, and (b) the ways in which analysts might address them. I use the idea of policy analyst archetypes to compare a variety of possible responses.

Key constraints to policy analytical capacity

Terms like ‘bounded rationality’ highlight major limits on the ability of humans and organisations to process information.

• Humans use heuristics or cognitive shortcuts to process enough information to make choices, and institutions are the rules used by organisations to limit information processing.

Terms like policymaking ‘context’, ‘environments’, and multi-centric policymaking suggest that the policy process is beyond the limits of policymaker understanding and control.

• Policy actors need to find ways to act, with incomplete information about the problem they seek to solve and the likely impact of their ‘solution’.
• They gather information to help reduce uncertainty, but problem definition is really about exercising power to reduce ambiguity: select one way to interpret a problem (at the expense of most others), and limit therefore limit the relevance and feasibility of solutions.
• This context informs how actors might use the tools of policy analysis. Key texts in this series highlight the use of tools to establish technical feasibility (will it work as intended?), but policymakers also select tools for their political feasibility (who will support or oppose this measure?).

How might policy analysts address these constraints ethically?

Most policy analysis texts (in this series) consider the role of professional ethics and values during the production of policy analysis. However, they also point out that there is not a clearly defined profession and associated code of conduct (e.g. see Adachi). In that context, let us begin with some questions about the purpose of policy analysis and your potential role:

1. Is your primary role to serve individual clients or some notion of the ‘public good’?
2. Should you maximise your role as an individual or play your part in a wider profession?
3. What is the balance between the potential benefits of individual ‘entrepreneurship’ and collective ‘co-productive’ processes?
4. Which policy analysis techniques should you prioritise?
5. What forms of knowledge and evidence count in policy analysis?
6. What does it mean to communicate policy analysis responsibly?
7. Should you provide a clear recommendation or encourage reflection?

 

Policy analysis archetypes: pragmatists, entrepreneurs, manipulators, storytellers, and decolonisers

In that context, I have created a story of policy analysis archetypes to identify the elements that each text emphasises.

The pragmatic policy analyst

• Bardach provides the classic simple, workable, 8-step system to present policy analysis to policymakers while subject to time and resource-pressed political conditions.
• Dunn also uses Wildavsky’s famous phrase ‘art and craft’ to suggest that scientific and ‘rational’ methods can only take us so far.

The professional, client–oriented policy analyst

• Weimer and Vining provide a similar 7-step client-focused system, but incorporating a greater focus on professional development and economic techniques (such as cost-benefit-analysis) to emphasise a particular form of professional analyst.
• Meltzer and Schwartz also focus on advice to clients, but with a greater emphasis on a wide variety of methods or techniques (including service design) to encourage the co-design of policy analysis with clients.

The communicative policy analyst

•  C. Smith focuses on how to write and communicate policy analysis to clients in a political context.
• Compare with Spiegelhalter and Gigerenzer on how to communicate responsibly when describing uncertainty, probability, and risk.

The manipulative policy analyst.

• Riker helps us understand the relationship between two aspects of agenda setting: the rules/ procedures to make choice, and the framing of policy problems and solutions.

The entrepreneurial policy analyst

• Mintrom shows how to combine insights from studies of policy entrepreneurship and policy analysis, to emphasise the benefits of collaboration and creativity.

The questioning policy analyst

• Bacchi  analyses the wider context in which people give and use such advice, to identify the emancipatory role of analysis and encourage policy analysts to challenge dominant social constructions of problems and populations.

The storytelling policy analyst

• Stone identifies the ways in which people use storytelling and argumentation techniques to define problems and justify solutions. This process is about politics and power, not objectivity and optimal solutions.

The decolonizing policy analyst.

• L.T. Smith does not describe policy analysis directly, but shows how the ‘decolonization of research methods’ can inform the generation and use of knowledge.
• Compare with Hindess on the ways in which knowledge-based hierarchies rely on an untenable, circular logic.
• Compare with Michener’s thread, discussing Doucet’s new essay on (a) the role of power and knowledge in limiting (b) the ways in which we gather evidence to analyse policy problems.

Incisive essay from @bailabomba on studying the use of research evidence through critical perspectives that center the marginalized. There is so too much good stuff in here to summarize via twitter (you should just read it). But let me point out a few things that resonated (1/n) https://t.co/nIahyIjwBo

— Jamila Michener (@povertyscholar) December 17, 2019

Using archetypes to define the problem of policy analysis

Studies of the field (e.g. Radin plus Brans, Geva-May, and Howlett) suggest that there are many ways to do policy analysis. Further, as Thissen and Walker describe, such roles are not mutually exclusive, your views on their relative value could change throughout the process of analysis, and you could perform many of these roles.

Further, each text describes multiple roles, and some seem clustered together:

• pragmatic, client-orientated, and communicative could sum-up the traditional 5-8 step approaches, while
• questioning, storytelling, and decolonizing could sum up an important (‘critical’) challenge to narrow ways of thinking about policy analysis and the use of information.

Still, the emphasis matters.

Each text is setting an agenda or defining the problem of policy analysis more-or-less in relation to these roles. Put simply, the more you are reading about economic theory and method, the less you are reading about dominance and manipulation.

How can you read further?

Michener’s ‘Policy Feedback in a Racialized Polity’ connects to studies of historical institutionalism, and reminds us to use insights from policy theories to identify the context for policy analysis.

I have co-authored a lot about uncertainty/ ambiguity in relation to ‘evidence based policymaking’, including:

• Fostering Evidence-informed Policy Making: Uncertainty Versus Ambiguity(PDF)
• To Bridge the Divide between Evidence and Policy: Reduce Ambiguity as Much as Uncertainty
• How to communicate effectively with policymakers: combine insights from psychology and policy studies

See also The new policy sciences for a discussion of how these issues inform Lasswell’s original vision for the policy sciences (combining the analysis of and for policy).

Policy Analysis in 750 words: Deborah Stone (2012) Policy Paradox

Please see the Policy Analysis in 750 words series overview before reading the summary. This post is 750 words plus a bonus 750 words plus some further reading that doesn’t count in the word count even though it does.

Deborah Stone (2012) Policy Paradox: The Art of Political Decision Making 3^rd edition (Norton)

‘Whether you are a policy analyst, a policy researcher, a policy advocate, a policy maker, or an engaged citizen, my hope for Policy Paradox is that it helps you to go beyond your job description and the tasks you are given – to think hard about your own core values, to deliberate with others, and to make the world a better place’ (Stone, 2012: 15)

Stone (2012: 379-85) rejects the image of policy analysis as a ‘rationalist’ project, driven by scientific and technical rules, and separable from politics. Rather, every policy analyst’s choice is a political choice – to define a problem and solution, and in doing so choosing how to categorise people and behaviour – backed by strategic persuasion and storytelling.

The Policy Paradox: people entertain multiple, contradictory, beliefs and aims

Stone (2012: 2-3) describes the ways in which policy actors compete to define policy problems and public policy responses. The ‘paradox’ is that it is possible to define the same policies in contradictory ways.

‘Paradoxes are nothing but trouble. They violate the most elementary principle of logic: something can’t be two different things at once. Two contradictory interpretations can’t both be true. A paradox is just such an impossible situation, and political life is full of them’ (Stone, 2012: 2).

This paradox does not refer simply to a competition between different actors to define policy problems and the success or failure of solutions. Rather:

• The same actor can entertain very different ways to understand problems, and can juggle many criteria to decide that a policy outcome was a success and a failure (2012: 3).
• Surveys of the same population can report contradictory views – encouraging a specific policy response and its complete opposite – when asked different questions in the same poll (2012: 4; compare with Riker)

Policy analysts: you don’t solve the Policy Paradox with a ‘rationality project’

Like many posts in this series (Smith, Bacchi, Hindess), Stone (2010: 9-11) rejects the misguided notion of objective scientists using scientific methods to produce one correct answer (compare with Spiegelhalter and Weimer & Vining). A policy paradox cannot be solved by ‘rational, analytical, and scientific methods’ because:

• We can produce scientific evidence to reduce uncertainty, but not ambiguity.
• We can seek order and clarity in analysis and communication, but should not expect it in political systems (2012: 10).
• 5-step policy analysis (identify objectives, identify alternatives, predict their effects, evaluate alternatives, choose) is pervasive, but it ‘ignores our emotional feelings and moral intuitions’ (2012: 11).
• A policy cycle model does not explain why solutions often chase problems, or the continuous struggle over ideas even when a cycle is ostensibly complete (2012: 12-13)

Further, Stone (2012: 10-11) rejects the over-reliance, in policy analysis, on the misleading claim that:

• policymakers are engaging primarily with markets rather than communities (see 2012: 35 on the comparison between a ‘market model’ and ‘polis model’),
• economic models can sum up political life, and
• cost-benefit-analysis can reduce a complex problem into the sum of individual preferences using a single unambiguous measure.

Rather, many factors undermine such simplicity:

1. People do not simply act in their own individual interest. Nor can they rank-order their preferences in a straightforward manner according to their values and self-interest.

• Instead, they maintain a contradictory mix of objectives, which can change according to context and their way of thinking – combining cognition and emotion – when processing information (2012: 12; 30-4).

2. People are social actors. Politics is characterised by ‘a model of community where individuals live in a dense web of relationships, dependencies, and loyalties’ and exercise power with reference to ideas as much as material interests (2012: 10; 20-36; compare with Ostrom, more Ostrom, and Lubell; and see Sousa on contestation).
3. Morals and emotions matter. If people juggle contradictory aims and measures of success, then a story infused with ‘metaphor and analogy’, and appealing to values and emotions, prompts people ‘to see a situation as one thing rather than another’ and therefore draw attention to one aim at the expense of the others (2012: 11; compare with Gigerenzer).

Policy analysis reconsidered: the ambiguity of values and policy goals

Stone (2012: 14) identifies the ambiguity of the criteria for success used in 5-step policy analyses. They do not form part of a solely technical or apolitical process to identify trade-offs between well-defined goals (compare Bardach, Weimer and Vining, and Mintrom). Rather, ‘behind every policy issue lurks a contest over conflicting, though equally plausible, conceptions of the same abstract goal or value’ (2012: 14). Examples of competing interpretations of valence issues include definitions of:

1. Equity, according to: (a) which groups should be included, how to assess merit, how to identify key social groups, if we should rank populations within social groups, how to define need and account for different people placing different values on a good or service, (b) which method of distribution to use (competition, lottery, election), and (c) how to balance individual, communal, and state-based interventions (2012: 39-62).
2. Efficiency, to use the least resources to produce the same objective, according to: (a) who determines the main goal and how to balance multiple objectives, (a) who benefits from such actions, and (c) how to define resources while balancing equity and efficiency – for example, does a public sector job and a social security payment represent a sunk cost to the state or a social investment in people? (2012: 63-84).
3. Welfare or Need, according to factors including (a) the material and symbolic value of goods, (b) short term support versus a long term investment in people, (c) measures of absolute poverty or relative inequality, and (d) debates on ‘moral hazard’ or the effect of social security on individual motivation (2012: 85-106)
4. Liberty, according to (a) a general balancing of freedom from coercion and freedom from the harm caused by others, (b) debates on individual and state responsibilities, and (c) decisions on whose behaviour to change to reduce harm to what populations (2012: 107-28)
5. Security, according to (a) our ability to measure risk scientifically (see Spiegelhalter and Gigerenzer), (b) perceptions of threat and experiences of harm, (c) debates on how much risk to safety to tolerate before intervening, (d) who to target and imprison, and (e) the effect of surveillance on perceptions of democracy (2012: 129-53).

Policy analysis as storytelling for collective action

Actors use policy-relevant stories to influence the ways in which their audience understands (a) the nature of policy problems and feasibility of solutions, within (b) a wider context of policymaking in which people contest the proper balance between state, community, and market action. Stories can influence key aspects of collective action, including:

1. Defining interests and mobilising actors, by drawing attention to – and framing – issues with reference to an imagined social group and its competition (e.g. the people versus the elite; the strivers versus the skivers) (2012: 229-47)
2. Making decisions, by framing problems and solutions (2012: 248-68). Stone (2012: 260) contrasts the ‘rational-analytic model’ with real-world processes in which actors deliberately frame issues ambiguously, shift goals, keep feasible solutions off the agenda, and manipulate analyses to make their preferred solution seem the most efficient and popular.
3. Defining the role and intended impact of policies, such as when balancing punishments versus incentives to change behaviour, or individual versus collective behaviour (2012: 271-88).
4. Setting and enforcing rules (see institutions), in a complex policymaking system where a multiplicity of rules interact to produce uncertain outcomes, and a powerful narrative can draw attention to the need to enforce some rules at the expense of others (2012: 289-310).
5. Persuasion, drawing on reason, facts, and indoctrination. Stone (2012: 311-30) highlights the context in which actors construct stories to persuade: people engage emotionally with information, people take certain situations for granted even though they produce unequal outcomes, facts are socially constructed, and there is unequal access to resources – held in particular by government and business – to gather and disseminate evidence.
6. Defining human and legal rights, when (a) there are multiple, ambiguous, and intersecting rights (in relation to their source, enforcement, and the populations they serve) (b) actors compete to make sure that theirs are enforced, (c) inevitably at the expense of others, because the enforcement of rights requires a disproportionate share of limited resources (such as policymaker attention and court time) (2012: 331-53)
7. Influencing debate on the powers of each potential policymaking venue – in relation to factors including (a) the legitimate role of the state in market, community, family, and individual life, (b) how to select leaders, (c) the distribution of power between levels and types of government – and who to hold to account for policy outcomes (2012: 354-77).

Key elements of storytelling include:

1. Symbols, which sum up an issue or an action in a single picture or word (2012:157-8)
2. Characters, such as heroes or villain, who symbolise the cause of a problem or source of solution (2012:159)
3. Narrative arcs, such as a battle by your hero to overcome adversity (2012:160-8)
4. Synecdoche, to highlight one example of an alleged problem to sum up its whole (2012: 168-71; compare the ‘welfare queen’ example with SCPD)
5. Metaphor, to create an association between a problem and something relatable, such as a virus or disease, a natural occurrence (e.g. earthquake), something broken, something about to burst if overburdened, or war (2012: 171-78; e.g. is crime a virus or a beast?)
6. Ambiguity, to give people different reasons to support the same thing (2012: 178-82)
7. Using numbers to tell a story, based on political choices about how to: categorise people and practices, select the measures to use, interpret the figures to evaluate or predict the results, project the sense that complex problems can be reduced to numbers, and assign authority to the counters (2012:183-205; compare with Speigelhalter)
8. Assigning Causation, in relation to categories including accidental or natural, ‘mechanical’ or automatic (or in relation to institutions or systems), and human-guided causes that have intended or unintended consequences (such as malicious intent versus recklessness)

• ‘Causal strategies’ include to: emphasise a natural versus human cause, relate it to ‘bad apples’ rather than systemic failure, and suggest that the problem was too complex to anticipate or influence
• Actors use these arguments to influence rules, assign blame, identify ‘fixers’, and generate alliances among victims or potential supporters of change (2012: 206-28).

Wider Context and Further Reading: 1. Policy analysis

This post connects to several other 750 Words posts, which suggest that facts don’t speak for themselves. Rather, effective analysis requires you to ‘tell your story’, in a concise way, tailored to your audience.

For example, consider two ways to establish cause and effect in policy analysis:

One is to conduct and review multiple randomised control trials.

Another is to use a story of a hero or a villain (perhaps to mobilise actors in an advocacy coalition).

2. Evidence-based policymaking

Stone (2012: 10) argues that analysts who try to impose one worldview on policymaking will find that ‘politics looks messy, foolish, erratic, and inexplicable’. For analysts, who are more open-minded, politics opens up possibilities for creativity and cooperation (2012: 10).

This point is directly applicable to the ‘politics of evidence based policymaking’. A common question to arise from this worldview is ‘why don’t policymakers listen to my evidence?’ and one answer is ‘you are asking the wrong question’.

3. Policy theories highlight the value of stories (to policy analysts and academics)

Policy problems and solutions necessarily involve ambiguity:

1. There are many ways to interpret problems, and we resolve such ambiguity by exercising power to attract attention to one way to frame a policy problem at the expense of others (in other words, not with reference to one superior way to establish knowledge).

• Punctuated equilibrium theory shows that policy change is primarily a function of such shifts in policymaker attention.

2. Policy is actually a collection of – often contradictory – policy instruments and institutions, interacting in complex systems or environments, to produce unclear messages and outcomes. As such, what we call ‘public policy’ (for the sake of simplicity) is subject to interpretation and manipulation as it is made and delivered, and we struggle to conceptualise and measure policy change. Indeed, it makes more sense to describe competing narratives of policy change.

4. Policy theories and storytelling

People communicate meaning via stories. Stories help us turn (a) a complex world, which provides a potentially overwhelming amount of information, into (b) something manageable, by identifying its most relevant elements and guiding action (compare with Gigerenzer on heuristics).

The Narrative Policy Framework identifies the storytelling strategies of actors seeking to exploit other actors’ cognitive shortcuts, using a particular format – containing the setting, characters, plot, and moral – to focus on some beliefs over others, and reinforce someone’s beliefs enough to encourage them to act.

Compare with Tuckett and Nicolic on the stories that people tell to themselves.

 

 

Policy Analysis in 750 words: Linda Tuhiwai Smith (2012) Decolonizing Methodologies

Please see the  Policy Analysis in 750 words series overview before reading the summary. The reference to 750 words is increasingly misleading.

Linda Tuhiwai Smith (2012) Decolonizing Methodologies 2^nd edition (London: Zed Books)

 ‘Whose research is it? Who owns it? Whose interests does it serve? Who will benefit from it? Who has designed its questions and framed its cope? Who will carry it out? Who will write it up? How will its results be disseminated?’ (Smith, 2012: 10; see also 174-7)

Many texts in this series highlight the politics of policy analysis, but few (such as Bacchi) identify the politics of the research that underpins policy analysis.

You can find some discussion of these issues in the brief section on ‘co-production’, in wider studies of co-produced research and policy, and ‘evidence based policymaking’, and in posts on power and knowledge and feminist institutionalism. However, the implications rarely feed into standard policy analysis texts. This omission is important, because the production of knowledge – and the exercise of power to define whose knowledge counts – is as political as it gets.

Smith (2012) demonstrates this point initially by identifying multiple, often hidden, aspects of politics and power that relate to ‘research’ and ‘indigenous peoples’:

 

1. The term ‘indigenous peoples’ is contested, and its meaning-in-use can range from

• positive self-identification, to highlight common international experiences and struggles for self-determination but distinctive traditions; other terms include ‘First Nations’ in Canada or, in New Zealand, ‘Maori’ as opposed to ‘Pakeha’ (the colonizing population) (2012: 6)
• negative external-identification, including – in some cases – equating ‘indigenous’ (or similar terms) with ‘dirtiness, savagery, rebellion and, since 9/11, terrorism’ (2012: xi-xii).

 

2. From the perspective of ‘the colonized’, “the term ‘research’ is inextricably linked to European imperialism and colonialism” (2012: 1; 21-6). Western research practices (and the European ‘Enlightenment’) reflect and reinforce political practices associated with colonial rule (2012: 2; 23).

‘To the colonized, the ways in which academic research has been implicated in the throes of imperialism remains a painful memory’ (2012: back cover).

“The word itself, ‘research’, is probably one of the dirtiest words in the indigenous world’s vocabulary” (2012: xi).

 

3. People in indigenous communities describe researchers who exploit ‘their culture, their knowledge, their resources’ (and, in some cases, their bodies) to bolster their own income, career or profession (2012: xi; 91-4; 102-7), in the context of a long history of subjugation and slavery that makes such practices possible (2012: 21-6; 28-9; 176-7), and “justified as being for ‘the good of mankind’” (2012: 26).

 

 

4. Western researchers think – hubristically – that they can produce a general understanding of the practices and cultures of indigenous peoples (e.g. using anthropological methods). Instead, they produce – irresponsibly or maliciously – negative and often dehumanizing images that feed into policies ‘employed to deny the validity of indigenous peoples’ claim to existence’ and solve the ‘indigenous problem’ (2012: 1; 8-9; 26-9; 62-5; 71-2; 81-91; 94-6).

For example, research contributes to a tendency for governments to

• highlight, within indigenous communities, indicators of inequality (in relation to factors such as health, education, crime, and family life), and relate it to
• indigenous cultures and low intelligence, rather than
• the ways in which colonial legacy and current policy contributes to poverty and marginalisation (2012: 4; 12; compare with Social Construction and Policy Design).

 

5. Western researchers’ views on how to produce high-quality scientific evidence lead them to ‘see indigenous peoples, their values and practices as political hindrances that get in the way of good research’ (2012: xi; 66-71; compare with ‘hierarchy of evidence’). Similarly, the combination of a state’s formal laws and unwritten rules and assumptions can serve to dismiss indigenous community knowledge as not meeting evidential standards (2012: 44-9).

 

6. Many indigenous researchers need to negotiate the practices and expectations of different groups, such as if they are portrayed as:

• ‘insiders’ in relation to an indigenous community (and, for example, expected by that community to recognise the problems with Western research traditions)
• ‘outsiders’, by (a) an indigenous community in relation to their ‘Western education’ (2012: 5), or (b) by a colonizing state commissioning insider research
• less technically proficient or less likely to maintain confidentiality than a ‘non-indigenous researcher’ (2012: 12)

Can policy analysis be informed by a new research agenda?

In that context, Smith (2012: xiii; 111-25) outlines a new agenda built on the recognition that research is political and connected explicitly to political and policy aims (2012: xiii; compare with Feminism, Postcolonialism, and Critical Policy Studies)

At its heart is a commitment to indigenous community ‘self-determination’, ‘survival’, ‘recovery’, and ‘development’, aided by processes such as social movement mobilization and decolonization (2012: 121). This agenda informs the meaning of ethical conduct, signalling that research:

• serves explicit political goals and requires researchers to reflect on their role as activists in an emancipatory project, in contrast to the disingenuous argument that science or scientists are objective (2012: 138-42; 166-77; 187-8; 193-5; 198-215; 217-26)
• is not ‘something done only by white researchers to indigenous peoples’ (2012: 122),
• is not framed so narrowly, in relation to specific methods or training, that it excludes (by definition) most indigenous researchers, community involvement in research design, and methods such as storytelling (2012: 127-38; 141; for examples of methods, see 144-63; 170-1)
• requires distinctive methods and practices to produce knowledge, reinforced by mutual support during the nurturing of such practices
• requires a code of respectful conduct that extends ‘beyond issues of individual consent and confidentiality’) (2012: 124; 179-81).

Wider context: informing the ‘steps’ to policy analysis

This project informs directly the ‘steps’ to policy analysis described in Bardach, Weimer and Vining, and Mintrom, including:

Problem definition

Mintrom describes the moral and practical value of engaging with stakeholders to help frame policy problems and design solutions (as part of a similarly-worded aim to transform and improve the world).

However, Smith (2012: 228-32; 13) describes such a profound gulf, in the framing of problems, that cannot be bridged simply via consultation or half-hearted ‘co-production’ exercises.

For example, if a government policy analyst relates poor health to individual and cultural factors in indigenous communities, and people in those communities relate it to colonization, land confiscation, minimal self-determination, and an excessive focus on individuals, what could we realistically expect from set-piece government-led stakeholder analyses built on research that has already set the policy agenda (compare with Bacchi)?

Rather, Smith (2012: 15-16) describes the need, within research practices, for continuous awareness of, and respect for, a community’s ‘cultural protocols, values and behaviours’ as part of ‘an ethical and respectful approach’. Indeed, the latter could have mutual benefits which underpin the long-term development of trust: a community may feel less marginalised by the analysis-to-policy process, and future analysts may be viewed with less suspicion.

Even so, a more respectful policy process is not the same as accepting that some communities may benefit more from writing about their own experiences than contributing to someone else’s story. Writing about the past, present, and future is an exercise of power to provide a dominant perspective with which to represent people and problems (2012: 29-41; 52-9)

Analysing and comparing solutions

Imagine a cost-benefit analysis designed to identify the most efficient outcomes by translating all of the predicted impacts on people into a single unit of analysis (such as a dollar amount, or quality-adjusted-life-years). Assumptions include that we can: (a) assign the same value to a notionally similar experience, and (b) produce winners from policy and compensate losers.

Yet, this calculation hinges on the power to decide how we should understand such experiences and place relative values on outcomes, and to take a calculation of their value to one population and generalise it to others. Smith’s analysis suggests that such processes will not produce outcomes that we can describe honestly as societal improvements. Rather, they feed into a choice to produce winners from policy and fail to compensate losers in an adequate or appropriate manner.

See also:

1. In relation to policy theories

This post – Policy Concepts in 1000 Words: Feminism, Postcolonialism, and Critical Policy Studies – provides a tentative introduction to the ways in which many important approaches can inform policy theories, such as by

• identifying the unequal competition to define the rules of knowledge gathering, within
• a wider political context in which some social groups, countries, and ways of thinking dominate proceedings, from the direct exercise of power to the maintenance of ‘hegemonic’ or ‘paradigmatic’ ideas.

The 2^nd edition of Understanding Public Policy summarises these themes as follows:

2. In relation to policy analysis

If you look back to the Policy Analysis in 750 words series overview, you will see that a popular way to address policy issues is through the ‘coproduction’ of research and policy, perhaps based on a sincere commitment to widen a definition of useful knowledge/ ways of thinking and avoid simply making policy from the ‘centre’ or ‘top down’.

Yet, the post you are now reading, summarising Decolonizing Methodologies, should prompt us to question the extent to which a process could be described sincerely as ‘coproduction’ if there is such an imbalance of power and incongruence of ideas between participants.

Although many key texts do not discuss ‘policy analysis’ directly, they provide ways to reflect imaginatively on this problem. I hope that I am not distorting their original messages, but please note that the following are my stylized interpretations of key texts.

Audre Lorde (2018*) The Master’s Tools Will Never Dismantle the Master’s House (Penguin) (*written from 1978-82)

One issue with very quick client-oriented policy analysis is that it encourages analysts to (a) work with an already-chosen definition of the policy problem, and (b) use well-worn methods to collect information, including (c) engaging with ideas and people with whom they are already familiar.

Some forms of research and policy analysis may be more conducive to challenging existing frames and encouraging wider stakeholder engagement. Still, compare this mild shift from the status quo with a series of issues and possibilities identified by Lorde (2018):

• Some people are so marginalised and dismissed that they struggle to communicate – about the ways in which they are oppressed, and how they might contribute to imagining a better world – in ways that would be valued (or even noticed) during stakeholder consultation (2018: 1-5 ‘Poetry is not a luxury’).
• The ‘european-american male tradition’ only allows for narrowly defined (‘rational’) means of communication (2018: 6-15 ‘Uses of the Erotic’)

A forum can be designed ostensibly to foster communication and inclusivity, only to actually produce the opposite, by signalling to some participants that

• they are a token afterthought, whose views and experiences are – at best – only relevant to a very limited aspect of a wide discussion, and
• their differences will be feared, not celebrated, becoming a source of conflict, not mutual nurture or cooperation.

It puts marginalised people in the position of having to work hard simply to be heard. They learn that powerful people are only willing to listen if others do the work for them, because (a) they are ignorant of experiences other than their own, and/or (b) they profess ignorance strategically to suck the energy from people whose views they fear and do not understand. No one should feel immune from such criticism even if they profess to be acting with good intentions (2018: 16-21 ‘The Master’s Tools Will Never Dismantle the Master’s House’).

• The correct response to racism is anger. Therefore, do not prioritise (a) narrow rules of civility, or the sensibilities of the privileged, if (b) your aim is to encourage conversations with people who are trying to express the ways in which they deal with overwhelming and continuous hatred, violence, and oppression (2018: 22-35, ‘Uses of Anger: Women Responding to Racism’)

Boaventura de Sousa Santos (2014) Epistemologies of the South: Justice Against Epistemicide (Routledge)

Imagine global policy processes and policy analysis, in which some countries and international organisations negotiate agreements, influenced (or not) by critical social movements in pursuit of social justice. Santos (2014) identifies a series of obstacles including:

• A tendency for Western (as part of the Global North) ways of thinking to dominate analysis, at the expense of insights from the Global South (2014: viii), producing
• A tendency for ‘Western centric’ ideas to inform the sense that some concepts and collective aims – such as human dignity and human rights – can be understood universally, rather than through the lens of struggles that are specific to some regions (2014: 21; 38)
• A lack of imagination or willingness to imagine different futures and conceptions of social justice (2014: 24)

Consequently, actors may come together to discuss major policy change on ostensibly the same terms, only for some groups to – intentionally and unintentionally – dominate thought and action and reinforce the global inequalities they propose to reduce.

Sarah Ahmed (2017) Living a Feminist Life (Duke University Press)

Why might your potential allies in ‘coproduction’ be suspicious of your motives, or sceptical about the likely outcomes of such an exchange? One theme throughout Smith’s (2012) book is that people often co-opt key terms (such as ‘decolonizing’) to perform the sense that they care about social change, to try to look like they are doing something important, while actually designing ineffective or bad faith processes to protect the status of themselves or their own institution or profession.

Ahmed (2017: 103) describes comparable initiatives – such as to foster ‘equality and diversity’ – as a public relations exercise for organisations, rather than a sincere desire to do the work. Consequently, there is a gap ‘between a symbolic commitment and a lived reality’ (2017: 90). Indeed, the aim may be to project a sense of transformation to hinder that transformation (2017: 90), coupled with a tendency to use a ‘safe’ and non-confrontational language (‘diversity’) to project the sense that we can only push people so far, at the expense of terms such as ‘racism’ that would signal challenge, confrontation, and a commitment to high impact (2017: chapter 4).

..

Putting these insights together suggests that a stated commitment to co-produced research and policy might begin with good intentions. Even so, a commitment to sincere engagement does not guarantee an audience or prevent you from exacerbating the very problems you profess to solve.

A general theory of public policy

This is a placeholder for future work and discussion. It tails off at the end.

People sometimes talk about a ‘general theory’ of public policy to put in our minds a comparison with the physical sciences. Usually, the punchline is that there are ‘no general theories of public policy that are not bounded by space or time’ (p21). There may be some reference to the accumulation of knowledge or wisdom in policy studies, but based rarely on the understanding that policy studies contain the equivalent of general laws (I can only think of one possible exception).

This outcome is not too surprising in the social sciences, in which context really matters and we would expect a lot of variation in policy, policymaking, and outcomes.

On the other hand, we still need a way to communicate our findings, relate them to other studies, compare them, and wonder what it all adds up to. Few people go as far as expressing the sense that every study is unique (to the point of non-comparability) and that every description of policymaking does not compare to another.

In other words, we may be looking for a happy medium, to reject the idea of general laws but encourage – when appropriate or necessary – enough of a sense of common outlook and experience to help us communicate with each other (without descending too quickly into heated debate on our cross-purposes). Or, we can at least tell a story of policy studies and invite others to learn from, or challenge, its insights.

In my case, there are two examples in which it is necessary to project some sense of a common and initially-not-too-complicated story:

1. When describing policy theory insights to students, on the assumption that it may be their gateway to more reading.

It is possible to choose how many words to devote to each topic, including 500 Words, 1000 Words, a 9000 word Understanding Public Policy chapter, more in the source material, and even more if students start to ‘snowball’.

It is also possible, if you have a clearly defined audience, to introduce some level of uncertainty about these descriptions and their limitations.

For example, I try to describe ‘the policy process’ in 500 words and 1000 words, but in the context of a wider discussion of images of the policy process.

It is also possible to provide more context, such as in this kind of introductory box, coupled with 12 things to know about studying public policy

(from Chapter 1)

You can also get into the idea that my story is one of many, particularly after students have invested in many versions of that story by the end of an introductory book

(from Chapter 13)

2. When describing these insights to people – from other disciplines or professions – who do not have the time, inclination, or frame of reference to put in that kind of work.

In this case, one presentation or article may be the limit. People may want to know the answer to a question – e.g. Why don’t policymakers listen to your evidence? – rather than hear all about the explanation for the answer.

You do your best, and then – if there is time – you talk about what you missed out.

For example, in this talk, the first question was: why didn’t you mention the role of power?

 

A general theory or a general understanding? Two key issues

That was a long-winded introduction to a more philosophical point about what we might want from general theories. My impression is that you might be seeking one of these two possibilities:

1. To use theories and concepts to describe material reality. In producing a general theory, we seek a general understanding of the ways in which the real world works. If so, we may focus primarily on how well these concepts describe the world, and the extent to which we can produce methods to produce systematic and consistent findings. The lack of a general theory denotes too much complexity and context.
2. To use theories and concepts to represent a useful story. In producing a general understanding, we focus on the ways in which people generate and communicate their understanding. If so, we may focus more on how people come together to produce and share meaning through concepts. The lack of a general theory could reflect the lack of agreement on how to study policymaking. Or, the presence of a general understanding could represent the exercise of power, to set the agenda and limit scholarly attention to a small number of theories.

I describe this distinction in the following audio clip, produced halfway through a run with the dogs, while jetlagged. The large gap in the middle happens when I am trying to see if the voice to text is working well enough for me to copy/paste it here (no).

Key examples of the exercise of power include:

1. The act of dismissing an individual, social group, or population by undermining the value of their knowledge or claim to knowledge (discussed in power and knowledge and Chapter 3).
2. Ongoing discussions about how we deal with (a) a relatively new focus (among the most-established policy theories) on policy studies in countries in the Global South, given that (b) the dominant interpretations of policymaking come from experiences in the Global North.

So, if you read these posts or Chapter 13 you will find a story of a general understanding of policy followed, almost immediately, by a list of reasons for why you should engage with it critically and perhaps not accept it. I’m setting your agenda but also reminding you that I’m doing it.

That’s it really. To be continued.

Beware the well-intentioned advice of unusually successful academics

This post – by Dr Kathryn Oliver and me – originally appeared on the LSE Impact Blog. I have replaced the picture of a thumb up with a cat hanging in there. 

Many academics want to see their research have an impact on policy and practice, and there is a lot of advice on how to seek it. It can be helpful to take advice from experienced and successful people. However, is this always the best advice? Guidance based on best practice and success stories in particular, often reflect unequal access to policymakers, institutional support, and credibility attached to certain personal characteristics.

To take stock of the vast amount of advice being offered to academics, we decided to compare it with the more systematic analyses available in the peer-reviewed literature, on the ‘barriers’ between evidence and policy, and policy studies. This allowed us to situate this advice in a wider context, see whether it was generalisable across settings and career stages, and to think through the inconsistencies and dilemmas which underlie these suggestions.

The advice: Top tips on influencing policy

The key themes and individual recommendations we identified from the 86 most-relevant publications are:

1. Do high quality research: Use well-established research designs, methods, or metrics.
2. Make your research relevant and readable: Provide easily-understandable, clear, relevant and high-quality research. Aim for the general reader. Produce good stories based on emotional appeals or humour.
3. Understand the policymaking context. Note the busy and constrained lives of policy actors. Maximise established ways to engage, such as in advisory committees. Be pragmatic, accepting that research rarely translates directly into policy.
4. Be ‘accessible’ to policymakers. This may involve discussing topics beyond your narrow expertise. Be humble, courteous, professional, and recognise the limits to your skills.
5. Decide if you want to be an ‘issue advocate’. Decide whether to simply explain the evidence, remain an ‘honest broker, or recommend specific policy options. Negative consequences may include peer criticism, being seen as an academic lightweight, being used to add legitimacy to a policy position, and burnout.
6. Build relationships (and ground rules) with policymakers: Relationship-building requires investment and skills, but working collaboratively is often necessary. Academics could identify policy actors to provide insights into policy problems, act as champions for their research, and identify the most helpful policy actors.
7. Be ‘entrepreneurial’ or find someone who is. Be a daring, persuasive scientist, comfortable in policy environments and available when needed. Or, seek brokers to act on your behalf.
8. Reflect continuously: should you engage, do you want to, and is it working? Academics may enjoy the work or are passionate about the issue. Even so, keep track of when and how you have had impact, and revise your practices continuously.

Inconsistencies and dilemmas

This advice tends not to address wider issues. For example, there is no consensus over what counts as good evidence for policy, or therefore how best to communicate good evidence. We know little about how to gain the wide range of skills that researchers and policymakers need to act collectively, including to: produce evidence syntheses, manage expert communities, ‘co-produce’ research and policy with a wide range of stakeholders, and be prepared to offer policy recommendations as well as scientific advice. Further, a one-size fits-all model won’t help researchers navigate a policymaking environment where different venues have different cultures and networks. Researchers therefore need to decide what policy engagement is for—to frame problems or simply measure them according to an existing frame—and how far researchers should go to be useful and influential. If academics need to go ‘all in’ to secure meaningful impact, we need to reflect on the extent to which they have the resources and support to do so. This means navigating profound dilemmas:

Source: The dos and don’ts of influencing policy: a systematic review of advice to academics

 

Can academics try to influence policy? The financial costs of seeking impact are prohibitive for junior or untenured researchers, while women and people of colour may be more subject to personal abuse. Such factors undermine the diversity of voices available.

How should academics influence policy? Many of these new required skills – such as storytelling – are not a routine part of academic training, and may be looked down on by our colleagues.  

What is the purpose of academics engagement in policymaking? To go beyond tokenistic and instrumental engagement is to build genuine rapport with policymakers, which may require us to co-produce knowledge and cede some control over the research process. It involves a fundamentally different way of doing public engagement: one with no clear aim in mind other than to listen and learn, with the potential to transform research practices and outputs.

Where is the evidence that this advice helps us improve impact?

The existing advice offered to academics on how to create impact is – although often well-meaning – not based on systematic research or comprehensive analysis of empirical evidence. Few advice-givers draw clearly on key literatures on policymaking or evidence use. This leads to significant misunderstandings, which can have potentially costly repercussions for research, researchers and policy.  These limitations matter, as they lead to advice which fails to address core dilemmas for academics—whether to engage, how to engage, and why—which have profound implications for how scientists and universities should respond to the calls for increased impact.

Most tips focus on individual experience, whereas engagement between research and policy is driven by systemic factors. Many of the tips may be sensible and effective, but often only within particular settings. The advice is likely to be useful mostly to a relatively similar group of people who are confident and comfortable in policy environments, and have access and credibility within policy arenas. Thus, the current advice and structures may help reproduce and reinforce existing power dynamics and an underrepresentation of people who do not fit a very narrow mould.

The overall result may be that each generation of scientists has to fight the same battles, and learn the same lessons over again. Our best response as a profession is to interrogate current advice, shape and frame it, and to help us all to find ways to navigate the complex practical, political, moral and ethical challenges associated with being researchers today. The ‘how to’ literature can help, but only if authors are cognisant of their wider role in society and complex policymaking systems.

This blog post is based on the authors’ co-written articles, The dos and don’ts of influencing policy: a systematic review of advice to academics, published in Palgrave Communications, and ‘How should academics engage in policymaking to achieve impact?’  published in Political Studies Review ​

About the authors

Kathryn Oliver is Associate Professor of Sociology and Public Health, London School of Hygiene and Tropical Medicine (@oliver_kathryn ). Her interest is in how knowledge is produced, mobilized and used in policy and practice, and how this affects the practice of research. She co-runs the research collaborative Transforming Evidence with Annette Boaz. https://transformure.wordpress.com and her writings can be found here: https://kathrynoliver.wordpress.com

Paul Cairney is Professor of Politics and Public Policy, University of Stirling, UK (@Cairneypaul).  His research interests are in comparative public policy and policy theories, which he uses to explain the use of evidence in policy and policymaking, in one book (The Politics of Evidence-Based Policy Making, 2016), several articles, and many, many blog posts: https://paulcairney.wordpress.com/ebpm/

See also:

1. Adam Wellstead, Paul Cairney, and Kathryn Oliver (2018) ‘Reducing ambiguity to close the science-policy gap’, Policy Design and Practice, 1, 2, 115-25 PDF
2. Paul Cairney and Kathryn Oliver (2017) ‘Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy?’ Health Research Policy and Systems (HARPS), DOI: 10.1186/s12961-017-0192-x PDF AM
3. Paul Cairney, Kathryn Oliver, and Adam Wellstead (2016) ‘To Bridge the Divide between Evidence and Policy: Reduce Ambiguity as Much as Uncertainty’, Public Administration Review, 76, 3, 399–402 DOI:10.1111/puar.12555 PDF

Policy in 500 Words: Power and Knowledge

Classic studies suggest that the most profound and worrying kinds of power are the hardest to observe. We often witness highly visible political battles and can use pluralist methods to identify who has material resources, how they use them, and who wins. However, key forms of power ensure that many such battles do not take place. Actors often use their resources to reinforce social attitudes and policymakers’ beliefs, to establish which issues are policy problems worthy of attention and which populations deserve government support or punishment. Key battles may not arise because not enough people think they are worthy of debate. Attention and support for debate may rise, only to be crowded out of a political agenda in which policymakers can only debate a small number of issues.

Studies of power relate these processes to the manipulation of ideas or shared beliefs under conditions of bounded rationality (see for example the NPF). Manipulation might describe some people getting other people to do things they would not otherwise do. They exploit the beliefs of people who do not know enough about the world, or themselves, to know how to identify and pursue their best interests. Or, they encourage social norms – in which we describe some behaviour as acceptable and some as deviant – which are enforced by the state (for example, via criminal justice and mental health policy), but also social groups and individuals who govern their own behaviour with reference to what they feel is expected of them (and the consequences of not living up to expectations).

Such beliefs, norms, and rules are profoundly important because they often remain unspoken and taken for granted. Indeed, some studies equate them with the social structures that appear to close off some action. If so, we may not need to identify manipulation to find unequal power relationships: strong and enduring social practices help some people win at the expense of others, by luck or design.

In practice, these more-or-less-observable forms of power co-exist and often reinforce each other:

Example 1. The control of elected office is highly skewed towards men. Male incumbency, combined with social norms about who should engage in politics and public life, signal to women that their efforts may be relatively unrewarded and routinely punished – for example, in electoral campaigns in which women face verbal and physical misogyny – and the oversupply of men in powerful positions tends to limit debates on feminist issues.

Example 2. ‘Epistemic violence’ describes the act of dismissing an individual, social group, or population by undermining the value of their knowledge or claim to knowledge. Specific discussions include: (a) the colonial West’s subjugation of colonized populations, diminishing the voice of the subaltern; (b) privileging scientific knowledge and dismissing knowledge claims via personal or shared experience; and (c) erasing the voices of women of colour from the history of women’s activism and intellectual history.

It is in this context that we can understand ‘critical’ research designed to ‘produce social change that will empower, enlighten, and emancipate’ (p51). Powerlessness can relate to the visible lack of economic material resources and factors such as the lack of opportunity to mobilise and be heard.

See also:

Policy Concepts in 1000 Words: Power and Ideas

Evidence-informed policymaking: context is everything

Evidence-informed policymaking: context is everything

I thank James Georgalakis for inviting me to speak at the inaugural event of IDS’ new Evidence into Policy and Practice Series, and the audience for giving extra meaning to my story about the politics of ‘evidence-based based policymaking’. The talk (using powerpoint) and Q&A is here:

 

James invited me to respond to some of the challenges raised to my talk – in his summary of the event – so here it is.

I’m working on a ‘show, don’t tell’ approach, leaving some of the story open to interpretation. As a result, much of the meaning of this story – and, in particular, the focus on limiting participation – depends on the audience.

For example, consider the impact of the same story on audiences primarily focused on (a) scientific evidence and policy, or (b) participation and power.

Normally, when I talk about evidence and policy, my audience is mostly people with scientific or public health backgrounds asking why do policymakers ignore scientific evidence? I am usually invited to ruffle feathers, mostly by challenging a – remarkably prevalent – narrative that goes like this:

• We know what the best evidence is, since we have produced it with the best research methods (the ‘hierarchy of evidence’ argument).
• We have evidence on the nature of the problem and the most effective solutions (the ‘what works’ argument).
• Policymakers seems to be ignoring our evidence or failing to act proportionately (the ‘evidence-policy barriers’ argument).
• Or, they cherry-pick evidence to suit their agenda (the ‘policy based evidence’ argument).

In that context, I suggest that there are many claims to policy-relevant knowledge, policymakers have to ignore most information before making choices, and they are not in control of the policy process for which they are ostensibly in charge.

Limiting participation as a strategic aim

Then, I say to my audience that – if they are truly committed to maximising the use of scientific evidence in policy – they will need to consider how far they will go to get what they want. I use the metaphor of an ethical ladder in which each rung offers more influence in exchange for dirtier hands: tell stories and wait for opportunities, or demonise your opponents, limit participation, and humour politicians when they cherry-pick to reinforce emotional choices.

It’s ‘show don’t tell’ but I hope that the take-home point for most of the audience is that they shouldn’t focus so much on one aim – maximising the use of scientific evidence – to the detriment of other important aims, such as wider participation in politics beyond a reliance on a small number of experts. I say ‘keep your eyes on the prize’ but invite the audience to reflect on which prizes they should seek, and the trade-offs between them.

Limited participation – and ‘windows of opportunity’ – as an empirical finding

I did suggest that most policymaking happens away from the sphere of ‘exciting’ and ‘unruly’ politics. Put simply, people have to ignore almost every issue almost all of the time. Each time they focus their attention on one major issue, they must – by necessity – ignore almost all of the others.

For me, the political science story is largely about the pervasiveness of policy communities and policymaking out of the public spotlight.

The logic is as follows. Elected policymakers can only pay attention to a tiny proportion of their responsibilities. They delegate the rest to bureaucrats at lower levels of government. Bureaucrats lack specialist knowledge, and rely on other actors for information and advice. Those actors trade information for access. In many cases, they develop effective relationships based on trust and a shared understanding of the policy problem.

Trust often comes from a sense that everyone has proven to be reliable. For example, they follow norms or the ‘rules of the game’. One classic rule is to contain disputes within the policy community when actors don’t get what they want: if you complain in public, you draw external attention and internal disapproval; if not, you are more likely to get what you want next time.

For me, this is key context in which to describe common strategic concerns:

• Should you wait for a ‘window of opportunity’ for policy change? Maybe. Or, maybe it will never come because policymaking is largely insulated from view and very few issues reach the top of the policy agenda.
• Should you juggle insider and outsider strategies? Yes, some groups seem to do it well and it is possible for governments and groups to be in a major standoff in one field but close contact in another. However, each group must consider why they would do so, and the trade-offs between each strategy. For example, groups excluded from one venue may engage (perhaps successfully) in ‘venue shopping’ to get attention from another. Or, they become discredited within many venues if seen as too zealous and unwilling to compromise. Insider/outsider may seem like a false dichotomy to experienced and well-resourced groups, who engage continuously, and are able to experiment with many approaches and use trial-and-error learning. It is a more pressing choice for actors who may have only one chance to get it right and do not know what to expect.

Where is the power analysis in all of this?

I rarely use the word power directly, partly because – like ‘politics’ or ‘democracy’ – it is an ambiguous term with many interpretations (see Box 3.1). People often use it without agreeing its meaning and, if it means everything, maybe it means nothing.

However, you can find many aspects of power within our discussion. For example, insider and outsider strategies relate closely to Schattschneider’s classic discussion in which powerful groups try to ‘privatise’ issues and less powerful groups try to ‘socialise’ them. Agenda setting is about using resources to make sure issues do, or do not, reach the top of the policy agenda, and most do not.

These aspects of power sometimes play out in public, when:

• Actors engage in politics to turn their beliefs into policy. They form coalitions with actors who share their beliefs, and often romanticise their own cause and demonise their opponents.
• Actors mobilise their resources to encourage policymakers to prioritise some forms of knowledge or evidence over others (such as by valuing scientific evidence over experiential knowledge).
• They compete to identify the issues most worthy of our attention, telling stories to frame or define policy problems in ways that generate demand for their evidence.

However, they are no less important when they play out routinely:

• Governments have standard operating procedures – or institutions – to prioritise some forms of evidence and some issues routinely.
• Many policy networks operate routinely with few active members.
• Certain ideas, or ways of understanding the world and the nature of policy problems within it, becomes so dominant that they are unspoken and taken for granted as deeply held beliefs. Still, they constrain or facilitate the success of new ‘evidence based’ policy solutions.

In other words, the word ‘power’ is often hidden because the most profound forms of power often seem to be hidden.

In the context of our discussion, power comes from the ability to define some evidence as essential and other evidence as low quality or irrelevant, and therefore define some people as essential or irrelevant. It comes from defining some issues as exciting and worthy of our attention, or humdrum, specialist and only relevant to experts. It is about the subtle, unseen, and sometimes thoughtless ways in which we exercise power to harness people’s existing beliefs and dominate their attention as much as the transparent ways in which we mobilise resources to publicise issues. Therefore, to ‘maximise the use of evidence’ sounds like an innocuous collective endeavour, but it is a highly political and often hidden use of power.

See also:

I discussed these issues at a storytelling workshop organised by the OSF:

See also:

Policy in 500 Words: Power and Knowledge

The politics of evidence-based policymaking

Palgrave Communications: The politics of evidence-based policymaking

Using evidence to influence policy: Oxfam’s experience

The UK government’s imaginative use of evidence to make policy

 

Evidence-based policymaking and the ‘new policy sciences’

I have given a series of talks to explain why we should think of ‘evidence-based policymaking’ as a myth or political slogan, not an ideal scenario or something to expect from policymaking in the real world. They usually involve encouraging framing and storytelling rather than expecting evidence to speak for itself, and rejecting the value of simple models like the policy cycle. I then put up an image of my own and encourage people to think about the implications of each concept:

I describe the advice as simple-sounding and feasible at first glance, but actually a series of Herculean* tasks:

• There are many policymakers and influencers spread across government, so find out where the action is, or the key venues in which people are making authoritative decisions.
• Each venue has its own ‘institutions’ – the formal and written, or informal and unwritten rules of policymaking – so learn the rules of each venue in which you engage.
• Each venue is guided by a fundamental set of ideas – as paradigms, core beliefs, monopolies of understanding – so learn that language.
• Each venue has its own networks – the relationships between policy makers and influencers – so build trust and form alliances within networks.
• Policymaking attention is often driven by changes in socioeconomic factors, or routine/ non-routine events, so be prepared to exploit the ‘windows of opportunity’ to present your solution during heightened attention to a policy problem.

In most cases, we don’t have time to discuss a more fundamental issue (at least for researchers using policy theory and political science concepts):

From where did these concepts come, and how well do we know them?

To cut a long story short, each concept represents its own literature: see these short explainers on the psychology of policymaking, actors spread across multi-level governance, institutions, networks, ideas, and socioeconomic factors/ events. What the explainers don’t really project is the sense of debate within the literature about how best to conceptualise each concept. You can pick up their meaning in a few minutes but would need a few years to appreciate the detail and often-fundamental debate.

Ideally, we would put all of the concepts together to help explain policymaker choice within a complex policymaking environment (how else could I put up the image and present is as one source of accumulated wisdom from policy studies?). Peter John describes such accounts as ‘synthetic’. I have also co-authored work with Tanya Heikkila – in 2014 and 2017 to compare the different ways in which ‘synthetic’ theories conceptualise the policy process. However, note the difficulty of putting together a large collection of separate and diverse literatures into one simple model (e.g. while doing a PhD).

The new policy sciences

More recently, in the ‘new policy sciences’, Chris Weible and I present a more provocative story of these efforts, in which we advocate:

• a return to Lasswell’s vision of combining policy analysis (to recommend policy change) and policy theory (to explain policy change), but
• focusing on a far larger collection of actors (beyond a small group at the centre),
• recognising new developments in studies of the psychology of policymaker choice, and
• building into policy analysis the recognition that any policy solution is introduced in a complex policymaking environment over which no-one has control.

This focus on psychology is not new …

• PET shows the overall effect of policymaker psychology on policy change: they combine cognition and emotion to pay disproportionate attention to a small number of issues (contributing to major change) and ignore the rest (contributing to ‘hyperincremental’ change).
• The IAD focuses partly on the rules and practices that actors develop to build up trust in each other.
• The ACF describes actors going into politics to turn their beliefs into policy, forming coalitions with people who share their beliefs, then often romanticising their own cause and demonising their opponents.
• The NPF describes the relative impact of stories on audiences who use cognitive shortcuts to (for example) identify with a hero and draw a simple moral.
• SCPD describes policymakers drawing on gut feeling to identify good and bad target populations.
• Policy learning involves using cognition and emotion to acquire new knowledge and skills.

… but the pace of change in psychological research often seems faster than the ways in which policy studies can incorporate new and reliable insights.

Perhaps more importantly, policy studies help us understand the context in which people make such choices. For example, consider the story that Kathryn Oliver and I tell about the role of evidence in policymaking environments:

If there are so many potential authoritative venues, devote considerable energy to finding where the ‘action’ is (and someone specific to talk to). Even if you find the right venue, you will not know the unwritten rules unless you study them intensely. Some networks are close-knit and difficult to access because bureaucracies have operating procedures that favour some sources of evidence. Research advocates can be privileged insiders in some venues and excluded completely in others. If your evidence challenges an existing paradigm, you need a persuasion strategy good enough to prompt a shift of attention to a policy problem and a willingness to understand that problem in a new way. You can try to find the right time to use evidence to exploit a crisis leading to major policy change, but the opportunities are few and chances of success low.  In that context, policy studies recommend investing your time over the long term – to build up alliances, trust in the messenger, knowledge of the system, and to seek ‘windows of opportunity’ for policy change – but offer no assurances that any of this investment will ever pay off

Then, have a look at this discussion of ‘synthetic’ policy theories, designed to prompt people to consider how far they would go to get their evidence into policy.

Theory-driven policy analysis

As described, this focus on the new policy sciences helps explain why ‘the politics of evidence-based policymaking’ is equally important to civil servants (my occasional audience) as researchers (my usual audience).

To engage in skilled policy analysis, and give good advice, is to recognise the ways in which policymakers combine cognition/emotion to engage with evidence, and must navigate a complex policymaking environment when designing or selecting technically and politically feasible solutions. To give good advice is to recognise what you want policymakers to do, but also that they are not in control of the consequences.

Epilogue

Well, that is the last of the posts for my ANZOG talks. If I’ve done this properly, there should now be a loop of talks. It should be possible to go back to the first one in Auckland and see it as a sequel to this one in Brisbane!

Or, for more on theory-informed policy analysis – in other words, where the ‘new policy sciences’ article is taking us – here is how I describe it to students doing a policy analysis paper (often for the first time).

Or, have a look at the earlier discussion of images of the policy process. You may have noticed that there is a different image in this post (knocked up in my shed at the weekend). It’s because I am experimenting with shapes. Does the image with circles look more relaxing? Does the hexagonal structure look complicated even though it is designed to simplify? Does it matter? I think so. People engage emotionally with images. They share them. They remember them. So, I need an image more memorable than the policy cycle.

 

Paul Cairney Brisbane EBPM New Policy Sciences 25.10.18

 

 

*I welcome suggestions on another word to describe almost-impossibly-hard

Emotion and reason in politics: the rational/ irrational distinction

In ‘How to communicate effectively with policymakers’, Richard Kwiatkowski and I use the distinction between ‘rational’ and ‘irrational’ cognitive shortcuts ‘provocatively’. I sort of wish we had been more direct, because I have come to realise that:

1. My attempts to communicate with sarcasm and facial gestures may only ever appeal to a niche audience, and
2. even if you use the scare quotes – around a word like ‘irrational’ – to denote the word’s questionable use, it’s not always clear what I’m questioning, because
3. you need to know the story behind someone’s discussion to know what they are questioning.*

So, here are some of the reference points I’m using when I tell a story about ‘irrationality’:

1. I’m often invited to be the type of guest speaker that challenges the audience, it is usually a scientific audience, and the topic is usually evidence based policymaking.

So, when I say ‘irrational’, I’m speaking to (some) scientists who think of themselves as rational and policymakers as irrational, and use this problematic distinction to complain about policy-based evidence, post-truth politics, and perhaps even the irrationality of voters for Brexit. Action based on this way of thinking would be counterproductive. In that context, I use the word ‘irrational’ as a way into some more nuanced discussions including:

• all humans combine cognition and emotion to make choices; and,
• emotions are one of many sources of ‘fast and frugal heuristics’ that help us make some decisions very quickly and often very well.

In other words, it is silly to complain that some people are irrational, when we are all making choices this way, and such decision-making is often a good thing.

2. This focus on scientific rationality is part of a wider discussion of what counts as good evidence or valuable knowledge. Examples include:

• Policy debates on the value of bringing together many people with different knowledge claims – such as through user and practitioner experience – to ‘co-produce’ evidence.
• Wider debates on the ‘decolonization of knowledge’ in which narrow ‘Western’ scientific principles help exclude the voices of many populations by undermining their claims to knowledge.

3. A focus on rationality versus irrationality is still used to maintain sexist and racist caricatures or stereotypes, and therefore dismiss people based on a misrepresentation of their behaviour.

I thought that, by now, we’d be done with dismissing women as emotional or hysterical, but apparently not. Indeed, as some recent racist and sexist coverage of Serena Williams demonstrates, the idea that black women are not rational is still tolerated in mainstream discussion.

4. Part of the reason that we can only conclude that people combine cognition and emotion, without being able to separate their effects in a satisfying way, is that the distinction is problematic.

It is difficult to demonstrate empirically. It is also difficult to assign some behaviours to one camp or the other, such as when we consider moral reasoning based on values and logic.

To sum up, I’ve been using the rational/irrational distinction explicitly to make a simple point that is relevant to the study of politics and policymaking:

• All people use cognitive shortcuts to help them ignore almost all information about the world, to help them make decisions efficiently.
• If you don’t understand and act on this simple insight, you’ll waste your time by trying to argue someone into submission or giving them a 500-page irrelevant report when they are looking for one page written in a way that makes sense to them.

Most of the rest has been mostly implicit, and communicated non-verbally, which is great when you want to keep a presentation brief and light, but not if you want to acknowledge nuance and more serious issues.

 

 

 

 

*which is why I’m increasingly interested in Riker’s idea of heresthetics, in which the starting point of a story is crucial. We can come to very different conclusions about a problem and its solution by choosing different starting points, to accentuate one aspect of a problem and downplay another, even when our beliefs and preferences remain basically the same.

 

 

Policy in 500 words: uncertainty versus ambiguity

In policy studies, there is a profound difference between uncertainty and ambiguity:

• Uncertainty describes a lack of knowledge or a worrying lack of confidence in one’s knowledge.
• Ambiguity describes the ability to entertain more than one interpretation of a policy problem.

Both concepts relate to ‘bounded rationality’: policymakers do not have the ability to process all information relevant to policy problems. Instead, they employ two kinds of shortcut:

• ‘Rational’. Pursuing clear goals and prioritizing certain sources of information.
• ‘Irrational’. Drawing on emotions, gut feelings, deeply held beliefs, and habits.

I make an artificially binary distinction, uncertain versus ambiguous, and relate it to another binary, rational versus irrational, to point out the pitfalls of focusing too much on one aspect of the policy process:

1. Policy actors seek to resolve uncertainty by generating more information or drawing greater attention to the available information.

Actors can try to solve uncertainty by: (a) improving the quality of evidence, and (b) making sure that there are no major gaps between the supply of and demand for evidence. Relevant debates include: what counts as good evidence?, focusing on the criteria to define scientific evidence and their relationship with other forms of knowledge (such as practitioner experience and service user feedback), and what are the barriers between supply and demand?, focusing on the need for better ways to communicate.

2. Policy actors seek to resolve ambiguity by focusing on one interpretation of a policy problem at the expense of another.

Actors try to solve ambiguity by exercising power to increase attention to, and support for, their favoured interpretation of a policy problem. You will find many examples of such activity spread across the 500 and 1000 words series:

• Actors combine facts with emotional appeals to draw attention to their issues, at the expense of attention to other issues in a crowded political environment.
• Actors form coalitions with people who share their beliefs. They romanticise their own cause and demonise their opponents.
• Actors tell simple stories with a hero and moral to appeal to the emotional and ideological biases of policymakers.
• Policy entrepreneurs use skills of persuasion to frame problems, make sure their favoured solutions are feasible, and exploit times when policymakers have the motive and opportunity to act.
• Dominant ways of thinking act as a lens through which we process information.
• Policymakers exploit stereotypes of target populations.

A focus on reducing uncertainty gives the impression that policymaking is a technical process in which people need to produce the best evidence and deliver it to the right people at the right time.

In contrast, a focus on reducing ambiguity gives the impression of a more complicated and political process in which actors are exercising power to compete for attention and dominance of the policy agenda. Uncertainty matters, but primarily to describe the role of a complex policymaking system in which no actor truly understands where they are or how they should exercise power to maximise their success.

Further reading:

For a longer discussion, see Fostering Evidence-informed Policy Making: Uncertainty Versus Ambiguity (PDF)

Or, if you fancy it in French: Favoriser l’élaboration de politiques publiques fondées sur des données probantes : incertitude versus ambiguïté (PDF)

Framing

The politics of evidence-based policymaking

To Bridge the Divide between Evidence and Policy: Reduce Ambiguity as Much as Uncertainty

How to communicate effectively with policymakers: combine insights from psychology and policy studies

Here is the relevant opening section in UPP:

#EU4Facts: 3 take-home points from the JRC annual conference

See EU4FACTS: Evidence for policy in a post-fact world

Evidence-based policy making in an post fact world. Follow the #EU4facts debate today https://t.co/LaszStnsgA #posttruth #fakenews pic.twitter.com/uKecgbh39r

— DigitalSingleMarket (@DSMeu) September 26, 2017

The JRC’s annual conference has become a key forum in which to discuss the use of evidence in policy. At this scale, in which many hundreds of people attend plenary discussions, it feels like an annual mass rally for science; a ‘call to arms’ to protect the role of science in the production of evidence, and the protection of evidence in policy deliberation. There is not much discussion of storytelling, but we tell each other a fairly similar story about our fears for the future unless we act now.

For me, the failure scenario is not an option.
Because every time we mistrust #science, it undermines the progress of humanity.#EU4Facts

— Carlos Moedas (@Moedas) September 26, 2017

Last year, the main story was of fear for the future of heroic scientists: the rise of Trump and the Brexit vote prompted many discussions of post-truth politics and reduced trust in experts. An immediate response was to describe attempts to come together, and stick together, to support each other’s scientific endeavours during a period of crisis. There was little call for self-analysis and reflection on the contribution of scientists and experts to barriers between evidence and policy.

This year was a bit different. There was the same concern for reduced trust in science, evidence, and/ or expertise, and some references to post-truth politics and populism, but with some new voices describing the positive value of politics, often when discussing the need for citizen engagement, and of the need to understand the relationship between facts, values, and politics.

My address to #EU4FACTS at JRC on role of science advice as a bastion against the post-truth/post-trust polemic; https://t.co/vJWoXAI7EB

— peter gluckman (@PeterGluckman) September 27, 2017

For example, a panel on psychology opened up the possibility that we might consider our own politics and cognitive biases while we identify them in others, and one panellist spoke eloquently about the importance of narrative and storytelling in communicating to audiences such as citizens and policymakers.

Excellent walk the talk example by @KuperSimon when describing policy making lessons from football. #EU4FACTS pic.twitter.com/BlDL2nSJM9

— Mona Wallin (@monawallin) September 26, 2017

A focus on narrative is not new, but it provides a challenging agenda when interacting with a sticky story of scientific objectivity. For the unusually self-reflective, it also reminds us that our annual discussions are not particularly scientific; the usual rules to assess our statements do not apply.

Storytelling & policy: lessons from (1) the grey literature https://t.co/oAgmjvR7FR (2) polsci literature https://t.co/2mpTY3Sbho #eu4facts

— ProfessorPaulCairney (@CairneyPaul) September 26, 2017

As in studies of policymaking, we can say that there is high support for such stories when they remain vague and driven more by emotion than the pursuit of precision. When individual speakers try to make sense of the same story, they do it in different – and possibly contradictory – ways. As in policymaking, the need to deliver something concrete helps focus the mind, and prompts us to make choices between competing priorities and solutions.

I describe these discussions in two ways: tables, in which I try to boil down each speaker’s speech into a sentence or two (you can get their full details in the programme and the speaker bios); and a synthetic discussion of the top 3 concerns, paraphrasing and combining arguments from many speakers:

1. What are facts?

The key distinction began as between politics-values-facts which is impossible to maintain in practice.

#EU4FACTS I hope people reflect on @senseaboutsci Tracey Brown's pt about the problematic fact v value distinction https://t.co/qf1E1LUMkW

— ProfessorPaulCairney (@CairneyPaul) September 26, 2017

Yet, subsequent discussion revealed a more straightforward distinction between facts and opinion, ‘fake news’, and lies. The latter sums up an ever-present fear of the diminishing role of science in an alleged ‘post truth’ era.

2. What exactly is the problem, and what is its cause?

The tables below provide a range of concerns about the problem, from threats to democracy to the need to communicate science more effectively. A theme of growing importance is the need to deal with the cognitive biases and informational shortcuts of people receiving evidence: communicate with reference to values, beliefs, and emotions; build up trust in your evidence via transparency and reliability; and, be prepared to discuss science with citizens and to be accountable for your advice. There was less discussion of the cognitive biases of the suppliers of evidence.

3. What is the role of scientists in relation to this problem?

Not all speakers described scientists as the heroes of this story:

• Some described scientists as the good people acting heroically to change minds with facts.
• Some described their potential to co-produce important knowledge with citizens (although primarily with like-minded citizens who learn the value of scientific evidence?).
• Some described the scientific ego as a key barrier to action.
• Some identified their low confidence to engage, their uncertainty about what to do with their evidence, and/ or their scientist identity which involves defending science as a cause/profession and drawing the line between providing information and advocating for policy. This hope to be an ‘honest broker’ was pervasive in last year’s conference.
• Some (rightly) rejected the idea of separating facts/ values and science/ politics, since evidence is never context free (and gathering evidence without thought to context is amoral).

Often in such discussions it is difficult to know if some scientists are naïve actors or sophisticated political strategists, because their public statements could be identical. For the former, an appeal to objective facts and the need to privilege science in EBPM may be sincere. Scientists are, and should be, separate from/ above politics. For the latter, the same appeal – made again and again – may be designed to energise scientists and maximise the role of science in politics.

Yet, energy is only the starting point, and it remains unclear how exactly scientists should communicate and how to ‘know your audience’: would many scientists know who to speak to, in governments or the Commission, if they had something profoundly important to say?

MEP Rubig: the attitude ‘I have made up my mind, don’t bother me with facts’ undermines quality of policy making #EU4FACTS @EU_ScienceHub

— Maive Rute (@MaiveRute) September 26, 2017

Keynotes and introductory statements from panel chairs

Vladimír Šucha: We need to understand the relationship between politics, values, and facts. Facts are not enough. To make policy effectively, we need to combine facts and values.

Tibor Navracsics: Politics is swayed more by emotions than carefully considered arguments. When making policy, we need to be open and inclusive of all stakeholders (including citizens), communicate facts clearly and at the right time, and be aware of our own biases (such as groupthink).

Sir Peter Gluckman: ‘Post-truth’ politics is not new, but it is pervasive and easier to achieve via new forms of communication. People rely on like-minded peers, religion, and anecdote as forms of evidence underpinning their own truth. When describing the value of science, to inform policy and political debate, note that it is more than facts; it is a mode of thinking about the world, and a system of verification to reduce the effect of personal and group biases on evidence production. Scientific methods help us define problems (e.g. in discussion of cause/ effect) and interpret data. Science advice involves expert interpretation, knowledge brokerage, a discussion of scientific consensus and uncertainty, and standing up for the scientific perspective.

Carlos Moedas: Safeguard trust in science by (1) explaining the process you use to come to your conclusions; (2) provide safe and reliable places for people to seek information (e.g. when they Google); (3) make sure that science is robust and scientific bodies have integrity (such as when dealing with a small number of rogue scientists).

Pascal Lamy: 1. ‘Deep change or slow death’ We need to involve more citizens in the design of publicly financed projects such as major investments in science. Many scientists complain that there is already too much political interference, drowning scientists in extra work. However, we will face a major backlash – akin to the backlash against ‘globalisation’ – if we do not subject key debates on the future of science and technology-driven change (e.g. on AI, vaccines, drone weaponry) to democratic processes involving citizens. 2. The world changes rapidly, and evidence gathering is context-dependent, so we need to monitor regularly the fitness of our scientific measures (of e.g. trade).

Jyrki Katainen: ‘Wicked problems’ have no perfect solution, so we need the courage to choose the best imperfect solution. Technocratic policymaking is not the solution; it does not meet the democratic test. We need the language of science to be understandable to citizens: ‘a new age of reason reconciling the head and heart’.

Former commissioner Kuneva We need evidence to govern better as we cannot afford to fail #EU4Facts #evidence4gov We need better policies

— Stephane Jacobzone (@SJacobzone) September 26, 2017

Panel: Why should we trust science?

Jonathan Kimmelman: Some experts make outrageous and catastrophic claims. We need a toolbox to decide which experts are most reliable, by comparing their predictions with actual outcomes. Prompt them to make precise probability statements and test them. Only those who are willing to be held accountable should be involved in science advice.

Johannes Vogel: We should devote 15% of science funding to public dialogue. Scientific discourse, and a science-literature population, is crucial for democracy. EU Open Society Policy is a good model for stakeholder inclusiveness.

Tracey Brown: Create a more direct link between society and evidence production, to ensure discussions involve more than the ‘usual suspects’. An ‘evidence transparency framework’ helps create a space in which people can discuss facts and values. ‘Be open, speak human’ describes showing people how you make decisions. How can you expect the public to trust you if you don’t trust them enough to tell them the truth?

Francesco Campolongo: Claude Juncker’s starting point is that Commission proposals and activities should be ‘based on sound scientific evidence’. Evidence comes in many forms. For example, economic models provide simplified versions of reality to make decisions. Economic calculations inform profoundly important policy choices, so we need to make the methodology transparent, communicate probability, and be self-critical and open to change.

Frmr comm'r Geoghegan-Quinn stresses importance of "winning hearts & minds" 2combat posttruth. Scientists must communicate better #EU4Facts pic.twitter.com/g2luVqR55h

— Graeme Taylor (@leekandpotato) September 26, 2017

Panel: the politician’s perspective

Janez Potočnik: The shift of the JRC’s remit allowed it to focus on advocating science for policy rather than policy for science. Still, such arguments need to be backed by an economic argument (this policy will create growth and jobs). A narrow focus on facts and data ignores the context in which we gather facts, such as a system which undervalues human capital and the environment.

Máire Geoghegan-Quinn: Policy should be ‘solidly based on evidence’ and we need well-communicated science to change the hearts and minds of people who would otherwise rely on their beliefs. Part of the solution is to get, for example, kids to explain what science means to them.

EC Audiovisual Service – Video https://t.co/YpdDxsCb5D via @EC_AVService#eu4facts@michaelbossetta@simonkuper

— Mary I. William (@MIWilliamauthor) September 26, 2017

Panel: Redesigning policymaking using behavioural and decision science

Steven Sloman: The world is complex. People overestimate their understanding of it, and this illusion is burst when they try to explain its mechanisms. People who know the least feel the strongest about issues, but if you ask them to explain the mechanisms their strength of feeling falls. Why? People confuse their knowledge with that of their community. The knowledge is not in their heads, but communicated across groups. If people around you feel they understand something, you feel like you understand, and people feel protective of the knowledge of their community. Implications? 1. Don’t rely on ‘bubbles’; generate more diverse and better coordinated communities of knowledge. 2. Don’t focus on giving people full information; focus on the information they need at the point of decision.

Stephan Lewandowsky: 97% of scientists agree that human-caused climate change is a problem, but the public thinks it’s roughly 50-50. We have a false-balance problem. One solution is to ‘inoculate’ people against its cause (science denial). We tell people the real figures and facts, warn them of the rhetorical techniques employed by science denialists (e.g. use of false experts on smoking), and mock the false balance argument. This allows you to reframe the problem as an investment in the future, not cost now (and find other ways to present facts in a non-threatening way). In our lab, it usually ‘neutralises’ misinformation, although with the risk that a ‘corrective message’ to challenge beliefs can entrench them.

Françoise Waintrop: It is difficult to experiment when public policy is handed down from on high. Or, experimentation is alien to established ways of thinking. However, our 12 new public innovation labs across France allow us to immerse ourselves in the problem (to define it well) and nudge people to action, working with their cognitive biases.

Simon Kuper: Stories combine facts and values. To change minds: persuade the people who are listening, not the sceptics; find go-betweens to link suppliers and recipients of evidence; speak in stories, not jargon; don’t overpromise the role of scientific evidence; and, never suggest science will side-line human beings (e.g. when technology costs jobs).

2 page recap of 10 hours of debate for those who could not participate in #EU4Facts https://t.co/jqzN3M4FPF

— Vladimir Sucha (@VladimirSucha) September 26, 2017

Panel: The way forward

Jean-Eric Paquet: We describe ‘fact based evidence’ rather than ‘science based’. A key aim is to generate ‘ownership’ of policy by citizens. Politicians are more aware of their cognitive biases than we technocrats are.

Anne Bucher: In the European Commission we used evidence initially to make the EU more accountable to the public, via systematic impact assessment and quality control. It was a key motivation for better regulation. We now focus more on generating inclusive and interactive ways to consult stakeholders.

Ann Mettler: Evidence-based policymaking is at the heart of democracy. How else can you legitimise your actions? How else can you prepare for the future? How else can you make things work better? Yet, a lot of our evidence presentation is so technical; even difficult for specialists to follow. The onus is on us to bring it to life, to make it clearer to the citizen and, in the process, defend scientists (and journalists) during a period in which Western democracies seem to be at risk from anti-democratic forces.

Mariana Kotzeva: Our facts are now considered from an emotional and perception point of view. The process does not just involve our comfortable circle of experts; we are now challenged to explain our numbers. Attention to our numbers can be unpredictable (e.g. on migration). We need to build up trust in our facts, partly to anticipate or respond to the quick spread of poor facts.

Rush Holt: In society we can find the erosion of the feeling that science is relevant to ‘my life’, and few US policymakers ask ‘what does science say about this?’ partly because scientists set themselves above politics. Politicians have had too many bad experiences with scientists who might say ‘let me explain this to you in a way you can understand’. Policy is not about science based evidence; more about asking a question first, then asking what evidence you need. Then you collect evidence in an open way to be verified.

What can EU learn from US experience in trying to reconcile the worlds of #science & #politics? Listen to @RushHolt, @aaas CEO #EU4Facts pic.twitter.com/U7GuFjMLkE

— EU_ScienceHub (@EU_ScienceHub) September 27, 2017

Phew!

That was 10 hours of discussion condensed into one post. If you can handle more discussion from me, see:

Psychology and policymaking: Three ways to communicate more effectively with policymakers

The role of evidence in policy: EBPM and How to be heard  

Practical Lessons from Policy Theories

The generation of many perspectives to help us understand the use of evidence

How to be an ‘entrepreneur’ when presenting evidence

 

 

 

Policy Concepts in 500 Words: Social Construction and Policy Design

Why would a democratic political system produce ‘degenerative’ policy that undermines democracy? Social Construction and Policy Design (SCPD) describes two main ways in which policymaking alienates many citizens:

1. The Social Construction of Target Populations

High profile politics and electoral competition can cause alienation:

1. Political actors compete to tell ‘stories’ to assign praise or blame to groups of people. For example, politicians describe value judgements about who should be rewarded or punished by government. They base them on stereotypes of ‘target populations’, by (a) exploiting the ways in which many people think about groups, or (b) making emotional and superficial judgements, backed up with selective use of facts.
2. These judgements have a ‘feed-forward’ effect: they are reproduced in policies, practices, and institutions. Such ‘policy designs’ can endure for years or decades. The distribution of rewards and sanctions is cumulative and difficult to overcome.
3. Policy design has an impact on citizens, who participate in politics according to how they are characterised by government. Many know they will be treated badly; their engagement will be dispiriting.

Some groups have the power to challenge the way they are described by policymakers (and the media and public), and receive benefits behind the scenes despite their poor image. However, many people feel powerless, become disenchanted with politics, and do not engage in the democratic process.

SCTP depicts this dynamic with a 2-by-2 table in which target populations are described positively/ negatively and more or less able to respond:

2. Bureaucratic and expert politics

Most policy issues are not salient and politicised in this way. Yet, low salience can exacerbate problems of citizen exclusion. Policies dominated by bureaucratic interests often alienate citizens receiving services. Or a small elite dominates policymaking when there is high acceptance that (a) the best policy is ‘evidence based’, and (b) the evidence should come from experts.

Overall, SCPD describes a political system with major potential to diminish democracy, containing key actors (a) politicising issues to reward or punish populations or (b) depoliticising issues with reference to science and objectivity. In both cases, policy design is not informed by routine citizen participation.

Take home message for students: SCPD began as Schneider and Ingram’s description of the US political system’s failure to solve major problems including inequality, poverty, crime, racism, sexism, and effective universal healthcare and education. Think about how its key drivers apply elsewhere: (1) some people make and exploit quick and emotional judgements for political gain, and others refer to expertise to limit debate; (2) these judgements inform policy design; and, (3) policy design sends signals to citizens which can diminish or boost their incentive to engage in politics.

For more, see the 1000-word and 5000-word versions. The latter has a detailed guide to further reading.

 

 

 

 

Evidence based policymaking: 7 key themes

I looked back at my blog posts on the politics of ‘evidence based policymaking’ and found that I wrote quite a lot (particularly from 2016). Here is a list based on 7 key themes.

1. Use psychological insights to influence the use of evidence

My most-current concern. The same basic theme is that (a) people (including policymakers) are ‘cognitive misers’ seeking ‘rational’ and ‘irrational’ shortcuts to gather information for action, so you won’t get far if you (b) bombard them with information, or (c) call them idiots.

Three ways to communicate more effectively with policymakers (shows how to use psychological insights to promote evidence in policymaking)

Using psychological insights in politics: can we do it without calling our opponents mental, hysterical, or stupid? (yes)

The Psychology of Evidence Based Policymaking: Who Will Speak For the Evidence if it Doesn’t Speak for Itself? (older paper, linking studies of psychology with studies of EBPM)

Older posts on the same theme:

Is there any hope for evidence in emotional debates and chaotic government? (yes)

We are in danger of repeating the same mistakes if we bemoan low attention to ‘facts’

These complaints about ignoring science seem biased and naïve – and too easy to dismiss

How can we close the ‘cultural’ gap between the policymakers and scientists who ‘just don’t get it’?

2. How to use policy process insights to influence the use of evidence

I try to simplify key insights about the policy process to show to use evidence in it. One key message is to give up on the idea of an orderly policy process described by the policy cycle model. What should you do if a far more complicated process exists?

Why don’t policymakers listen to your evidence?

The Politics of Evidence Based Policymaking: 3 messages (3 ways to say that you should engage with the policy process that exists, not a mythical process that will never exist)

Three habits of successful policy entrepreneurs (shows how entrepreneurs are influential in politics)

Why doesn’t evidence win the day in policy and policymaking? and What does it take to turn scientific evidence into policy? Lessons for illegal drugs from tobacco and There is no blueprint for evidence-based policy, so what do you do? (3 posts describing the conditions that must be met for evidence to ‘win the day’)

Writing for Impact: what you need to know, and 5 ways to know it (explains how our knowledge of the policy process helps communicate to policymakers)

How can political actors take into account the limitations of evidence-based policy-making? 5 key points (presentation to European Parliament-European University Institute ‘Policy Roundtable’ 2016)

Evidence Based Policy Making: 5 things you need to know and do (presentation to Open Society Foundations New York 2016)

What 10 questions should we put to evidence for policy experts? (part of a series of videos produced by the European Commission)

3. How to combine principles on ‘good evidence’, ‘good governance’, and ‘good practice’

My argument here is that EBPM is about deciding at the same time what is: (1) good evidence, and (2) a good way to make and deliver policy. If you just focus on one at a time – or consider one while ignoring the other – you cannot produce a defendable way to promote evidence-informed policy delivery.

Kathryn Oliver and I have just published an article on the relationship between evidence and policy (summary of and link to our article on this very topic)

We all want ‘evidence based policy making’ but how do we do it? (presentation to the Scottish Government on 2016)

The ‘Scottish Approach to Policy Making’: Implications for Public Service Delivery

The politics of evidence-based best practice: 4 messages

The politics of implementing evidence-based policies

Policy Concepts in 1000 Words: the intersection between evidence and policy transfer

Key issues in evidence-based policymaking: comparability, control, and centralisation

The politics of evidence and randomised control trials: the symbolic importance of family nurse partnerships

What Works (in a complex policymaking system)?

How Far Should You Go to Make Sure a Policy is Delivered?

4. Face up to your need to make profound choices to pursue EBPM

These posts have arisen largely from my attendance at academic-practitioner conferences on evidence and policy. Many participants tell the same story about the primacy of scientific evidence challenged by post-truth politics and emotional policymakers. I don’t find this argument convincing or useful. So, in many posts, I challenge these participants to think about more pragmatic ways to sum up and do something effective about their predicament.

Political science improves our understanding of evidence-based policymaking, but does it produce better advice? (shows how our knowledge of policymaking clarifies dilemmas about engagement)

The role of ‘standards for evidence’ in ‘evidence informed policymaking’ (argues that a strict adherence to scientific principles may help you become a good researcher but not an effective policy influencer)

How far should you go to secure academic ‘impact’ in policymaking? From ‘honest brokers’ to ‘research purists’ and Machiavellian manipulators (you have to make profound ethical and strategic choices when seeking to maximise the use of evidence in policy)

Principles of science advice to government: key problems and feasible solutions (calling yourself an ‘honest broker’ while complaining about ‘post-truth politics’ is a cop out)

What sciences count in government science advice? (political science, obvs)

I know my audience, but does my other audience know I know my audience? (compares the often profoundly different ways in which scientists and political scientists understand and evaluate EBPM – this matters because, for example, we rarely discuss power in scientist-led debates)

Is Evidence-Based Policymaking the same as good policymaking? (no)

Idealism versus pragmatism in politics and policymaking: … evidence-based policymaking (how to decide between idealism and pragmatism when engaging in politics)

Realistic ‘realist’ reviews: why do you need them and what might they look like? (if you privilege impact you need to build policy relevance into systematic reviews)

‘Co-producing’ comparative policy research: how far should we go to secure policy impact? (describes ways to build evidence advocacy into research design)

The Politics of Evidence (review of – and link to – Justin Parkhurt’s book on the ‘good governance’ of evidence production and use)

5. For students and researchers wanting to read/ hear more

These posts are relatively theory-heavy, linking quite clearly to the academic study of public policy. Hopefully they provide a simple way into the policy literature which can, at times, be dense and jargony.

‘Evidence-based Policymaking’ and the Study of Public Policy

Policy Concepts in 1000 Words: ‘Evidence Based Policymaking’

Practical Lessons from Policy Theories (series of posts on the policy process, offering potential lessons for advocates of evidence use in policy)

Writing a policy paper and blog post 

12 things to know about studying public policy

Can you want evidence based policymaking if you don’t really know what it is? (defines each word in EBPM)

Can you separate the facts from your beliefs when making policy? (no, very no)

Policy Concepts in 1000 Words: Success and Failure (Evaluation) (using evidence to evaluate policy is inevitably political)

Policy Concepts in 1000 Words: Policy Transfer and Learning (so is learning from the experience of others)

Four obstacles to evidence based policymaking (EBPM)

What is ‘Complex Government’ and what can we do about it? (read about it)

How Can Policy Theory Have an Impact on Policy Making? (on translating policy theories into useful advice)

The role of evidence in UK policymaking after Brexit (argues that many challenges/ opportunities for evidence advocates will not change after Brexit)

Why is there more tobacco control policy than alcohol control policy in the UK? (it’s not just because there is more evidence of harm)

Evidence Based Policy Making: If You Want to Inject More Science into Policymaking You Need to Know the Science of Policymaking and The politics of evidence-based policymaking: focus on ambiguity as much as uncertainty and Revisiting the main ‘barriers’ between evidence and policy: focus on ambiguity, not uncertainty and The barriers to evidence based policymaking in environmental policy (early versions of what became the chapters of the book)

6. Using storytelling to promote evidence use

This is increasingly a big interest for me. Storytelling is key to the effective conduct and communication of scientific research. Let’s not pretend we’re objective people just stating the facts (which is the least convincing story of all). So far, so good, except to say that the evidence on the impact of stories (for policy change advocacy) is limited. The major complication is that (a) the story you want to tell and have people hear interacts with (b) the story that your audience members tell themselves.

Combine Good Evidence and Emotional Stories to Change the World

Storytelling for Policy Change: promise and problems

Is politics and policymaking about sharing evidence and facts or telling good stories? Two very silly examples from #SP16

7. The major difficulties in using evidence for policy to reduce inequalities

These posts show how policymakers think about how to combine (a) often-patchy evidence with (b) their beliefs and (c) an electoral imperative to produce policies on inequalities, prevention, and early intervention. I suggest that it’s better to understand and engage with this process than complain about policy-based-evidence from the side-lines. If you do the latter, policymakers will ignore you.

The UK government’s imaginative use of evidence to make policy 

What do you do when 20% of the population causes 80% of its problems? Possibly nothing.

The theory and practice of evidence-based policy transfer: can we learn how to reduce territorial inequalities?

We need better descriptions than ‘evidence-based policy’ and ‘policy-based evidence’: the case of UK government ‘troubled families’ policy

How can you tell the difference between policy-based-evidence and evidence-based-policymaking?

Early intervention policy, from ‘troubled families’ to ‘named persons’: problems with evidence and framing ‘valence’ issues

Key issues in evidence-based policymaking: comparability, control, and centralisation

The politics of evidence and randomised control trials: the symbolic importance of family nurse partnerships

Two myths about the politics of inequality in Scotland

Social investment, prevention and early intervention: a ‘window of opportunity’ for new ideas?

A ‘decisive shift to prevention’: how do we turn an idea into evidence based policy?

Can the Scottish Government pursue ‘prevention policy’ without independence?

Note: these issues are discussed in similar ways in many countries. One example that caught my eye today:

"Changing concepts of 'evidence'" visual summary @fletchermj @PowerToPersuade @PSResearchG #P2P2017 pic.twitter.com/6M9QgTXAyW

— Lisa Carson (@LisaC_Research) August 23, 2017

 

All of this discussion can be found under the EBPM category: https://paulcairney.wordpress.com/category/evidence-based-policymaking-ebpm/T

See also the special issue on maximizing the use of evidence in policy

Telling Stories that Shape Public Policy

This is a guest post by Michael D. Jones (left) and Deserai Anderson Crow (right), discussing how to use insights from the Narrative Policy Framework to think about how to tell effective stories to achieve policy goals. The full paper has been submitted to the series for Policy and Politics called Practical Lessons from Policy Theories.

Imagine. You are an ecologist. You recently discovered that a chemical that is discharged from a local manufacturing plant is threatening a bird that locals love to watch every spring. Now, imagine that you desperately want your research to be relevant and make a difference to help save these birds. All of your training gives you depth of expertise that few others possess. Your training also gives you the ability to communicate and navigate things such as probabilities, uncertainty, and p-values with ease.

But as NPR’s Robert Krulwich argues, focusing on this very specialized training when you communicate policy problems could lead you in the wrong direction. While being true to the science and best practices of your training, one must also be able to tell a compelling story.  Perhaps combine your scientific findings with the story about the little old ladies who feed the birds in their backyards on spring mornings, emphasizing the beauty and majesty of these avian creatures, their role in the community, and how the toxic chemicals are not just a threat to the birds, but are also a threat to the community’s understanding of itself and its sense of place.  The latest social science is showing that if you tell a good story, your policy communications are likely to be more effective.

Why focus on stories?

The world is complex. We are bombarded with information as we move through our lives and we seek patterns within that information to simplify complexity and reduce ambiguity, so that we can make sense of the world and act within it.

The primary means by which human beings render complexity understandable and reduce ambiguity is through the telling of stories. We “fit” the world around us and the myriad of objects and people therein, into story patterns. We are by nature storytelling creatures. And if it is true of us as individuals, then we can also safely assume that storytelling matters for public policy where complexity and ambiguity abound.

Based on our (hopefully) forthcoming article (which has a heavy debt to Jones and Peterson, 2017 and Catherine Smith’s popular textbook) here we offer some abridged advice synthesizing some of the most current social science findings about how best to engage public policy storytelling. We break it down into five easy steps and offer a short discussion of likely intervention points within the policy process.

The 5 Steps of Good Policy Narrating

1. Tell a Story: Remember, facts never speak for themselves. If you are presenting best practices, relaying scientific information, or detailing cost/benefit analyses, you are telling or contributing to a story.  Engage your storytelling deliberately.
2. Set the Stage: Policy narratives have a setting and in this setting you will find specific evidence, geography, legal parameters, and other policy consequential items and information.  Think of these setting items as props.  Not all stages can hold every relevant prop.  Be true to science; be true to your craft, but set your stage with props that maximize the potency of your story, which always includes making your setting amenable to your audience.
3. Establish the Plot: In public policy plots usually define the problem (and polices do not exist without at least a potential problem). Define your problem. Doing so determines the causes, which establishes blame.
4. Cast the Characters:  Having established a plot and defined your problem, the roles you will need your characters to play become apparent. Determine who the victim is (who is harmed by the problem), who is responsible (the villain) and who can bring relief (the hero). Cast characters your audience will appreciate in their roles.
5. Clearly Specify the Moral: Postmodern films might get away without having a point.  Policy narratives usually do not. Let your audience know what the solution is.

Public Policy Intervention Points

There are crucial points in the policy process where actors can use narratives to achieve their goals. We call these “intervention points” and all intervention points should be viewed as opportunities to tell a good policy story, although each will have its own constraints.

These intervention points include the most formal types of policy communication such as crafting of legislation or regulation, expert testimony or statements, and evaluation of policies. They also include less formal communications through the media and by citizens to government.

Each of these interventions can frequently be dry and jargon-laden, but it’s important to remember that by employing effective narratives within any of them, you are much more likely to see your policy goals met.

When considering how to construct your story within one or more of the various intervention points, we urge you to first consider several aspects of your role as a narrator.

1. Who are you and what are your goals? Are you an outsider trying to affect change to solve a problem or push an agency to do something it might not be inclined to do?  Are you an insider trying to evaluate and improve policy making and implementation? Understanding your role and your goals is essential to both selecting an appropriate intervention point and optimizing your narrative therein.
2. Carefully consider your audience. Who are they and what is their posture towards your overall goal? Understanding your audience’s values and beliefs is essential for avoiding invoking defensiveness.
3. There is the intervention point itself – what is the best way to reach your audience? What are the rules for the type of communication you plan to use? For example, media communications can be done with lengthy press releases, interviews with the press, or in the confines of a simple tweet.  All of these methods have both formal and informal constraints that will determine what you can and can’t do.

Without deliberate consideration of your role, audience, the intervention point, and how your narrative links all of these pieces together, you are relying on chance to tell a compelling policy story.

On the other hand, thoughtful and purposeful storytelling that remains true to you, your values, your craft, and your best understanding of the facts, can allow you to be both the ecologist and the bird lover.

 

Three habits of successful policy entrepreneurs

This post is one part of a series – called Practical Lessons from Policy Theories and it summarizes this article (PDF).

‘Policy entrepreneurs’ invest their time wisely for future reward, and possess key skills that help them adapt particularly well to their environments. They are the agents for policy change who possess the knowledge, power, tenacity, and luck to be able to exploit key opportunities. They draw on three strategies:

1. Don’t focus on bombarding policymakers with evidence.

Scientists focus on making more evidence to reduce uncertainty, but put people off with too much information. Entrepreneurs tell a good story, grab the audience’s interest, and the audience demands information.

2. By the time people pay attention to a problem it’s too late to produce a solution.

So, you produce your solution then chase problems.

3. When your environment changes, your strategy changes.

For example, in the US federal level, you’re in the sea, and you’re a surfer waiting for the big wave. In the smaller subnational level, on a low attention and low budget issue, you can be Poseidon moving the ‘streams’. In the US federal level, you need to ‘soften’ up solutions over a long time to generate support. In subnational or other countries, you have more opportunity to import and adapt ready-made solutions.

It all adds up to one simple piece of advice – timing and luck matters when making a policy case – but policy entrepreneurs know how to influence timing and help create their own luck.

Full paper: Three habits of successful policy entrepreneurs

(Note: the previous version was friendlier and more focused on entrepreneurs)

For more on ‘multiple streams’ see:

Paul Cairney and Michael Jones (2016) ‘Kingdon’s Multiple Streams Approach: What Is the Empirical Impact of this Universal Theory?’ Policy Studies Journal, 44, 1, 37-58 PDF (Annex to Cairney Jones 2016) (special issue of PSJ)

Paul Cairney and Nikos Zahariadis (2016) ‘Multiple streams analysis: A flexible metaphor presents an opportunity to operationalize agenda setting processes’ in Zahariadis, N. (eds) Handbook of Public Policy Agenda-Setting (Cheltenham: Edward Elgar) PDF see also

I use a space launch metaphor in the paper. If you prefer different images, have a look at 5 images of the policy process. If you prefer a watery metaphor (it’s your life, I suppose), click Policy Concepts in 1000 Words: Multiple Streams Analysis

For more on entrepreneurs:

So you want to be a policy entrepreneur? https://t.co/y8RG7MOSKV by @MikeMintrom

— Professor Paul Cairney (@CairneyPaul) October 29, 2019

Writing for Impact: what you need to know, and 5 ways to know it

This is a post for my talk at the ‘Politheor: European Policy Network’ event Write For Impact: Training In Op-Ed Writing For Policy Advocacy. There are other speakers with more experience of, and advice on, ‘op-ed’ writing. My aim is to describe key aspects of politics and policymaking to help the audience learn why they should write op-eds in a particular way for particular audiences.

A key rule in writing is to ‘know your audience’, but it’s easier said than done if you seek many sympathetic audiences in many parts of a complex policy process. Two simple rules should help make this process somewhat clearer:

1. Learn how policymakers simplify their world, and
2. Learn how policy environments influence their attention and choices.

We can use the same broad concepts to help explain both processes, in which many policymakers and influencers interact across many levels and types of government to produce what we call ‘policy’:

1. Policymaker psychology: tell an evidence-informed story

Policymakers receive too much information, and seek ways to ignore most of it while making decisions. To do so, they use ‘rational’ and ‘irrational’ means: selecting a limited number of regular sources of information, and relying on emotion, gut instinct, habit, and familiarity with information. In other words, your audience combines cognition and emotion to deal with information, and they can ignore information for long periods then quickly shift their attention towards it, even if that information has not really changed.

Consequently, an op-ed focusing solely ‘the facts’ can be relatively ineffective compared to an evidence-informed story, perhaps with a notional setting, plot, hero, and moral. Your aim shifts from providing more and more evidence to reduce uncertainty about a problem, to providing a persuasive reason to reduce ambiguity. Ambiguity relates to the fact that policymakers can understand a policy problem in many different ways – such as tobacco as an economic good, issue of civil liberties, or public health epidemic – but often pay exclusive attention to one.

So, your aim may be to influence the simple ways in which people understand the world, to influence their demand for more information. An emotional appeal can transform a factual case, but only if you know how people engage emotionally with information. Sometimes, the same story can succeed with one audience but fail with another.

2. Institutions: learn the ‘rules of the game’

Institutions are the rules people use in policymaking, including the formal, written down, and well understood rules setting out who is responsible for certain issues, and the informal, unwritten, and unclear rules informing action. The rules used by policymakers can help define the nature of a policy problem, who is best placed to solve it, who should be consulted routinely, and who can safely be ignored. These rules can endure for long periods and become like habits, particularly if policymakers pay little attention to a problem or why they define it in a particular way.

3. Networks and coalitions: build coalitions and establish trust

Such informal rules, about how to understand a problem and who to speak with about it, can be reinforced in networks of policymakers and influencers.

‘Policy community’ partly describes a sense that most policymaking is processed out of the public spotlight, often despite minimal high level policymaker interest. Senior policymakers delegate responsibility for policymaking to bureaucrats, who seek information and advice from groups. Groups exchange information for access to, and potential influence within, government, and policymakers have ‘standard operating procedures’ that favour particular sources of evidence and some participants over others

‘Policy community’ also describes a sense that the network seems fairly stable, built on high levels of trust between participants, based on factors such as reliability (the participant was a good source of information, and did not complain too much in public about decisions), a common aim or shared understanding of the problem, or the sense that influencers represent important groups.

So, the same policy case can have a greater impact if told by a well trusted actor in a policy community. Or, that community member may use networks to build key coalitions behind a case, use information from the network to understand which cases will have most impact, or know which audiences to seek.

4. Ideas: learn the ‘currency’ of policy argument

This use of networks relates partly to learning the language of policy debate in particular ‘venues’, to learn what makes a convincing case. This language partly reflects a well-established ‘world view’ or the ‘core beliefs’ shared by participants. For example, a very specific ‘evidence-based’ language is used frequently in public health, while treasury departments look for some recognition of ‘value for money’ (according to a particular understanding of how you determine VFM). So, knowing your audience is knowing the terms of debate that are often so central to their worldview that they take them for granted and, in contrast, the forms of argument that are more difficult to pursue because they are challenging or unfamiliar to some audiences. Imagine a case that challenges completely someone’s world view, or one which is entirely consistent with it.

5. Socioeconomic factors and events: influence how policymakers see the outside world

Some worldviews can be shattered by external events or crises, but this is a rare occurrence. It may be possible to generate a sense of crisis with reference to socioeconomic changes or events, but people will interpret these developments through the ‘lens’ of their own beliefs. In some cases, events seem impossible to ignore but we may not agree on their implications for action. In others, an external event only matters if policymakers pay attention to them. Indeed, we began this discussion with the insight that policymakers have to ignore almost all such information available to them.

Know your audience revisited: practical lessons from policy theories

To take into account all of these factors, while trying to make a very short and persuasive case, may seem impossible. Instead, we might pick up some basic rules of thumb from particular theories or approaches. We can discuss a few examples from ongoing work on ‘practical lessons from policy theories’.

Storytelling for policy impact

If you are telling a story with a setting, plot, hero, and moral, it may be more effective to focus on a hero than villain. More importantly, imagine two contrasting audiences: one is moved by your personal and story told to highlight some structural barriers to the wellbeing of key populations; another is unmoved, judges that person harshly, and thinks they would have done better in their shoes (perhaps they prefer to build policy on stereotypes of target populations). ‘Knowing your audience’ may involve some trial-and-error to determine which stories work under which circumstances.

Appealing to coalitions

Or, you may decide that it is impossible to write anything to appeal to all relevant audiences. Instead, you might tailor it to one, to reinforce its beliefs and encourage people to act. The ‘advocacy coalition framework’ describes such activities as routine: people go into politics to translate their beliefs into policy, they interpret the world through those beliefs, and they romanticise their own cause while demonising their opponents. If so, would a bland op-ed have much effect on any audience?

Learning from entrepreneurs

‘Policy entrepreneurs’ draw on three rules, two of which seem counterintuitive:

1. Don’t focus on bombarding policymakers with evidence. Scientists focus on making more evidence to reduce uncertainty, but put people off with too much information. Entrepreneurs tell a good story, grab the audience’s interest, and the audience demands information.
2. By the time people pay attention to a problem it’s too late to produce a solution. So, you produce your solution then chase problems.
3. When your environment changes, your strategy changes. For example, in the US federal level, you’re in the sea, and you’re a surfer waiting for the big wave. In the smaller subnational level, on a low attention and low budget issue, you can be Poseidon moving the ‘streams’. In the US federal level, you need to ‘soften’ up solutions over a long time to generate support. In subnational or other countries, you have more opportunity to import and adapt ready-made solutions.

It all adds up to one simple piece of advice – timing and luck matters when making a policy case – but policy entrepreneurs know how to influence timing and help create their own luck.

On the day, we can use such concepts to help us think through the factors that you might think about while writing op-eds, even though it is very unlikely that you would mention them in your written work.

I know my audience, but does my other audience know I know my audience?

‘Know your audience’ is a key phrase for anyone trying to convey a message successfully. To ‘know your audience’ is to understand the rules they use to make sense of your message, and therefore the adjustments you have to make to produce an effective message. Simple examples include:

• The sarcasm rules. The first rule is fairly explicit. If you want to insult someone’s shirt, you (a) say ‘nice shirt, pal’, but also (b) use facial expressions or unusual speech patterns to signal that you mean the opposite of what you are saying. Otherwise, you’ve inadvertently paid someone a compliment, which is just not on. The second rule is implicit. Sarcasm is sometimes OK – as a joke or as some nice passive aggression – and a direct insult (‘that shirt is shite, pal’) as a joke is harder to pull off.
• The joke rule. If you say that you went to the doctor because a strawberry was growing out of your arse and the doctor gave you some cream for it, you’d expect your audience to know you were joking because it’s such a ridiculous scenario and there’s a pun. Still, there’s a chance that, if you say it quickly, with a straight face, your audience is not expecting a joke, and/ or your audience’s first language is not English, your audience will take you seriously, if only for a second. It’s hilarious if your audience goes along with you, and a bit awkward if your audience asks kindly about your welfare.
• Keep it simple stupid. If someone says KISS, or some modern equivalent – ‘it’s the economy, stupid’, the rule is that, generally, they are not calling you stupid (even though the insertion of the comma, in modern phrases, makes it look like they are). They are referring to the value of a simple design or explanation that as many people as possible can understand. If your audience doesn’t know the phrase, they may think you’re calling them stupid, stupid.

These rules can be analysed from various perspectives: linguistics, focusing on how and why rules of language develop; and philosophy, to help articulate how and why rules matter in sense making.

There is also a key role for psychological insights, since – for example – a lot of these rules relate to the routine ways in which people engage emotionally with the ‘signals’ or information they receive.

Think of the simple example of twitter engagement, in which people with emotional attachments to one position over another (say, pro- or anti- Brexit), respond instantly to a message (say, pro- or anti- Brexit). While some really let themselves down when they reply with their own tweet, and others don’t say a word, neither audience is immune from that emotional engagement with information. So, to ‘know your audience’ is to anticipate and adapt to the ways in which they will inevitably engage ‘rationally’ and ‘irrationally’ with your message.

I say this partly because I’ve been messing around with some simple ‘heuristics’ built on insights from psychology, including Psychology Based Policy Studies: 5 heuristics to maximise the use of evidence in policymaking .

Two audiences in the study of ‘evidence based policymaking’

I also say it because I’ve started to notice a big unintended consequence of knowing my audience: my one audience doesn’t like the message I’m giving the other. It’s a bit like gossip: maybe you only get away with it if only one audience is listening. If they are both listening, one audience seems to appreciate some new insights, while the other wonders if I’ve ever read a political science book.

The problem here is that two audiences have different rules to understand the messages that I help send. Let’s call them ‘science’ and ‘political science’ (please humour me – you’ve come this far). Then, let’s make some heroic binary distinctions in the rules each audience would use to interpret similar issues in a very different way.

I could go on with these provocative distinctions, but you get the idea. A belief taken for granted in one field will be treated as controversial in another. In one day, you can go to one workshop and hear the story of objective evidence, post-truth politics, and irrational politicians with low political will to select evidence-based policies, then go to another workshop and hear the story of subjective knowledge claims.

Or, I can give the same presentation and get two very different reactions. If these are the expectations of each audience, they will interpret and respond to my messages in very different ways.

So, imagine I use some psychology insights to appeal to the ‘science’ audience. I know that,  to keep it on side and receptive to my ideas, I should begin by being sympathetic to its aims. So, my implicit story is along the lines of, ‘if you believe in the primacy of science and seek evidence-based policy, here is what you need to do: adapt to irrational policymaking and find out where the action is in a complex policymaking system’. Then, if I’m feeling energetic and provocative, I’ll slip in some discussion about knowledge claims by saying something like, ‘politicians (and, by the way, some other scholars) don’t share your views on the hierarchy of evidence’, or inviting my audience to reflect on how far they’d go to override the beliefs of other people (such as the local communities or service users most affected by the evidence-based policies that seem most effective).

The problem with this story is that key parts are implicit and, by appearing to go along with my audience, I provoke a reaction in another audience: don’t you know that many people have valid knowledge claims? Politics is about values and power, don’t you know?

So, that’s where I am right now. I feel like I ‘know my audience’ but I am struggling to explain to my original political science audience that I need to describe its insights in a very particular way to have any traction in my other science audience. ‘Know your audience’ can only take you so far unless your other audience knows that you are engaged in knowing your audience.

If you want to know more, see:

Kathryn Oliver and I have just published an article on the relationship between evidence and policy

How far should you go to secure academic ‘impact’ in policymaking? From ‘honest brokers’ to ‘research purists’ and Machiavellian manipulators

Why doesn’t evidence win the day in policy and policymaking?

The Science of Evidence-based Policymaking: How to Be Heard

When presenting evidence to policymakers, engage with the policy process that exists, not the process you wish existed

 

 

Racism and Stories in Scottish Politics

Brexit boosts the case for Scottish independence because it can now be framed more easily as the cosmopolitan choice: vote Yes to get away from a ‘little England’ mentality. This possibility was perhaps in Labour’s mind when it described Scottish nationalism as anything but cosmopolitan. For example, Sadiq Khan seemed to link Scottish nationalism strongly with the divisiveness of Trump and Leave campaigns, prompting some debate about how far he went to equate it with bigotry and racism, while former Labour minister Douglas Alexander left less room for doubt.

These claims prompted a small number of commentary pieces supporting or rejecting the idea that Scottish nationalism fosters racism, bigotry, and/ or social division:

• Claire Heuchan welcomed Khan’s intervention cautiously, highlighting a tendency of Scottish actors to assert their superiority over their English counterparts, and using the opportunity to expand the debate, to highlight important issues that we often ignore, from personal stories of racist abuse and examples of more limited education and employment opportunities for people of colour, to the role of Scotland in the British empire’s colonial past built on slavery and exploitation.
• In contrast, Robert Somynne identified a civic Scottish nationalism far apart from a ‘western trend towards populism based on tribal and ethnic divisions’, arguing that Khan’s description ‘doesn’t bear out the experience of so many people of colour in Scotland who campaigned in the grassroots’.
• Kevin McKenna ridiculed Khan’s argument, rejecting the idea of nationalism underpinned by anti-Englishness, identifying a more divisive UK politics of which Labour is a key part, and dismissing Khan and others as part of ‘the leftwing intelligentsia’. John McKee argued that current Scottish nationalism is more about rejecting the British state than British people, while Eric Joyce links it more to rejecting more worrying forms of nationalism pursued by parties like UKIP.

The debate rages on in twitter, but the discussion has not been driven primarily by a willingness to listen, engage constructively, or talk about issues that challenge our beliefs. I don’t suppose you need me to explain why, but it’s worth highlighting three analytically-separate explanations that will likely be present throughout all debates like this:

1. The devil shift undermines debate

People form coalitions with the people who share their beliefs, and they compete with people who don’t. The ‘devil shift’ describes a form of ‘groupthink’; a tendency of actors in those coalitions to romanticise their own cause and demonise the cause of their opponents: ‘anyone who disagrees with them must be mistaken about the facts, operating from the wrong value premises, or acting from evil motive’ (Sabatier, Hunter and McLaughlin). They question the motives of their opponents but not their allies, subject only their opponent’s arguments to criticism, and think their opponents are more powerful than they are.

If all debates are interpreted through this lens of Scottish independence, you can predict the results: Yes groups will see Khan’s intervention as threatening to their beliefs and aims, No groups will embrace it because it supports their beliefs and aims, and there is little scope for conflict resolution. Indeed, during such debates, we’ll put up with some sketchy characters if they support our cause while denouncing their equivalents in the other coalition.

This dynamic will be apparent each time we interpret each issue. For example, imagine a semi-honest and open discussion about Scotland and the colonial empire. One side might combine two points – Scotland ‘punches above its weight’ in every endeavour, and Scotland was part of the empire – to argue that Scotland played a disproportionate role in colonialism and slavery. The other will remind us of the unequal and coerced alliance of 1707 and/ or blame an unelected elite in Scotland for the Union and empire, to argue that Scottish independence is the best way to reject the colonial past. The same history has a very different villain and moral.

2. There are two colliding roles of storytelling

We can identify two main roles of personal storytelling: (1) to empower an individual, when they share their experiences of life and feel listened to, and (2) to take forward a political agenda, when they identify a hero/ villain and moral that suits one coalition’s beliefs and aims.

In our case, it is difficult to separate the two, and most people are not willing or able to do so. This may be understandable with Khan’s recent intervention since, although he generally has an important story to tell from his perspective, this specific speech seemed designed to bolster the position of Scottish Labour at its party conference. Heuchan’s experience is more worrying, her motives seemed far less instrumental and her personal story was worth listening to, but almost no-one has simply said ‘thank you for your story’ without adding conditions or objections. There are very few spaces in which people will listen rather than judge.

3. Some topics are unusually personal

In part, this is because few people like to think that they are racist or bigoted. Some won’t think about the ways in which they benefit from the systematic effects of racism – in which some groups benefit disproportionately from education/ employment opportunities and face a smaller risk of personal abuse – partly because they don’t have to, and it’s generally not an enjoyable experience. Others will flinch at the idea that they are privileged because they are white, often because they have vivid memories of personal experiences of abuse or disadvantage linked to another part of their background, such as their gender, class, religion, or disability.

So, we often want to tell our stories without listing to those of others. If there is no such space in which to exchange the details of such stories, we soon end up with heated, futile debates based on the sense that you don’t understand my experience or perspective before you criticise it. We can’t solve this problem, but we should at least be aware of it, and perhaps be aware that, although the Scottish debate has some unusual features, it is one of many examples of routine divisive politics.