Category Archives: Storytelling

Policy Analysis in 750 Words: policy analysis for marginalized groups in racialized political systems

Note: this post forms one part of the Policy Analysis in 750 words series overview.

For me, this story begins with a tweet by Professor Jamila Michener, about a new essay by Dr Fabienne Doucet, ‘Centering the Margins: (Re)defining Useful Research Evidence Through Critical Perspectives’:

Research and policy analysis for marginalized groups

For Doucet (2019: 1), it begins by describing the William T. Grant Foundation’s focus on improving the ‘use of research evidence’ (URE), and the key questions that we should ask when improving URE:

  1. For what purposes do policymakers find evidence useful?

Examples include to: inform a definition of problems and solutions, foster practitioner learning, support an existing political position, or impose programmes backed by evidence (compare with How much impact can you expect from your analysis?).

  1.   Who decides what to use, and what is useful?

For example, usefulness could be defined by the researchers providing evidence, the policymakers using it, the stakeholders involved in coproduction, or the people affected by research and policy (compare with Bacchi, Stone and Who should be involved in the process of policy analysis?).

  1. How do critical theories inform these questions? (compare with T. Smith)

First, they remind us that so-called ‘rational’ policy processes have incorporated research evidence to help:

‘maintain power hierarchies and accept social inequity as a given. Indeed, research has been historically and contemporaneously (mis)used to justify a range of social harms from enslavement, colonial conquest, and genocide, to high-stakes testing, disproportionality in child welfare services, and “broken windows” policing’ (Doucet, 2019: 2)

Second, they help us redefine usefulness in relation to:

‘how well research evidence communicates the lived experiences of marginalized groups so that the understanding of the problem and its response is more likely to be impactful to the community in the ways the community itself would want’ (Doucet, 2019: 3)

In that context, potential responses include to:

  1. Recognise the ways in which research and policy combine to reproduce the subordination of social groups.
  • General mechanisms include: the reproduction of the assumptions, norms, and rules that produce a disproportionate impact on social groups (compare with Social Construction and Policy Design).
  • Specific mechanism include: judging marginalised groups harshly according to ‘Western, educated, industrialized, rich and democratic’ norms (‘WEIRD’)
  1. Reject the idea that scientific research can be seen as objective or neutral (and that researchers are beyond reproach for their role in subordination).
  2. Give proper recognition to ‘experiential knowledge’ and ‘transdiciplinary approaches’ to knowledge production, rather than privileging scientific knowledge.
  3. Commit to social justice, to help ‘eliminate oppressions and to emancipate and empower marginalized groups’, such as by disrupting ‘the policies and practices that disproportionately harm marginalized groups’ (2019: 5-7)
  4. Develop strategies to ‘center race’, ‘democratize’ research production, and ‘leverage’ transdisciplinary methods (including poetry, oral history and narrative, art, and discourse analysis – compare with Lorde) (2019: 10-22)

Policy analysis in a ‘racialized polity’

A key way to understand these processes is to use, and improve, policy theories to explain the dynamics and impacts of a racialized political system. For example, ‘policy feedback theory’ (PFT) draws on elements from historical institutionalism and SCPD to identify the rules, norms, and practices that reinforce subordination.

In particular, Michener’s (2019: 424) ‘Policy Feedback in a Racialized Polity’ develops a ‘racialized feedback framework (RFF)’ to help explain the ‘unrelenting force with which racism and White supremacy have pervaded social, economic, and political institutions in the United States’. Key mechanisms include (2019: 424-6):

  1. Channelling resources’, in which the rules, to distribute government resources, benefit some social groups and punish others.
  • Examples include: privileging White populations in social security schemes and the design/ provision of education, and punishing Black populations disproportionately in prisons (2019: 428-32).
  • These rules also influence the motivation of social groups to engage in politics to influence policy (some citizens are emboldened, others alienated).
  1. Generating interests’, in which ‘racial stratification’ is a key factor in the power of interest groups (and balance of power in them).
  2. Shaping interpretive schema’, in which race is a lens through which actors understand, interpret, and seek to solve policy problems.
  3. The ways in which centralization (making policy at the federal level) or decentralization influence policy design.
  • For example, the ‘historical record’ suggests that decentralization is more likely to ‘be a force of inequality than an incubator of power for people of color’ (2019: 433).

Insufficient attention to race and racism: what are the implications for policy analysis?

One potential consequence of this lack of attention to race, and the inequalities caused by racism in policy, is that we place too much faith in the vague idea of ‘pragmatic’ policy analysis.

Throughout the 750 words series, you will see me refer generally to the benefits of pragmatism:

In that context, pragmatism relates to the idea that policy analysis consists of ‘art and craft’, in which analysts assess what is politically feasible if taking a low-risk client-oriented approach.

In this context, pragmatism may be read as a euphemism for conservatism and status quo protection.

In other words, other posts in the series warn against too-high expectations for entrepreneurial and systems thinking approaches to major policy change, but they should not be read as an excuse to reject ambitious plans for much-needed changes to policy and policy analysis (compare with Meltzer and Schwartz, who engage with this dilemma in client-oriented advice).

Connections to blog themes

This post connects well to:

 

 

Leave a comment

Filed under 750 word policy analysis, Evidence Based Policymaking (EBPM), public policy, Storytelling

Policy Analysis in 750 Words: Reflecting on your role as a policy analyst

This post forms one part of the Policy Analysis in 750 words series overview.

One aim of this series is to combine insights from policy research (1000, 500) and policy analysis texts.

If we take key insights from policy theories seriously, we can use them to identify (a) the constraints to policy analytical capacity, and (b) the ways in which analysts might address them. I use the idea of policy analyst archetypes to compare a variety of possible responses.

Key constraints to policy analytical capacity

Terms like ‘bounded rationality’ highlight major limits on the ability of humans and organisations to process information.

Terms like policymaking ‘context’, ‘environments’, and multi-centric policymaking suggest that the policy process is beyond the limits of policymaker understanding and control.

  • Policy actors need to find ways to act, with incomplete information about the problem they seek to solve and the likely impact of their ‘solution’.
  • They gather information to help reduce uncertainty, but problem definition is really about exercising power to reduce ambiguity: select one way to interpret a problem (at the expense of most others), and limit therefore limit the relevance and feasibility of solutions.
  • This context informs how actors might use the tools of policy analysis. Key texts in this series highlight the use of tools to establish technical feasibility (will it work as intended?), but policymakers also select tools for their political feasibility (who will support or oppose this measure?).

box 2.3 2nd ed UPP

How might policy analysts address these constraints ethically?

Most policy analysis texts (in this series) consider the role of professional ethics and values during the production of policy analysis. However, they also point out that there is not a clearly defined profession and associated code of conduct (e.g. see Adachi). In that context, let us begin with some questions about the purpose of policy analysis and your potential role:

  1. Is your primary role to serve individual clients or some notion of the ‘public good’?
  2. Should you maximise your role as an individual or play your part in a wider profession?
  3. What is the balance between the potential benefits of individual ‘entrepreneurship’ and collective ‘co-productive’ processes?
  4. Which policy analysis techniques should you prioritise?
  5. What forms of knowledge and evidence count in policy analysis?
  6. What does it mean to communicate policy analysis responsibly?
  7. Should you provide a clear recommendation or encourage reflection?

 

Policy analysis archetypes: pragmatists, entrepreneurs, manipulators, storytellers, and decolonisers

In that context, I have created a story of policy analysis archetypes to identify the elements that each text emphasises.

The pragmatic policy analyst

  • Bardach provides the classic simple, workable, 8-step system to present policy analysis to policymakers while subject to time and resource-pressed political conditions.
  • Dunn also uses Wildavsky’s famous phrase ‘art and craft’ to suggest that scientific and ‘rational’ methods can only take us so far.

The professional, clientoriented policy analyst

  • Weimer and Vining provide a similar 7-step client-focused system, but incorporating a greater focus on professional development and economic techniques (such as cost-benefit-analysis) to emphasise a particular form of professional analyst.
  • Meltzer and Schwartz also focus on advice to clients, but with a greater emphasis on a wide variety of methods or techniques (including service design) to encourage the co-design of policy analysis with clients.

The communicative policy analyst

  •  C. Smith focuses on how to write and communicate policy analysis to clients in a political context.
  • Compare with Spiegelhalter and Gigerenzer on how to communicate responsibly when describing uncertainty, probability, and risk.

The manipulative policy analyst.

  • Riker helps us understand the relationship between two aspects of agenda setting: the rules/ procedures to make choice, and the framing of policy problems and solutions.

The entrepreneurial policy analyst

  • Mintrom shows how to combine insights from studies of policy entrepreneurship and policy analysis, to emphasise the benefits of collaboration and creativity.

The questioning policy analyst

  • Bacchi  analyses the wider context in which people give and use such advice, to identify the emancipatory role of analysis and encourage policy analysts to challenge dominant social constructions of problems and populations.

The storytelling policy analyst

  • Stone identifies the ways in which people use storytelling and argumentation techniques to define problems and justify solutions. This process is about politics and power, not objectivity and optimal solutions.

The decolonizing policy analyst.

  • L.T. Smith does not describe policy analysis directly, but shows how the ‘decolonization of research methods’ can inform the generation and use of knowledge.
  • Compare with Hindess on the ways in which knowledge-based hierarchies rely on an untenable, circular logic.
  • Compare with Michener’s thread, discussing Doucet’s new essay on (a) the role of power and knowledge in limiting (b) the ways in which we gather evidence to analyse policy problems.

Using archetypes to define the problem of policy analysis

Studies of the field (e.g. Radin plus Brans, Geva-May, and Howlett) suggest that there are many ways to do policy analysis. Further, as Thissen and Walker describe, such roles are not mutually exclusive, your views on their relative value could change throughout the process of analysis, and you could perform many of these roles.

Further, each text describes multiple roles, and some seem clustered together:

  • pragmatic, client-orientated, and communicative could sum-up the traditional 5-8 step approaches, while
  • questioning, storytelling, and decolonizing could sum up an important (‘critical’) challenge to narrow ways of thinking about policy analysis and the use of information.

Still, the emphasis matters.

Each text is setting an agenda or defining the problem of policy analysis more-or-less in relation to these roles. Put simply, the more you are reading about economic theory and method, the less you are reading about dominance and manipulation.

How can you read further?

Michener’s ‘Policy Feedback in a Racialized Polity’ connects to studies of historical institutionalism, and reminds us to use insights from policy theories to identify the context for policy analysis.

I have co-authored a lot about uncertainty/ ambiguity in relation to ‘evidence based policymaking’, including:

See also The new policy sciences for a discussion of how these issues inform Lasswell’s original vision for the policy sciences (combining the analysis of and for policy).

2 Comments

Filed under 750 word policy analysis, agenda setting, Evidence Based Policymaking (EBPM), feminism, public policy, Storytelling

Policy Analysis in 750 words: Deborah Stone (2012) Policy Paradox

Please see the Policy Analysis in 750 words series overview before reading the summary. This post is 750 words plus a bonus 750 words plus some further reading that doesn’t count in the word count even though it does.

Stone policy paradox 3rd ed cover

Deborah Stone (2012) Policy Paradox: The Art of Political Decision Making 3rd edition (Norton)

‘Whether you are a policy analyst, a policy researcher, a policy advocate, a policy maker, or an engaged citizen, my hope for Policy Paradox is that it helps you to go beyond your job description and the tasks you are given – to think hard about your own core values, to deliberate with others, and to make the world a better place’ (Stone, 2012: 15)

Stone (2012: 379-85) rejects the image of policy analysis as a ‘rationalist’ project, driven by scientific and technical rules, and separable from politics. Rather, every policy analyst’s choice is a political choice – to define a problem and solution, and in doing so choosing how to categorise people and behaviour – backed by strategic persuasion and storytelling.

The Policy Paradox: people entertain multiple, contradictory, beliefs and aims

Stone (2012: 2-3) describes the ways in which policy actors compete to define policy problems and public policy responses. The ‘paradox’ is that it is possible to define the same policies in contradictory ways.

‘Paradoxes are nothing but trouble. They violate the most elementary principle of logic: something can’t be two different things at once. Two contradictory interpretations can’t both be true. A paradox is just such an impossible situation, and political life is full of them’ (Stone, 2012: 2).

This paradox does not refer simply to a competition between different actors to define policy problems and the success or failure of solutions. Rather:

  • The same actor can entertain very different ways to understand problems, and can juggle many criteria to decide that a policy outcome was a success and a failure (2012: 3).
  • Surveys of the same population can report contradictory views – encouraging a specific policy response and its complete opposite – when asked different questions in the same poll (2012: 4; compare with Riker)

Policy analysts: you don’t solve the Policy Paradox with a ‘rationality project’

Like many posts in this series (Smith, Bacchi, Hindess), Stone (2010: 9-11) rejects the misguided notion of objective scientists using scientific methods to produce one correct answer (compare with Spiegelhalter and Weimer & Vining). A policy paradox cannot be solved by ‘rational, analytical, and scientific methods’ because:

Further, Stone (2012: 10-11) rejects the over-reliance, in policy analysis, on the misleading claim that:

  • policymakers are engaging primarily with markets rather than communities (see 2012: 35 on the comparison between a ‘market model’ and ‘polis model’),
  • economic models can sum up political life, and
  • cost-benefit-analysis can reduce a complex problem into the sum of individual preferences using a single unambiguous measure.

Rather, many factors undermine such simplicity:

  1. People do not simply act in their own individual interest. Nor can they rank-order their preferences in a straightforward manner according to their values and self-interest.
  • Instead, they maintain a contradictory mix of objectives, which can change according to context and their way of thinking – combining cognition and emotion – when processing information (2012: 12; 30-4).
  1. People are social actors. Politics is characterised by ‘a model of community where individuals live in a dense web of relationships, dependencies, and loyalties’ and exercise power with reference to ideas as much as material interests (2012: 10; 20-36; compare with Ostrom, more Ostrom, and Lubell; and see Sousa on contestation).
  2. Morals and emotions matter. If people juggle contradictory aims and measures of success, then a story infused with ‘metaphor and analogy’, and appealing to values and emotions, prompts people ‘to see a situation as one thing rather than another’ and therefore draw attention to one aim at the expense of the others (2012: 11; compare with Gigerenzer).

Policy analysis reconsidered: the ambiguity of values and policy goals

Stone (2012: 14) identifies the ambiguity of the criteria for success used in 5-step policy analyses. They do not form part of a solely technical or apolitical process to identify trade-offs between well-defined goals (compare Bardach, Weimer and Vining, and Mintrom). Rather, ‘behind every policy issue lurks a contest over conflicting, though equally plausible, conceptions of the same abstract goal or value’ (2012: 14). Examples of competing interpretations of valence issues include definitions of:

  1. Equity, according to: (a) which groups should be included, how to assess merit, how to identify key social groups, if we should rank populations within social groups, how to define need and account for different people placing different values on a good or service, (b) which method of distribution to use (competition, lottery, election), and (c) how to balance individual, communal, and state-based interventions (2012: 39-62).
  2. Efficiency, to use the least resources to produce the same objective, according to: (a) who determines the main goal and how to balance multiple objectives, (a) who benefits from such actions, and (c) how to define resources while balancing equity and efficiency – for example, does a public sector job and a social security payment represent a sunk cost to the state or a social investment in people? (2012: 63-84).
  3. Welfare or Need, according to factors including (a) the material and symbolic value of goods, (b) short term support versus a long term investment in people, (c) measures of absolute poverty or relative inequality, and (d) debates on ‘moral hazard’ or the effect of social security on individual motivation (2012: 85-106)
  4. Liberty, according to (a) a general balancing of freedom from coercion and freedom from the harm caused by others, (b) debates on individual and state responsibilities, and (c) decisions on whose behaviour to change to reduce harm to what populations (2012: 107-28)
  5. Security, according to (a) our ability to measure risk scientifically (see Spiegelhalter and Gigerenzer), (b) perceptions of threat and experiences of harm, (c) debates on how much risk to safety to tolerate before intervening, (d) who to target and imprison, and (e) the effect of surveillance on perceptions of democracy (2012: 129-53).

Policy analysis as storytelling for collective action

Actors use policy-relevant stories to influence the ways in which their audience understands (a) the nature of policy problems and feasibility of solutions, within (b) a wider context of policymaking in which people contest the proper balance between state, community, and market action. Stories can influence key aspects of collective action, including:

  1. Defining interests and mobilising actors, by drawing attention to – and framing – issues with reference to an imagined social group and its competition (e.g. the people versus the elite; the strivers versus the skivers) (2012: 229-47)
  2. Making decisions, by framing problems and solutions (2012: 248-68). Stone (2012: 260) contrasts the ‘rational-analytic model’ with real-world processes in which actors deliberately frame issues ambiguously, shift goals, keep feasible solutions off the agenda, and manipulate analyses to make their preferred solution seem the most efficient and popular.
  3. Defining the role and intended impact of policies, such as when balancing punishments versus incentives to change behaviour, or individual versus collective behaviour (2012: 271-88).
  4. Setting and enforcing rules (see institutions), in a complex policymaking system where a multiplicity of rules interact to produce uncertain outcomes, and a powerful narrative can draw attention to the need to enforce some rules at the expense of others (2012: 289-310).
  5. Persuasion, drawing on reason, facts, and indoctrination. Stone (2012: 311-30) highlights the context in which actors construct stories to persuade: people engage emotionally with information, people take certain situations for granted even though they produce unequal outcomes, facts are socially constructed, and there is unequal access to resources – held in particular by government and business – to gather and disseminate evidence.
  6. Defining human and legal rights, when (a) there are multiple, ambiguous, and intersecting rights (in relation to their source, enforcement, and the populations they serve) (b) actors compete to make sure that theirs are enforced, (c) inevitably at the expense of others, because the enforcement of rights requires a disproportionate share of limited resources (such as policymaker attention and court time) (2012: 331-53)
  7. Influencing debate on the powers of each potential policymaking venue – in relation to factors including (a) the legitimate role of the state in market, community, family, and individual life, (b) how to select leaders, (c) the distribution of power between levels and types of government – and who to hold to account for policy outcomes (2012: 354-77).

Key elements of storytelling include:

  1. Symbols, which sum up an issue or an action in a single picture or word (2012:157-8)
  2. Characters, such as heroes or villain, who symbolise the cause of a problem or source of solution (2012:159)
  3. Narrative arcs, such as a battle by your hero to overcome adversity (2012:160-8)
  4. Synecdoche, to highlight one example of an alleged problem to sum up its whole (2012: 168-71; compare the ‘welfare queen’ example with SCPD)
  5. Metaphor, to create an association between a problem and something relatable, such as a virus or disease, a natural occurrence (e.g. earthquake), something broken, something about to burst if overburdened, or war (2012: 171-78; e.g. is crime a virus or a beast?)
  6. Ambiguity, to give people different reasons to support the same thing (2012: 178-82)
  7. Using numbers to tell a story, based on political choices about how to: categorise people and practices, select the measures to use, interpret the figures to evaluate or predict the results, project the sense that complex problems can be reduced to numbers, and assign authority to the counters (2012:183-205; compare with Speigelhalter)
  8. Assigning Causation, in relation to categories including accidental or natural, ‘mechanical’ or automatic (or in relation to institutions or systems), and human-guided causes that have intended or unintended consequences (such as malicious intent versus recklessness)
  • ‘Causal strategies’ include to: emphasise a natural versus human cause, relate it to ‘bad apples’ rather than systemic failure, and suggest that the problem was too complex to anticipate or influence
  • Actors use these arguments to influence rules, assign blame, identify ‘fixers’, and generate alliances among victims or potential supporters of change (2012: 206-28).

Wider Context and Further Reading: 1. Policy analysis

This post connects to several other 750 Words posts, which suggest that facts don’t speak for themselves. Rather, effective analysis requires you to ‘tell your story’, in a concise way, tailored to your audience.

For example, consider two ways to establish cause and effect in policy analysis:

One is to conduct and review multiple randomised control trials.

Another is to use a story of a hero or a villain (perhaps to mobilise actors in an advocacy coalition).

  1. Evidence-based policymaking

Stone (2012: 10) argues that analysts who try to impose one worldview on policymaking will find that ‘politics looks messy, foolish, erratic, and inexplicable’. For analysts, who are more open-minded, politics opens up possibilities for creativity and cooperation (2012: 10).

This point is directly applicable to the ‘politics of evidence based policymaking’. A common question to arise from this worldview is ‘why don’t policymakers listen to my evidence?’ and one answer is ‘you are asking the wrong question’.

  1. Policy theories highlight the value of stories (to policy analysts and academics)

Policy problems and solutions necessarily involve ambiguity:

  1. There are many ways to interpret problems, and we resolve such ambiguity by exercising power to attract attention to one way to frame a policy problem at the expense of others (in other words, not with reference to one superior way to establish knowledge).
  1. Policy is actually a collection of – often contradictory – policy instruments and institutions, interacting in complex systems or environments, to produce unclear messages and outcomes. As such, what we call ‘public policy’ (for the sake of simplicity) is subject to interpretation and manipulation as it is made and delivered, and we struggle to conceptualise and measure policy change. Indeed, it makes more sense to describe competing narratives of policy change.

box 13.1 2nd ed UPP

  1. Policy theories and storytelling

People communicate meaning via stories. Stories help us turn (a) a complex world, which provides a potentially overwhelming amount of information, into (b) something manageable, by identifying its most relevant elements and guiding action (compare with Gigerenzer on heuristics).

The Narrative Policy Framework identifies the storytelling strategies of actors seeking to exploit other actors’ cognitive shortcuts, using a particular format – containing the setting, characters, plot, and moral – to focus on some beliefs over others, and reinforce someone’s beliefs enough to encourage them to act.

Compare with Tuckett and Nicolic on the stories that people tell to themselves.

 

 

6 Comments

Filed under 750 word policy analysis, Evidence Based Policymaking (EBPM), Psychology Based Policy Studies, public policy, Storytelling

Policy Analysis in 750 words: Linda Tuhiwai Smith (2012) Decolonizing Methodologies

Please see the  Policy Analysis in 750 words series overview before reading the summary. The reference to 750 words is increasingly misleading.

Linda Tuhiwai Smith (2012) Decolonizing Methodologies 2nd edition (London: Zed Books)

 ‘Whose research is it? Who owns it? Whose interests does it serve? Who will benefit from it? Who has designed its questions and framed its cope? Who will carry it out? Who will write it up? How will its results be disseminated?’ (Smith, 2012: 10; see also 174-7)

Many texts in this series highlight the politics of policy analysis, but few (such as Bacchi) identify the politics of the research that underpins policy analysis.

You can find some discussion of these issues in the brief section on ‘co-production’, in wider studies of co-produced research and policy, and ‘evidence based policymaking’, and in posts on power and knowledge and feminist institutionalism. However, the implications rarely feed into standard policy analysis texts. This omission is important, because the production of knowledge – and the exercise of power to define whose knowledge counts – is as political as it gets.

Smith (2012) demonstrates this point initially by identifying multiple, often hidden, aspects of politics and power that relate to ‘research’ and ‘indigenous peoples’:

 

  1. The term ‘indigenous peoples’ is contested, and its meaning-in-use can range from
  • positive self-identification, to highlight common international experiences and struggles for self-determination but distinctive traditions; other terms include ‘First Nations’ in Canada or, in New Zealand, ‘Maori’ as opposed to ‘Pakeha’ (the colonizing population) (2012: 6)
  • negative external-identification, including – in some cases – equating ‘indigenous’ (or similar terms) with ‘dirtiness, savagery, rebellion and, since 9/11, terrorism’ (2012: xi-xii).

 

  1. From the perspective of ‘the colonized’, “the term ‘research’ is inextricably linked to European imperialism and colonialism” (2012: 1; 21-6). Western research practices (and the European ‘Enlightenment’) reflect and reinforce political practices associated with colonial rule (2012: 2; 23).

To the colonized, the ways in which academic research has been implicated in the throes of imperialism remains a painful memory’ (2012: back cover).

“The word itself, ‘research’, is probably one of the dirtiest words in the indigenous world’s vocabulary” (2012: xi).

 

  1. People in indigenous communities describe researchers who exploit ‘their culture, their knowledge, their resources’ (and, in some cases, their bodies) to bolster their own income, career or profession (2012: xi; 91-4; 102-7), in the context of a long history of subjugation and slavery that makes such practices possible (2012: 21-6; 28-9; 176-7), and “justified as being for ‘the good of mankind’” (2012: 26).

 

 

  1. Western researchers think – hubristically – that they can produce a general understanding of the practices and cultures of indigenous peoples (e.g. using anthropological methods). Instead, they produce – irresponsibly or maliciously – negative and often dehumanizing images that feed into policies ‘employed to deny the validity of indigenous peoples’ claim to existence’ and solve the ‘indigenous problem’ (2012: 1; 8-9; 26-9; 62-5; 71-2; 81-91; 94-6).

For example, research contributes to a tendency for governments to

  • highlight, within indigenous communities, indicators of inequality (in relation to factors such as health, education, crime, and family life), and relate it to
  • indigenous cultures and low intelligence, rather than
  • the ways in which colonial legacy and current policy contributes to poverty and marginalisation (2012: 4; 12; compare with Social Construction and Policy Design).

 

  1. Western researchers’ views on how to produce high-quality scientific evidence lead them to ‘see indigenous peoples, their values and practices as political hindrances that get in the way of good research’ (2012: xi; 66-71; compare with ‘hierarchy of evidence’). Similarly, the combination of a state’s formal laws and unwritten rules and assumptions can serve to dismiss indigenous community knowledge as not meeting evidential standards (2012: 44-9).

 

  1. Many indigenous researchers need to negotiate the practices and expectations of different groups, such as if they are portrayed as:
  • ‘insiders’ in relation to an indigenous community (and, for example, expected by that community to recognise the problems with Western research traditions)
  • ‘outsiders’, by (a) an indigenous community in relation to their ‘Western education’ (2012: 5), or (b) by a colonizing state commissioning insider research
  • less technically proficient or less likely to maintain confidentiality than a ‘non-indigenous researcher’ (2012: 12)

Can policy analysis be informed by a new research agenda?

In that context, Smith (2012: xiii; 111-25) outlines a new agenda built on the recognition that research is political and connected explicitly to political and policy aims (2012: xiii; compare with Feminism, Postcolonialism, and Critical Policy Studies)

At its heart is a commitment to indigenous community ‘self-determination’, ‘survival’, ‘recovery’, and ‘development’, aided by processes such as social movement mobilization and decolonization (2012: 121). This agenda informs the meaning of ethical conduct, signalling that research:

  • serves explicit political goals and requires researchers to reflect on their role as activists in an emancipatory project, in contrast to the disingenuous argument that science or scientists are objective (2012: 138-42; 166-77; 187-8; 193-5; 198-215; 217-26)
  • is not ‘something done only by white researchers to indigenous peoples’ (2012: 122),
  • is not framed so narrowly, in relation to specific methods or training, that it excludes (by definition) most indigenous researchers, community involvement in research design, and methods such as storytelling (2012: 127-38; 141; for examples of methods, see 144-63; 170-1)
  • requires distinctive methods and practices to produce knowledge, reinforced by mutual support during the nurturing of such practices
  • requires a code of respectful conduct that extends ‘beyond issues of individual consent and confidentiality’) (2012: 124; 179-81).

Wider context: informing the ‘steps’ to policy analysis

This project informs directly the ‘steps’ to policy analysis described in Bardach, Weimer and Vining, and Mintrom, including:

Problem definition

Mintrom describes the moral and practical value of engaging with stakeholders to help frame policy problems and design solutions (as part of a similarly-worded aim to transform and improve the world).

However, Smith (2012: 228-32; 13) describes such a profound gulf, in the framing of problems, that cannot be bridged simply via consultation or half-hearted ‘co-production’ exercises.

For example, if a government policy analyst relates poor health to individual and cultural factors in indigenous communities, and people in those communities relate it to colonization, land confiscation, minimal self-determination, and an excessive focus on individuals, what could we realistically expect from set-piece government-led stakeholder analyses built on research that has already set the policy agenda (compare with Bacchi)?

Rather, Smith (2012: 15-16) describes the need, within research practices, for continuous awareness of, and respect for, a community’s ‘cultural protocols, values and behaviours’ as part of ‘an ethical and respectful approach’. Indeed, the latter could have mutual benefits which underpin the long-term development of trust: a community may feel less marginalised by the analysis-to-policy process, and future analysts may be viewed with less suspicion.

Even so, a more respectful policy process is not the same as accepting that some communities may benefit more from writing about their own experiences than contributing to someone else’s story. Writing about the past, present, and future is an exercise of power to provide a dominant perspective with which to represent people and problems (2012: 29-41; 52-9)

Analysing and comparing solutions

Imagine a cost-benefit analysis designed to identify the most efficient outcomes by translating all of the predicted impacts on people into a single unit of analysis (such as a dollar amount, or quality-adjusted-life-years). Assumptions include that we can: (a) assign the same value to a notionally similar experience, and (b) produce winners from policy and compensate losers.

Yet, this calculation hinges on the power to decide how we should understand such experiences and place relative values on outcomes, and to take a calculation of their value to one population and generalise it to others. Smith’s analysis suggests that such processes will not produce outcomes that we can describe honestly as societal improvements. Rather, they feed into a choice to produce winners from policy and fail to compensate losers in an adequate or appropriate manner.

See also:

  1. In relation to policy theories

This post – Policy Concepts in 1000 Words: Feminism, Postcolonialism, and Critical Policy Studies – provides a tentative introduction to the ways in which many important approaches can inform policy theories, such as by

The 2nd edition of Understanding Public Policy summarises these themes as follows:

p49 2nd ed UPPp50 2nd ed UPP

  1. In relation to policy analysis

If you look back to the Policy Analysis in 750 words series overview, you will see that a popular way to address policy issues is through the ‘coproduction’ of research and policy, perhaps based on a sincere commitment to widen a definition of useful knowledge/ ways of thinking and avoid simply making policy from the ‘centre’ or ‘top down’.

Yet, the post you are now reading, summarising Decolonizing Methodologies, should prompt us to question the extent to which a process could be described sincerely as ‘coproduction’ if there is such an imbalance of power and incongruence of ideas between participants.

Although many key texts do not discuss ‘policy analysis’ directly, they provide ways to reflect imaginatively on this problem. I hope that I am not distorting their original messages, but please note that the following are my stylized interpretations of key texts.

Audre Lorde (2018*) The Master’s Tools Will Never Dismantle the Master’s House (Penguin) (*written from 1978-82)

Lorde Masters Tools

One issue with very quick client-oriented policy analysis is that it encourages analysts to (a) work with an already-chosen definition of the policy problem, and (b) use well-worn methods to collect information, including (c) engaging with ideas and people with whom they are already familiar.

Some forms of research and policy analysis may be more conducive to challenging existing frames and encouraging wider stakeholder engagement. Still, compare this mild shift from the status quo with a series of issues and possibilities identified by Lorde (2018):

  • Some people are so marginalised and dismissed that they struggle to communicate – about the ways in which they are oppressed, and how they might contribute to imagining a better world – in ways that would be valued (or even noticed) during stakeholder consultation (2018: 1-5 ‘Poetry is not a luxury’).
  • The ‘european-american male tradition’ only allows for narrowly defined (‘rational’) means of communication (2018: 6-15 ‘Uses of the Erotic’)

A forum can be designed ostensibly to foster communication and inclusivity, only to actually produce the opposite, by signalling to some participants that

  • they are a token afterthought, whose views and experiences are – at best – only relevant to a very limited aspect of a wide discussion, and
  • their differences will be feared, not celebrated, becoming a source of conflict, not mutual nurture or cooperation.

It puts marginalised people in the position of having to work hard simply to be heard. They learn that powerful people are only willing to listen if others do the work for them, because (a) they are ignorant of experiences other than their own, and/or (b) they profess ignorance strategically to suck the energy from people whose views they fear and do not understand. No one should feel immune from such criticism even if they profess to be acting with good intentions (2018: 16-21 ‘The Master’s Tools Will Never Dismantle the Master’s House’).

  • The correct response to racism is anger. Therefore, do not prioritise (a) narrow rules of civility, or the sensibilities of the privileged, if (b) your aim is to encourage conversations with people who are trying to express the ways in which they deal with overwhelming and continuous hatred, violence, and oppression (2018: 22-35, ‘Uses of Anger: Women Responding to Racism’)

Boaventura de Sousa Santos (2014) Epistemologies of the South: Justice Against Epistemicide (Routledge)

Sousa cover

Imagine global policy processes and policy analysis, in which some countries and international organisations negotiate agreements, influenced (or not) by critical social movements in pursuit of social justice. Santos (2014) identifies a series of obstacles including:

  • A tendency for Western (as part of the Global North) ways of thinking to dominate analysis, at the expense of insights from the Global South (2014: viii), producing
  • A tendency for ‘Western centric’ ideas to inform the sense that some concepts and collective aims – such as human dignity and human rights – can be understood universally, rather than through the lens of struggles that are specific to some regions (2014: 21; 38)
  • A lack of imagination or willingness to imagine different futures and conceptions of social justice (2014: 24)

Consequently, actors may come together to discuss major policy change on ostensibly the same terms, only for some groups to – intentionally and unintentionally – dominate thought and action and reinforce the global inequalities they propose to reduce.

Sarah Ahmed (2017) Living a Feminist Life (Duke University Press)

Ahmed cover.jpg

Why might your potential allies in ‘coproduction’ be suspicious of your motives, or sceptical about the likely outcomes of such an exchange? One theme throughout Smith’s (2012) book is that people often co-opt key terms (such as ‘decolonizing’) to perform the sense that they care about social change, to try to look like they are doing something important, while actually designing ineffective or bad faith processes to protect the status of themselves or their own institution or profession.

Ahmed (2017: 103) describes comparable initiatives – such as to foster ‘equality and diversity’ – as a public relations exercise for organisations, rather than a sincere desire to do the work. Consequently, there is a gap ‘between a symbolic commitment and a lived reality’ (2017: 90). Indeed, the aim may be to project a sense of transformation to hinder that transformation (2017: 90), coupled with a tendency to use a ‘safe’ and non-confrontational language (‘diversity’) to project the sense that we can only push people so far, at the expense of terms such as ‘racism’ that would signal challenge, confrontation, and a commitment to high impact (2017: chapter 4).

..

Putting these insights together suggests that a stated commitment to co-produced research and policy might begin with good intentions. Even so, a commitment to sincere engagement does not guarantee an audience or prevent you from exacerbating the very problems you profess to solve.

5 Comments

Filed under 750 word policy analysis, Evidence Based Policymaking (EBPM), public policy, Research design, Storytelling

A general theory of public policy

This is a placeholder for future work and discussion. It tails off at the end.

People sometimes talk about a ‘general theory’ of public policy to put in our minds a comparison with the physical sciences. Usually, the punchline is that there are ‘no general theories of public policy that are not bounded by space or time’ (p21). There may be some reference to the accumulation of knowledge or wisdom in policy studies, but based rarely on the understanding that policy studies contain the equivalent of general laws (I can only think of one possible exception).

This outcome is not too surprising in the social sciences, in which context really matters and we would expect a lot of variation in policy, policymaking, and outcomes.

On the other hand, we still need a way to communicate our findings, relate them to other studies, compare them, and wonder what it all adds up to. Few people go as far as expressing the sense that every study is unique (to the point of non-comparability) and that every description of policymaking does not compare to another.

In other words, we may be looking for a happy medium, to reject the idea of general laws but encourage – when appropriate or necessary – enough of a sense of common outlook and experience to help us communicate with each other (without descending too quickly into heated debate on our cross-purposes). Or, we can at least tell a story of policy studies and invite others to learn from, or challenge, its insights.

In my case, there are two examples in which it is necessary to project some sense of a common and initially-not-too-complicated story:

  1. When describing policy theory insights to students, on the assumption that it may be their gateway to more reading.

It is possible to choose how many words to devote to each topic, including 500 Words, 1000 Words, a 9000 word Understanding Public Policy chapter, more in the source material, and even more if students start to ‘snowball’.

It is also possible, if you have a clearly defined audience, to introduce some level of uncertainty about these descriptions and their limitations.

For example, I try to describe ‘the policy process’ in 500 words and 1000 words, but in the context of a wider discussion of images of the policy process.

Circle image policy process 24.10.18

It is also possible to provide more context, such as in this kind of introductory box, coupled with 12 things to know about studying public policy

Introduction box

(from Chapter 1)

You can also get into the idea that my story is one of many, particularly after students have invested in many versions of that story by the end of an introductory book

conclusion box

(from Chapter 13)

  1. When describing these insights to people – from other disciplines or professions – who do not have the time, inclination, or frame of reference to put in that kind of work.

In this case, one presentation or article may be the limit. People may want to know the answer to a question – e.g. Why don’t policymakers listen to your evidence?rather than hear all about the explanation for the answer.

You do your best, and then – if there is time – you talk about what you missed out.

For example, in this talk, the first question was: why didn’t you mention the role of power?

 

A general theory or a general understanding? Two key issues

That was a long-winded introduction to a more philosophical point about what we might want from general theories. My impression is that you might be seeking one of these two possibilities:

  1. To use theories and concepts to describe material reality. In producing a general theory, we seek a general understanding of the ways in which the real world works. If so, we may focus primarily on how well these concepts describe the world, and the extent to which we can produce methods to produce systematic and consistent findings. The lack of a general theory denotes too much complexity and context.
  2. To use theories and concepts to represent a useful story. In producing a general understanding, we focus on the ways in which people generate and communicate their understanding. If so, we may focus more on how people come together to produce and share meaning through concepts. The lack of a general theory could reflect the lack of agreement on how to study policymaking. Or, the presence of a general understanding could represent the exercise of power, to set the agenda and limit scholarly attention to a small number of theories.

I describe this distinction in the following audio clip, produced halfway through a run with the dogs, while jetlagged. The large gap in the middle happens when I am trying to see if the voice to text is working well enough for me to copy/paste it here (no).

Key examples of the exercise of power include:

  1. The act of dismissing an individual, social group, or population by undermining the value of their knowledge or claim to knowledge (discussed in power and knowledge and Chapter 3).
  2. Ongoing discussions about how we deal with (a) a relatively new focus (among the most-established policy theories) on policy studies in countries in the Global South, given that (b) the dominant interpretations of policymaking come from experiences in the Global North.

box 13.4 part 1box 13.4 part 2

So, if you read these posts or Chapter 13 you will find a story of a general understanding of policy followed, almost immediately, by a list of reasons for why you should engage with it critically and perhaps not accept it. I’m setting your agenda but also reminding you that I’m doing it.

That’s it really. To be continued.

Leave a comment

Filed under 1000 words, Academic innovation or navel gazing, agenda setting, public policy, Storytelling

Beware the well-intentioned advice of unusually successful academics

This post – by Dr Kathryn Oliver and me – originally appeared on the LSE Impact Blog. I have replaced the picture of a thumb up with a cat hanging in there. 

Many academics want to see their research have an impact on policy and practice, and there is a lot of advice on how to seek it. It can be helpful to take advice from experienced and successful people. However, is this always the best advice? Guidance based on best practice and success stories in particular, often reflect unequal access to policymakers, institutional support, and credibility attached to certain personal characteristics.

To take stock of the vast amount of advice being offered to academics, we decided to compare it with the more systematic analyses available in the peer-reviewed literature, on the ‘barriers’ between evidence and policy, and policy studies. This allowed us to situate this advice in a wider context, see whether it was generalisable across settings and career stages, and to think through the inconsistencies and dilemmas which underlie these suggestions.

The advice: Top tips on influencing policy

The key themes and individual recommendations we identified from the 86 most-relevant publications are:

  1. Do high quality research: Use well-established research designs, methods, or metrics.
  2. Make your research relevant and readable: Provide easily-understandable, clear, relevant and high-quality research. Aim for the general reader. Produce good stories based on emotional appeals or humour.
  3. Understand the policymaking context. Note the busy and constrained lives of policy actors. Maximise established ways to engage, such as in advisory committees. Be pragmatic, accepting that research rarely translates directly into policy.
  4. Be ‘accessible’ to policymakers. This may involve discussing topics beyond your narrow expertise. Be humble, courteous, professional, and recognise the limits to your skills.
  5. Decide if you want to be an ‘issue advocate’. Decide whether to simply explain the evidence, remain an ‘honest broker, or recommend specific policy options. Negative consequences may include peer criticism, being seen as an academic lightweight, being used to add legitimacy to a policy position, and burnout.
  6. Build relationships (and ground rules) with policymakers: Relationship-building requires investment and skills, but working collaboratively is often necessary. Academics could identify policy actors to provide insights into policy problems, act as champions for their research, and identify the most helpful policy actors.
  7. Be ‘entrepreneurial’ or find someone who is. Be a daring, persuasive scientist, comfortable in policy environments and available when needed. Or, seek brokers to act on your behalf.
  8. Reflect continuously: should you engage, do you want to, and is it working? Academics may enjoy the work or are passionate about the issue. Even so, keep track of when and how you have had impact, and revise your practices continuously.

hang-in-there-baby

Inconsistencies and dilemmas

This advice tends not to address wider issues. For example, there is no consensus over what counts as good evidence for policy, or therefore how best to communicate good evidence. We know little about how to gain the wide range of skills that researchers and policymakers need to act collectively, including to: produce evidence syntheses, manage expert communities, ‘co-produce’ research and policy with a wide range of stakeholders, and be prepared to offer policy recommendations as well as scientific advice. Further, a one-size fits-all model won’t help researchers navigate a policymaking environment where different venues have different cultures and networks. Researchers therefore need to decide what policy engagement is for—to frame problems or simply measure them according to an existing frame—and how far researchers should go to be useful and influential. If academics need to go ‘all in’ to secure meaningful impact, we need to reflect on the extent to which they have the resources and support to do so. This means navigating profound dilemmas:

Source: The dos and don’ts of influencing policy: a systematic review of advice to academics

 

Can academics try to influence policy? The financial costs of seeking impact are prohibitive for junior or untenured researchers, while women and people of colour may be more subject to personal abuse. Such factors undermine the diversity of voices available.

How should academics influence policy? Many of these new required skills – such as storytelling – are not a routine part of academic training, and may be looked down on by our colleagues.  

What is the purpose of academics engagement in policymaking? To go beyond tokenistic and instrumental engagement is to build genuine rapport with policymakers, which may require us to co-produce knowledge and cede some control over the research process. It involves a fundamentally different way of doing public engagement: one with no clear aim in mind other than to listen and learn, with the potential to transform research practices and outputs.

Where is the evidence that this advice helps us improve impact?

The existing advice offered to academics on how to create impact is – although often well-meaning – not based on systematic research or comprehensive analysis of empirical evidence. Few advice-givers draw clearly on key literatures on policymaking or evidence use. This leads to significant misunderstandings, which can have potentially costly repercussions for research, researchers and policy.  These limitations matter, as they lead to advice which fails to address core dilemmas for academics—whether to engage, how to engage, and why—which have profound implications for how scientists and universities should respond to the calls for increased impact.

Most tips focus on individual experience, whereas engagement between research and policy is driven by systemic factors. Many of the tips may be sensible and effective, but often only within particular settings. The advice is likely to be useful mostly to a relatively similar group of people who are confident and comfortable in policy environments, and have access and credibility within policy arenas. Thus, the current advice and structures may help reproduce and reinforce existing power dynamics and an underrepresentation of people who do not fit a very narrow mould.

The overall result may be that each generation of scientists has to fight the same battles, and learn the same lessons over again. Our best response as a profession is to interrogate current advice, shape and frame it, and to help us all to find ways to navigate the complex practical, political, moral and ethical challenges associated with being researchers today. The ‘how to’ literature can help, but only if authors are cognisant of their wider role in society and complex policymaking systems.

This blog post is based on the authors’ co-written articles, The dos and don’ts of influencing policy: a systematic review of advice to academics, published in Palgrave Communications, and ‘How should academics engage in policymaking to achieve impact?’  published in Political Studies Review 

About the authors

Kathryn Oliver is Associate Professor of Sociology and Public Health, London School of Hygiene and Tropical Medicine (@oliver_kathryn ). Her interest is in how knowledge is produced, mobilized and used in policy and practice, and how this affects the practice of research. She co-runs the research collaborative Transforming Evidence with Annette Boaz. https://transformure.wordpress.com and her writings can be found here: https://kathrynoliver.wordpress.com

Paul Cairney is Professor of Politics and Public Policy, University of Stirling, UK (@Cairneypaul).  His research interests are in comparative public policy and policy theories, which he uses to explain the use of evidence in policy and policymaking, in one book (The Politics of Evidence-Based Policy Making, 2016), several articles, and many, many blog posts: https://paulcairney.wordpress.com/ebpm/

See also:

  1. Adam Wellstead, Paul Cairney, and Kathryn Oliver (2018) ‘Reducing ambiguity to close the science-policy gap’, Policy Design and Practice, 1, 2, 115-25 PDF
  2. Paul Cairney and Kathryn Oliver (2017) ‘Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy?’ Health Research Policy and Systems (HARPS), DOI: 10.1186/s12961-017-0192-x PDF AM
  3. Paul Cairney, Kathryn Oliver, and Adam Wellstead (2016) ‘To Bridge the Divide between Evidence and Policy: Reduce Ambiguity as Much as Uncertainty’, Public Administration Review, 76, 3, 399–402 DOI:10.1111/puar.12555 PDF

1 Comment

Filed under Evidence Based Policymaking (EBPM), Folksy wisdom, public policy, Storytelling, UK politics and policy

Policy in 500 Words: Power and Knowledge

Classic studies suggest that the most profound and worrying kinds of power are the hardest to observe. We often witness highly visible political battles and can use pluralist methods to identify who has material resources, how they use them, and who wins. However, key forms of power ensure that many such battles do not take place. Actors often use their resources to reinforce social attitudes and policymakers’ beliefs, to establish which issues are policy problems worthy of attention and which populations deserve government support or punishment. Key battles may not arise because not enough people think they are worthy of debate. Attention and support for debate may rise, only to be crowded out of a political agenda in which policymakers can only debate a small number of issues.

Studies of power relate these processes to the manipulation of ideas or shared beliefs under conditions of bounded rationality (see for example the NPF). Manipulation might describe some people getting other people to do things they would not otherwise do. They exploit the beliefs of people who do not know enough about the world, or themselves, to know how to identify and pursue their best interests. Or, they encourage social norms – in which we describe some behaviour as acceptable and some as deviant – which are enforced by the state (for example, via criminal justice and mental health policy), but also social groups and individuals who govern their own behaviour with reference to what they feel is expected of them (and the consequences of not living up to expectations).

Such beliefs, norms, and rules are profoundly important because they often remain unspoken and taken for granted. Indeed, some studies equate them with the social structures that appear to close off some action. If so, we may not need to identify manipulation to find unequal power relationships: strong and enduring social practices help some people win at the expense of others, by luck or design.

In practice, these more-or-less-observable forms of power co-exist and often reinforce each other:

Example 1. The control of elected office is highly skewed towards men. Male incumbency, combined with social norms about who should engage in politics and public life, signal to women that their efforts may be relatively unrewarded and routinely punished – for example, in electoral campaigns in which women face verbal and physical misogyny – and the oversupply of men in powerful positions tends to limit debates on feminist issues.

Example 2. ‘Epistemic violencedescribes the act of dismissing an individual, social group, or population by undermining the value of their knowledge or claim to knowledge. Specific discussions include: (a) the colonial West’s subjugation of colonized populations, diminishing the voice of the subaltern; (b) privileging scientific knowledge and dismissing knowledge claims via personal or shared experience; and (c) erasing the voices of women of colour from the history of women’s activism and intellectual history.

It is in this context that we can understand ‘critical’ research designed to ‘produce social change that will empower, enlighten, and emancipate’ (p51). Powerlessness can relate to the visible lack of economic material resources and factors such as the lack of opportunity to mobilise and be heard.

See also:

Policy Concepts in 1000 Words: Power and Ideas

Evidence-informed policymaking: context is everything

8 Comments

Filed under 500 words, agenda setting, public policy, Storytelling