What is essential reading in policy process research?

The first Conference on Policy Process Research (COPPR) takes place in January 2023, online and in person (Denver).

I’m thinking of proposing an online panel (without paper presentations) that asks: what is essential reading in policy process research?

Put another way, if you were guiding students who were relatively new to public policy studies, what would you want them to know?

My motivation is fairly instrumental. I will be writing the third edition of Understanding Public Policy, reflecting on what is in there, what is absent, and what changes to make as a result (and thinking of how to update the 500 and 1000 word summaries). What do I need to discuss more, and what should I cut to make space? For example:

1. The book focuses on ‘mainstream’ policy theories and does not have much discussion of interpretive or critical approaches (although this series does more). If I were to shift the balance, what insights would be essential?
2. If I were to devote a lot more space to (say) the study of gender or race, should it be mainstreamed throughout every chapter (e.g. many chapters discuss a major policy theory) or consolidated in dedicated chapters (e.g. the chapter on power)?
3. The first edition’s conclusion focused on how to combine theoretical insights. The second focused on the dominance of the field by authors in a small number of Global North countries (especially in the US and Western Europe). What should be the concluding theme of the third?
4. From what approaches (e.g. to teaching public policy with the aid of written material) can I learn?
5. Are there better ways to foster learning than someone like me writing textbooks and explainers?

These are just some early (and perhaps self-indulgent) thoughts on what to discuss.

If you were thinking of proposing something with a similar theme, or would like to collaborate to design this panel proposal, please let me know.

Or, if you were to attend a panel like this, who else would you want to hear from? For example, it could be a panel composed of people who write introductory textbooks or (say) people who use them for teaching or read them during their studies.

Any ideas welcome (by August), either by commenting below or emailing me directly (p.a.cairney@stir.ac.uk).

Policy Analysis in 750 Words: Separating facts from values

This post begins by reproducing Can you separate the facts from your beliefs when making policy?(based on the 1^st edition of Understanding Public Policy) …

A key argument in policy studies is that it is impossible to separate facts and values when making policy. We often treat our beliefs as facts, or describe certain facts as objective, but perhaps only to simplify our lives or support a political strategy (a ‘self-evident’ fact is very handy for an argument). People make empirical claims infused with their values and often fail to realise just how their values or assumptions underpin their claims.

This is not an easy argument to explain. One strategy is to use extreme examples to make the point. For example, Herbert Simon points to Hitler’s Mein Kampf as the ultimate example of value-based claims masquerading as facts. We can also identify historic academic research which asserts that men are more intelligent than women and some races are superior to others. In such cases, we would point out, for example, that the design of the research helped produce such conclusions: our values underpin our (a) assumptions about how to measure intelligence or other measures of superiority, and (b) interpretations of the results.

‘Wait a minute, though’ (you might say). “What about simple examples in which you can state facts with relative certainty – such as the statement ‘there are X number of words in this post’”. ‘Fair enough’, I’d say (you will have to speak with a philosopher to get a better debate about the meaning of your X words claim; I would simply say that it is trivially true). But this statement doesn’t take you far in policy terms. Instead, you’d want to say that there are too many or too few words, before you decided what to do about it.

In that sense, we have the most practical explanation of the unclear fact/ value distinction: the use of facts in policy is to underpin evaluations (assessments based on values). For example, we might point to the routine uses of data to argue that a public service is in ‘crisis’ or that there is a public health related epidemic (note: I wrote the post before COVID-19; it referred to crises of ‘non-communicable diseases’). We might argue that people only talk about ‘policy problems’ when they think we have a duty to solve them.

Or, facts and values often seem the hardest to separate when we evaluate the success and failure of policy solutions, since the measures used for evaluation are as political as any other part of the policy process. The gathering and presentation of facts is inherently a political exercise, and our use of facts to encourage a policy response is inseparable from our beliefs about how the world should work.

… It continues with an edited excerpt from p59 of Understanding Public Policy, which explores the implications of bounded rationality for contemporary accounts of ‘evidence-based policymaking’:

‘Modern science remains value-laden … even when so many people employ so many systematic methods to increase the replicability of research and reduce the reliance of evidence on individual scientists. The role of values is fundamental. Anyone engaging in research uses professional and personal values and beliefs to decide which research methods are the best; generate research questions, concepts and measures; evaluate the impact and policy relevance of the results; decide which issues are important problems; and assess the relative weight of ‘the evidence’ on policy effectiveness. We cannot simply focus on ‘what works’ to solve a problem without considering how we used our values to identify a problem in the first place. It is also impossible in practice to separate two choices: (1) how to gather the best evidence and (2) whether to centralize or localize policymaking. Most importantly, the assertion that ‘my knowledge claim is superior to yours’ symbolizes one of the most worrying exercises of power. We may decide to favour some forms of evidence over others, but the choice is value-laden and political rather than objective and innocuous’.

Implications for policy analysis

Many highly-intelligent and otherwise-sensible people seem to get very bothered with this kind of argument. For example, it gets in the way of (a) simplistic stories of heroic-objective-fact-based-scientists speaking truth to villainous-stupid-corrupt-emotional-politicians, (b) the ill-considered political slogan that you can’t argue with facts (or ‘science’), (c) the notion that some people draw on facts while others only follow their feelings, and (d) the idea that you can divide populations into super-facty versus post-truthy people.

A more sensible approach is to (1) recognise that all people combine cognition and emotion when assessing information, (2) treat politics and political systems as valuable and essential processes (rather than obstacles to technocratic policymaking), and (3) find ways to communicate evidence-informed analyses in that context. This article and 750 post explore how to reflect on this kind of communication.

Most relevant posts in the 750 series

Linda Tuhiwai Smith (2012) Decolonizing Methodologies 

Carol Bacchi (2009) Analysing Policy: What’s the problem represented to be? 

Deborah Stone (2012) Policy Paradox

Who should be involved in the process of policy analysis?

William Riker (1986) The Art of Political Manipulation

Using Statistics and Explaining Risk (David Spiegelhalter and Gerd Gigerenzer)

Barry Hindess (1977) Philosophy and Methodology in the Social Sciences

See also

To think further about the relevance of this discussion, see this post on policy evaluation, this page on the use of evidence in policymaking, this book by Douglas, and this short commentary on ‘honest brokers’ by Jasanoff.

Perspectives on academic impact and expert advice to policymakers

A blog post prompted by this fascinating post by Dr Christiane Gerblinger: Are experts complicit in making their advice easy for politicians to ignore?

There is a lot of advice out there for people seeking to make an ‘impact’ on policy with their research, but some kinds of advice must seem like they are a million miles apart.

For the sake of brevity, here are some exemplars of the kinds of discussion that you might find:

Advice from former policymakers

Here is what you could have done to influence my choices when I was in office. Almost none of you did it.

(for a nicer punchline see How can we demonstrate the public value of evidence-based policy making when government ministers declare that the people ‘have had enough of experts’?)

Advice from former civil servants

If you don’t know and follow the rules here, people will ignore your research. We despair when you just email your articles.

(for nicer advice see Creating and communicating social research for policymakers in government)

Advice from training courses on communication

Be concise and engaging.

Advice from training courses on policy impact

Find out where the action is, learn the rules, build up relationships and networks, become a trusted guide, be in the right place at the right time to exploit opportunities, give advice rather than sitting on the fence.

(see for example Knowledge management for policy impact: the case of the European Commission’s Joint Research Centre)

Advice from researchers with some experience of engagement

Do great research, make it relevant and readable, understand your policymaking context, decide how far you want to go have an impact, be accessible, build relationships, be entrepreneurial.

(see Beware the well-intentioned advice of unusually successful academics)

Advice from academic-practitioner exchanges

Note the different practices and incentives that undermine routine and fruitful exchanges between academics, practitioners, and policymakers.

(see Theory and Practice: How to Communicate Policy Research beyond the Academy and ANZOG Wellington).

Advice extrapolated from policy studies

Your audience decides if your research will have impact; policymakers will necessarily ignore almost all of it; a window of opportunity may never arise; and, your best shot may be to tailor your research findings to policymakers whose beliefs you may think are abhorrent.

(discussed in how much impact can you expect from your analysis? and book The Politics of Policy Analysis)

Inference from my study of UK COVID-19 policy

Very few expert advisers had a continuous impact on policy, some had decent access, but almost all were peripheral players or outsiders by choice.

(see The UK government’s COVID-19 policy: what does ‘guided by the science’ mean in practice? and COVID-19 page)

Inference from Dr Gerblinger

Experts ensure that they ignored when: ‘focussing extensively on one strand of enquiry while sidestepping the wider context; expunging complexity; and routinely raising the presence of inconclusiveness’.

What can we make of all of this advice?

One way to navigate all of this material is to make some basic distinctions between:

Sensible basic advice to early career researchers

Know your audience, and tailor your communication accordingly; see academic-practitioner exchange as two-way conversation rather than one-way knowledge transfer.

Take home message: here are some sensible ways to share experiences with people who might find your research useful.

Reflections from people with experience

It will likely not reflect your position or experience (but might be useful sometimes).

Take home message: I think this stuff worked for me, but I am not really sure, and I doubt you will have the same resources.

Reflections from studies of academic-practitioner exchange

It tends to find minimal evidence that people are (a) evaluating research engagement projects, and (b) finding tangible evidence of success (see Research engagement with government: insights from research on policy analysis and policymaking)

Take home message: there is a lot of ‘impact’ work going on, but no one is sure what it all adds up to.

Policy initiatives such as the UK Research Excellence Framework, which requires case studies of policy (or other) impact to arise directly from published research.

Take home message: I have my own thoughts, but see Rethinking policy ‘impact’: four models of research-policy relations

Reflections from people like me

Policy studies can be quite dispiriting. It often looks like I am saying that none of these activities will make much of a difference to policy or policymaking. Rather, I am saying to beware the temptation to turn (a) studies that describe policymaking complexity (e.g. 500 Words) into an agent-centred story of heroically impactful researchers (see for example the Discussion section of this article on health equity policy).

Take home message: don’t confuse studies of policymaking with advice for policy participants.

In other words, identify what you are after before you start to process all of this advice. If you want to engage more with policymakers, you will find some sensible practical advice. If you want to be responsible for a fundamental change of public policy in your field, I doubt any of the available advice will help (unless you seek an explanation for failure).

A general theory of public policy

This is a placeholder for future work and discussion. It tails off at the end.

People sometimes talk about a ‘general theory’ of public policy to put in our minds a comparison with the physical sciences. Usually, the punchline is that there are ‘no general theories of public policy that are not bounded by space or time’ (p21). There may be some reference to the accumulation of knowledge or wisdom in policy studies, but based rarely on the understanding that policy studies contain the equivalent of general laws (I can only think of one possible exception).

This outcome is not too surprising in the social sciences, in which context really matters and we would expect a lot of variation in policy, policymaking, and outcomes.

On the other hand, we still need a way to communicate our findings, relate them to other studies, compare them, and wonder what it all adds up to. Few people go as far as expressing the sense that every study is unique (to the point of non-comparability) and that every description of policymaking does not compare to another.

In other words, we may be looking for a happy medium, to reject the idea of general laws but encourage – when appropriate or necessary – enough of a sense of common outlook and experience to help us communicate with each other (without descending too quickly into heated debate on our cross-purposes). Or, we can at least tell a story of policy studies and invite others to learn from, or challenge, its insights.

In my case, there are two examples in which it is necessary to project some sense of a common and initially-not-too-complicated story:

1. When describing policy theory insights to students, on the assumption that it may be their gateway to more reading.

It is possible to choose how many words to devote to each topic, including 500 Words, 1000 Words, a 9000 word Understanding Public Policy chapter, more in the source material, and even more if students start to ‘snowball’.

It is also possible, if you have a clearly defined audience, to introduce some level of uncertainty about these descriptions and their limitations.

For example, I try to describe ‘the policy process’ in 500 words and 1000 words, but in the context of a wider discussion of images of the policy process.

It is also possible to provide more context, such as in this kind of introductory box, coupled with 12 things to know about studying public policy

(from Chapter 1)

You can also get into the idea that my story is one of many, particularly after students have invested in many versions of that story by the end of an introductory book

(from Chapter 13)

2. When describing these insights to people – from other disciplines or professions – who do not have the time, inclination, or frame of reference to put in that kind of work.

In this case, one presentation or article may be the limit. People may want to know the answer to a question – e.g. Why don’t policymakers listen to your evidence? – rather than hear all about the explanation for the answer.

You do your best, and then – if there is time – you talk about what you missed out.

For example, in this talk, the first question was: why didn’t you mention the role of power?

 

A general theory or a general understanding? Two key issues

That was a long-winded introduction to a more philosophical point about what we might want from general theories. My impression is that you might be seeking one of these two possibilities:

1. To use theories and concepts to describe material reality. In producing a general theory, we seek a general understanding of the ways in which the real world works. If so, we may focus primarily on how well these concepts describe the world, and the extent to which we can produce methods to produce systematic and consistent findings. The lack of a general theory denotes too much complexity and context.
2. To use theories and concepts to represent a useful story. In producing a general understanding, we focus on the ways in which people generate and communicate their understanding. If so, we may focus more on how people come together to produce and share meaning through concepts. The lack of a general theory could reflect the lack of agreement on how to study policymaking. Or, the presence of a general understanding could represent the exercise of power, to set the agenda and limit scholarly attention to a small number of theories.

I describe this distinction in the following audio clip, produced halfway through a run with the dogs, while jetlagged. The large gap in the middle happens when I am trying to see if the voice to text is working well enough for me to copy/paste it here (no).

Key examples of the exercise of power include:

1. The act of dismissing an individual, social group, or population by undermining the value of their knowledge or claim to knowledge (discussed in power and knowledge and Chapter 3).
2. Ongoing discussions about how we deal with (a) a relatively new focus (among the most-established policy theories) on policy studies in countries in the Global South, given that (b) the dominant interpretations of policymaking come from experiences in the Global North.

So, if you read these posts or Chapter 13 you will find a story of a general understanding of policy followed, almost immediately, by a list of reasons for why you should engage with it critically and perhaps not accept it. I’m setting your agenda but also reminding you that I’m doing it.

That’s it really. To be continued.

I know my audience, but does my other audience know I know my audience?

‘Know your audience’ is a key phrase for anyone trying to convey a message successfully. To ‘know your audience’ is to understand the rules they use to make sense of your message, and therefore the adjustments you have to make to produce an effective message. Simple examples include:

• The sarcasm rules. The first rule is fairly explicit. If you want to insult someone’s shirt, you (a) say ‘nice shirt, pal’, but also (b) use facial expressions or unusual speech patterns to signal that you mean the opposite of what you are saying. Otherwise, you’ve inadvertently paid someone a compliment, which is just not on. The second rule is implicit. Sarcasm is sometimes OK – as a joke or as some nice passive aggression – and a direct insult (‘that shirt is shite, pal’) as a joke is harder to pull off.
• The joke rule. If you say that you went to the doctor because a strawberry was growing out of your arse and the doctor gave you some cream for it, you’d expect your audience to know you were joking because it’s such a ridiculous scenario and there’s a pun. Still, there’s a chance that, if you say it quickly, with a straight face, your audience is not expecting a joke, and/ or your audience’s first language is not English, your audience will take you seriously, if only for a second. It’s hilarious if your audience goes along with you, and a bit awkward if your audience asks kindly about your welfare.
• Keep it simple stupid. If someone says KISS, or some modern equivalent – ‘it’s the economy, stupid’, the rule is that, generally, they are not calling you stupid (even though the insertion of the comma, in modern phrases, makes it look like they are). They are referring to the value of a simple design or explanation that as many people as possible can understand. If your audience doesn’t know the phrase, they may think you’re calling them stupid, stupid.

These rules can be analysed from various perspectives: linguistics, focusing on how and why rules of language develop; and philosophy, to help articulate how and why rules matter in sense making.

There is also a key role for psychological insights, since – for example – a lot of these rules relate to the routine ways in which people engage emotionally with the ‘signals’ or information they receive.

Think of the simple example of twitter engagement, in which people with emotional attachments to one position over another (say, pro- or anti- Brexit), respond instantly to a message (say, pro- or anti- Brexit). While some really let themselves down when they reply with their own tweet, and others don’t say a word, neither audience is immune from that emotional engagement with information. So, to ‘know your audience’ is to anticipate and adapt to the ways in which they will inevitably engage ‘rationally’ and ‘irrationally’ with your message.

I say this partly because I’ve been messing around with some simple ‘heuristics’ built on insights from psychology, including Psychology Based Policy Studies: 5 heuristics to maximise the use of evidence in policymaking .

Two audiences in the study of ‘evidence based policymaking’

I also say it because I’ve started to notice a big unintended consequence of knowing my audience: my one audience doesn’t like the message I’m giving the other. It’s a bit like gossip: maybe you only get away with it if only one audience is listening. If they are both listening, one audience seems to appreciate some new insights, while the other wonders if I’ve ever read a political science book.

The problem here is that two audiences have different rules to understand the messages that I help send. Let’s call them ‘science’ and ‘political science’ (please humour me – you’ve come this far). Then, let’s make some heroic binary distinctions in the rules each audience would use to interpret similar issues in a very different way.

I could go on with these provocative distinctions, but you get the idea. A belief taken for granted in one field will be treated as controversial in another. In one day, you can go to one workshop and hear the story of objective evidence, post-truth politics, and irrational politicians with low political will to select evidence-based policies, then go to another workshop and hear the story of subjective knowledge claims.

Or, I can give the same presentation and get two very different reactions. If these are the expectations of each audience, they will interpret and respond to my messages in very different ways.

So, imagine I use some psychology insights to appeal to the ‘science’ audience. I know that,  to keep it on side and receptive to my ideas, I should begin by being sympathetic to its aims. So, my implicit story is along the lines of, ‘if you believe in the primacy of science and seek evidence-based policy, here is what you need to do: adapt to irrational policymaking and find out where the action is in a complex policymaking system’. Then, if I’m feeling energetic and provocative, I’ll slip in some discussion about knowledge claims by saying something like, ‘politicians (and, by the way, some other scholars) don’t share your views on the hierarchy of evidence’, or inviting my audience to reflect on how far they’d go to override the beliefs of other people (such as the local communities or service users most affected by the evidence-based policies that seem most effective).

The problem with this story is that key parts are implicit and, by appearing to go along with my audience, I provoke a reaction in another audience: don’t you know that many people have valid knowledge claims? Politics is about values and power, don’t you know?

So, that’s where I am right now. I feel like I ‘know my audience’ but I am struggling to explain to my original political science audience that I need to describe its insights in a very particular way to have any traction in my other science audience. ‘Know your audience’ can only take you so far unless your other audience knows that you are engaged in knowing your audience.

If you want to know more, see:

Kathryn Oliver and I have just published an article on the relationship between evidence and policy

How far should you go to secure academic ‘impact’ in policymaking? From ‘honest brokers’ to ‘research purists’ and Machiavellian manipulators

Why doesn’t evidence win the day in policy and policymaking?

The Science of Evidence-based Policymaking: How to Be Heard

When presenting evidence to policymakers, engage with the policy process that exists, not the process you wish existed

 

 

Five advantages of blogging

Reminder of our Academic Blogging event tomorrow morning https://t.co/k07z8oWueJ w/ @CairneyPaul @ThomsonPat @janetinkler #uwsblogging

— David McGillivray (@dgmcgillivray) March 6, 2017

This is my third ‘hey, let’s blog’ event, so it finally dawned on me to write a blog post about it. See also Fiona Miller’s account of the Stirling event.

I don’t know much about blogging research, so will focus on my personal experience of its advantages. One frequent academic argument against blogging is that it takes you away from more important parts of the job, such as teaching and research. My argument is that it helps you do both things more effectively.

See also the accounts of the disadvantages, which often relate to the ways in which they make you vulnerable to personal abuse on social media (examples 1, 2, 3).

Advantage 1: Clarity

Writing a blog has improved my academic writing. When you blog, you write for a non-specialist audience. You use less jargon or explain its meaning and value. You assume that people will not read your work unless you front-load the ‘reveal’. You need a catchy and tweetable title, to provide a ‘hook’ in the first sentence, and to show your work in a few hundred words (perhaps to encourage people to read more of your work). When you develop these skills, you can use them while writing journal article titles, abstracts, and introductions.

If you like, you can also write a blog post instead of relying on the paper/ powerpoint combo for workshops and conferences, since a 4-paper panel at conferences is usually an endurance test, and a blog post reminds you to say why people should be interested in the paper (e.g. recent examples on evidence/ policy and Scottish independence).

Blogging is good practice at writing in an accessible way – because academic publications should be accessible too. @CairneyPaul #rbah15

— Diljeet Bhachu (@DiljeetB_Flute) August 20, 2015

Advantage 2: Timeliness

It can take years for people to read an article you publish in a top journal. Sometimes the article is worth the wait. In other cases, I think it’s best to see this work as part of a package in which the article is one of the last things to appear. There is a good case to be made for taking your time to get articles right, but a less good case to keep it a secret while you do so.

Advantage 3: Exposure

It’s now common to say that we make better links with practitioners and policymakers by making our writing more accessible (short, punchy, and one click away). In my experience, the biggest payoff has been with other academics. Politics colleagues will mention my blog (and textbook) more than my articles. I can also use introductory blog posts to communicate ideas with colleagues in other disciplines – and/ or in other countries – without expecting them to do weeks of homework on the foundational texts. In each case, it works partly because we struggle to find the time to read, and appreciate a short story. Indeed, my articles are one click away on my website, but very, very, very, very few people read them.

However, you don’t need a personal blog. In fact, my most exposureyish posts have been elsewhere, including two in the Guardian’s political science blog (on evidence-based policymaking, and (with Kathryn Oliver) the dilemmas that arise when we seek it), some on the LSE blog (I tried really hard to compare tobacco and alcohol policy – look! There’s a video!), and many in The Conversation.

Advantage 4: Teaching and Learning

Teaching. The most-used page of my website hosts a series of 1000 Word summaries of policy concepts (the ‘policy cycle’ got 26000 hits in 2016). I use them, like a gateway drug, to teach undergraduate and MPP modules: they can get a feel for the concept quickly then do further reading. They now come with podcasts, which I use instead of lectures (for workshops). Other academics also use the podcasts, particularly when their students are new to policy studies (e.g. David P. Carter).

Learning. I also ask my students to write blog posts as part of their coursework, to help them learn how to write in a concise and punchy way for a non-academic audience. In most cases, students excel at this kind of work, as part of a package of assessment in which they learn how to communicate the same insights in many different ways.

Advantage 5: Unexpected benefits

When I started blogging I didn’t really know what it was for. I used to copy and paste my article abstracts, or complain about David Cameron’s handling of Scottish independence. This was at a time in which colleagues at my former University were reticent about self-publicity, and sending round a link to a new journal article via the departmental email was pushing it a bit. Now, self-promotion seems to be part of the job, and we might expect some benefits without really knowing what they’ll be. For example, my links with some very interesting people in places like the European Commission and Alliance for Useful Evidence have arisen largely from blogging.

We all have different things that tickle us in life. For me, the most tickling part of the unexpected benefit of blogging is that I now (almost!) top the following google searches: policy cycle, multiple streams, advocacy coalition framework, punctuated equilibrium theory, the politics of evidence based policymaking, and the psychology of policymaking. I’m also doing my best to push out the other Paul Cairney from the first page of google, but Wikipedia is getting in the way. The more serious point is that a personal blog might need to generate attention through social media first, before it catches fire and rises up the search engine pages.

Why the pollsters got it wrong

We have a new tradition in politics in which some people glory in the fact that the polls got it wrong. It might begin with ‘all these handsome experts with all their fancy laptops and they can’t even tell us exactly how an election will turn out’, and sometimes it ends with, ‘yet, I knew it all along’. I think that the people who say it most are the ones that are pleased with the result and want to stick it to the people who didn’t predict it: ‘if, like me, they’d looked up from their laptops and spoken to real people, they’d have seen what would happen’.

To my mind, it’s always surprising when so many polls seem to do so well. Think for a second about what ‘pollsters’ do: they know they can’t ask everyone how they will vote (and why), so they take a small sample and use it as a proxy for the real world. To make sure the sample isn’t biased by selection, they develop methods to generate respondents randomly. To try to make the most of their resources, and make sure that their knowledge is cumulative, they use what they think they know about the population to make sure that they get enough responses from a ‘representative’ sample of the population. In many cases, that knowledge comes from things like focus groups or one-to-one interviews to get richer (qualitative) information than we can achieve from asking everyone the same question, often super-quickly, in a larger survey.

This process involves all sorts of compromises and unintended consequences when we have a huge population but limited resources: we’d like to ask everyone in person, but it’s cheaper to (say) get a 4-figure response online or on the phone; and, if we need to do it quickly, our sample will be biased towards people willing to talk to us.* So, on top of a profound problem – the possibility of people not telling the truth in polls – we have a potentially less profound but more important problem: the people we need to talk to us aren’t talking to us. So, we get a misleading read because we’re asking an unrepresentative sample (although it is nothing like as unrepresentative as proxy polls from social media, the word ‘on the doorstep’, or asking your half-drunk mates how they’ll vote).

Sensible ‘pollsters’ deal with such problems by admitting that they might be a bit off: highlighting their estimated ‘margin of error’ from the size of their sample, then maybe crossing their fingers behind their backs if asked about the likelihood of more errors based on non-random sampling. So, ignore this possibility for error at your peril. Yet, people do ignore it despite the peril! Here are two reasons why.

1. Being sensible is boring.

In a really tight-looking two-horse race, the margin of error alone might suggest that either horse might win. So, a sensible interpretation of a poll might be (say), ‘either Clinton or Trump will get the most votes’. Who wants to hear or talk about that?! You can’t fill a 24-hour news cycle and keep up shite Twitter conversations by saying ‘who knows?’ and then being quiet. Nor will anyone pay much attention to a quietly sensible ‘pollster’ or academic telling them about the importance of embracing uncertainty. You’re in the studio to tell us what will happen, pal. Otherwise, get lost.

2. Recognising complexity and uncertainty is boring.

You can heroically/ stupidly break down the social scientific project into two competing ideas: (1) the world contains general and predictable patterns of behaviour that we can identify with the right tools; or (2) the world is too complex and unpredictable to produce general laws of behaviour, and maybe your best hope is to try to make sense of how other people try to make sense of it. Then, maybe (1) sounds quite exciting and comforting while (2) sounds like it is the mantra of a sandal-wearing beansprout-munching hippy academic. People seem to want a short, confidently stated, message that is easy to understand. You can stick your caveats.

Can we take life advice from this process?

These days I’m using almost every topic as a poorly-constructed segue into a discussion about the role of evidence in politics and policy. This time, the lesson is about using evidence correctly for the correct purpose. In our example, we can use polls effectively for their entertainment value. Or, campaigners can use them as the best-possible proxies during their campaigns: if their polls tell them they are lagging in one area, give it more attention; if they seem to have a big lead in another area; give it less attention. The evidence won’t be totally accurate, but it gives you enough to generate a simple campaigning strategy. Academics can also use the evidence before and after a campaign to talk about how it’s all going. Really, the only thing you don’t expect poll evidence to do is predict the result. For that, you need the Observers from Fringe.

The same goes for evidence in policymaking: people use rough and ready evidence because they need to act on what they think is going on. There will never be enough evidence to make the decision for you, or let you know exactly what will happen next. Instead, you combine good judgement with your values, sprinkle in some evidence, and off you go. It would be silly to expect a small sample of evidence – a snapshot of one part of the world – to tell you exactly what will happen in the much larger world. So, let’s not kid ourselves about the ability of science to tell us what’s what and what to do. It’s better, I think, to recognise life’s uncertainties and act accordingly. It’s better than blaming other people for not knowing what will happen next.

 

*I say ‘we’ and ‘us’ but I’ve never conducted a poll in my life. I interview elites in secret and promise them anonymity.

A Stern review for everyone?

The Stern review ‘was commissioned by the government to carry out the review of the Research Excellence Framework (REF) to ensure future university research funding is allocated more efficiently, offers greater rewards for excellent research and reduces the administrative burden on institutions’. In this post, I explain why no single policy can solve these problems uniformly: they affect scholars of different seniority, and different disciplines, very differently. The punchline is at the end.

My initial impression of the Stern review is that it has gone to great lengths to address the unintended consequences of the previous Research Excellence Framework. One of its key aims now is to try to anticipate the potential unintended consequences of its reduction of other unintended consequences! This is remarkably common in policymaking, perhaps summed up by Aaron Wildavsky’s phrase ‘policy as its own cause’: we enter a never-ending process of causing ripple effects when trying to fix previous problems.

The example of non-portability (#sternreview portable)

Take the example of one of the biggest problems:

Problem: there was a large incentive for Universities to ‘game’ the REF towards the end of the cycle: paying for 20% of the time of big name academics, or appointing them at huge salaries, to gain access to their 4 best publications. A policy of rewarding research excellence became a policy to (a) reward big transfers, undermining the efforts of other Universities and reducing their incentive to invest for the long term, and (b) boost the salaries of senior scholars (many of whom were already on 6-figure salaries), often to ridiculous levels.

Solution: non-portability. The idea is that you can move but your former employer gets to use the texts that you published while in your last job. So, there may now be less incentive to buy up the big names in the run up to the next REF.

Unintended consequence: uncertainty for early career researchers (ECRs) or scholars without permanent (open-ended) contracts. Many ECRs have expressed the concern that their incentives may suddenly change, from generating a portfolio of up to 4 excellent publications to secure a permanent post, to perhaps holding back publications and promising their delivery when in post. This could be addressed by the present review (ECRs could be included in the REF but be under no obligation to submit any publications), or perhaps by exempting ECRs since the policy is aimed at senior scholars (but the exemption would also have unintended consequences!).

Interpreting the problem through the lens of precarious positions

We will now enter a phase of debate driven by uncertainty and anxiety about the end result, and a lot of the discussion will be emotionally charged because many people will have spent maybe 8 years in education (and several years in low paid posts after it) and not know what to do next. It is relatively easy for people like me to say that the proposed new system is better, and for senior scholars to look on Stern as a big improvement, because people like me will be rewarded in either system (I will leave it to you to decide what I mean by ‘people like me’). It is more difficult for ECRs who are genuinely uncertain about their prospects.

The punchline: how does this look through the eyes of scholars in different disciplines?

The disciplinary lens is the factor that can often be most important but least discussed, for two key reasons:

1. The general differences. Scholars operate in different ways. The ‘STEM’ (science, technology, engineering and mathematics) subjects are often described in these terms: you have large teams headed by a senior scholar; there is a hierarchy; you all work on the same research question; you publish many short articles as a team (with senior authors listed at the very start and end of the list of authors); you are increasingly driven by key metrics, such as personal citations and the ‘impact factor’ of journals . The humanities is often described like this: you are a lone scholar; you work on your own research question; you publish single-author work, and the big status symbol is the research monograph (book); these journal or other metrics do not work as well in your discipline. I am verging on caricature here (many ‘STEM’ scholars will work alone, and some humanities or social science scholars will operate laboratory-style teams), but you get the idea.

These differences feed into other practices: only in some subjects can it make sense for a University to ‘poach’ a whole team or unit; only in some subjects do ECRs need to develop their own portfolio of work; in some subjects, a PhD student or ECR effectively works for a senior scholar, while in others the PhD student has a supervisor but can set their own research agenda; in some subjects, it is automatic to include the senior scholar in an article you wrote, in others it would be seen as exploitation of the work of a PhD student or ECR. In some subjects, a CV with your name on team publications is the norm, while in others it would look like you do not have your own ideas.

2. The REF reinforces these differences. You often find the impression that research exercises and metrics are there for the STEM subjects (or ‘hard sciences’) and not the humanities or social sciences: a process or review for Universities does not take into account the differences in incentives and practices across the disciplines, and some disciplines might lose out.

So, if you follow this debate on twitter, I recommend that you look at the bios of each participant to check their level of seniority and discipline because the Stern review is for all but it will affect us all in very different ways.

See also: James Wilsdon ‘The road to REF 2021: why I welcome Lord Stern’s blueprint for research assessment‘

 

Q. Should PhD students blog? A. Yes.

I wish I could go back and rewrite everything I have published, including my PhD. If I knew then what I know now: I would get to the point quicker and describe its importance to a far wider audience than my supervisor and a few dedicated journal readers. To do so, I would exhibit the skills you develop when you write frequently for an ‘intelligent lay’ audience.

These are the writing traits that I think you develop when just writing for academics:

1. You assume a specialist audience, familiar with key terms. So, you use jargon as shorthand without explaining its meaning. The downside is that the jargon often doesn’t have a particularly clear meaning. When you blog, you assume a non-specialist audience. You use less jargon, or you explain its meaning and value.
2. You treat the exercise as a detective novel with a big reveal: a nice, vague opening discussion (passive tense optional), a main body of text to build up the suspense, and finally the big twist at the end. Ta da! Wow, I didn’t see that coming. When you blog, you assume that people will not read your work unless you front-load the reveal. You have a catchy and tweetable title, you provide a hook in the first sentence, and you only have a few hundred words in which to show your work (and encourage people to read the longer report).
3. Or, you describe your hypotheses in a way that suggests that even you don’t know what will happen. Wow – I confirmed that hypothesis! Who knew? When you blog, it seems more sensible to use the language of hypotheses (or an equivalent) more simply, to explain what factors are most important to your explanation.

You can develop this skill by using a personal blog to describe your research progress and the value of your findings. However, it is also worth blogging in at least two other venues:

1. Somewhere like the LSE blog, or Democratic Audit, in which the editors will try to summarise your argument in a short opening statement. This is very handy for you: did they summarise the main argument? If so, good. If not, look again to see if you explained it well.
2. Somewhere like The Conversation, in which the editors will try to mess around with the title (to encourage more traffic) and wording (to make it punchier and quotable). This is a good exercise in which you can think about how far you want to go. Are you confident enough in your research to make such stark statements? Or, do you want to obfuscate and fill the argument with caveats? If the latter, you can think about the extent to which your argument is clear and defendable (it may well be – sometimes caveats and humility can be good!).

I also encourage advanced undergraduates and taught postgraduates to produce a blog post (albeit unpublished) alongside an essay or policy paper, because it is difficult to be concise, and the exercise helps develop a good life skill. Even without the blog exercise, I’d still encourage dissertation students (at the start of their research) to write up their argument/ plan/ work in a half-page document, so that we can see if it adds up to a coherent argument. You can do the same thing with a blog post, with the added (potential) benefit of some feedback from outside sources.

See also: there are resource sites which go into far more aspects of the writing process, such as medium.com/@Write4Research and patthomson.wordpress.com

The Art and Skill of Academic Translation: it’s harder when you move beyond English

I have been writing about the idea of ‘translation’ in terms of ‘knowledge transfer’ or ‘diffusion’, which often suggests that there is a linear process of knowledge production and dissemination: knowledge is held by one profession which has to find the right language to pass it on to another. This approach has often been reflected in the strategies of academic and government bodies. Yet, the process is two-way. Both groups offer knowledge and the potential to have a meaningful conversation that suits both parties. If so, ‘translation’ becomes a way for them to engage in a meaningful way, to produce a common language that they can both ‘own’ and use. Examples include: the need for scientists to speak with policymakers about how the policy process works; the need for ‘complexity’ theorists to understand the limits to policymaker action in Westminster systems; the separate languages of institutions which struggle to come together during public service integration (key to local partnership and ‘joined up government’); and, the difference in the language used by service providers and users. We might also worry about the language we use to maintain interdisciplinary discussion (such as when ‘first order change’ means something totally different in physics and politics).

It’s not the same thing, but translating into another language, such as when conversing in English and Japanese, reinforces the point in an immediately visible way. In both directions, English to Japanese, and vice versa, it is clear that the recipient only receives a version of the original statement – even when people use a highly skilled interpreter. Further, if the statement is quite technical, or designed to pass on knowledge, the gap between original intention and the relayed message is wider still.

This point can be made more strongly in a short lecture using interpretation. As academics, many of us have been to conferences in English, and witnessed a presenter trying to cram in too much information in 15 minutes. They give a long introduction for 10, then race through the slides without explaining them, simply say that they can’t explain what they hoped, or keep going until the chair insists they stop. You don’t really get a good sense of the key arguments.

In another language, you have to reduce your time to less than half, to speak slowly and account for translation (simultaneous translation is quicker, but you still have to speak very slowly). You have to minimise the jargon (and the idioms) to allow effective translation. Or, you need to find the time to explain each specialist word. For example, while I would often provide an 8000 word paper to accompany a lecture/ workshop, this one is 1500. There is no visible theory, although theory tends to underpin what you focus on and how you explain it. It took 40 minutes to present, largely because I left a lot of topics for Q&A. I still had a hard time explaining some things. I predicted some (such as the difference between ‘federalism’ and ‘federacy’, and the meaning of ‘poll tax’ and ‘bedroom tax’) but realised, late on, that I’d struggle to explain others (such as ‘fracking’, or the unconventional drilling used to access and extract shale gas).

This sort of exercise is fantastically useful, to force you to think about the essential points in an argument, keep it short without referring to shorthand jargon, and explain them without assuming much prior knowledge in the audience, in the knowledge that things will just mean different things to different audiences. It is a skill like any other, and it forces on you a sense of discipline (one might develop a comparable skill when explaining complex issues to pre-University students).

Indeed, I have now done it so much, alongside writing short blog posts, that I find it hard to go back from Tokyo to jargon city. Each time I read something dense (on, for example, ‘meta-governance’), I ask myself if I could explain it to an audience whose first language is not English. If not, I wonder how useful it is, or if it is ever translated outside of a very small group.

This is increasingly important in the field of policy theory, when we consider the use of theories, developed in English and applied to places such as the US and UK, and applied to countries around the globe (see Using Traditional Policy Theories and Concepts in Untraditional Ways). If you can’t explain them well, how can you work out if the same basic concepts are being used to explain things in different countries?

Further, we don’t know, until we listen to our audience, what they want to know and how they will understand what we say. Let me give you simple examples from my Hokkaido lecture. One panellist was a journalist from Okinawa. He used what I said to argue that we should learn from the Scots; to develop a national identity-based social movement, and to be like Adam Smith (persevering with a regional accent, and a specific view of the world, in the face of snobbishness and initial scepticism; note that I hadn’t mentioned Adam Smith). Another panelist, a journalist from Hokkaido, argued that the main lesson from Scotland is that you have to be tenacious; the Scots faced many obstacles to self-determination, but they persevered and saw the results, and still persevere despite the setback (for some) of the referendum result (I pointed out that ‘the 45%’ are not always described as tenacious!). Another contributor wondered why Thatcherism was so unpopular in Scotland when we can see that, for example, it couldn’t have saved Scottish manufacturing and was perhaps proved correct after not trying to do so. Others use the Scottish experience to highlight a similar sense of central government imposition or aloofness in Japan (from the perspective of the periphery).

In general, this problem of academic translation is difficult enough when you share a common language, but the need to translate, in two ways, brings it to the top of the agenda. In short, if we take the idea of translation seriously, it is not just about a technical process in which words are turned into a direct equivalent in another language and you expect the audience to be informed or do the work to become informed. It is about thinking again about what we think we know, and how much of that knowledge we can share with other people.

Reviews of My Books

A review of Understanding Public Policy and Global Tobacco Control in Public Administration: Painter review of 2 Cairney books 2013

A review of Global Tobacco Control in Governance: Kurzer review of GTC in Governance 2014

A review of Understanding Public Policy from an early career academic: http://ezinearticles.com/?Book-Review:-Understanding-Public-Policy-by-Paul-Cairney&id=7116636

Two reviews of Understanding Public Policy in Political Studies Review:

 

(they are both here http://onlinelibrary.wiley.com/doi/10.1111/1478-9302.12000_9/abstract)

From someone keeping it succinct and numeric: http://www.goodreads.com/book/show/12570034-understanding-public-policy#other_reviews

The Shami Chakrabarti Greeting Someone at a Political Studies Conference Rule

Update 22.3.30: No.

One of the few things funnier than Shami Chakrabarti’s speech (scroll down here) at the Political Studies Association annual dinner was the sight of a succession of men kissing her, politely but clumsily, on each cheek, as they received awards for excellent scholarship. Women received awards too, but they generally had the greeting down to a fine art. It raised, by far, the most important issue of the annual conference for me: how should I greet colleagues? For me, men are easy because the exchange is predictable. You shake their hands. In some cases, you get a bone cruncher, but that’s just physical rather than social discomfort. The same goes, almost always, for women I meet as colleagues. However, on a small number of occasions, we hug. I thought I had solved this problem by simply hugging the same people each time. As long as I know what we’re doing, I’d happily greet someone in any way they like. I’d even high 10 someone, up high and down low, and then the bit round the back, if I knew it was always going to be that way. Yet, things change: you sometimes miss your window to hug some people (awkward) and, on very rare occasions (at least for me) you hug someone spontaneously. It’s a fraught situation in a room packed with introverts. So, drawing on historical institutionalism, I propose the Shami Chakrabarti Greeting Someone at a Political Studies Conference Rule, which comes in two parts:

1. The default rule should be handshakes all round, at a Dutch (not Swedish) level of strength and eye contact. Or, if you’re Scottish, it’s OK to say ‘alright?’ in a slightly too loud voice.

2. If, at a critical juncture, you’ve hugged in the past, the default should be that you hug each time you meet, for the rest of your lives. Back slap optional.

Journal Article Acceptance(s) After 5 Rejections and 25 months

Update: the title is now less catchy but more accurate. See the italicised bits for the update. I have also added this poster:

You might have to be a glass-is-half-full kind of person to take something positive from this story of publication success after a long run of failure. After 18 months, 5 rejections, 4 substantial redrafts, 2-3 changes of journal direction, and minus 8000 words, we had it accepted (update: add another 7 months,  and three substantial redrafts and additions, for the 2nd article acceptance).

It began with our submission to World Politics, which is a high status journal, in politics and international relations, with a high rejection rate, so this was a gamble. I thought we had done the double: produced something interesting to say about ‘evolutionary’ policymaking, building on work I began for Understanding Public Policy; and, produced a wealth of new information on global tobacco policy, built on work led by Donley Studlar and Hadii Mamudu, and informing Global Tobacco Control. So, HM and I put both together to produce this paper, submitted 11^th September 2012:

World Politics Evolutionary Theory International Agreements 11September2012

It was rejected on the 4^th December (not a bad turnaround). The rejection came with substantial reviewer comments – World Politics decision letter – which we used to revise the next version substantially. My impression, from this review, was that the combination of evolutionary theory and the case study was not working. In fact, I may have been pushing us into a position that I advise PhD students and early career researchers to avoid: a paper suggesting a new theoretical angle, reinforced by a single case . In my defence, I wasn’t proposing a new theory. Instead, I was trying to present the approach as a reflection of accumulated knowledge, in both theory and case.

Still, it wasn’t working, so we separated the two elements somewhat. I chopped about 3000 words of theory – something made easier by the fact that I had submitted (February 2012) a separate paper on evolutionary theory to Policy and Politics, which was reviewed (July 2012) and accepted after a minor revise-and-resubmit (23 October) then published early 2013 – ‘What is Evolutionary Theory and How Does it Inform Policy Studies?’ Policy and Politics, 41, 2, 279-98 Paywall Green

We hummed and hawed about policy journals before I made the mistake of sending it to Public Administration and Development, partly because we were focusing on contrasts in implementation based on the simple developed/ developing country distinction, partly because it was interdisciplinary, and partly because its description seemed really close to our topic.

Cairney Mamudu Evolution Tobacco Control PAD submission 5Feb2013

It was rejected without going to review, described by the editor as ‘out of scope’.

So, we sent it, almost immediately (21 Feb 2013), to Governance, which had been HM’s (more sensible) preference. Again, this is a high status political/ policy science journal with a high rejection rate, so we were still confident enough to take the usual gamble.

Anonymous Evolution Tobacco Control Governance submission 21Feb2013

It was rejected on 26^th May after substantial review (which seems more critical than the reviewers of World Politics, so we were no further forward) –  governance rejection

We figured that we had to do two things based on the reviews: (1) strip out the discussion of evolutionary theory more and focus on the basic political science concepts (implementation, networks, agendas, etc.), shifting back the focus to the case study and evidence so far (particularly since I had now published an article separately on evolutionary theory); (2) be super-clear on key terms (leading/ laggard; developed/ developing) to anticipate future concerns, and clarify the narrative on the origins and role of the FCTC.

By this time, my University had made available some funds for Open Access, and I was keen to go this route, partly because OA seems good, and partly because I had recently co-authored an article in the OA journal Implementation Science and it was a very positive experience.

We chose Globalization and Health – based at the LSE, interdisciplinary, covering our topic and focus – and submitted on 12^th September 2013. It was rejected on 29^th October, which is a good turnaround, but the reviews were too brief to be useful – except it is still clear that our attempts to address the developed/ developing distinction are still needling our referee audience.

GH rejection letter GH referee 1 GH referee 2

Our solution was twofold: (1) to check with the editor of the next journal if there would be a problem with our approach, and (2) to get away from the developed/ developing sticking point by presenting an even more nuanced account, taking every opportunity to show that we weren’t providing naïve caricatures, and going super-conceptual to describe an ideal-type of a leading implementing country rather than identifying ‘leaders’ and ‘laggards’.

I emailed the editors of the Journal of Public Health Policy in November and got a good assurance on the developed/ developing point. The only problem is that the word limit is 4000, which is about one-third of the length of our original paper. Still, we revised the paper again.

By then, HM reckoned that Tobacco Control was a better fit, since they had begun to publish a series of papers on the ‘endgame’. We submitted there on the 20^th December.

1 TCJ-Endgame_CoverLetter-14Dec2013 2 Cairney Mamudu Checklist cover letter 3 Cairney-Mamudu_22Dec2013

They rejected it on the 8^th January 2014 without sending it to review TC rejection

We sent it to the JPHP on the 10^th January – 1 CAirney Mamuducover letter JPHP 10JAn14 2 CAirney Mamudu Submitted article JPHP 10JAn14

We got a revise and resubmit on the 17^th February – a very decent turnaround indeed. We got the classic binary response: one thought it was great, and one thought it was mince – JPHP reviews 17.2.14

We resubmitted on March 13 – 1 cover and rebuttal letter 2 resubmitted JPHP    – and got the thumbs up by the 27^th.

Update, November 2014. We submitted a much better paper on the same theme (more developed theoretical argument, more data, a better refined argument) to Public Administration (special issue on global public policy) in June. After two resubmissions (and, unusually, a referral to a member of the editorial board – to deal with comments made by the third reviewer), we had it accepted in November.

So what did we learn?

1. It is natural to blame journals, editors and reviewers for these long, drawn out processes – but I need to take some responsibility for the journal choices and the quality of submissions.
2. Even a rejection can give you useful material for a redraft, as long as it actually goes to review.
3. It is worth persevering. This is a very unusual case of 5 rejections, but it seem fairly normal to get 1 or 2 before success. For a while, I went on a good run of acceptances-after-revision, then a run of acceptances after rejection. I have almost always published each paper by the end.
4. I think the article is, in many ways, a far better paper than when it began – but it also changed so much that we reckon we can go back and submit some of the chopped material (the new data) elsewhere.
5. Final lesson – you need a thick skin for this process, particularly when you get one or two cranky anonymous reviewers, and particularly when you go interdisciplinary and invite comment from people who often don’t respect your discipline.
6. Final, final, updated lesson: don’t lose your confidence and settle for a second-best result. Our first acceptance was for an article that stripped away a lot of what was good in the original idea (partly to meet the 4000 word limit), and it was rewritten for a public health audience in a way that I don’t entirely like. The Public Administration article (9000 words) is the one I’ll send to people and be proud of. It was accepted more than two years after we first made the mistake to send it to a different journal.

 

 

 

Comparing Theories of the Policy Process: A Brief Guide for Postgraduates

When you do a PhD in policymaking, it is likely that you have to engage significantly with the literature on policy theory. This often prompts bursts of enthusiasm, when some theories seem to be spot on, and periods of frustration, when theories seem to be deficient in some way. For example, they might only explain one part of the ‘policy process’ when you want to explain it all, or they are difficult to operationalise and apply to specific cases. This may be a particular problem for PhDs focused primarily on a substantive case study. You will find that the case study is too complicated to be explained fully by a relatively simple theory designed to be applied across a range of cases. You may then think about what to do, focusing on two main possibilities:

First, should I propose my own theory, which takes insights from other theories but puts them together in a new way, perhaps with a new terminology? My advice is: no (or not unless you are convinced that you are a genius, destined to become the new Sabatier, Kingdon, Ostrom, Schneider, Ingram, Baumgartner or Jones). Don’t confuse a better explanation for your case with a better overall explanation. Your theory is not backed up in the same way by multiple applications and long term refinement.

Second, should I use the insights from a range of established policy theories to inform my case study? My advice is: yes, but be careful. Read this Policy Concepts in 1000 Words: Combining Theories. Then, consider what you are trying to do. In most cases, your aim is to account for a significant part of the policy process. Tanya Heikkila and I describe what that might entail. In Theories of the Policy Process 3^rd ed.  we identify (with a lot of input from Chris Weible) what we think are the main elements of the policy process that a theory or case should cover:

1. “Actors making choices.  We need to simplify a policymaking world which may include thousands of people, into a set of categories and/ or discussion of the key actors involved. Actors can be individuals or collectives, and collectives can range from private companies to interest groups to governments bodies. We also need to account for the ways in which people act; their calculations and motivations. For example, most theories use ‘bounded rationality’ as a springboard for explanation, while others focus on motivations such as beliefs.
2. Institutions. These are the rules, norms, practices and relationships that influence individual and collective behaviour.  The choices of actors is explained to some extent by their understanding of, and adherence to, such rules. Those rules can be formal and widely understood, such as when enshrined in law or a constitution. Or, they can be informal and only understood in particular organisations. Policies can be considered a subset of the broad concept of institutions, but institutions at one level (e.g. constitutional rules) can also shape the policymaking activities or decisions at another level (e.g. legislation or regulation). Similarly, institutions can establish the types of venues where policy decisions are made and the rules that allow particular types of actors or information and ideas to enter into the policy process.
3. Networks or subsystems. These are the relationships between actors responsible for policy decisions and the ‘pressure participants’ such as interest groups with which they consult and negotiate.  Senior policymakers delegate responsibility for policy making to bureaucrats, who seek information and advice from groups. Groups exchange information for access to, and potential influence within, government or other collective choice processes. It is through these networks where collective action often emerges in policy processes.  Many theories describe a process in which groups exchange information for access to, and potential influence within, government. Bureaucracies and other public bodies may have particular operating procedures that favour particular sources of evidence and some participants over others.
4. Ideas. This is a broad term to describe beliefs, or ways of thinking, and the extent to which they are shared within groups, organisations, networks and political systems. It can refer to two intertwined processes. First, shared ideas (knowledge, world views, language) appear to structure political activity when they are almost taken for granted or rarely questioned – for example, as core beliefs, paradigms and monopolies of understanding. Second, new ideas or ways of thinking can be used to prompt actors to rethink their beliefs to some extent – such as when a proposed new solution challenges the way that a problem is framed or understood, and therefore how much attention it receives and how it is solved. So, for example, we may identify relative stability when shared ideas are not questioned and instability when different groups with different beliefs interact.
5. Policy Context. This is a broad category to describe the extent to which a policymaker’s environment is in her control. It can refer to the often-changing policy conditions that policymakers take into account when identifying problems and deciding how to address them, such as a political system’s: geography, biophysical and demographic profile; economy; and, mass attitudes and behaviour.  It can also refer to a sense of policymaker ‘inheritance’ – of laws, rules, institutions and programs – when they enter office.
6. Events. Events can be routine and anticipated, such as the elections which produce limited change or introduce new actors with different ideas about policy problems and solutions. Or, they can be unanticipated incidents, including social or natural crises, or major scientific breakthroughs and technological change. Their unpredictability makes them difficult to theorise, and they can often effectively be treated as ‘error’, or as external factors providing an additional source of explanation to a policy theory. Or, they can be incorporated within theories which focus on how actors interpret and respond to events. To some extent, an event is only significant if actors within political systems pay attention to them”.

From there, we can consider how each theory accounts for the nature of these elements, and their interaction, to explain policy dynamics and outcomes. You can find summaries of many of these relevant theories here: https://paulcairney.wordpress.com/1000-words/ and a copy of the 3rd and 4th edition chapters here: Cairney and Heikkila 2014, Heikkila and Cairney 2017.

For what it’s worth, when I co-authored the book Global Tobacco Control (with Studlar and Mamudu), we had chapters on the role of policy theory but structured the book according to those elements of the policy process, not specific theories. I’m not saying that you should take this simple approach. Rather, I am asking you to think about, and explain, why yours is better.

What Works (in a complex policymaking system)?

The Scottish Government and ESRC held an event yesterday to publicise their proposed new What Works centre. Its role is ‘is to deepen the impact of the emergent Scottish approach to public service delivery and reform, by evaluating evidence in delivery of that approach’. As you might expect from a series of presentations (9 in total), the speakers presented ideas which had the potential to vary in meaning and emphasis. Consequently, there seemed to be some potential tensions between things such as:

1. A government trying to ‘scale up’ from the experience of successful pilots, and a government devolving responsibility to local bodies (such as local authorities), giving them the space to choose how to meet broad Scottish Government aims – unless ‘scaling up’ simply means encouraging ‘best’ or ‘good’ practice. Scaling up may have a different meaning in different academic disciplines and different policy areas. For example, healthcare and public health may be associated more with larger and more uniform policies than ‘community’ based projects.
2. A focus on learning from successful projects, quite quickly, when the Scottish Government and ESRC are looking for examples of policies that produce good outcomes after, 10, 20 or more years. The former may involve drawing relatively quick conclusions about the success of projects, based on a mixture of their reputations and evidence limited to a small number of years. This is a general feature of ‘lesson-drawing’ or ‘transfer’ from one government to another – borrowers are often very quick to judge the success of lenders, based on some rough and ready indicators.

What struck me in particular was Harry Burns’ (Chief Medical Officer, Scottish Government) emphasis on the importance of complexity, complex systems and ‘complex systems thinking’. As I have found recently, when co-editing a book on complexity and public policy, complexity is a remarkably vague term which can mean very different things to different people. To many, it means something akin to ‘complicated’. To others, to identify a complex system is to identify a very specific set of arguments and properties, including:

1. A complex system is greater than the sum of its parts; those parts are interdependent – elements interact with each other, share information and combine to produce systemic behaviour.
2. Some attempts to influence complex systems are dampened (negative feedback) while others are amplified (positive feedback). Small actions can have large effects and large actions can have small effects.
3. Complex systems are particularly sensitive to initial conditions that produce a long-term momentum or ‘path dependence’.
4. They exhibit ‘emergence’, or behaviour that results from the interaction between elements at a local level rather than central direction.
5. They may contain ‘strange attractors’ or demonstrate extended regularities of behaviour which may be interrupted by short bursts of change.

Similarly, the meaning of ‘systems thinking’ is often unclear or unhelpful when we seek to go beyond the instantly intuitive notion of thinking about the broader context of policymaking. Here are some further examples of unresolved issues regarding complexity and public policy:

• Some people describe the natural and social world (the latter is often the thing that policymakers want, but struggle, to influence) as complex, but neglect to consider the idea that policymaking systems are complex systems. The particular relevance to What Works is that it would be a mistake to recognise the complicated set of problems to be solved by policies, but ignore the complicated set of processes to which the policies themselves are subject. This is often a key feature of wider debates on ‘evidence based policymaking’.
• One tenet of complexity theory is that law-like behaviour is difficult to identify – so a policy that was successful in one time or place may not have the same effect in another. This places important limits on the idea of ‘scaling up’ from the experience of one study. If scaling-up means encouraging good practice, adapted to local experience, all well and good. If it means ‘rolling out’ a successful policy to many areas, it may be problematic.
• One solution, of sorts, to such problems, is to accept that we don’t know the individual effects of individual policy instruments. Rather, we introduce a wide range of policy instruments and hope that they interact to produce a positive overall outcome. This sort of approach can be found in areas such as tobacco control (in which there may be six main types of policy, each with several elements), but may be less common or applicable in other policy areas. The complication is that it is difficult to say ‘what works’ if we mean ‘what particular policy instruments work?’.
• ‘Systems thinking’ may prompt us deal with uncertainty and change by encouraging trial-and-error projects, or pilots, that can provide lessons, and be adopted, amended or rejected, relatively quickly. This encourages us to think about the potentially very limited role of a central government. A lot of the literature uses complexity theory to present almost the opposite idea to the ‘Westminster model’ in which power rests, and should rest, in the hands of a small number of elected policymakers, accountable to the public via Parliament.  Many advocate relying less on central government driven targets, in favour of giving local organisations more freedom to learn from their experience and adapt to their rapidly-changing environment.
• A lot of the literature uses complexity theory to “challenge particular brands of ‘positivism’ which present a ‘vision of society based on order, laws and progress’ (Geyer and Rihani, 2010, p. 5); to suggest that ‘quantitative and reductionist methodologies’ may be useful to explain topics such as elections with ‘rules and orderly structures’, but not issues that contain unpredictable political events, significant levels of uncertainty and ambiguity (Geyer and Rihani, 2010, pp. 74–5) or factors outside the control of policy makers (Room, 2011)” (excerpt from Cairney 2012 PSR Complexity Theory). This raises the issue of methods and their underlying philosophy. At the event, he general response to a question on methods was the usual idea that we can mix them. Yet, the underlying ideas behind particular approaches may be complementary and others contradictory (see here and here Cairney 2013 PSJ Standing on the shoulder of giants for a wider discussion). A mix of methods is not a solution in itself.

A positive interpretation of these problems is that all governments face them, and some may respond better than others. This is certainly the reputation that the Scottish Government has developed, and this reputation is articulated in the What Works call:

• The Scottish Government is unusually positioned to respond favourably because successive Scottish Governments have appeared to be much more open to this sort of advice (or, at least, they have engaged in behaviour consistent with it).
• In particular, they have relied more on ‘partnership working’ and less on stringent performance management regimes linked to targets and punitive measures for not meeting them.
• The SNP Government in particular (from 2007) signalled a willingness to devolve more responsibility to local authorities, reduce the proportion of ‘ring fenced’ budgets  and develop less-top-down ‘single outcome agreements’ (Remember Alex Salmond in 2007: ‘The days of top-down diktats are over’).
• This difference of attitude might reflect cultural differences in Scotland’s political system or simply the different policymaking environment in which Scottish Governments operate.  In particular, Scotland is smaller and its policymakers have fewer responsibilities; both factors may allow them to develop quite meaningful horizontal and vertical relationships and rely less on more impersonal and inflexible policy measures.

Related posts:

Policy Concepts in 1000 Words: Policy Transfer and Learning

Policy Concepts in 1000 Words: ‘Evidence Based Policymaking’

Policy Concepts in 1000 Words: Complex Systems

Related paper: Paul Cairney 13.1.14 How Can Policy Theory Inform Policymaking

Academic Branding

The word ‘branding’ applied to academics is so pretentious that I feel dirty just typing it, but it’s a good way into thinking about how you are seen as an academic based on the decisions you make. I think that a lot of assume that we should brand ourselves according to our specialism: there’s Professor X, who knows the most about mutation, or there’s Professor Green who knows the most about mangling songs. However, there are now strongish incentives (in some disciplines – mostly social science?) to maintain some sort of generalist knowledge to further comparative, interdisciplinary and/ or ‘impact’ work. In each case, your audience may not necessarily benefit from your fiddly knowledge of a subject but, instead, appreciate how it fits into a wider picture. Let me give you some rationalised and glossed-over examples of my career to make the point:

1. Comparative work. I’ve been paid to go to Japan 4 times (3 since 2011, 2 this year) on the back of my work on Scottish devolution. The most useful stuff here, even for academic colleagues, is the Scottish Politics textbook I did with Neil McGarvey. My most recent trip was to give talks to a public and practitioner audience where, again, few are interested in the Scottish navel.
2. Interdisciplinary work. I’ve been working on collaborations with colleagues in subjects such as physics and psychology, The most useful stuff here is a public policy textbook I did in 2011.
3. Specialist work. I’ve been making some good links with US colleagues, including Chris Weible (editor of Policy Studies Journal), who asked me to co-author the ‘Schlager chapter’ in the 3^rd ed. of Theories of the Policy Process (you may have to trust me on how good that is). Again, this came out of work I did bringing together a discussion of theories after doing a textbook.

So, I’m convinced, based on some anecdotal evidence (which I’ve manipulated to suit my argument) that this more general textbook (and bloggy) work has been crucial to my career development – perhaps since it has helped me write in a more accessible/ less jargony way. Perhaps more importantly, it has allowed me to maintain a broad knowledge in particular fields, including policy theory and one particular area (Scotland/ UK). It is no substitute for the REF-type work we all have to do but, then again, it may not hurt either (my big book is underpinned by the policy theory work I analysed while writing a textbook).

Addressing Academic Assumptions: Revisiting That Presentation (on the Scottish Parliament)

Presenting to colleagues in other disciplines is interesting because it makes you think about your disciplinary assumptions – what you often take for granted and would assume that your home audience knows too. This came up at a workshop led by legal academics. A small group of us sort of pointed out that we were political scientists and would approach the same topic (in this case, constitutional design) in different way. By the time it got to my talk, I felt that the paper I had prepared was incomplete because, from a policy studies perspective, I had my own starting point and my audience would not know about it:

• Most policy is made in policy networks/ communities/ subsystems
• The state is so large that it may become unmanageable. So, policymakers divide administration into departments and policymaking into sectors and subsector
• Ministers can only pay attention to a tiny proportion of issues for which they are responsible. So, they delegate almost all policymaking to civil servants. Senior civil servants do the same to relatively junior civil servants.
• Civil servants rely on participants such as interest group for information and advice. They may also seek a degree of group ‘ownership’ of policy. Civil servants and groups may form fairly close relationships over time.
• The key point regards ‘parallel and serial processing’. Policymakers can only engage in serial (considering one issue at a time) while governments as a whole engage in parallel (processing many issues at once).

So, this is the context for a consideration of Parliament. An ill resourced Parliament can engage in serial processing but will struggle to engage in parallel – and therefore to hold the Government to account. The attention of ministers and parliamentarians will lurch from issue to issue, often with important consequences, but their attention to one issue means that they must ignore the others.

This context is so familiar to many policy/ politics scholars that they may be surprised if one takes Parliaments seriously. In fact, when I gave my presentation for a lectureship at Aberdeen in 2004, one audience member pretty much said ‘since parliaments are peripheral to the policy process, why are you bothering with this topic?’. I have two answers:

1. Parliaments perform other functions – deliberative, participatory, symbolic and, most importantly, they legitimize the outputs of government. Without Parliament, the government would struggle to maintain a wider sense of public legitimacy for its decisions. Consequently, a parliament can appear, simultaneously, to be highly ineffective (in relation to scrutiny) and profoundly effective (at legitimization).
2. The Scottish Parliament was sold as a powerful institution at the centre of a range of ‘new politics’ initiatives.

So that is the context for the rest of my talk here –

 https://paulcairney.wordpress.com/2013/11/12/the-role-of-the-scottish-parliament-in-a-devolved-or-independent-scotland/

UPDATE: Questions from the audience

(1) What would you do about these weaknesses in scrutiny. Potential remedies?

I gave three main solutions: (a) lower your expectations about what can happen; (b) increase parliamentary resources (permanent staff seem like better value than elected MSPs in this context) to increase MSP incentives to engage in scrutiny; and (c) learn from other countries and decide if you want to transfer their practices. As I discuss with colleagues in a comparison with Sweden, we may not be willing to give up what we have (clear lines of accountability) to secure what they have (more cross-party cooperation).

(2) Should there be a ‘big bang’ reform of the Scottish political system to address these problems? I tried very much not to answer this question. It doesn’t seem likely to happen or to change the fundamental relationship between government and parliament.

 

A Concise Response to Any Question

Sometimes you need to give a quick response to a media inquiry. Often journalists spread their net a bit, looking for something quick from the first responder (to add to an almost-finished audio piece). There’s no need to be bitter about this – better to be prepared. And concise. And you don’t want to say something too specific or controversial – particularly if you are responding to questions on a sensitive topic. You also want to give the sense that the question might be simple but, by gosh, the answer is complex. It’s hard to give that impression in a sound-bite response. Until now. If said properly, this is a maximum of one word in under one second.

 

The Academic Interview

I have seen a few posts recently giving advice and perspective on applying for academic posts – such as Nadine Muller on her early interview experience and Steve Joy’s annoying-but-good advice on CVs . I can’t give any useful advice on doing a good interview, since I have messed up every time (even the feedback from my 2 successes was that, to some extent, I got the job despite the interview). Only point 7 is based on my experience at that side of the table. However, I have sat on a few panels in the last few years, so can give some practical advice from that side:

1. There is a range of processes, from the entry-level to the higher grades (and from University to University), which affect your preparation. Most notably, you don’t always do a presentation and interview to the same people. For a lectureship, you might present to the department then have a completely separate interview process (perhaps the following day) with people who did not see the presentation. For a research job, the (often very brief) presentation is part of the whole. If so, you may be doing a presentation to 3-5 people.
2. In either case, assume that the interview audience is the one to impress – the departmental audience may have a minimal input.
3. To a panel of 3-5 people, interviewing 5 people in a day, a lengthy powerpoint presentation can be particularly awkward. If you are swithering about keeping to time, or cramming in information, go for the former. If you get it right, it will show that you can explain complex ideas concisely.
4. Doing an interview remotely is very high bar. It is difficult to maintain a constant connection between you and your audience, and to pick up vital social cues.
5. Don’t haver when you are asked the generic question: why here, why now? For many panels, this is not just an ice breaker. We want to know if you have done your homework on our institution. We want to hear particular evidence that, at least, you have checked our website for a sense of who is here and what we are doing – then how you might fit in and make a contribution (although, note that University websites are often out of date).  We want to know if this is just one of many applications you have made. We want to feel special.
6. At the entry-level, the interview performance may count a little bit more. At the chair level, a lot has already been decided since you are being judged on your long record in your CV (or, in the UK, if you are appointed near the census date for the research assessment exercise, you may be judged on your ‘best 4’). At the entry level, there is a bigger judgement to be made about potential, which is difficult (but not impossible) to judge on short CVs – particularly when all the shortlisted candidates have a PhD and a small number of publications. I have seen a few people rocket up the short-list rankings after a good interview performance, where they are well prepared for questions and can think on their feet (since many of us academic types like to think that we are creative; that it’s not just a job that anyone can do well).
7. Don’t drink too much coffee then tell the panel that you are wired.

10 Reasons Why You Should Vote For/ Against Scottish Independence