Category Archives: Uncategorized

POLU9RM Asking the right research question

When I supervise dissertation students, I try to get them to do things in a specific order:

  • get the research question right
  • write an abstract to see if you can answer it (and explain how you structure the dissertation to allow you to answer it)
  • write the introduction to see if you can explain the whole rationale for your dissertation before you do most of the research.

Now, I don’t want to get into a big debate with the deviants who want to write or rewrite their introductions at the end. You can do what you like, pal.

Instead, I want to emphasise the benefits of the early investment. If you get the research question spot-on, in relation to the introduction, you can do the following:

  • make your project manageable from the start, without learning the hard way that you’ve bitten off more than you can chew
  • save a remarkably hellish amount of time on your ‘literature review’ by producing a clear sense of what is relevant/ to be skipped over
  • boast to your friends that you finished on time.

There is some good advice out there on designing a question to speak to a big question and a narrow research project at the same time.

For example, most of my projects follow roughly the same format: what is policy, how much has it changed, and why?

We can then narrow it down in several ways:

  • Choose, say, tobacco policy (quite specific) versus health policy (very broad indeed)
  • Choose one political system or one region, or limit your comparison of systems
  • Choose one time period
  • Choose what aspect of change you want to explain.

The latter is often the most important, because (in my case) it can make the difference between (a) feeling the need to explain many, many theories to give the whole picture (an impossible task) or (b) narrowing down theory selection by focusing on a small number of causes/ dynamics.

Ideally, the question should be super-important and sophisticated, but a dissertation also takes a lot of time and attention. So, my best advice is to choose a question to which you actually want to know the answer. If so, you should end up very satisfied in your result. If you don’t find the question interesting, you may come to resent your dissertation.

A final thought is that students often don’t know what question to ask, and they talk quite broadly about a very general topic. In such cases, it’s important to work with your supervisor until you’re both happy with the final result. My most memorable example is a student who, above all else, wanted to write about Beyoncé (and it worked out very well indeed).

Leave a comment

Filed under Research design, Uncategorized

People come to my show for the joy of being hoodwinked

Ladies and gents, roll up, roll up, as I offer to you a fantastic and surprising story. You’ll think you’re seeing one story. Then you’ll think it’s another. Actually it’s another story entirely.

Our story begins to be about a circus owner, before appearing to turn into a story about giving voice to people once considered to be deviants.

Yet, it would fail at both tales, because to romanticize historic circus owners would be wrong (and it couldn’t be clearer that the film is not a biopic) and to really give voice to so many people would require a different film entirely.

Instead, we have a fairly safe and conventional film in which the main characters are white, male, heterosexual, and going on the usual voyage of self discovery.

As such, its popularity can be ensured by appealing widely to a fairly conventional family audience without being too threatening to the people buying the tickets.

Yet, there is an unusually radical morality tale hidden within this safe morality tale if you conclude eventually that the white male heroes are not heroic at all.

At first glance, you think you’re being asked to root for them as they go on a journey at great risk to themselves for the benefit of other people.

At second glance, they act out of selfishness and ignorance, yet still do well despite their faults because, for them, the rewards are unusually high and the stakes will always be relatively low.

The main character is PT Barnum. At first, there is a lot to root for. He emerges from poverty after showing much tenacity and imagination. In the process, he sticks it to the upper classes – symbolised by Charity Barnum’s violently oppressive father – and shows that anyone from humble beginnings can end up rich and famous if they have a big enough chip on their shoulder.

Yet, really, his is a story of male hubris and selfishness, in which he manipulates the people he claims to care for, and takes all sorts of risks which affect the people around him more than himself. He is in the remarkably privileged position – afforded to very few other characters – to be able to fail then succeed; to emerge as the wise hero at the end.

The superficial moral is to treat people with respect, and be thankful for the love of a small group of people around you (a message handed to him by the heroically supportive Charity). The deeper moral is that our hero is not to be admired for his actions. Our main character is a clown who (*spoiler alert*) emerged victorious despite his faults. He did not get there on merit.

The second main character is Phillip Carlyle, a rich white man putting on theatrical plays and drinking a lot until he inherits massive wealth from his father. At first glance, he is cast in the heroic role of a man willing to give it all up for his dreams and for love. Yet, his economic position never really seems in doubt, wavering from super wealthy to merely wealthy.

More importantly, he appears heroic because he is (eventually) willing to undermine his social position to form a relationship with a black woman, Anne Wheeler.

Yet, Phillip quickly serves as the film’s representation of the ignorance of white men who do not understand the unequal negative effects, or unintended consequences, of collective positive action.

For him, the choice is simple: to declare their love, be defiant, and not care about the reactions of other people (including his racist and classist parents).

This act exposes his ignorance and prompts Anne to explain that she would bear the brunt of their decision. She would be continuously vulnerable to violence and abuse (and would have to choose to be totally reliant on a white man for support and protection, despite having no previous life experience prompting her to trust white men). She knows this because it is a constant part of her life experience, of which he is blissfully ignorant.

[The importance of racism to this story is reinforced in three brief scenes with WD Wheeler, Anne’s brother. In one scene, he warns correctly about the likely hostile and violent reaction of a white audience to black performers, even though Barnum clearly designs his recruitment of performers to shock.

In another, his brief knowing smile confirms to Anne their belief that black people will be excluded from their visit to the Queen, because exclusion has been their routine experience

In another, he punches a white man – part of a mob – who threatens him and calls him a ‘spook’].

This context does not occur to Charles at all. Then, when she explains, he dismisses her concerns (albeit melodically) before she rejects him.

In the end, Anne decides (*spoiler alert*)  that a lifetime of external abuse for her is preferable to his horrible death, which really makes her the hero.

By the end, the two main male characters are totally delighted because their lives turned out brilliantly despite their ignorance and hubris. So, the film provides a realistic moral from a deliberately unrealistic premise.

🎤🎵🎶🎼 It also has some brilliant songs which make me cry every time I see it. I’ve seen it 7 times so far and I’ve lined up 2 more visits. I honestly don’t know why it’s so good, but I love it. Love it. Absolutely love it. If you go, sit in the front section. It has better sound and you can weep liberally and have a little dance when no one’s looking.

Leave a comment

Filed under Uncategorized

Of my two favourite NE Scotland beaches, this one is less touched by Trump

I love St Cyrus beach. This fact may explain why I took so many short videos along the 90 minute round trip. On days like today, you almost get the whole beach to yourself because of the weather. Yet, it’s the best time to go because the waves are so dramatic, the water against the rocks is hypnotic, and the sand blows along the surface in such a beautiful way. Coco and Mabel are keen too.

 


 

Leave a comment

Filed under Uncategorized

The Politics of Evidence revisited

This is a guest post by Dr Justin Parkhurst, responding to a review of our books by Dr Joshua Newman, and my reply to that review.

I really like that Joshua Newman has done this synthesis of 3 recent books covering aspects of evidence use in policy. Too many book reviews these days just describe the content, so some critical comments are welcome, as is the comparative perspective.

I’m also honoured that my book was included in the shortlist (it is available here, free as an ebook: bit.ly/2gGSn0n for interested readers) and I’d like to follow on from Paul to add some discussion points to the debate here – with replies to both Joshua and Paul (hoping first names are acceptable).

Have we heard all this before?

Firstly, I agree with Paul that saying ‘we’ve heard this all before’ risks speaking about a small community of active researchers who study these issues, and not the wider community. But I’d also add that what we’ve heard before is a starting point to many of these books, not where they end up.

In terms of where we start: I’m sure many of us who work in this field are somewhat frustrated at meetings when we hear people making statements that are well established in the literature. Some examples include:

  • “There can be many types of evidence, not just scientific research…”
  • “In the legal field, ‘evidence’ means something different…”
  • “We need evidence-based policy, not policy-based evidence…”
  • “We need to know ‘what works’ to get evidence into policy…”

Thus, I do think there is still a need to cement the foundations of the field more strongly – in essence, to establish a disciplinary baseline that people weighing in on a subject should be expected to know about before providing additional opinions. One way to help do this is for scholars to continue to lay out the basic starting points in our books – typically in the first chapter or two.

Of course, other specialist fields and disciplines have managed to establish their expertise to a point that individuals with opinions on a subject typically have some awareness that there is a field of study out there which they don’t necessarily know about. This is most obvious in the natural sciences (and perhaps in economics). E.g. most people (current presidents of some large North American countries aside?) are aware that don’t know a lot about engineering, medicine, or quantum physics – so they won’t offer speculative or instinctive opinions about why airplanes stay in the air, how to do bypass surgery, or what was wrong with the ‘Ant-Man’ film. Or when individuals do offer views, they are typically expected to know the basics of the subject.

For the topic of evidence and policy, I often point people to Huw Davies, Isabel Walter, and Sandra Nutley’s book Using Evidence, which is a great introduction to much of this field, as well as Carol Weiss’ insights from the late 70s on the many meanings of research utilisation. I also routinely point people to read The Honest Broker by Roger Pielke Jr. (which I, myself, failed to read before writing my book and, as such, end up repeating many of his points – I’ve apologised to him personally).

So yes, I think there is space for work like ours to continue to establish a baseline, even if some of us know this, because the expertise of the field is not yet widely recognised or established. Yet I think is it not accurate for Joshua to argue we end up repeating what is known, considering our books diverge in key ways after laying out some of the core foundations.

Where do we go from there?

More interesting for this discussion, then, is to reflect on what our various books try to do beyond simply laying out the basics of what we know about evidence use and policy. It is here where I would disagree with Joshua’s point claiming we don’t give a clear picture about the ‘problem’ that ‘evidence-based policy’ (his term – one I reject) is meant to address. Speaking only for my own book, I lay out the problem of bias in evidence use as the key motivation driving both advocates of greater evidence use as well as policy scholars critical of (oversimplified) knowledge translation efforts. But I distinguish between two forms of bias: technical bias – whereby evidence is used in ways that do not adhere to scientific best practice and thus produce sub-optimal social outcomes; and issue bias – whereby pieces of evidence, or mechanisms of evidence use, can obscure the important political choices in decision making, skewing policy choices towards those things that have been measured, or are conducive to measurement. Both of these forms of bias are violations of widely held social values – values of scientific fidelity on the one hand, and of democratic representation on the other. As such, for me, these are the problems that I try to consider in my book, exploring the political and cognitive origins of both, in order to inform thinking on how to address them.

That said, I think Joshua is right in some of the distinctions he makes between our works in how we try to take this field forward, or move beyond current challenges in differing ways. Paul takes the position that researchers need to do something, and one thing they can do is better understand politics and policy making. I think Paul’s writings about policy studies for students is superb (see his book and blog posts about policy concepts). But in terms of applying these insights to evidence use, this is where we most often diverge. I feel that keeping the focus on researchers puts too much emphasis on achieving ‘uptake’ of researcher’s own findings. In my view, I would point to three potential (overlapping) problems with this.

  • First – I do not think it is the role or responsibility of researchers to do this, but rather a failure to establish the right system of evidence provision;
  • Second – I feel it leaves unstated the important but oft ignored normative question of how ‘should’ evidence be used to inform policy;
  • Third – I believe these calls rest on often unstated assumptions about the answer to the second point which we may wish to challenge.

In terms of the first point: I’m more of an institutionalist (as Joshua points out). My view is that the problems around non-use or misuse of evidence can be seen as resulting from a failure to establish appropriate systems that govern the use of evidence in policy processes. As such, the solution would have to lie with institutional development and changes (my final chapter advocates for this) that establish systems which serve to achieve the good governance of evidence.

Paul’s response to Joshua says that researchers are demanding action, so he speaks to them. He wants researchers to develop “useful knowledge of the policy process in which they might want to engage” (as he says above).  Yet while some researchers may wish to engage with policy processes, I think it needs to be clear that doing so is inherently a political act – and can take on a role of issue advocacy by promoting those things you researched or measured over other possible policy considerations (points made well by Roger Pielke Jr. in The Honest Broker). The alternative I point towards is to consider what good systems of evidence use would look like. This is the difference between arguing for more uptake of research, vs. arguing for systems through which all policy relevant evidence can be seen and considered in appropriate ways – regardless of the political savvy, networking, or activism of any given researcher (in my book I have chapters reflecting on what appropriate evidence for policy might be, and what a good process for its use might be, based on particular widely shared values).

In terms of the second and third points – my book might be the most explicit in its discussion of the normative values guiding efforts to improve evidence, and I am more critical than some about the assumption that getting researchers work ‘used’ by policymakers is a de-facto good thing. This is why I disagree with Joshua’s conclusion that my work frames the problem as ‘bridging the gap’. Rather I’d say I frame the problem as asking the question of ‘what does a better system of evidence use look like from a political perspective?’ My ‘good governance of evidence’ discussion presents an explicitly normative framework based the two sets of values mentioned above – those around democratic accountability and around fidelity to scientific good practice – both of which have been raised as important in discussions about evidence use in political processes.

Is the onus on researchers?

Finally, I also would argue against Joshua’s conclusion that my work places the burden of resolving the problems on researchers. Paul argues above that he does this but with good reason. I try not to do this. This is again because my book is not making an argument for more evidence to be ‘used’ per se. (and I don’t expect policy makers to just want to use it either). Rather I focus on identifying principles by which we can judge systems of evidence use, calling for guided incremental changes within national systems.

While I think academics can play an important role in establishing ‘best practice’ ideas, I explicitly argue that the mandate to establish, build, or incrementally change evidence advisory systems lies with the representatives of the people. Indeed, I include ‘stewardship’ as a core principle of my good governance of evidence framework to show that it should be those individuals who are accountable to the public that build these systems in different countries. Thus, the burden lies not with academics, but rather with our representatives – and, indirectly with all of us through the demands we make on them – to improve systems of evidence use.

 

1 Comment

Filed under Evidence Based Policymaking (EBPM), Uncategorized

Monetizing my website

This could go very wrong, but hubris has taken me this far in life. I’ve decided to monetize my website in a nicer way than ‘monetize your website’ seems to suggest. For every 1000 hits, I’ll give £1 to a good cause.

Assuming that my blog only ever reaches ‘cult status’ and people keep buying expensive academic books, I can cover the donation comfortably with royalties. The worst that can go wrong is that my website starts to do very well indeed and it costs a bit more, but that outcome doesn’t seem like a likely or bad thing right now.

I made a £724 donation to Women on Waves (please feel free to set up your own blog to share your views on my choice) on the assumption that my all time hits will trickle into 724k by the end of the year.

I’ll make the donation once per year. If my website stays at around 200k per year, and the price of Toffifee doesn’t rocket after Bexit, I’ll treble the rate of donation to 3p per 1000. Then let’s see how it goes.

Leave a comment

Filed under Uncategorized

IMAJINE

I’m now working with Professor Michael Keating and Dr Emily St Denny on a work package for IMAJINE (Integrative Mechanisms for Addressing Spatial Justice and Territorial Inequalities in Europe).

We are leading WP6: Multilevel Policy-making and Inequalities, which “will explore how states design fiscal regimes and public services to mitigate the effects of socio-spatial inequalities”, through case studies in areas including Greece, Poland, Scotland and Wales”.

The website is about to grow. In the meantime, here are two posts with preliminary thoughts:

 

The theory and practice of evidence-based policy transfer: can we learn how to reduce territorial inequalities?

 

‘Co-producing’ comparative policy research: how far should we go to secure policy impact?

 

And here is a picture of many of us. One of our work plans is to look less menacing by year 4.

imajine-launch-photo-jan-2017

2 Comments

Filed under Uncategorized

How to design ‘maps’ for policymakers relying on their ‘internal compass’

This is a guest post by Dr Richard Simmons (below, in between Profs Alex Marsh and Catherine Farrell), discussing how to use insights from ‘cultural theory’ to think about how to design institutions to help policymakers think and make good decisions. The full paper has been submitted to the series for Policy and Politics called Practical Lessons from Policy Theories.

My reading of Richard’s argument is through the lens of debates on ‘evidence-based policymaking’ and policymaker psychology. Policymakers can only pay attention to a tiny proportion of the world, and a small proportion of their responsibilities. They combine ‘rational’ and ‘irrational’ informational shortcuts to act quickly and make ‘good enough’ decisions despite high uncertainty. You can choose to criticize their reliance on ‘cognitive frailties’ (and perhaps design institutions to limit their autonomous powers) or praise their remarkable ability to develop ‘fast and frugal’ heuristics (and perhaps work with them to refine such techniques). I think Richard takes a relatively sympathetic approach to describing ‘thinking, fast and slow’:

  1. His focus on an ‘internal compass’ describes aspects of fast thinking (using gut or instinct, emotion, habit, familiarity) but without necessarily equating a compass with negative cognitive ‘biases’ that get in the way of ‘rationality’.
  2. Instead, an internal compass can be remarkably useful, particularly if combined with a ‘map’ of the ‘policymaking terrain’. Terrain can describe the organisations, rules, and other sources of sources of direction and learning in a policymaking system.
  3. Both compass and map are necessary; they reinforce the value of each other.
  4. However, perhaps unlike a literal map, we cannot simply design one-size-fits-all advice for policymakers. We need to speak with them in some depth, to help them work out what they think the policy problem is and probe how they would like to solve it.
  5. In that context, the role of policy analysis is to help policymakers think about and ask the right questions as it is to provide tailor-made answers.

It is a paradox that in a world where there are often more questions than answers, policymakers more often seek to establish and then narrow the range of possible answers than to establish and then narrow the range of possible questions. There are different explanations for this:

  • One is that policymakers occupy a ‘rational’, ‘technical’ space, in which everything from real-time data to scientific evidence can be balanced in ‘problem-solving’. This means doing the background work to support authoritative choice between policy alternatives, perhaps via ‘structured interactions’, as a way to bring order to the weight of evidence and expertise.
  • Another is that policymakers occupy a ‘formally structured’, ‘political’ space, in which the contest to have ‘agenda-setting’ power has already been decided. For policy actors, this means learning not to ‘question why’ – accepting the legitimacy, if not the substantive nature, of their political masters’ concerns and (outwardly, at least) directing their attention accordingly.
  • A third explanation, however, is that policymakers occupy a ‘complex’ and ‘uncertain’ space, in which ‘What is a good question?’ is itself a good question. Yet often we lack good ways to ask questions about questions – at least, without encountering accusations of either ‘avoiding the problem’ or ‘re-politicising technical concerns’.

Given that questions are logically prior to a technical search for the ‘best answer’, it seems sensible that the search for the ‘best question’ should start away from the realm of ‘the technical’ (cf. Explanation 1). As a result, two possible options remain in response to ‘What is a good question?’:

  1. That it is a subjectively-normative question that depends on the eyes of the beholder, best aggregated and understood in the realm of ‘the political’ (which returns us to Explanation 2).
  2. That it is an objectively-normative question that depends on ‘social construction’ in policy work, best aggregated and understood in the realm of ‘the institutional’ (which returns us to Explanation 3).

Option 1 is the stuff of basic politics; it will not be explored further here. This leaves the ‘objectively-normative’ Option2 , which is less often explored. This option is ‘normative’ in the sense that it gives space to framing a problem in ways that acknowledges different sets of values and beliefs, that may be socially constructed in different ways. It is ‘objective’ in the sense that it seeks to resolve tensions between these different sets of values and beliefs in without re-opening the kind of explicit competition normally reserved for the realm of politics. Yet in its basis in the realm of institutions, some might ask: how is ‘objective’ analysis even possible?

Step Forward, Cultural Theory (CT)?

There are still, perhaps, a few ‘flat-earth’ policy actors who doubt the importance of institutions. Yet even for those who do not deny their influence, the prospect of ‘objective’ institutional analysis seems remote. By their very nature hard to define and intangible to the eye, institutions can seem esoteric, ephemeral, and resistant to meaningful measurement. However, new developments in Cultural Theory (CT) can help policymakers get a grip. Now well-established in policy circles, CT constitutes institutions along two dimensions into four (and only four) rival ‘cultural biases’ – hierarchy, individualism, egalitarianism and fatalism:

Simmons 1

Importantly, biases combine in different institutional patterns – and the mix matters. Dominant patterns tend to structure policy problems and guide policymakers’ response in different ways. Through exposure and experience, institutional patterns can become internalised in their ‘thought styles’; as an ‘internal compass’ that directs ‘fast-thinking’. No bad thing, perhaps – unless and until this sends them off course. Faulty compass readings arise when narrow thought-styles become ‘cultural blinkers’. As ‘practical wisdom’ may be present in more than one location, navigation risks arise if a course ahead is plotted that blocks out other constructions of the problem.

How would policymakers realise when they have led themselves astray? One way might be ‘slower thinking’ – reflection on their actions to question their constructions and promote dynamic learning. CT provides a parsimonious way of framing such reflection. Simplifying complex criteria into just four cultural categories, skilled ‘reflective policymakers’ are facilitated more quickly to ask ‘good’ questions. However, space for such ‘slow thinking’ is often limited in practical policy work. When this is closed-out by constraints of time and attention, what more has Cultural Theory to offer?

Recent work operationalises CT to both map institutions and chart ‘internal compass’ bearings. Using stakeholder surveys to ‘materialise the intangible’, institutions are mapped by visually overlaying policy actors’ perceptions of how policy problems ‘actually are’ governed, with those of how they ‘should be’ governed:

Simmons 2

Meanwhile, as points of congruence and dissonance emerge in this institutional environment, policymakers internal compass bearings show the likelihood that they might actually see them. Together, these tools up-the-odds of asking ‘good’ questions even further than reflection. Actors learn to navigate both change and the obstacles to change.

But is this not still too slow? This process may indeed seem slow, but intelligent investment in institutional analysis potentially has payoffs that can make it worth the wait. The ‘map’ immediately provides a provocation for more valid and reliable policy practice – definitively directing policy attention, no matter where the compass is pointing. Speed and accuracy increases. Not only this; over time, such purposive action serves to maintain, create and disrupt institutions. As new patterns emerge that subconsciously subvert existing thought-styles, the compass itself is recalibrated. There are fewer faulty readings to direct ‘fast thinking’. Speed and accuracy increases again…

For some, the tools provided by CT may seem blunt; for others, as esoteric and ephemeral as the institutions this theory purports to portray. The recent work reported here certainly requires further refinement to reinforce its validity and reliability. But the effort of doing so may be a small price to pay. The practical potential of CT’s meaningful measurement makes further progress a beguiling prospect.

 

 

1 Comment

Filed under Uncategorized