Putting it all together: dissertation research question and research design (POLU9RM)

Writing a dissertation can be daunting. It is likely the longest piece of work you will plan as an undergraduate (10000 words plus bibliography) but, when you are done, it will not seem long enough.

On the one hand, it is a joyous exploration of research, in which you receive supervision but are in charge. On the other, you don’t want it to go horribly wrong.

It seems unlikely that reading my blog will spark joy, but I can at least give you some tips to avoid unnecessary problems and make your dissertation manageable.

Other advice (such as the reading in your module guide) is available, and I suggest you take it. Indeed, whenever I speak with colleagues about my approach to supervision, it seems relatively conservative and joyless.

On the other hand, why not play it safe with the dissertation then use all the time you’ve saved by seeking joy in a lovely meadow or a summer’s day?

1. Ask the right research question.

Most undergraduate coursework involves answering your lecturer’s rather generic question. Your task is to produce something a bit different, with some of these characteristics:

• You should find it interesting and want to answer it.
• It should be something that you can answer.
• It should be specific enough to help you manage your time well and answer it with the resources you have.

Compare with Halperin and Heath’s (p164) criteria, in which it should be important, ‘researchable’, and it has not been ‘answered definitively’.

(also compare with Dunleavy’s call for full narrative titles)

For example, many projects that I supervise follow roughly the same format: what is policy, how much has it changed, and why?

We can then narrow it down in several ways by choosing a specific issue, political system, time period, and/ or aspect of policy change.

This narrowing can make the difference between:

(a) feeling the need to explain many theories in the literature review, versus

(b) limiting theory selection by focusing on a small number of political system dynamics.

Action point 1

Describe your initial question or theme with your supervisor, and work with them until you are both happy with the question.

2. Write the abstract and the introduction first?

Many people suggest that your first main piece of work should be the literature review, for quite good reasons:

• It allows you to gain enough initial knowledge to help you guide your research
• It allows you to get writing – often a major stumbling block – and then edit later

I suggest that your first piece of work should be the abstract and introduction for these reasons:

• Writing a half page abstract allows you to describe what your project adds up to.
• It really helps you discuss your plans with your supervisor
• Writing the introduction allows you to describe your research design in enough depth to reflect on it is coherence and feasibility.
• All going well, it will be only a small jump from your POLU9RM ‘research project design’ exercise.
• It allows you to make sense of a quite general format for research publications (in many fields): theory, method, results.

Action point 2

Write the question/ title and abstract, share it with your likely supervisor, and talk about how coherent and feasible your plan looks.

3. Identify the relevant theory or literature.

In some cases, the potentially relevant literature is vast if you have, for example:

1. A too-general question about political parties or elections.

• One good solution is to select a subfield like ‘pledge fulfilment’

2. A too-general question about policy change.

• One good solution is to ask if (say) the advocacy coalition framework or multiple streams analysis helps explain your case study.
• Or, I have supervised many good dissertations asking: what can X government learn from Y government about solving Z policy problem?

Action point 3.

Make sure to connect your research question to a well-defined literature (and do a preliminary literature search to see what is out there)

4. Identify your method to gather information.

Halperin and Heath’s chapter 7 goes into some depth about the principles of research design:

• what data collection is appropriate
• what we can deduce from certain data
• how confident you can be about cause/effect in this case (internal validity)
• and cause-and-effect more generally (external validity)
• if someone could do your research and get the same results (reliability)

They also describe the types of design you can likely not do (well, 1 and 2) in a UG dissertation, but can get the data to analyse:

1. Experimental (like an RCT)
2. Cross-sectional and longitudinal
3. Comparative (for which I did a separate post)

Then they describe data gathering strategies that you might be tempted to do (subject to ethical clearance):

• Surveys
• Interviews
• Focus groups
• Ethnographic
• Discourse analysis

In the lecture, I will put on my dour face and warn you against most of these methods, for reasons such as:

• Doing a proper survey takes a lot of time and resources, someone has likely already done a better one, and it would be a shame not to find it
• You can often find things in the public record without interviewing someone (and maybe they will only repeat what is out there)
• The ethical clearance will be a major issue with ethnographic (and other) methods

I won’t try to put you off entirely. Rather, I will encourage you to ask yourself:

• Why are you choosing this method?

1. Does it relate clearly to your research question?
2. Or, have you begun with the most interesting sounding method?
3. Or, do you have some sort of connection that gets you access, which seems a shame not to use?

• Are you prepared to do a literature review on your chosen method?
• What do you realistically expect to get from your method?
• What will you do if it goes wrong?

Action point 4

Discuss your choice of data collection with your supervisor.

5. Think about how you will analyse and interpret the results.

This part tends to make the difference between a very good or an excellent dissertation.

• Raul Pacheco-Vega gives many useful examples in Distinguishing between description and analysis in academic writing

Put most simply, simple description involves summarising things. Analysis is about telling the reader what the results mean. For example, you might:

• Evaluate the size of the results according to your expectations. Does a survey result seem unusual?
• Describe how much one should rely on the results. Does the result seem important after taking into account a margin of error?
• Describe the wider context. Does the result mark a change over time, or seem different from another country?
• Relate a case study result to your literature review. Is your case unusual, or as expected?

Action point 5

Clarify the difference between summary and analysis

6. Be clear about the conclusion.

Don’t just to the dissertation equivalent of saying ‘cheerio’ (or, my favourite thing, leaving without saying cheerio).

The conclusion differs from:

• The introduction, because you should use it to summarise your question and approach (perhaps quite briefly) and relate it in some depth to the results.
• The analysis of results, because you relate the results much more clearly to your overall project.

Don’t think of it as saying: ‘as I have said before …’

Think of it as saying: ‘here is what it all adds up to …’

7. The end.

Remember to add your bibliography and ask yourself if you need an appendix for your data (which does not count towards the word count).

PS some of my supervisees write policy analysis reports, which differ somewhat from regular dissertations. If you are keen, please see me and/ or read more here.

 

 

 

 

 

 

A Stern review for everyone?

The Stern review ‘was commissioned by the government to carry out the review of the Research Excellence Framework (REF) to ensure future university research funding is allocated more efficiently, offers greater rewards for excellent research and reduces the administrative burden on institutions’. In this post, I explain why no single policy can solve these problems uniformly: they affect scholars of different seniority, and different disciplines, very differently. The punchline is at the end.

My initial impression of the Stern review is that it has gone to great lengths to address the unintended consequences of the previous Research Excellence Framework. One of its key aims now is to try to anticipate the potential unintended consequences of its reduction of other unintended consequences! This is remarkably common in policymaking, perhaps summed up by Aaron Wildavsky’s phrase ‘policy as its own cause’: we enter a never-ending process of causing ripple effects when trying to fix previous problems.

The example of non-portability (#sternreview portable)

Take the example of one of the biggest problems:

Problem: there was a large incentive for Universities to ‘game’ the REF towards the end of the cycle: paying for 20% of the time of big name academics, or appointing them at huge salaries, to gain access to their 4 best publications. A policy of rewarding research excellence became a policy to (a) reward big transfers, undermining the efforts of other Universities and reducing their incentive to invest for the long term, and (b) boost the salaries of senior scholars (many of whom were already on 6-figure salaries), often to ridiculous levels.

Solution: non-portability. The idea is that you can move but your former employer gets to use the texts that you published while in your last job. So, there may now be less incentive to buy up the big names in the run up to the next REF.

Unintended consequence: uncertainty for early career researchers (ECRs) or scholars without permanent (open-ended) contracts. Many ECRs have expressed the concern that their incentives may suddenly change, from generating a portfolio of up to 4 excellent publications to secure a permanent post, to perhaps holding back publications and promising their delivery when in post. This could be addressed by the present review (ECRs could be included in the REF but be under no obligation to submit any publications), or perhaps by exempting ECRs since the policy is aimed at senior scholars (but the exemption would also have unintended consequences!).

Interpreting the problem through the lens of precarious positions

We will now enter a phase of debate driven by uncertainty and anxiety about the end result, and a lot of the discussion will be emotionally charged because many people will have spent maybe 8 years in education (and several years in low paid posts after it) and not know what to do next. It is relatively easy for people like me to say that the proposed new system is better, and for senior scholars to look on Stern as a big improvement, because people like me will be rewarded in either system (I will leave it to you to decide what I mean by ‘people like me’). It is more difficult for ECRs who are genuinely uncertain about their prospects.

The punchline: how does this look through the eyes of scholars in different disciplines?

The disciplinary lens is the factor that can often be most important but least discussed, for two key reasons:

1. The general differences. Scholars operate in different ways. The ‘STEM’ (science, technology, engineering and mathematics) subjects are often described in these terms: you have large teams headed by a senior scholar; there is a hierarchy; you all work on the same research question; you publish many short articles as a team (with senior authors listed at the very start and end of the list of authors); you are increasingly driven by key metrics, such as personal citations and the ‘impact factor’ of journals . The humanities is often described like this: you are a lone scholar; you work on your own research question; you publish single-author work, and the big status symbol is the research monograph (book); these journal or other metrics do not work as well in your discipline. I am verging on caricature here (many ‘STEM’ scholars will work alone, and some humanities or social science scholars will operate laboratory-style teams), but you get the idea.

These differences feed into other practices: only in some subjects can it make sense for a University to ‘poach’ a whole team or unit; only in some subjects do ECRs need to develop their own portfolio of work; in some subjects, a PhD student or ECR effectively works for a senior scholar, while in others the PhD student has a supervisor but can set their own research agenda; in some subjects, it is automatic to include the senior scholar in an article you wrote, in others it would be seen as exploitation of the work of a PhD student or ECR. In some subjects, a CV with your name on team publications is the norm, while in others it would look like you do not have your own ideas.

2. The REF reinforces these differences. You often find the impression that research exercises and metrics are there for the STEM subjects (or ‘hard sciences’) and not the humanities or social sciences: a process or review for Universities does not take into account the differences in incentives and practices across the disciplines, and some disciplines might lose out.

So, if you follow this debate on twitter, I recommend that you look at the bios of each participant to check their level of seniority and discipline because the Stern review is for all but it will affect us all in very different ways.

See also: James Wilsdon ‘The road to REF 2021: why I welcome Lord Stern’s blueprint for research assessment‘

 

Policy Concepts in 1000 Words: Feminism, Postcolonialism, and Critical Policy Studies

See also three more recent posts:

1. Policy in 500 Words: Power and Knowledge
2. Policy in 500 Words: Feminist Institutionalism
3. Policy Analysis in 750 words: Linda Tuhiwai Smith (2012) Decolonizing Methodologies

In this post, let’s begin with a transition from two others: combining theories, and critical policy studies/ the NPF. Both posts raise the same basic question: what is science? This question leads to a series of concerns about the criteria we use to determine which theories are most worthy of our investment, and the extent to which social scientific criteria should emulate those in natural science.

One set of criteria, which you can find in the ‘policy shootout!’, relates to the methods and principles we might associate with some branches of natural science (and use, for example, to support astronomy but not astrology):

• A theory’s methods should be explained so that they can be replicated by others.
• Its concepts should be clearly defined, logically consistent, and give rise to empirically falsifiable hypotheses.
• Its propositions should be as general as possible.
• It should set out clearly what the causal processes are.
• It should be subject to empirical testing and revision

If we were to provide a caricature of this approach, we might associate it with other explicit or implicit principles, such as:

1. The world exists independently of our knowledge of it, and our role is to develop theories to help us understand its properties (for example, discover its general laws).
2. These principles help us produce objective science: if the methods and results can be replicated, they do not depend on individual scientists.

In other words, the caricature is of a man in a white lab coat gathering knowledge of his object of study while remaining completely separate from it. Such principles are generally difficult to maintain, and relatively tricky in the study of the social world (and it seems increasingly common for one part of PhD training to relate to reflexivity – see what is our role in social scientific research)? However, critical challenges go far beyond this point about false objectivity.

The challenge to objective science: 1. the role of emancipatory research

One aspect of feminist and postcolonial social science is to go beyond the simple rejection of the idea of objective social science: a further key (or perhaps primary) aim is to generate research with emancipatory elements. This may involve producing research questions with explicit normative elements and combining research with recommendations on social and political change.

The challenge to objective science: 2. a rejection of the dominant scientific method?

A second aspect is the challenge to the idea that one dominant conception of scientific method is correct. Instead, one might describe the scientific rules developed by one social group to the exclusion of others. This may involve historical analysis to identify the establishment of an elite white male dominance of science in the ‘West’, and the ‘Western’ dominance of science across the world.

To such scientists, a challenge to these criteria seems ridiculous: why reject the scientific principles that help us produce objective science and major social and technological advances? To their challengers, this response may reflect a desire to protect the rules associated with elite privilege, and to maintain dominance over the language we use to establish which social groups should be respected as the generators of knowledge (the recipients of prestige and funding, and perhaps the actors most influential in policy).

The challenge to objective science: 3. the democratisation of knowledge production

A third is the challenge to the idea that only well-trained scientists can produce valuable knowledge. This may involve valuing the knowledge of lived experience as a provider of new perspectives (particularly when people are in the unusual position to understand and compare their perspective and those of others). It also involves the development of new research methods and principles, combined with a political challenge to the dominance of a small number of scientific methods (for example, see rejections of the hierarchy of knowledge at which the systematic review of randomised control trials is often at the top).

Revisiting the live debate on the NPF and critical/ interpretive studies

This seems like good context for some of the debate on the NPF (see this special issue). One part of the debate may be about fundamentally different ideas about how we do research: do we adhere to specific scientific principles, or reject them in favour of a focus on, for example, generating meaning from statements and actions in particular contexts?

Another part may reflect wider political views on what these scientific principles represent (an elitist and exclusionary research agenda, whose rules reinforce existing privileges) and the role of alternative methods, in which critical policy studies may play an important part. In other words, we may be witnessing such a heated debate because critical theorists see the NPF as symbolic of attempts by some scholars to (a) reassert a politically damaging approach to academic research and (b) treat other forms of research as unscientific.

Where do we go from here?

If so, we have raised the stakes considerably. When I wrote previously about the problems of combining the insights and knowledge from different theories, it often related to the practical problems of research resources and potential for conceptual misunderstanding. Now, we face a more overt political dimension to social research and some fundamentally different understandings of its role by different social groups.

Can these understandings be reconciled, or will they remain ‘incommensurable’, in which we cannot generate agreement on the language to use to communicate research, and therefore the principles on which to compare the relative merits of approaches? I don’t know.

Initial further reading

Paying attention to this intellectual and political challenge provides a good way ‘in’ to reading that may seem relatively unfamiliar, at least for students with (a) some grounding in the policy theories I describe, and (b) looking to expand their horizons.

Possibly the closest link to our focus is when:

• Bacchi (1999) describes ‘the politics of policy studies’ by applying insights from policy studies (including power, framing, and social construction) to the study of policy.
• See also Policy Analysis in 750 words: Carol Bacchi’s (2009) WPR Approach

First, we know that policy problems do not receive policymaker attention because they are objectively important. Instead, actors compete to define issues and maximise attention to that definition. Second, we do the same when we analyse public policy: we decide which issues are worthy of study and how to define problems. Bacchi (1999) argues that the ‘conventional’ policy theorists (including Simon, Bardach, Lindblom, Wildavsky) try to ‘stand back from the policy process’ to give advice from afar, while others (including Fischer, Drysek, Majone) “recognise the analysts’ necessarily normative involvement in advice giving” (1999: 200). Combining both points, Bacchi argues that feminists should engage in both processes – to influence how policymakers and analysts define issues – to, for example, challenge ‘constructions of problems which work to disempower women’ (1999: 204). This is a topic (how should academics engage in the policy process?) which I follow up in a study of EBPM.

For a wider discussion of feminist studies and methods, see:

• Fonow and Cook’s ‘pragmatic’ discussion about how to do feminist public policy research based on key principles:

‘Our original analysis of feminist approaches to social science research in women’s studies revealed some commonalities, which we articulated as guiding principles of feminist methodology: first, the necessity of continuously and reflexively attending to the significance of gender and gender asymmetry as a basic feature of all social life, including the conduct of research; second, the centrality of consciousness-raising or debunking as a specific methodological tool and as a general orientation or way of seeing; third, challenging the norm of objectivity that assumes that the subject and object of research can be separated from each other and that personal and/or grounded experiences are unscientific; fourth, concern for the ethical implications of feminist research and recognition of the exploitation of women as objects of knowledge; and finally, emphasis on the empowerment of women and transformation of patriarchal social institutions through research and research results’ (Fonow and Cook, 2005: 2213).

• Lovenduski on early attempts to reinterpret political science through the lens of feminist theory/ research.

Note the links between our analysis of power/ideas and institutions as the norms and rules (informal and formal, written and unwritten) which help produce regular patterns of behaviour which benefit some and exclude others (and posts on bounded rationality, EBPM and complexity: people use simple rules to turn a complex world into manageable strategies, but to whose benefit?).

With feminist research comes a shift of focus from sex (as a primarily biological definition) and gender (as a definition based on norms and roles performed by individuals), and therefore the (ideal-type) ‘codes of masculinity and femininity’ which underpin political action and even help define which aspects of public policy are public or private. This kind of research links to box 3.3 in Understanding Public Policy (note that it relates to my discussion of Schattschneider and the privatisation/ socialisation of conflict, which he related primarily to ‘big business’).

• A good way into the study of feminist activism, and to explore the fact that ‘feminism’ sums up a wide range of activity and approaches (just like, say, ‘new institutionalism’) is to read Elizabeth Evans’ The Politics of Third Wave Feminism. You might also read Evans’ discussion of method (comparing the UK and US) with my post on comparative public policy research.
• See also Stevi Jackson on Women’s Studies, Gender Studies and Feminism
• Of course, another good way ‘in’ to a discussion is to begin with something really well written, thoughtful, and informative. For example, see bell hooks’ Feminism is for Everybody

Then see two articles which continue our theme of combining theories and insights carefully:

• Kenny’s discussion of feminist institutionalism, which seems like one of many variants of new institutionalism (e.g. this phrase could be found in many discussions of new institutionalism: ‘seemingly neutral institutional processes and practices are in fact embedded in hidden norms and values, privileging certain groups over others’ – Kenny, 2007: 95) but may involve ‘questioning the very foundations and assumptions of mainstream institutional theory’. Kenny argues that few studies of new institutionalism draw on feminist research (‘there has been little dialogue between the two fields’) and, if they were to do so, may produce very different analyses of power and ‘the political’. This point reinforces the problems I describe in combining theories when we ignore the different meanings that people attach to apparently identical concepts.
• Mackay and Meier’s concern (quoted here) that new institutionalism could be ‘an enabling framework – or an intellectual strait-jacket” for feminist scholarship’. Kenny and Mackay identify similar issues about ‘epistemological incompatibilities’ when we combine approaches such as feminist research and rational choice institutionalism.
• These approaches receive more coverage in the 2nd edition of Understanding Public Policy, and are summarised in Policy in 500 Words: Feminist Institutionalism

Here is one example of a link between ‘postcolonial’ studies and public policy:

• Munshi and Kurian’s identify the use of ‘postcolonial filters’ to reinterpret the framing of corporate social responsibility, describing ‘the old colonial strategy of reputation management among elite publics at the expense of marginalized publics’ which reflects a ‘largely Western, top-down way of doing or managing things’. In this case, we are talking about frames as structures or dominant ways to understand the world. Actors exercise power to reinforce a particular way of thinking which benefits some at the expense of others. Munshi and Kurian describe a ‘dominant, largely Western, model of economic growth and development’ which corporations seek to protect with reference to, for example, the ‘greenwashing’ of their activities to divert attention from the extent to which ‘indigenous peoples and poorer communities in a number of developing countries “are generally the victims of environmental degradation mostly caused by resource extractive operations of MNCs in the name of global development”’ (see p516).

It is also worth noting that I have, in some ways, lumped feminism and postcolonialism together when they are separate fields with different (albeit often overlapping and often complementary) traditions. See for example Emejulu’s Beyond Feminism’s White Gaze.

For more discussion, please see

Policy Analysis in 750 words: Linda Tuhiwai Smith (2012) Decolonizing Methodologies

Master in Public Policy (MPP) at the University of Stirling

Here is the extra marketing information to go with this survey. We’ll have a glossy leaflet out there in no time:

A new degree which applies the rigour of academic research to real world policy problems

Why Should You Study for an MPP?

The MPP is an advanced qualification in research and policy analysis. Studying for an MPP allows you to develop the conceptual, analytical and practical skills required to flourish in the policymaking world. It prepares you for a career in the public sector or in sectors that make a contribution to the development or delivery of public policy (such as non-profit or professional bodies).  You can also use it as a springboard for further postgraduate research. The MPP combines core modules in policy and policymaking with a suite of modules in social research and policy-relevant disciplines. If you want to use the degree to focus on research (for example, to pursue a PhD) you can take five modules in applied social research. If you want to pursue an interest in other policy-relevant disciplines, you can combine a focus on policy and research with module options in areas such as law, economics, behavioural science, gender studies, social marketing, energy, environmental and international politics. The programme is designed to meet your specific requirements. The norm in the core modules is small group teaching in weekly seminars – to help produce a group identity and a collegiate approach to your studies.  You complete the course by completing eight taught modules, then producing a dissertation which applies intellectual rigour to a real world policy problem and speaks to a policymaker audience.

Applied Research Opportunities

The MPP gives you the opportunity to apply your research to real world problems. We have excellent links with a range of organisations in the public, third and private sectors. When you begin your course, we will discuss how you want to make use of them. If you seek as many practitioner links as possible, we will explore how to apply your studies and coursework to a range of problems identified by those organisations – and arrange, in negotiations with organisations, how best to use your developing skills. You may also be taking the MPP to pursue a more ‘traditional’ academic path, with the knowledge that academic ‘impact’ is a key part of a postgraduate degree. We will discuss how best to balance the theoretical, empirical and practical aspects of your study.

Programme Overview

The programme (of 180 credits) combines core modules on policy theory and practice with a suite of modules in social research and policy-relevant disciplines. The norm is to maintain a meaningful level of contact between students engaged in the MPP and a small cohort of staff (teaching core and common ASR modules), but with the flexibility to take your own path. Core modules are delivered on the same day and there is a high degree of flexibility over optional modules to allow both full-time and part-time students to work around other commitments.

Core modules (45 credits) focus on multi-level policymaking, identifying the responsibilities and policies of local, devolved, national and international decision-makers. We identify the concepts, models and theories used to study policy and policymaking. We compare theories in political science with a range of policy-relevant disciplines (including economics, communication, psychology, management and social marketing). We combine theory and practice by inviting a range of policy actors to give guest seminars as an integral part of the core modules.

Research modules. You can choose up to five 15-credit modules in applied social research (ASR), including qualitative and quantitative analysis, research design and the philosophy of science. If appropriate, you can also choose to replace some ASR modules with research methods modules in your chosen subject – such as the MSc Gender Studies module ‘Feminist Research’ which is a prerequisite for its Research Placement module.

Policy relevant modules. You can choose two 15-credit modules in law, economics, behavioural science, social marketing, gender studies, energy, environmental or international politics.

Dissertation. You complete the course by producing a 60-credit dissertation (around 10000 words) which applies intellectual rigour to a real world policy problem. You will have the option to pursue a placement with a relevant organisation to allow you to tailor your research to a policymaker or policy influencer audience.

Staff: The Director for the MPP is Paul Cairney, Professor of Politics and Public Policy in the Division of History and Politics, School of Arts and Humanities. Paul will deliver the majority of the core module content and oversee the completion of your dissertation (and, if appropriate, as the first supervisor). He will work closely with Richard Simmons, the director of the School of Arts and Social Science’s applied social research programme.

Five reasons why you should choose the MPP at Stirling

1.You will be taught by experienced and committed staff, teaching in a field they are passionate about. All contributing staff are engaged in research at the forefront of their disciplines, including Professor Cairney, who is currently funded by the ESRC to research the Scottish Government’s policymaking capacity.

2. You will develop a range of research skills that enhance further study and employability.

3. You will engage with debates from a wide variety of different disciplines.

4. You will have the opportunity to apply your knowledge and skills in real world settings.

5. You will enjoy studying on one of the most beautiful campuses in Europe.

Fees and Funding

Details of tuition fees can be found at: http://www.stir.ac.uk/postgraduate/financial-information/tuition-fees/. A variety of scholarships and bursaries may be available in any given year, including scholarships in the School of Arts and Humanities. You can find out more about possible sources of funding here: http://www.stir.ac.uk/postgraduate/financial-information/scholarships/

Entrance Requirements

Normally an upper second class Honours degree, or equivalent qualification from a university recognised by the University of Stirling. Degrees can be in any relevant discipline. If English is not your first language, you must provide evidence of proficiency such as a minimum IELTS score of 7.0 with a minimum of 6.5 in any individual test.

To find out more about this programme please contact:

Professor Paul Cairney (Director) p.a.cairney@stir.ac.uk