Putting it all together: dissertation research question and research design (POLU9RM)

Writing a dissertation can be daunting. It is likely the longest piece of work you will plan as an undergraduate (10000 words plus bibliography) but, when you are done, it will not seem long enough.

On the one hand, it is a joyous exploration of research, in which you receive supervision but are in charge. On the other, you don’t want it to go horribly wrong.

It seems unlikely that reading my blog will spark joy, but I can at least give you some tips to avoid unnecessary problems and make your dissertation manageable.

Other advice (such as the reading in your module guide) is available, and I suggest you take it. Indeed, whenever I speak with colleagues about my approach to supervision, it seems relatively conservative and joyless.

On the other hand, why not play it safe with the dissertation then use all the time you’ve saved by seeking joy in a lovely meadow or a summer’s day?

1. Ask the right research question.

Most undergraduate coursework involves answering your lecturer’s rather generic question. Your task is to produce something a bit different, with some of these characteristics:

• You should find it interesting and want to answer it.
• It should be something that you can answer.
• It should be specific enough to help you manage your time well and answer it with the resources you have.

Compare with Halperin and Heath’s (p164) criteria, in which it should be important, ‘researchable’, and it has not been ‘answered definitively’.

(also compare with Dunleavy’s call for full narrative titles)

For example, many projects that I supervise follow roughly the same format: what is policy, how much has it changed, and why?

We can then narrow it down in several ways by choosing a specific issue, political system, time period, and/ or aspect of policy change.

This narrowing can make the difference between:

(a) feeling the need to explain many theories in the literature review, versus

(b) limiting theory selection by focusing on a small number of political system dynamics.

Action point 1

Describe your initial question or theme with your supervisor, and work with them until you are both happy with the question.

2. Write the abstract and the introduction first?

Many people suggest that your first main piece of work should be the literature review, for quite good reasons:

• It allows you to gain enough initial knowledge to help you guide your research
• It allows you to get writing – often a major stumbling block – and then edit later

I suggest that your first piece of work should be the abstract and introduction for these reasons:

• Writing a half page abstract allows you to describe what your project adds up to.
• It really helps you discuss your plans with your supervisor
• Writing the introduction allows you to describe your research design in enough depth to reflect on it is coherence and feasibility.
• All going well, it will be only a small jump from your POLU9RM ‘research project design’ exercise.
• It allows you to make sense of a quite general format for research publications (in many fields): theory, method, results.

Action point 2

Write the question/ title and abstract, share it with your likely supervisor, and talk about how coherent and feasible your plan looks.

3. Identify the relevant theory or literature.

In some cases, the potentially relevant literature is vast if you have, for example:

1. A too-general question about political parties or elections.

• One good solution is to select a subfield like ‘pledge fulfilment’

2. A too-general question about policy change.

• One good solution is to ask if (say) the advocacy coalition framework or multiple streams analysis helps explain your case study.
• Or, I have supervised many good dissertations asking: what can X government learn from Y government about solving Z policy problem?

Action point 3.

Make sure to connect your research question to a well-defined literature (and do a preliminary literature search to see what is out there)

4. Identify your method to gather information.

Halperin and Heath’s chapter 7 goes into some depth about the principles of research design:

• what data collection is appropriate
• what we can deduce from certain data
• how confident you can be about cause/effect in this case (internal validity)
• and cause-and-effect more generally (external validity)
• if someone could do your research and get the same results (reliability)

They also describe the types of design you can likely not do (well, 1 and 2) in a UG dissertation, but can get the data to analyse:

1. Experimental (like an RCT)
2. Cross-sectional and longitudinal
3. Comparative (for which I did a separate post)

Then they describe data gathering strategies that you might be tempted to do (subject to ethical clearance):

• Surveys
• Interviews
• Focus groups
• Ethnographic
• Discourse analysis

In the lecture, I will put on my dour face and warn you against most of these methods, for reasons such as:

• Doing a proper survey takes a lot of time and resources, someone has likely already done a better one, and it would be a shame not to find it
• You can often find things in the public record without interviewing someone (and maybe they will only repeat what is out there)
• The ethical clearance will be a major issue with ethnographic (and other) methods

I won’t try to put you off entirely. Rather, I will encourage you to ask yourself:

• Why are you choosing this method?

1. Does it relate clearly to your research question?
2. Or, have you begun with the most interesting sounding method?
3. Or, do you have some sort of connection that gets you access, which seems a shame not to use?

• Are you prepared to do a literature review on your chosen method?
• What do you realistically expect to get from your method?
• What will you do if it goes wrong?

Action point 4

Discuss your choice of data collection with your supervisor.

5. Think about how you will analyse and interpret the results.

This part tends to make the difference between a very good or an excellent dissertation.

• Raul Pacheco-Vega gives many useful examples in Distinguishing between description and analysis in academic writing

Put most simply, simple description involves summarising things. Analysis is about telling the reader what the results mean. For example, you might:

• Evaluate the size of the results according to your expectations. Does a survey result seem unusual?
• Describe how much one should rely on the results. Does the result seem important after taking into account a margin of error?
• Describe the wider context. Does the result mark a change over time, or seem different from another country?
• Relate a case study result to your literature review. Is your case unusual, or as expected?

Action point 5

Clarify the difference between summary and analysis

6. Be clear about the conclusion.

Don’t just to the dissertation equivalent of saying ‘cheerio’ (or, my favourite thing, leaving without saying cheerio).

The conclusion differs from:

• The introduction, because you should use it to summarise your question and approach (perhaps quite briefly) and relate it in some depth to the results.
• The analysis of results, because you relate the results much more clearly to your overall project.

Don’t think of it as saying: ‘as I have said before …’

Think of it as saying: ‘here is what it all adds up to …’

7. The end.

Remember to add your bibliography and ask yourself if you need an appendix for your data (which does not count towards the word count).

PS some of my supervisees write policy analysis reports, which differ somewhat from regular dissertations. If you are keen, please see me and/ or read more here.

 

 

 

 

 

 

Policy in 500 Words: The advocacy coalition framework

Here is the ACF story.

People engage in politics to turn their beliefs into policy. They form advocacy coalitions with people who share their beliefs, and compete with other coalitions. The action takes place within a subsystem devoted to a policy issue, and a wider policymaking process that provides constraints and opportunities to coalitions.

The policy process contains multiple actors and levels of government. It displays a mixture of intensely politicized disputes and routine activity. There is much uncertainty about the nature and severity of policy problems. The full effects of policy may be unclear for over a decade. The ACF sums it up in the following diagram:

Policy actors use their beliefs to understand, and seek influence in, this world. Beliefs about how to interpret the cause of and solution to policy problems, and the role of government in solving them, act as a glue to bind actors together within coalitions.

If the policy issue is technical and humdrum, there may be room for routine cooperation. If the issue is highly charged, then people romanticise their own cause and demonise their opponents.

The outcome is often long-term policymaking stability and policy continuity because the ‘core’ beliefs of coalitions are unlikely to shift and one coalition may dominate the subsystem for long periods.

There are two main sources of change.

1. Coalitions engage in policy learning to remain competitive and adapt to new information about policy. This process often produces minor change because coalitions learn on their own terms. They learn how to retain their coalition’s strategic advantage and use the information they deem most relevant.
2. ‘Shocks’ affect the positions of coalitions within subsystems. Shocks are the combination of events and coalition responses. External shocks are prompted by events including the election of a new government with different ideas, or the effect of socioeconomic change. Internal shocks are prompted by policy failure. Both may prompt major change as members of one coalition question their beliefs in the light of new evidence. Or, another coalition may adapt more readily to its new policy environment and exploit events to gain competitive advantage.

The ACF began as the study of US policymaking, focusing largely on environmental issues. It has changed markedly to reflect the widening of ACF scholarship to new policy areas, political systems, and methods.

For example, the flow diagram’s reference to the political system’s long term coalition opportunity structures is largely the response to insights from comparative international studies:

• A focus on the ‘degree of consensus needed for major policy change’ reflects applications in Europe that highlighted the important of proportional electoral systems
• A focus on the ‘openness of the political system’ partly reflects applications to countries without free and fair elections, and/ or systems that do not allow people to come together easily as coalitions to promote policy change.

As such, like all theories in this series, the ACF discusses elements that it would treat as (a) universally applicable, such as the use of beliefs to address bounded rationality, and (b) context-specific, such as the motive and opportunity of specific people to organize collectively to translate their beliefs into policy.

See also:

The 500 and 1000 Words series

Why Advocacy Coalitions Matter and How to Think about Them

Three lessons from a comparison of fracking policy in the UK and Switzerland

Bonus material

Scottish Independence and the Devil Shift

Image source: Weible, Heikkila, Ingold, and Fischer (2016: 6)

https://twitter.com/amwellstead/status/1095011852915011586

 

 

 

Policy in 500 Words: Ecology of Games

The ‘Ecology of Games Framework’ (EG) combines insights from many approaches to analyze ‘institutional complexity’ and ‘complex institutional systems’.

The focus is on actors learning how to secure ‘mutually beneficial outcomes’, cooperating to produce and deliver agreed solutions, and bargaining within a system over which no actor has control. Therefore, it is worth reading the posts on game theory, the IAD, and SES first (especially if, like me, you associated ‘game’ with tig, then Monopoly, then The Wire).

Insights from three key approaches

EG connects Norton’s ‘ecology of games’, the IAD, and insights from complexity theory to reinforce the idea that institutional arrangements are not simple and orderly.

In simple games, we need only analyse the interaction between a small number of actors with reference to one set of self-contained rules providing clear sanctions or payoffs. In real world policymaking, many different games take place at the same time in different venues.

Some policy games may be contained within a geographical area – such as California – but there are no self-contained collective action problems:

• Examples such as ‘biodiversity’, ‘ecology’ or ‘environmental’ policies command a collection of interdependent policies relating to issues like local planning, protected species, water management, air pollution, transport, energy use, and contributors to such policies or policy problems in other areas of government (such as public services).
• Each contributor to policy may come from different institutions associated with many policymaking venues spread across many levels and types of government.

Consequently, many games interact with each other. The same actor might participate in multiple games subject to different rules. Further, each game produces ‘externalities’ for the others; the ‘payoffs’ to each game are connected and complicated.

A focus on ‘complex adaptive systems’ suggests that central governments do not have the resources to control – or understand fully – interaction at this frequency and scale. Rather, policymaking influences are:

• Internal to the game, when actors (a) follow and shape the rules of each institution, and (b) learn through trial and error.
• External to the game, when physical resources change, or central levels of government change the resources of local actors.

Insights from the wider literature

The EG brings in wider insights – from theories in the 500 and 1000 Words series – to analyse this process. Examples include:

• Actors deal with bounded rationality primarily with reference to their ‘social tribal instincts’.
• They form coalitions with their allies to get what they want from different games
• They engage in ‘policy learning’ to update their policy knowledge and enhance their policymaking position.

Consequently, we have come a long way from simple assumptions about human behaviour outlined in our first post in this series.

As with the IAD, the EG emphasis is on (a) finding solutions to complex (largely environmental) policy problems, with reference to (b) initiatives consistent with self-organising systems such as ‘collaborative governance’. Like most posts in this series, it rejects a naïve attachment to a single powerful central government. Policymaking is multi-centric, and solutions to complex problems will emerge in that context.

See also:

Policy Concepts in 1000 Words: the Institutional Analysis and Development Framework (IAD) and Governing the Commons

Policy Concepts in 1000 Words: it’s time for some game theory

Policy in 500 Words: the Social-Ecological Systems Framework

Policy Concepts in 1000 Words: Rational Choice and the IAD (the older post for the 1^st edition)

Policy Concepts in 1000 Words: Multi-centric Policymaking

How to Navigate Complex Policy Designs

How can governments better collaborate to address complex problems?

Yay! I didn't realize that @CairneyPaul has summarized our work on the "ecology of games" in 500 words! Who needs another 7000 word article at this point? All you need is right here! https://t.co/Y5f9O4Gaub pic.twitter.com/CgELtL8CG7

— Mark Lubell (@EnvPolicyCenter) February 20, 2019

 

Policy in 500 Words: the Social-Ecological Systems Framework

Think of the SES framework as the combination of:

• the IAD approach to analyzing ‘the commons’, and
• ecological sciences approaches to ‘complex social-ecological systems’

The result is a framework that resembles CPR studies in key respects. Ostrom’s 2009 article in Science provides a visual emphasis on the interactions between ‘first-level’ concepts including users, their governance system, resource system (such as a protected park) and resource units (such as its trees):

It also raises similar questions, such as ‘When will the users of a resource invest time and energy to avert “a tragedy of the commons”’?

It answers them with reference to ‘second level’ concepts describing factors that encourage users to (a) value long term sustainability and (b) self-organize to secure this outcome. This table summarizes many of them:

Note that Ostrom describes their effect as indicative because, ‘As in most complex systems, the variables interact in a nonlinear fashion …Simple blueprint policies do not work’.

As a result, we have a super-complicated framework to help us understand an even more super-complicated world. For some, the SES framework serves to ‘diagnose’ the sustainability of social-ecological systems and explore the prospect of more effective self-organisation to manage resources. However, as with the IAD, effective use of the framework itself requires a fair amount of immersion in the language of analysis.

See also:

Policy Concepts in 1000 Words: the Institutional Analysis and Development Framework (IAD) and Governing the Commons

Policy in 500 Words: Ecology of Games

Policy Concepts in 1000 Words: it’s time for some game theory

Policy Concepts in 1000 Words: Rational Choice and the IAD (the older post for the 1^st edition)

Policy Concepts in 100 Words: Multi-centric Policymaking

How to Navigate Complex Policy Designs

How can governments better collaborate to address complex problems?

 

Policy Concepts in 1000 Words: the Institutional Analysis and Development Framework (IAD) and Governing the Commons

The IAD provides a language, and way of thinking, about the ways in which different institutions foster collective action. The language is so complicated that I have cheated by summarising key terms in this box (and describing polycentric governance in a different post) to stay within the 1000 words limit:

Governing the Commons

For me, the best way to understand the IAD is through the lens of Governing the Commons (and the research agenda it inspired), which explains how to rethink ‘tragedies of the commons’ and encourage better management of common pool resources (CPRs).

Ostrom rejects the uncritical use of rational choice games to conclude – too quickly – that disastrous collective action problems are inevitable unless we ‘privatize’ commons or secure major government intervention (which is tricky anyway when global problems require international cooperation). The tragedy of the commons presents a too-bleak view of humanity, in which it would be surprising to find cooperation even when the fate of the world is in human hands.

Alternatively, what if there is evidence that people often work collectively and effectively without major coercion? People are social beings who share information, build trust by becoming known as reliable and predictable, and come together to produce, monitor and enforce rules for the group’s benefit. They produce agreements with each other that could be enforced if necessary.

The IAD helps us analyse these cooperative arrangements. Ostrom describes 8 ‘design principles’ of enduring and effective CPR management shared by many real world examples:

1. CPRs have clear boundaries. Users know what they are managing, and can identify legitimate users.
2. The rules suit local conditions. Users know what they (a) are expected to contribute to management and (b) receive from CPRs.
3. The actors affected by the rules help shape them (at low cost).
4. CPR monitors are users or accountable to users. They monitor (a) the conduct of users and (b) the state of the CPR. The costs of mutual monitoring are low, and their consequences felt quickly.
5. The penalties for rule-breaking are low if the choice is a one-off and understandable under the circumstances (to avoid alienating the user). The penalties are high if the choice is part of a pattern which makes other users feel like ‘suckers’, or if rule-breaking would be catastrophic.
6. Conflict resolution is frequent, rapid and low cost.
7. Users have the right to self-organise without too much outside interference.
8. Many projects are connected geographically and at different scales – local, regional, national – in ways that do not undermine individual projects.

These design principles help explain why some communities manage CPRs successfully. They allow users to share the same commitment and expect the long-term benefits to be worthwhile.

However, Ostrom stressed that there is no blueprint – no hard and fast rules – to CPR management. There are three particular complications:

1. Trust

Good management requires high trust to encourage norms of reciprocity. Trust is crucial to minimizing the costs of compliance monitoring and enforcement. Trust may develop when participants communicate regularly, share an understanding of their common interests, reciprocate each other’s cooperation, and have proven reliable in the past.

Design principles are important to developing trust and solidarity, but so are ‘evolutionary’ changes to behaviour. Actors have often learned about rule efficacy – to encourage cooperation and punish opportunism – through trial-and-error over a long period, beginning with simple, low-cost operational rules producing quick wins.

2. Rules, rules on rules, more rules, then even more rules

Institutions contain a large, complicated set of rules that serve many different purposes, and need to be understood and analysed in different ways.

Different purposes include:

• how many actors are part of an action situation, and the role they play
• what they must/ must not do
• who is eligible to participate
• who can move from one role to another
• who controls membership, and how
• how many participants are involved in a choice
• what will happen if there is no agreement
• how to manage and communicate information
• the rewards or sanctions
• the range of acceptable actions or outcomes from action.

 

We also need to analyse the relative costs and simplicity of different rules, and the rules about the other rules, including

• ‘operational’ rules on day-to-day issues (such as specific payoffs/ sanctions for behaviour)
• ‘collective choice’ rules about how to make those rules
• ‘constitutional’ rules on who can decide those rules and who can monitor and enforce, and
• ‘metaconstitutional’ analysis of how to design these constitutions with reference to the wider political and social context.

 

3. The world is too complex to break down into simple pieces

By now, you may be thinking that the IAD – and analysis of resource management – is complicated. This is true, partly because each case study – of the physical conditions and social practices regarding resource management – is different in some way. We can use the IAD to compare experiences, but accept that a profoundly successful scheme in one context may fail miserably in another.

Simplicity versus complexity: the world is complex, but should our analysis follow suit?

Indeed, this is why we need to think about rational choice games and the IAD simultaneously, to understand the analytical trade-offs.

Game theory laboratory experiments – built on simple rules and relatively small numbers of parameters – produce parsimonious analysis and results that we can understand relatively easily.

We may reject simple games as unrealistic, but what if we take this criticism to its extreme?

IAD in-depth field studies embrace complexity to try to understand the key dimensions of each study’s context. When we put them all together, there are too many concepts, variables, global applications, and variations-by-context, to contain in a simple theory.

The IAD addresses this trade off by offering a language to help organize research, encouraging people to learn it then use it to apply many different theories to explain different parts of the whole picture.

In other words, it is OK to reject simple models as unrealistic, but to embrace real-world complexity may require a rather complicated language.

See also:

Policy Concepts in 1000 Words: it’s time for some game theory

Policy Concepts in 1000 Words: Rational Choice and the IAD (the older post for the 1^st edition)

Policy Concepts in 100 Words: Multi-centric Policymaking

Policy in 500 Words: the Social-Ecological Systems Framework

Policy in 500 Words: Ecology of Games

How to Navigate Complex Policy Designs

How can governments better collaborate to address complex problems?

 

 

Policy Concepts in 1000 Words: game theory and thought experiments

Rational choice theory provides a way of thinking about collective action problems. There is great potential for choices made by individuals to have an adverse societal effect when there is an absence of trust, obligation, or other incentives to cooperate. People may have collective aims that require cooperation, but individual incentives to defect. While the action of one individual makes little difference, the sum total of individual actions may be catastrophic.

Simple ‘games’ provide a way to think about these issues logically, by limiting analysis to very specific situations under rather unrealistic conditions, before we consider possible solutions under more realistic conditions. For example, in simple games we assume that individuals pursue the best means to fulfil their preferences: they are able to act ‘optimally’ by processing all relevant information to rank-order their preferences consistently.

Go with it just now, and then we can consider what to do next.

The ‘prisoner’s dilemma’

Two people are caught red-handed and arrested for a minor crime, placed in separate rooms and invited to confess to a major crime (they both did it and the police know it but can’t prove it). The payoffs are:

• If Paul confesses and Linda doesn’t, then Paul walks free and Linda receives a 10 year jail sentence (or vice versa)
• If both confess they receive a much higher sentence (8 years) than if neither confesses (1 year).

Also assume that they take no benefit from the shorter sentence of the other person (a non-cooperative game).

It demonstrates a collective action problem: although the best outcome for the group requires that neither confess (both would go to jail for a total of 2 years), the actual outcome is that both confess (16 years). The latter represents the ‘Nash equilibrium’ since neither would be better off by changing their strategy unilaterally. Think of it from an individual’s perspective:

• Imagine Paul will confess. Linda knows that if she stays silent, she gets suckered into 10 years. If she confesses, she gets 8.
• Imagine Paul will stay silent. Linda knows that if she stays silent, she gets 1 year. If she confesses, she suckers Paul into 10.

The effect of Paul and Linda acting as individuals is that they are worse off collectively. Both ‘defect’ (confess) when they should ‘cooperate’ (stay silent).

The ‘logic of collective action’

Olson argues that, as the membership of an interest group rises, so does:

(a) the belief among individuals that their contribution to the group would make little difference and

(b) their ability to ‘free ride’.

I may applaud the actions of a group, but can – and will try to – enjoy the outcomes without leaving my sofa, paying them, or worrying that they will fail without me or punish me for not getting involved.

The ‘tragedy of the commons’

The scenario is that a group of farmers share a piece of land that can only support so many cattle before deteriorating and becoming useless. Although each farmer recognizes the collective benefit to an overall maximum number of cattle, each calculates that the marginal benefit she takes from one extra cow for herself exceeds the extra cost of over grazing to the group. Individuals place more value on the resources they extract for themselves now than the additional rewards they could all extract in the future.

The tragedy is that if all farmers act on the same calculation then they will destroy the common resource. The group is too large to track individual behaviour, individuals place more value on current over future consumption, and there is low mutual trust, with minimal motive and opportunity to produce and enforce binding agreements

This ‘tragedy’ sums up current anxieties about one of the defining problems of our time: global ‘common pool resources’ are scarce and the world’s population and consumption levels are rising; there is no magic solution; and, collective action is necessary but not guaranteed. We may value sustainable water, air, energy, forests, crops, and fishing stocks, but find it difficult to imagine how our small contribution to consumption will make much difference. As a group we fear climate change and seek to change our ways but, as individuals, contribute to the problem.

Overall, these scenarios suggest that individuals have weak incentives to cooperate even if it is in their interests and they agree to do so. This problem famously prompted Hardin (to recommend ‘mutual coercion, mutually agreed upon’ to ensure collective action.

What happens when there are many connected games?

In real life, it is almost impossible to find such self-contained and one-off games.  In many repeated – or connected – games, the players know that thereare wider or longer-term consequences to defection.

• In ‘nested games’, the behaviour of individuals often seems weird in one game until we recognise their involvement in a series. It may pay off to act ‘irrationally’ in the short term to support a longer-term strategy, or to lose in one to win in another.
• In an ‘ecology of games’, many overlapping games take place at roughly the same time, and players to learn how to play one game while keeping an eye on many others, while some key players encourage a wider set of rules under which all games operate.

Evolutionary game theory explores how behaviour changes over multiple games to reflect factors such as (a) feedback and learning from trial and error, and (b) norms and norm enforcement.

For example, player 2 may pursue a ‘tit-for-tat’ strategy. She cooperates at first, then mimics the other player’s previous choice: defecting, to punish the other player’s defection, or cooperating if the other player cooperated. Knowledge of this strategy could provide player 1 with the incentive to cooperate. Further, norms develop when players enforce and expect sanctions for non-cooperation, foster socialisation to discourage norm violation, and some norms become laws.

In other words, this focus on the rules of repeated games gives us more hope than the tragedy of the commons. Indeed, it underpin Ostrom’s famous analysis of the conditions under which people can govern the commons more effectively.

See also:

This post is one of four updates to the post Policy Concepts in 1000 Words: Rational Choice and the IAD 

Policy Concepts in 1000 Words: the Institutional Analysis and Development Framework

Policy in 500 Words: the Social-Ecological Systems Framework

Policy in 500 Words: Ecology of Games

See also:

How to Navigate Complex Policy Designs

How can governments better collaborate to address complex problems?

See also this tweet – and many others paying homage to it – to explain the title of the post:

<THREAD> I’m now hearing this meme that says Obama, Clinton, et al. are doing nothing, just gave up.

Guys. It’s time for some game theory.

— Eric Garland (@ericgarland) December 11, 2016

See also this tweet thread on Hardin:

Something I've been meaning to say about The Tragedy of the Commons. Bear with me for a small thread on why our embrace of Hardin is a stain on environmentalism. tldr: we’ve let a flawed metaphor by a racist ecologist define environmental thinking for a half century. 1/

— Matto Mildenberger (@mmildenberger) March 4, 2019