Tag Archives: Case study

Research design: Case studies and comparative research

My aim is to tell you about the use and value of comparative research by combining (a) a summary of your POLU9RM course reading, and (b) some examples from my research. Of course, I wouldn’t normally blow my own trumpet, but Dr Margulis insisted that I do so. Here is the podcast (25 minutes) which you can download or stream:

The reading for this session is Halperin and Heath’s chapter 9, which makes the following initial points about comparative research:

  • ‘Comparative’ describes methods to identify and explain differences/ similarities between cases.
  • These cases are often – but need not be – countries (for example, comparisons over time, or by policy area, can be just as illuminating).
  • There are three main approaches: large-N (many cases), small-N (several cases), and single-N (or ‘case studies’).
  • Comparison is not an end in itself. Rather, we need to justify, or clarify the value of, each comparison. This involves identifying a clear theoretical reason for a particular comparison (and, as in all studies, producing a clear research question that can be answered).
  • It helps us avoid what Rose calls ‘false uniqueness’ (assuming rather than demonstrating that a case is exceptional) and ‘false universalism’ (assuming that a finding from one time/ place applies to all times or places).
  • Comparative methods are most frequently used to determine the extent to which a theory generated from one set of cases applies to other cases.

Key issues when you apply theories to new cases

There are two interesting implications of the final bullet point.

First, note the tendency of a small number of countries to dominate academic publication. For example, a lot of the policy theories to which I refer in this ‘1000 words’ series derive from studies of the US. With many theories, you can see interesting developments when scholars apply them outside of the US:

  • The ‘universal’ v ‘territorial’ issue. In my article with Michael Jones (summarised here) on Kingdon’s ‘multiple streams analysis’ we note that it began as a study of health/ transport policy in the US in the 1980s. It then morphed into a theory applied to many regions at many times. A large part of its success comes from the abstract nature of its key concepts, which can be applied at any place and time: for example, a lot of its discussion of agenda setting relates to ‘bounded rationality’ which applies to all people. Yet, there are also ‘territorial’ issues which apply to particular regions or types of government. For example, some studies adapt MSA’s concepts to explain the influence of supranational authorities or to describe the differences between EU and US policymaking.
  • The ‘beyond the West’ issue. In one interesting application of MSA to China, Zhu argues that a key concept is not applicable: policy change in the US requires policy solutions to be technically feasible; in China, they only prompt change if ‘infeasible’. The example highlights the limits to the explanatory power of theories derived from studies of a small number of ‘Western’ countries (I know this description is loaded – what other terms could we use to describe countries like the US and UK?). In such cases, scholars are now exploring the implications in some depth: how well do these policy theories travel?

Second, consider the extent to which we are accumulating knowledge when applying the same theory to many cases. There are now some major reviews or debates of key policy theories, in which the authors highlight the difficulties of systematic research to produce a large number of comparable case studies:

For me, the important common factor in all of these reviews is that many scholars pay insufficient attention to the theory when applying its insights to new cases. Consequently, when you try to review the body of work, you find it difficult to generate an overall sense of developments by comparing many case studies. So, in my humble opinion, we’d be a lot better off if people did a proper review of the literature – and made sure that their concepts were clear and able to be ‘operationalised’ – before jumping into case study analysis. I wrote these blog posts for established scholars and new PhD students, but the argument should also apply to you as an undergraduate: get the basics right (which includes understanding the theory you are applying) to get the comparative research right. This is just as important as your case selection.

From case study to large-N research: choosing between depth and breadth?

Case studies. It is in this context that we might understand Halperin and Heath’s point that case study research (single-N) is comparative. You might be going into much depth to identify key aspects of a single case, but also considering the extent to which it compares meaningfully with other case studies using the same theory. All that we require in such examples is that you justify your case selection. In some examples, you are trying to see if a theory drawn from one country applies to another. Or, you might be interested in how far theories travel, or how applicable they are to cases which seem unusual or ‘deviant’. In some examples, we seek the ‘crucial case study’ that is central to the ‘confirmation or disconfirmation of a theory’ (p207), but don’t make the mistake of concluding that you need a new theory because the old one can only explain so much. Further, although single case studies should not be dismissed, you can only conclude so much from them. So, be careful to spell out and justify any conclusions that you find to be ‘generalisable’ to other cases.

Example 1. In my research, I often use US-origin theories to help explain policymaking by the UK and devolved governments. For example, I used the same approach as Kingdon (documentary analysis and semi-structured interviews) to identify, in great detail, the circumstances under which 4 governments in the UK introduced the same ban on smoking in public places (the take home message: there was more to it than you might think!).

Small-N studies. The systematic comparison of several cases allows you to extend analysis often without compromising on depth. However, there is great potential to bias your outcomes by cherry-picking cases to suit particular theories. So, we look for ways to justify case selection. Halperin and Heath go to some length to identify the problems of ‘selection on the dependent variable’, which can be summed up as: don’t compare cases just because they seem to have the same outcome in common (focus instead on what causes such outcomes). Two well-established approaches are ‘most similar systems design’ (MSSD) and ‘most different systems design’ (MDSD) (p209). With MSSD, you choose cases which share key explanatory factors/ characteristics (e.g. they have the same socio-economic/ political context) so that you can track the effects of one or more difference (perhaps in the spirit of a randomised control trial, but without the substance). With MDSD you choose cases which do not share characteristics so that you can track the effect of a key similarity.

Aside from the problems of case study selection bias, it’s worth noting how difficult it is to produce a clear MSSD or MDSD research design, since you are making value judgements about which shared/ not shared characteristics are crucial (see their discussion of necessary/ sufficient factors).

Example 2. Take the example of a study I did with Karin Ingold and Manuel Fischer recently, comparing ‘fracking’ policy and policymaking in the UK and Switzerland.  We use the same theory (the ACF) and same method (documentary analysis and a survey of key actors) in both countries. We used a survey to allow us to quantify key relationships between actors: to what extent do they share the same beliefs with other actors, and how likely are they to share information frequently with their allies and competitors?

We describe the research design as MDSD because their political systems represent two contrasting political system archetypes – the ‘majoritarian’ UK and ‘consensus’ Switzerland – which Lijphart describes as key factors in their contrasting policymaking processes. Yet, central to our argument is that there are policymaking processes common to policy subsystems despite their system differences. In effect, we try to measure the effect of political system design on subsystem dynamics and find a subtle but important impact.

We also find that, although these differences exist, their policy outcomes are remarkably similar. So, ‘most different’ systems often produce very similar policymaking processes and policy outcomes. Then, we note that if we had our time again we would have extended the analysis to subnational governments in the UK. The ‘most different’ design prompted us to focus on the UK central government and Swiss Cantons (the alleged locus of power in both cases), but maybe we could have started from an assumption that they are not as different as they look. Have a look and you can see the dilemmas that still play out in (what I think is) a well-designed study.

Example 3. I face the same difficulties when comparing policy and policymaking by the UK and Scottish Governments. In some respects, they are ‘most different’: ‘new Scottish politics’ was designed to contrast with ‘old Westminster’ (particularly when it came to elections; the Scottish Parliament is also unicameral). In others, they are similar: the ‘architects of devolution’ introduced a system that seems to be of the Westminster family (particularly the executive-legislative relationships). Further, Scotland remains part of the UK, and the UK Government retains responsibility for many policies affecting Scotland. Overall, it is difficult to say for sure how similar/ different are their systems (which I discuss in a series of lectures). So, the comparison is fraught with difficulty. In such examples, they key solution is to ‘show your working’: describe these problems and state how you work within them (for example, in my case, I try to examine the extent to which policymaking reflects ‘territorial’ context or ‘universal’ drivers’, partly by interviewing policymakers in each government).

Large-N (quantitative) studies.  Halperin and Heath lay out some of the potential benefits of large-N research. For example, it is easier to come to general conclusions about many countries by studying many countries (rather than trying to generalise from a few, which might not be representative). They also highlight the pitfalls, including the problem of meaning: when you ‘operationalise’ a concept such as democracy or populism, can you provide a simple enough definition to allow you to give each system a number (to denote democratic/ undemocratic or X% democratic) that means the same thing in each case? To this problem, I would add the general issue of breadth and depth. With large-N studies you can examine the effects of a small number of variables, to explain a small part of the politics of many systems. With small-N you can study a large number of variables in a few systems. The classic trade-off is between breadth and depth. Of course, if you are doing an undergraduate dissertation the big Q is: what can you reasonably be expected to do? Maybe your highest aim should be to make sense of the studies which already exist.

 

 

 

 

6 Comments

Filed under Research design

Policy Concepts in 1000 Words: Punctuated Equilibrium Theory

cloud punctuated equilibrium

See also What is Policy? and the Policy concepts in 1000 words series

(podcast download)

Policymaking can appear stable for long periods, only to be destabilised profoundly. Most policies can stay the same for long periods while a small number change quickly and dramatically. Or, policy change in one issue may be minimal for decades, followed by profound change which sets policy on an entirely new direction. The aim of Baumgartner and Jones’ punctuated equilibrium theory is to measure and explain these long periods of policymaking stability, and policy continuity, disrupted by short but intense periods of instability and change. The key concepts are:

Bounded rationality. Policymakers cannot consider all problems and their solutions at all times. For example, government ministers can only pay attention to a tiny proportion of the issues for which they are responsible. They ignore most and promote few to the top of their agenda.

Disproportionate attention. Policymakers often ignore issues or pay them an unusual amount of attention. The lack of attention to most issues helps explain why most policies may not change. Intense periods of attention to some issues may prompt new ways to understand and seek to solve old problems.

Power and agenda setting. Some groups try to maintain their privileged position by minimizing attention to the policy solutions which benefit them. Others seek to expand attention, to encourage new audiences and participants, to generate debate and new action.

Framing. Groups compete to influence how a problem is framed (understood, defined, categorized and measured) and therefore solved by policymakers. For example, it may be framed as a problem that has largely been solved, leaving the technical details of implementation to experts, or a crisis which should generate widespread attention and immediate action.

Policy monopolies. Groups may enjoy a ‘monopoly of understanding’ when policymakers accept their preferred way to frame an issue for long periods, perhaps even taking it for granted. This monopoly may be ‘institutionalised’ when rules are created and resources devoted to solving the policy problem on those terms.

Venue shopping. To challenge a monopoly in one venue (such as the executive, or one type of government at a particular level), groups may seek an audience in another (such as the legislature, the courts, or another type or level of government).

In Agendas and Instability (1993; 2009), Baumgartner and Jones, use a case study approach to examine these processes in detail. For example, in postwar US nuclear power, they identify a period of major public attention, focused on the pressing need to solve a policy problem, followed by minimal attention – for decades – when the problem appeared to be solved. The government inspired public enthusiasm for nuclear power as a solution to several problems – including the need to reduce energy bills, minimise dependence on other countries for oil, reduce air pollution, and boost employment and economic activity. This positive image, and general sense that the policy problem was solved, supported the formation of a post-war policy monopoly involving the experts implementing policy. Public, media and most government attention fell and the details of policy were left to certain (mainly private sector) experts, federal agencies and congressional committees. The monopoly was only challenged in the 1970s following environmental activist and scientific concern about nuclear safety. Groups used this new, negative, portrayal of the nuclear solution to generate concerned interest in new venues, including the courts, congressional committees and, particularly following a major accident at Three Mile Island, the public. The policy monopoly – the way in which nuclear power was framed, and the institutions established to implement policy – was destroyed.  Then, a new, negative, image became dominant for decades – and a post-war policy of power plant expansion was replaced by a moratorium and increased regulation. Only recently has the resurgence of nuclear power become a serious possibility.

In The Politics of Attention (2005), Jones and Baumgartner’s focus shifts to more general observations of selective attention (they highlight over ‘400,000 observations collected as part of the Policy Agendas Project’). Policymakers are unwilling to focus on certain issues for ideological and pragmatic reasons (e.g. some solutions may be too unpopular to consider; there is an established view within government about how to address the issue). They are also unable to pay attention because the focus on one issue means ignoring 99 others. Change may require a critical mass of attention to overcome the conservatism of decision makers and shift their attention from competing problems. If levels of external pressure reach this tipping point, they can cause major and infrequent punctuations rather than smaller and more regular policy changes: the burst in attention and communication becomes self-reinforcing; new approaches are considered; different ‘weights’ are applied to the same types of information; policy is driven ideologically by new actors; and/or the ‘new’ issue sparks off new conflicts between political actors. Information processing is therefore characterized by ‘stasis interrupted by bursts of innovation’ and policy responses are unpredictable and episodic rather than continuous. One key example is the annual budget process which, like many other examples of political activity, does not display a ‘normal distribution’ of cases. Instead, it is characterized by a huge number of minimal changes and a small number of huge changes.

The trend in Baumgartner and Jones’ work has been to move from the specific to the ‘universal’; from some cases in one country to many cases in many countries. Their initial assumption was that many of the processes they observed in case studies resulted from the peculiarities of the US system. Yet, concepts such as bounded rationality, selective attention, policy monopolies and venue shopping should be applicable to all political systems. Punctuated equilibrium theory helps us to balance a focus on the specific and the general. We can use these concepts to generate empirical questions about why the policymakers, institutions and venues of specific political systems prompt particular problems and solutions to be addressed and others to be ignored. We can also use them to identify the same overall patterns – based on a mixture of stasis, stability and continuity disrupted by innovation, instability, and change – in many systems.

To read more, click here to get a Green Access version of chapter 9 of this book discussed here.

52 Comments

Filed under 1000 words, agenda setting, public policy