1

Chapter 3 Comparative Methods

Wyn Grant

[This chapter was written as a first draft in 2013 for a third edition of Burnham, Grant

et al Research Methods in Politics which was never completed. It does not,

therefore, take account of the most recent literature and there is too much quotation.

Nevertheless, it may be useful. Any citations should be to draft chapter, Leamington

2019].

The comparative method lies at the heart of the study of politics. ‘Even with an

increasing amount of statistical research in political science, a still significant amount

of case research, and a limited amount of experimental research, comparison remains

the fundamental laboratory for political science.’ (Peters, 2011, p. 38). It is ‘one of

the basic methods … of establishing general empirical propositions.’ (Lijphart, 1971,

p. 682). It is a means of discovering how things differ (or are similar), why things

differ (or do not) and whether we can learn from those differences and similarities to

improve policy processes and outcomes. For example, students of electoral systems

are interested in how different voting arrangements affect the distribution of seats in

elections, particularly where the system has changed from one form to another, as in

the case of the replacement of first past the post by a mixed-member system in New

Zealand. There is also an interest in whether different electoral systems affect

outcomes, for example do party list systems result in a higher proportion of women

elected to legislatures? ‘All good political economy is also necessarily comparative.’

(Phillips, 2005, p. 252).

It can be argued that methodologically, “everything” is comparable … there are

no limits to comparison. Everything – i.e. any case in the world at any point in time –
2

is, in principle, comparable.’ (Caramani, 2011, p. 16). Certainly there is a strong

case for a broad definition of comparative politics rather than one that excludes fields

such as policy analysis and political economy. (Caramani, 2011, p. 558). The

methodological problems that arise in, say, comparative political economy are in large

part general ones of comparison. Pushed to its conclusion, however, the principle of

comparing everything would lead to methodological and empirical incontinence. It

would certainly frustrate aspirations to create a political science that was cumulative,

i.e., one in which earlier theoretical and empirical discoveries provided building

blocks for later work, even if the final result was the repudiation of the original

findings. As was pointed out in Chapter *, the increasingly fashionable eclecticism

of approach needs some controls and boundaries if it is to be fruitful. One therefore

needs some notion of what it is that needs to be compared and political science has

given different answers at different times to this question throughout its history.

Historically comparative politics was thought of in terms of comparisons between

different nation-states as the standard unit of political organization. ‘Before World

War II it was mainly concerned with the analysis of the state and its institutions

defined in a narrow sense’. (Caramani, 2010, p. 37). There was an assumption of a

process of convergence and harmonization towards an Anglo-Saxon western model.

That assumption was no longer sustainable as research revealed a wider variety of

democratic models, e.g., consociationalism or patrimonialist regimes in the Middle

East and Africa and populist ones in South America (Carmani, 2010, p. 6), although

Peronism in Argentina received early attention in Lipset’s classic Political Man.

The unit of comparison

One of the challenges that comparative analysis faced were increasing doubts about

the reliability of the nation-state as a basis for comparative analysis in an era of

growing interdependence and globalization, leading to charges of theoretical

obsolescence. ‘Comparative politics was born on the methodological assumption

that cases – i.e. national political systems – are independent from each other.’

(Carmani, 2010, p. 41). This has led to a resurgence of interest in ‘Galton’s problem’

first stated in the 19th century. This formulation points out that associations between

observed phenomena can be ‘the result of diffusion effects between cases rather than

functional connections. Comparative political scientists realise that social

phenomena are not isolated and self-contained, but rather are affected by events

occurring within other societies.’ (Caramani, 2010, p. 42).

The unit of comparison does not, of course, have to be the nation-state or even

entities such as political parties within it. There was a long tradition in the United

States of comparison between different states and as sub-national units of government

have proliferated throughout the world, becoming more varied in form in multi-level

systems of government and have acquired greater powers, there has been a growing

interest in comparing these arrangements across different polities. The comparison

of different institutions, e.g., legislatures or processes, e.g., recruitment to political

office has also been an important feature of the study of comparative politics.

Although comparison between territorial entities has been the most important form of

comparison, there can also be comparisons between the politics of different policy

sectors within a particular country. However, comparison of that kind is more

fruitful when one extends the analysis to different countries so that, for example, one

compares health care systems in Britain, Germany and the United States.

The focus of comparison may, of course, be more specific than a whole policy

sector. There has been a revival in the study of policy instruments in political

science which are no longer seen merely as means of achieving goals but may be
4

constitutive of policy itself. (Le Gales). Thus, for example, one could study energy

saving and energy efficient improvement across different countries in terms of the

reduction of electricity consumption in private households and the contribution of

business to such policies. (Lang, Malang and Schneider, 2011).

The view which is sometimes heard that ‘The increasing interdependence of

national states and the decline of the autonomous power of public decision-making

are argued to imply that cross-national research is meaningless’ is clearly overstated.

(van Kersbergen, 2010, p. 57). The benefits of large-scale spatial comparison are

thrown into sharp relief when one considers cases where comparison is not feasible.

Whilst one can compare different global governance institutions, one cannot compare

the system of global governance as a whole: by definition there is one such system.

Nevertheless, ‘In an increasingly globalizing world the distinctions between the sub-

disciplines of comparative politics and international relations become more and more

blurred.’ (Berg-Schlosser, 2012, 223).

The EU is a polity of a very distinctive kind: it is more than an international

organization, but less than a state. There are other regional governance arrangements

in the world such as ASEAN, NAFTA and Mercosur, but they are much less

integrated and have far fewer powers over their member states. Hence, one view is

that alternative versions of the European Union can only be hypothetical constructs,

castles in the air, because there is nothing that it can properly be compared with.

However, it is possible to overstate the n=1 problem in the case of the EU. For

example, Simon Hix has compared its internal workings to those of states. The

European Parliament can be compared with other legislatures. The fact that it

functions in a different way through an innovative process of co-decision only adds to

the interest of the comparison, throwing into stark relief more traditional legislatures.
5

The benefits and challenges of comparison

A political science without comparison would be theoretically, methodologically and

empirically limited. One of its merits is that it provides a link between macro and

micro analysis that is generally absent in a discipline such as economics. Indeed, the

use of rational choice approaches in politics can lead to an excessive emphasis on the

choices and strategies of individual actors divorced from the social and political

context in which they make those choices. ‘The link between the micro and the

macro is crucial for comparative politics … the logic of comparison is primarily about

individual structures, and to think about how individuals interact within parliaments,

parties or bureaucracies.’ (Peters, 2011, p. 39).

Without the use of comparison political science would have difficulty in moving

beyond mere description or at best a historical account of a unique case. For

historians it is appropriate to discuss whether there was a modernizing revolution in

Tudor government with Thomas Cromwell alternatively cast in the role of ‘selfless

statesman’ (Marshall, 2009, p. 52) or self-serving rogue. For a political scientist the

question would be under what conditions do pre-modern forms of bureaucracy

emerge and what differences are there, for example, between a country that was

unified at an early date and one unified much later? Explaining a particular context

and its unique features may be satisfactory for a historian, but political scientists seek

to generalize. For example, what kinds of democratic arrangements work under what

circumstances? Can one make general statements about presidential or semi-

presidential compared with parliamentary systems of government?

Of course, in undertaking such an activity, we encounter the problem of

categorization. A basic methodological canon is that variation within a category

should not exceed variation between the categories used. The presidential systems
6

of, say, Chile, France and the United States have some similarities, but they also

display significant differences. For example, the legislature is weaker in France than

in the United States and the president of Chile cannot be re-elected for a second

consecutive term. A system like that of Britain has arguably become more quasi-

presidential over time. However, the exploration of the validity and usefulness of

such categories may of itself be instructive and lead to the development of new

explanatory categories, as has happened in the ‘Varieties of Capitalism’ debate which

seeks to compare the political economy of different countries. Schmidt was able to

suggest that the binary division between ‘liberal’ and ‘coordinated’ market economies

would benefit from the addition of a third category covering ‘state influenced’

economies.

We also encounter the problem of selection and that of selection bias which is

discussed further below in relation to case study selection. One of the problems that

arise is what is referred to as ‘convenience sampling’. In macro comparative

research, there is often ‘a focus on the OECD countries.’ (Collier, 2008, p. 2).

These offer a set of broadly similar advanced industrial countries, although with

enough economic and political variation between them to make within category

comparison rewarding. Perhaps even more enticing, one has a reliable data set

available from the OECD of key economic indicators which certainly facilitates the

study of comparative political economy.

However, deeper biases may be at work than those of convenience. In the case

of international political economy (IPE) there has been a focus on the advanced

industrializing countries at the expense of the Global South. This not just a question

of geographical breadth, but involves privileging a set of assumptions that emerges

from their experiences and the ‘specific context of “early industrialization”’.

(Phillips, 2005, p. 1). This in turn leads to a deficient treatment of key issues such as

globalization, production and finance. (Phillips, 2005, p. 6). Hence it has been

argued that there is a need to ‘globalize’ IPE.

The challenge of comparison

Although the imperative for comparison is a strong one, the task of achieving it in a

credible and defensible way is challenging. ‘The principal problem facing the

comparative method can be succinctly stated as: many variables, small number of

cases.’ (Lijphart, 1971, p. 685). As Lijphart points out, the former problem is

characteristic of social science. Our mean of screening out confounding variables are

still limited, notwithstanding advances in quantitative techniques and the wider use of

various forms of experiments. The latter problem is, however, ‘peculiar to the

comparative method.’ (Lijphart, 1971, p. 685). Indeed, the case study is widely

used in comparative research and is discussed in depth later in the chapter. ‘The

scientific status of the case study is somewhat ambiguous, however, because science

is a generalizing activity. A single case can constitute neither the basis for a valid

generalization nor the ground for disproving an established generalization.’ (Lijphart,

1971, p. 691).

One must also seek to avoid the risks of overly superficial comparison. Quality

comparative work needs to proceed from an in depth understanding of the particular

cases on which comparison is based. It does not obviate the need for single country

specialists for without such specialists genuine comparison is not possible. Of

course, the other side of this coin is the risk of treating a particular country as both

exceptional and as a sufficient basis for universal propositions. Peters notes (2011, p.

558) that a good portion of the considerable body of political science theory

developed in the US ‘does not appear relevant beyond the boundaries of the US’.
8

More generally, much of what is labelled as comparative politics ‘is not explicitly

comparative at all, consisting instead of “idiographic” studies (studies which are

limited to particular cases or events), often of individual countries. (Hopkin, 2011, p.

286).

As with any aspect of political science, comparative politics involves a complex

interaction between theoretical perspectives and empirical phenomena. In Chapter 2,

we emphasized the importance of theory in research design. Without theoretical

guidance, comparative analysis is likely to be merely descriptive. Peters notes

(2011, p. 39) that the interaction between theory and empirical research is ‘especially

important in comparative politics’. The choice of an appropriate theory is likely to be

influenced by the leverage that a particular perspective offers in relation to the

empirical phenomena that are the objects of explanation.

It should not be assumed, however, that the choice of an appropriate theoretical

perspective will necessarily increase the chances of good research being undertaken.

Theory can provide a set of blinkers which limit the range of any investigation.

‘Most research published in political science tends to find support for the theory or

model being investigated although in many ways negative findings would be more

useful.’ (Peters, 2011, p. 39).

It has, however, been argued that the study of comparative politics is relatively

atheoretical. Ross maintains (2009, p. 27), ‘Unlike sub-fields that are primarily

theoretically driven and ask questions derived from specific literatures, comparative

politics remains directly engaged with empirical puzzles and cases.’ That may well

be so, but does ‘the problem-focused and empirical foundation of comparative

politics’ rule out an engagement with theory? A scholar may start with an empirical

question such as why it is that electoral turnout is higher in some countries than others
9

or why is it that some party systems are more fragmented than others? Answering

some questions requires a theoretical apparatus if one is not to be overcome by the

mass of data or misled by the importance of specific details.

Consider the comparative analysis of different sub-national entities. Historically,

these typically formed part of federal arrangements in which the whole territory was

covered with subordinate units with broadly equivalent powers. However, more

recently one has seen arrangements in which only part of the territory has regional

autonomy, e.g., Scotland in the United Kingdom. Multi-level governance (MLG)

has emerged as a set of explanations which offer an alternative to federalism and offer

a vehicle for exploring new kinds of polities such as the European Union.

The literature on federalism has sought to explore the variety of vertical

relationships that exist between different levels of government in a range of settings.

MLG is about governance. It is less state oriented than federalism and focuses on

actors. It is concerned with the variety of ways in which modern government may be

performed, often involving public-private partnerships and a wide range of actors,

e.g., charities and non-governmental organizations. It is concerned not so much with

vertical relationships but with horizontal ones. For example, both in Europe and the

United States, cooperative relationships may develop across state boundaries.

MLG is not without its critics, some of whom see more traditional formulations

of federalism as having a continuing utility and a greater specificity which is helpful

in identifying and operationalizing research questions. As a heuristic device, it has

been criticized for narrowness and for being too descriptive. It has also been

criticized on normative grounds as a means of defending the status quo. In many

ways it would be unfortunate if a polarity developed between defenders of federalism

and intellectual advocates of MLG. The debate on MLG offers a means of renewing
10

and enriching the debate on federalism. Nevertheless, comparing subnational

governments involves making choices about the preferred theoretical perspective.

Because ‘Comparative politics is the broadest sub-discipline of political studies.’

it is probably a little harsh to characterize it as ‘something of a residual category for

research that cannot be classified as international relations or political philosophy’.

(Ross, 2009, p. 27). That is why one needs proper attention to theory, to

methodological questions and to principles of research design, starting with the

stipulations that Mill made about this form of analysis.

Mill’s three canons

Mill’s classic work needs to be taken seriously in comparative analysis given the

confusion that surrounds terminology and the designation of various sub-types.

(Moses and Knutsen, 2007, p. 96). Mill’s three canons have been extensively used by

successive generations of researchers although he ‘never intended for his canons to be

used in the social sciences because he did not believe that unit homogeneity could be

achieved in the field’. (Sekhon, 2008). In a book that had considerable influence,

Przeworski and Teune (1970) draw a distinction between ‘most similar’ and ‘most

different’ designs that has become embedded in the literature.

Mill set out five canons in all, but the indirect method of difference and the method

of residues are not discussed in depth here as they have not shaped the study of

comparative politics, although it has been argued that when scholars ‘assert they are

using the Method of Agreement and the Method of Difference they are using the

Indirect Method of Difference [which is] the weaker sibling of the Direct Method of

Difference.’ (Sekhon, 2008).

The Method of Agreement arises ‘If two or more instances of the phenomenon

under investigation have only one circumstance in common, the circumstance in

which alone all the instances agree is the cause (or effect) of the given phenomenon.’

(Mill, 1872, p. 255). ‘[T]he investigator merely begins to collect cases of a

particular phenomenon in an attempt to find common explanatory factors in cases that

are otherwise quite different.’ (Moses and Knutsen, 2007, p. 101). Thus, if a number

of observations reveal that the dependent variable has only one cause in common, we

can infer that that is the cause of the phenomenon. The approach involves the

identification of ‘a similarity in the independent variable in two or more cases.’

(George and Bennett, 2005, p. 153). If the independent variable has an effect on the

dependent variable, that effect should persist even if the cases differ on other

variables.

However, ‘If an instance in which the phenomenon under investigation occurs,

and in instance in which it does not occur, have every circumstance in common save

one, that only occurring in the former; the circumstance in which alone the two

instances differ is the effect, or the cause, or an indispensable part of the cause of the

phenomenon.’ (Mill, 1872, p. 256). This is the method of difference, often referred

to as the ‘most similar’ research design, where the attempt is made ‘to identify

independent variables associated with different outcomes.’ (George and Bennet,

2005, p. 153). ‘The Method of Difference compares political/social systems that

share a number of common features as a way of neutralizing some differences while

highlighting others. In other words, case selection is used in a way to control for

causal effect.’ (Moses and Knutsen, 2007, p.98). ‘Mill judged the method of

difference to be somewhat stronger than the method of agreement.’ (George and

Bennett, 2005, p. 154).

The method of concomitant variation is a more sophisticated version of the method

of difference which Mill proposed to deal ‘with some of the limitations of the other
12

two methods.’ (George and Bennett, 2005, p. 154). It is not confined to binary

cases and can take account of variations in the relative strength of a variable. It

occurs ‘If two variables tend to vary in the same pattern, then they are somehow

linked, either causally or through some other pattern of connection (linked to some

third variable).’ (Peters, 1998, p, 29). In terms of understanding underlying

commonalities rather than testing causal relationships it was used in a classic work of

comparative politics, Almond and Verba’s (1963) Civic Culture.

Box 3.1 Almond and Verba’s (1963) Civic Culture

This book is generally regarded as a landmark study in comparative politics. It was

the first time that the survey method had been used to make comparisons across five

widely varying countries to answer a set of questions about the conditions that

promoted democratic stability, in particular a type of political culture which the

authors called a ‘civic culture’. The study has subsequently been criticised from

many perspectives, including the use of the survey instrument (which had not been

extensively used in Italy and Mexico), its tendency to see American democracy as the

normatively preferable model against which all other variants must be judged and the

use of political culture as a residual ‘garbage can’ variable without its own theoretical

territory. Nevertheless, it remains a departure point for later studies. Verba

conceded retrospectively (1980, p. 396) that ‘The spurt of comparative work that

came after it would have come anyway. Nevertheless, The Civic Culture served, I

believe, as an example of what to do for some and what not to for others. In either

case it added to progress and to cumulation in the study of comparative politics.’

Verba admitted (1980, p. 398) that there was a trade-off between depth and breadth

which is always going to be a challenge in comparative politics: ‘One problem is that

a cross-national study can never give to each individual nation the full attention and

understanding that it deserves on its own.’ It should be noted that case selection was

on the dependent variable which is appropriate in a small-n study. The countries

studied had a variety of democratic experiences, but none of them (except in some

respects Mexico) was a non-democracy so the exploration of how their political

cultures differed was not possible.

Case studies

The use of the case study is central to qualitative comparative research. Yet it has

been viewed with great suspicion, particularly by positivists. From this perspective

there is a hierarchy of methods at the bottom of which is to be found the case study.

(Moses and Knutsen, 2007, p. 17). ‘A work that focuses its attention of a single

example of a broader phenomenon is apt to be described as a “mere” case study.’

Yet, ‘the discipline continues to produce a vast number of case studies, many of

which have entered the pantheon of classic works.’ (Gerring, 2004, p. 341).

Moreover, mixed methods research has given a significant boost to the centrality of

the case study in political research with a growing scepticism about ‘whether

statistical analysis alone can reveal causal relationships.’ (Hall, 2013, p, 28). The

defenders of single method research are more sceptical, seeing case studies as

primarily illustrative and unable to generate error reduction or cross-validation.

(Ahmed and Sil, 2013).

Any method can be used badly, and case studies are open to selection biases

which allow writers to justify a conclusion they have already arrived at. Arguably,

writers have been neglectful in many cases of the methodological implications of

what they are doing. They are happy to find the case study in the toolkit of social

research without reflecting how it might appropriately be used. It is all too easy to
14

produce a case study of limited value that is narrow in its geographical and functional

scope and offers neither ‘thick description’ nor a basis for generalization.

In practice there is a diversity of approaches to case study methodology, reflecting

disagreements about core theoretical and methodological issues and this is reflected in

a variety of definitions. Gerring seeks to provide much needed clarity. ‘Case

connotes a spatially delimited phenomenon (a unit) observed at a single point in time

or over some period of time.’ (Gerring, 2007a, p. 19). ‘A case study may be

understood as an intensive study of a single case where the purpose of the study is – at

least in part – to shed light on a larger class of cases (a population).’ (Gerring, 2007a,

p. 20). A case could be a nation-state, a local government authority, a political

party, a pressure group or the implementation of policy in a particular setting, e.g.,

controlling crime in a Brazilian favela. Thus, ‘A case study is a study of a subject

broader than the case itself.’ (Gerring, 2007b, p. 12). There is a trade off between the

intensity and depth of analysis that it permits and the limited range of comparison that

is possible. ‘Case studies are generally strong precisely where statistical and formal

methods are weak.’ (George and Bennett, 2005, p.19). For practical reasons, ‘the

case study research format is usually limited to a dozen cases or fewer. A single case

is unusual.’ (Gerring, 2007a, pp. 21-22).

Box 3.2 Allison’s Essence of Decision

The Cuban missile crisis of 1962 was a case of particular significance. It was the

closest the world has ever come to a nuclear war between the two then super powers.

Indeed, we now know that the risk was even greater than it was perceived to be at the

time as local Soviet commanders could have deployed what in effect were tactical

nuclear weapons on their own initiative in the event of an American invasion. The

significance of the event still leaves open the question of what it was a case of, e.g.,
15

nuclear deterrence, decision-making under conditions of crisis or relations between

the US and the USSR during the Cold War. In terms of explanation, Allison was

faced with the dominance of a rational actor model popular among both academics

and practitioners. Allison did not think it was without utility, but thought that it

needed to be supplemented or supplanted by two alternative models: the

organizational process model and the governmental politics model. The former

focuses on the way in which large organizations function according to regular patterns

of behaviour and the latter on the politics of a government. Allison acknowledged

(1971, p. 8) that ‘A single case can do no more that suggest the kinds of differences

among explanations produced by the three models.’ However, he argued that the

missile crisis provided a situation for which rational actor explanations seemed

particularly suited and therefore offered a tough test for the alternative models. It

does, however, pose the question of how far one can generalize from the special case

of the missile crisis to decision-making in general. George and Bennett argue (2005,

pp122-3) that is ‘a strong test case for the rational actor model, a moderate test of the

organizational process model, and a strong test of the bureaucratic politics model.

However, it is not the strongest possible test of any model and just how strong it is

depends on which Allison’s research questions is under consideration.’ In a sense

Allison asked the case to do too much, but in doing so he mounted a strong challenge

to conventional theoretical wisdom.

Types of case study

A broad distinction can be made between case-centric studies (aimed at understanding

a case, see Box 3.2) and theory-centric case studies (where the case is a case of X,

with the goal of making inferences that can travel to the population of the phenomena.

Indeed, one of the underlying tensions in case study research is that ‘they are
16

particularizing and generalizing.’ (Gerring, 2007, pp. 79-80). Lijphart (1971, p. 691)

distinguished between six types of case study:

1. Atheoretical

2. Interpretive. In these first two types ‘there need not be a generalizing

dimension in the cases.’ (Moses and Knutsen, 2012, p 136).

3. Hypothesis-generating with the objective of formulating hypotheses for further

testing.

4. Theory-confirming which tends to support the original theory and suggests

that it has a broader applicability.

5. Theory-infirming that does not support the original theory, hence weakening it

without disproving it.

6. Deviant, a case that in terms of our general understanding ‘where cases are

known to deviate from established generalizations’ (Moses and Knutsen,

2012, p 139).

Box 3.3 – Streeck and Thelen and theoretically informed case studies

In their book Beyond Continuity Streeck and Thelen (2005) tackle a central limitation

of the historical institutionalist literature, its reduction of explanations of change to

either incremental change or major exogenous shocks to the status quo. What they

did was to seek contributions ‘organized around a theoretically self-conscious

investigation of empirical cases of institutional change in advanced industrial

democracies that do not fit received conceptualizations.’ (Streeck and Thelen, 2005,

p.9). The cases are mostly of particular policies in single countries, e.g, welfare state

retrenchment in the US or early retirement in Germany, although some are

comparative across two countries. This is an approach which is theoretically rather

than empirically ambitious and not particularly reflective in terms of case study
17

methodology. The chapter authors were asked to provide empirically grounded

general insights into the character and mechanisms of institutional change. They

provide a corrective to existing theoretical frameworks and in that sense are theory

infirming. The chapters serve as mechanisms of hypothesis generation which

suggested to the Streeck and Thelen five modes of gradual but transformative change.

The chapters are used to provide confirmation and illustrations of the typology and in

that sense are theory confirming. No doubt the actual process of theory development

was messier and occurred less as a logical time sequence than this summary suggests.

There is a sense in which the process is circular and points to one outcome and in that

respect is open to methodological criticism, but the book is regarded as a powerful

contribution to historical institutionalism. [Box ends]

Even cases which may appear to be very distinctive can benefit from being placed

within a comparative context. Switzerland has proved something of a challenge to

political scientists because of its distinctive institutional arrangements such as the

multi-party Federal Council and direct democracy. It represents ‘a “deviant case” of

democracy: a democracy where citizens participate not only in the election of its

parliament and its government, but also vote on and ratify parliamentary decisions of

major importance.’ (Linder, 2010, p. xix). During the course of his substantial work

on democratization, Charles Tilly argued that Switzerland should be included in every

study about democratic processes because ‘If you get Switzerland, you get it.’

(Quoted in Helbig and Koller, 2009, p. 317). In other words, the Swiss case is so

complex and distinctive that it offers a tough test of any theory or proposition one

puts forward.

It is tempting to either ignore the Swiss case or to treat it as unique so that it is

‘rejected out of hand or gives rise to uncritical mystification.’ (Linder, 2010, p. xix).
18

Yet if we place Switzerland within a broader comparative context, we both

understand Switzerland better and enhance its contribution to our understanding of the

different ways in which democratic political institutions may function. The Swiss

case shows how political integration may be used to build a multicultural society, it is

‘instructive for the realisation of political unity while maintaining cultural diversity’.

(Linder, 2010, p.179). Of course, particular models often do not travel well into

different settings, but ‘While the model cannot be copied in its entirety, some of its

basic elements can be noted, adapted and used by others.’ (Linder, 2010, pp. xxii).

Similarly, our understanding of the merits and drawbacks of direct democracy may be

enhanced by comparing the Swiss model with its use in American states such as

California.

Case study selection

‘One of the most common critiques of case study methods is that they are particularly

prone to versions of “selection bias”’. (George and Bennett, 2005, p. 23). As with

all forms of research, selection should be guided by the research objectives of the

study. More generally, ‘the task of case selection is a matter of achieving two

objectives: representativeness (typicality) and variation (causal leverage).’ (Gerring,

2007, p. 149).

The textbook method of selecting cases would be through random sampling.

However, ‘When the objective is to achieve the greatest possible amount of

information on a given problem or phenomenon, a representative case may not be the

most appropriate strategy. This is because the typical or average case is often not the

richest in information.’ (Flyvberg, 2006, p. 229). Research based on a small

number of cases cannot use random sampling. ‘[There] is a high likelihood that the

(randomly) chosen cases will be wildly unrepresentative of the population’. (Gerring,

2008, p. 8). The sample may also not contain sufficient variation on theoretically

relevant dimensions.

As is often the case in social science research, the process by which cases are

selected for study is often much messier than a logical research design might imply.

All sorts of pragmatic considerations come into play such as access to the necessary

data, linguistic skills, familiarity with a particular country or institution and what

funding is available for. Quite reasonably, researchers want to minimize start up

costs which delay the conduct of the actual research. Lieberman (2007, p. 4) captures

this reality well when he states that ‘Sometimes case studies find us, we don’t look for

them. We may be working or traveling somewhere or reading something, and our

interest is piqued, perhaps because what we observe confirms or contradicts some

prevailing theory.’ Case studies are generated as ‘the product of life circumstances

and interests, personal and professional.’

Gerring retorts (2007b, p. 13) that such pragmatic imperatives ‘do not constitute

methodological defences of a particular research design.’ We need to be able to

provide a justification of the cases that we select that is not convenient or prudential

but methodological. Gerring acknowledges (2007b, p. 14) ‘that case-selection

strategies are sometimes justified methodologically only after the fact.’

Nevertheless, they still have to be justified. There has to be some reason for

choosing one case rather than another beyond simple expedience.

Helpful general advice is to look for ‘most likely’ or ‘least likely’ cases, ‘that is,

cases likely to either clearly confirm or irrefutably falsify propositions and hypotheses

… Cases of the “most likely” type are especially well suited to falsification of

propositions, whereas “least likely” cases are most appropriate to tests of

verification.’ (Flybjerg, 2006, p. 231). In practice ‘Cases usually fall somewhere in

between being most likely and least likely for particular theories, and so pose tests of

an intermediate degree of difficulty … researchers should be careful to specify, for

each alternative hypothesis, where the case at hand lies on the spectrum from most to

least likely for that theory’. (George and Bennet5t, 2005, p. 122).

One has to be cautious about irrefutably falsifying something, as a case study

rarely allows us to reject a theory outright. It tells us that the theory does not work in

that particular set of circumstances which may mean that the theory requires

qualification or modification. Of course, sometimes a case study may overturn an

existing theory. Until Lijphart’s (1968) study of the Netherlands, the prevailing

assumption about political stability was that it required homogeneity or at least cross-

cutting cleavages. Lijphart showed that in the Netherlands reinforcing social

cleavages were compatible with political stability and this led to the theory of

consociational democracy which reconfigured our understanding of democratic

societies.

Process tracing

[I know from peer review that process tracing is a very popular technique. I retain a

measure of scepticism about it because to me it involves too much in the way of

subjective judgment by the researcher. But I accept that I am probably behind the

curve].

Process tracing was first developed by George in 1979, but it has been increasingly

deployed as a technique in political science in the recent past. In part this may reflect

the increasing resort to intepretivist or constructivist approaches. However, its appeal

is broader than that and not confined to those approaches. ‘All kinds of plausible

mechanisms in human action – rational choice, constructivist or otherwise - make

interpretive claims about what people perceived and thought. Thus constructivist
21

methods are not fundamentally distinctive for including interpretive process-tracing.’

(Parsons, 2010, p. 92). Indeed, ‘To the extent that process tracing is meant to uncover

hidden causal patterns, or to document the nature of the causal links posited in larger

general studies, then this approach lends itself to naturalists.’ (Moses and Knutsen,

2012, p. 225).

Nevertheless, there is still considerable uncertainty about exactly what process

tracing offers or how it might be used. As Kittel and Kuehn observe (2013, p. 3),

‘there has been little success in formalising its methodology and defining standards …

a range of different understandings of process tracing seem to exist.’ At its least

satisfactory, it can offer an elaborate means of encapsulating a relatively simple set of

procedures: ‘many scholars who contend they are engaging in Process Tracing

research are in effect only tracing an empirical series of events between X and Y.’

Box 3.4 offers an example of well-designed and effectively executed process tracing.

Box 3.4 Using process tracing to validate and develop theory

Deters (2013) was interested in the paradox that the EU offers a high level of

environmental protection despite Scharpf’s joint decision trap. Three cases were

selected to maximize the variance on the outcome variable. The first case on energy

efficiency saw deadlock following opposition by two key member states. In the

second case on emission standards for cars the regulation was watered down by a

German-led coalition. In the third case on phasing out the conventional light bulb

the EU acted as a pacesetter and went beyond a minimal Council compromise.

Exhaustive case accounts traced the crucial phases of the decision-making process.

The second phase of the work attempted to identify causal process hypotheses that

could be tested on other cases. The level of decision-making, politicised or technical

comotology, was significant. Process tracing is seen as helping developing more

complex theoretical explanations and deriving more parsimonious versions of them.

However, Deters warns (2013, p 84) of ‘the risk of producing historically contingent

explanations at the first stage without merit for further examination during a second,

effects-of-causes analysis.’ [Box ends]

Process tracing potentially offers a more sophisticated way of making within-case

comparisons. It can be used to ‘shift the investigator’s focus from what happened, to

how and why it happened.’ (Moses and Knutsen, 2012, p. 225). Full description is

the starting point, but as a means of facilitating explanation and analysis What it

seeks to do is to explore and trace the causal mechanisms that link independent and

dependent variables, but in a different way from multivariate statistical analysis.

This does not mean that these qualitative and quantitative forms of analysis are

incompatible: in many respects they are complementary. As Hall has commented

(quoted in George and Bennett, 2005, p. 206), ‘[Process-tracing] is a method well-

suited to testing theories in a world marked by multiple interaction effects, where it is

difficult to explain outcomes in terms of two or three independent variables –

precisely the world that more and more social scientists believe we confront.’

However, the method is not without its critics, some of whom doubt whether it can

properly be called a method.

What does process tracing entail? ‘[The] cause-effect link that connects

independent variable and outcome is unwrapped and divided into smaller steps; then

the investigator looks for observable evidence of each step.’ (Van Evera, 1997, p.

64). This evidence need not be qualitative, it could be quantitative. It could come

from all the variety of data sources considered in this book: documents, elite

interviews, observations, surveys etc. Gerring (2007, p. 173) argues that ‘The

hallmark of process tracing … is that multiple types of evidence are employed for the
23

verification of a single inference – bits and pieces of evidence that embody different

units of analysis … Individual observations are therefore noncomparable.’

Gerring draws an analogy with detective work. One can see the point of this

analogy: various pieces of evidence are assembled from different sources – forensic,

cameras, witness statements etc. If all goes well, they point to a single conclusion,

the identity of the guilty party. However, the detective or team of detectives are

faced with a problem of information overload. The jigsaw may be difficult to fit

together and a crucial piece of evidence may be overlooked. In works of detective

fiction from Sherlock Holmes to Inspector Morse this is resolved by a flash of

intuition drawing on lateral thinking: it is the fact that the dog did not bark in the night

that is significant. This may produce a solution to the particular case, but one that

does not offer a method that is necessarily replicable to other cases.

George and Bennett (2005, p. 223) admit that process tracing ‘can require

enormous amounts of information and it is weakened when data is not accessible on

key steps in a hypothesized process.’ So how can it be undertaken? ‘The simplest

form of process-tracing takes the form of a detailed narrative or story that purports to

throw light on how an event came about.’ (George and Bennett, 2005, p. 210). This

atheoretical narrative might look remarkably similar to a traditional historical account.

However, its purpose is essentially different. It can serve as a first step, a ground

clearing operation that permits the construction of a more theoretically informed

account. Thus the historical narrative can be converted into ‘an analytical causal

explanation couched in explicit theoretical forms.’ (George and Bennett, 2015, p.

211).

A new approach to process tracing

Nevertheless, there is still considerable uncertainty about how process tracing can be

undertaken and what its relationship is to mixed method approaches more generally.

As Liebermann notes (2007, p. 5), ‘Political scientists would benefit a great deal by

breaking down a set of criteria which they believe would establish a reasonable

baseline for convincing or at least acceptable process-tracing evidence, including, say,

temporal proximity of links in a causal chain; the explicitness of actor intentions;

and/or the types of sources used.’

Beech and Perdersen (forthcoming) suggest that much of the difficulty arises

from the variety of approaches that are subsumed under a single heading. They

distinguish between three different variants of process tracing: theory-testing, theory-

building and explaining outcomes.

The theory-testing approach takes a theory that is already available in the

literature and examines the available evidence to see whether a hypothesized causal

mechanism is actually present in a given case. This, of course, reflects a well-

established use of case studies to confirm, question and hopefully refine existing

theoretical perspectives. The original empirical relationship could have been

discovered through a small-n comparative historical study, a medium-n qualitative

comparative analysis or a large-n regression based analysis. What this entails is

conceptualizing and operationalizing the causal mechanism and testing its presence

one step at a time.

Compared with the deductive approach of theory-testing process tracing, theory-

building process tracing is an inductive approach. There is an absence of explanation

in the existing literature about how it might be undertaken. The objective is to build

a parsimonious theory of a causal mechanism that is generalizable outside of the

context of the individual case. The ambition of theory-building process tracing is

relatively limited in the sense of building a middle-range theory of a causal

mechanism that is expected to function within a bounded context. This issue of

spatial or temporal boundaries poses fundamental challenges in case study work

whatever the particular methodology used. As Gerring notes (2007, p. 66), ‘the

boundaries of a case are sometimes – despite the researcher’s best efforts – open-

ended. This is particularly true of temporal boundaries, which may extend into the

future and into the past in rather indefinite ways.’ Beach and Pedersen suggest that

this variant of process tracing might be particularly useful when it is know that a

correlation exists, but the causal mechanism linking the independent and dependent

variable are not properly understood or where the outcome us unknown but its cause

is uncertain, in other words a deviant case.

Beach and Pedersen admit that there are some overlaps between theory-building

process tracing and explaining-outcome process tracing. However, in outcome

process tracing the emphasis is on building a minimally sufficient explanation of the

outcome in the individual case. In so far as the objective is to explain a particular

historical outcome, there are some analogies with historical studies, it is still guided

by theoretical perspectives that reach beyond the single case. In this type of work

various causal mechanisms usually need to be combined into an eclectic conglomerate

mechanism. It is usually necessary to non-systematic or case-specific mechanisms.

Beach and Perdersen seek to bring greater clarity to the literature on process

tracing. It is evident, however, that even if they are more clearly categorized

through their work, a variety of approaches can legitimately be used under the process

tracing umbrella. This is consistent with the licensed eclecticism that is characteristic

of contemporary political science, but it raises questions about whether process

tracing can properly be called a method.

The strengths and limitations of process tracing

The exploration of causal mechanisms is at the heart of case study work. Process

tracing is ‘one means of attempting to get closer to the mechanisms or

microfoundations behind observed phenomena.’ (George and Bennett, 2005, p. 147).

Its advocates see many advantages in its use:

Process-tracing has many advantages for theory development and theory testing

… some of them unique. It is a useful method for generating and analysing data

on causal mechanisms. It can check for spuriousness and permit causal

inference on the basis of a few cases or even a single case … It can point out

variables that were otherwise left out in the initial model or comparison of

cases, and it can lead inductively to the explanation of deviant cases and the

subsequent derivation of new hypotheses.’ (George and Bennett, 2005, p. 323)

Yet George and Bennett admit (2005, p. 222) that a ‘problem for process-

tracing is that there may be more than one hypothesized causal mechanism consistent

with any given set of process-tracing evidence. The researcher then faces the difficult

challenge of assessing whether alternative explanations are complementary in the

case, or whether the one is causal and the other spurious.’ Even with the work of

Beach and Pedersen, are there sufficient clear and agreed criteria to judge whether a

given exercise in process tracing meets acceptable standards of reliability and

validity?

Nevertheless, George and Bennett (2005, p. 208) are emphatic that ‘Process-

tracing is a research method.’ Gerring takes a more sceptical stance, referring to ‘its

apparently mysterious qualities’. For him ‘it may be impossible to arrive at a set of

standardized methodological rules – thus depriving process tracing of the status of a

“method”. ‘ (Gerring, 2007, p. 185). Challenged about his rather sceptical treatment
27

of process tracing Gerring replied that the empirical component of the technique ‘rests

largely upon background assumptions about the case and/or human behaviour in

general. Because it is so heavily context-dependent, it is not clear whether any

general methodological precepts apply. That is, one cannot tell a good process-

tracing study from a poor one unless one knows a great deal about the case under

investigation.’ (Gerring, 2007b, p. 14). If Gerring is correct in this argument, it has

worrying implications.

Of course, some of the difficulties are inherent in qualitative methods in general.

In the absence of statistical tests and the possibility of secondary analysis of the data,

one cannot readily undertake an independent assessment of the conclusions reached.

However, in the case of elite interviewing, normal practice is to explain how and why

the respondents were selected; how many there were; how long they were interviewed

for; and how structured the interview was. Participant observers can state how long

they spent at the research site and the conditions under which they did so. We have

less commonly accepted standards for process tracing, although there have clearly

been moves in that direction.

Gerring’s solution is to have research involving process tracing vetted by suitably

trained experts, but where these experts would come from, how they would be trained

and what criteria they would use remains unclear. In response to those who suggest

that process tracing is the defining characteristic of case study analysis, Gerring’s

response is that it is a tool, ‘but not the tool.’ (Gerring, 2007b, p. 14). Process

tracing has enhanced case study analysis, and posed important questions about how it

should be conducted, but it does not encompass it.

Conclusions
28

Case studies might seem to be prey to the individualistic fallacy, but in general

researchers are quite cautious about the generalizations that they make or, if they are

not, any flaws in their argument will soon be exposed by others. At its best there is a

healthy, dynamic dialogue between theory and case study. Case studies may be

useful at an exploratory stage of research where they can generate hypotheses which

can be tested further and may ultimately lead to new theoretical perspectives.

Theory will also inform the selection and conduct of case studies which will provide a

better understanding of the causal mechanisms at work and permit the refinement of

theory. ‘Case study methods involve a trade-off among the goals of attaining

theoretical parsimony, establishing explanatory richness and keeping the number [of]

the cases to be studied manageable.’ (George and Bennett, 2005, 31). As part of a

more general intellectual revival of qualitative methods, case studies have regained

methodological respect and process tracing has played a part in achieving that

outcome.

Comparative politics has the potential to address big questions of fundamental

concern to political scientists. For example, what conditions favour democratization

processes and how can those processes be sustained? There has been something of a

tension in comparative research between (relatively) ‘large-N’ research and ‘small-N’

research, often based on the in depth analysis of particular cases. However, we have

seen that our small-N research has become more methodologically sophisticated ‘less

and less about telling more or less compelling stories, and increasingly designed to

provide generalizable insights.’ Mixed methods approaches permit a coordinated

joint research strategy. In this approach, ‘case studies can be used to generate

theory, by examining a variety of possible causal mechanisms, and to test theoretical

propositions in greater detail and complexity than is possible in large-N studies. The
29

findings from case studies can be used to inform subsequent quantitative analysis,

thus generating more robust generalizations than would be possible from the case

study alone.’ (Hopkin, 2011, p. 306). Comparative analysis requires comparison

across cases using a variety of techniques and it also needs the careful exploration of

causal mechanisms in the single case.