All you need to know about the research methods in politics....
Research Methods in Politics A practical guide
Roger Pierce
Los Angeles • London • New Delhi • Singapore
To David, Kate and Tom Pierce
© Roger Pierce 2008 First Published 2008 Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act, 1988, this publication may be reproduced, stored or transmitted in any form, or by any means, only with the prior permission in writing of the publishers, or in the case of reprographic reproduction, in accordance with the terms of licences issued by the Copyright Licensing Agency. Enquiries concerning reproduction outside those terms should be sent to the publishers. SAGE Publications Ltd 1 Oliver’s Yard 55 City Road London EC1Y 1SP SAGE Publications Inc. 2455 Teller Road Thousand Oaks, California 91320 SAGE Publications India Pvt Ltd B 1/I 1 Mohan Cooperative Industrial Area Mathura Road New Delhi 110 044 SAGE Publications Asia-Pacific Pte Ltd 33 Pekin Street #02-01 Far East Square Singapore 048763 Library of Congress Control Number: 2007934984 British Library Cataloguing in Publication data A catalogue record for this book is available from the British Library ISBN 978-1-4129-3550-0 ISBN 978-1-4129-3551-7 (pbk) Typeset by CEPHA Imaging Pvt. Ltd., Bangalore, India Printed and bound in Great Britain by TJ International Ltd Printed on paper from sustainable resources
Contents
About This Book
v
Preface
x
PART I INTRODUCTION
1
1
Introduction
3
2
Power in Research, Ethics, Data Protection and Bias
9
3
The Philosophy and Principles of Research
PART II METHODOLOGIES 4
5
22 39
Qualitative Versus Quantitative Methods: A Relevant Argument?
41
Collecting Data Sets: Case Studies, Experimental, Comparative, Longitudinal and Action Research Methods
51
PART III COLLECTING INFORMATION
67
6
Critically Evaluating Published Research
69
7
Evaluating Information: Validity, Reliability, Accuracy, Triangulation
79
8
9
10
Completing a Literature Review: Accessing Published (β) Information
100
Asking Questions: Effective Elite Interviews, Other Interviews, Vignettes, Projective Questions, and Focus Groups
117
Questionnaire Surveys
140
iv
Contents
11
Observation
161
PART IV DATA ANALYSIS
173
12
175
Analysing Research Data: The Process
Part IVA Quantitative Analysis
181
13
Calculating and Interpreting Descriptive Statistics
183
14
Using and Understanding Inferential Statistics
197
15
Testing for Association
206
16
Applying Factor Analysis and Other Advanced Techniques
220
PART IVB Qualitative Analysis 17
239
Analysing Qualitative Information: Classifying, Coding and Interpreting Information
241
18
Using Content Analysis
263
19
Understanding and Adopting Discourse and Narrative Analysis
279
PART V COMMUNICATING RESEARCH
307
20
309
Writing-up
Glossary of Terms
319
Key Formulae and Symbols
329
Bibliography
330
Index
335
About This Book
This textbook has been written for students and researchers in Departments of Politics at UK and other ‘western’ universities where English is the language of instruction.3 It is designed to provide an introductory text for undergraduates, an intermediate text for graduates following ‘taught’ Masters’ programmes and firstyear postgraduate researchers. It is also designed to provide an additional teaching resource for busy teachers – especially those for whom teaching research methods is a duty rather than matter of choice. It seeks to meet the wide variety of readers’ prior training and training needs by adopting a multi-level approach. In this way, it seeks to bridge the present gap in the literature between the good introductory texts and the more advanced texts requiring substantial mathematical training, knowledge or sociolinguistics.4,5 It also seeks to be inclusive by including and giving equal coverage to quantitative (numeric) and qualitative ‘talk and text’ methods. It recognises the criticism of research methods texts of ‘eurocentricity’ by acknowledging that, while the underlying principles of research may be (almost) universal, the underlying philosophic roots and techniques must vary between different contexts. The book assumes a mixed readership of home English-speaking students and overseas students for whom English is a second language. Similarly, it assumes that some students – especially of ‘taught courses’ – may come from other disciplines and be unused to terms more specific to Politics. To meet the special needs of some students without patronising others, a system of comprehensive endnotes has been used to explain terms and to cite references fully.6 (Regrettably, today’s printing software precludes the use of footnotes.) Technical terms are written in italics. Where key technical terms are introduced for the first time, they are displayed in bold text, defined and discussed. Briefer definitions of terms are given in the glossary. Whilst the Harvard system of referencing is used, this has been augmented by providing the date of first publication so that the chronological significance is retained, e.g.: Hobbes, T. (1651/1996) Leviathan. Oxford, Oxford University Press.
Teaching and learning strategy Teaching and learning are often conceived as two sides of the same coin. Some students still prefer to learn passively from teaching through formal lectures. Others prefer to gain (arguably deeper) insights through a combination of individual study
vi
About This Book
(including reading and writing), exploration and heuristic learning (including direct practical experience). Most benefit from collective discussions at seminars or informal discussions with colleagues. The pedagogic 7 principle underlying this textbook is that we all learn best – in terms of acquiring knowledge and skills – through a mix of formal teaching, independent study, experience, and practice, in which the proportions will vary from student to student. So a blended (or distributed) teaching and learning strategy has been adopted. This offers readers a mix of: 1. text (supplemented by diagrams and other illustrations) communicating the underlying principles and to introduce various techniques 2. worked examples demonstrating how approaches and techniques can be applied 3. sources for independent study 4. topics for seminar and informal discussion 5. case studies providing practical assignments for individual or group work.
To assist readers, supplementary case studies and packages of PowerPoint slides for each of the chapters are available via the website www.sagepub.co.uk/pierce.
How to use the book Readers are not expected or encouraged to read this book from cover-to-cover. You are unlikely to really assimilate knowledge for its own sake. You are more likely to assimilate best what and when you ‘need to know’. So a strategy of timely, selective reading is recommended. And while this book seeks to be inclusive and authoritative, it makes no claims to be a definitive text. Users are therefore expected to follow the principles of best scholarship by considering and comparing other narratives before firming up their views. The relevant additional literature includes: •
•
•
Burnham, P., Grillard, K., Grant, W. and Layton-Henry, Z. (2004) Research Methods in Politics. Basingstoke: Palgrave Macmillan. This 308-page textbook has been written by members of the Department of Politics at the University of Warwick. It draws on their experience as teachers of research methods and their critique of the limitations of the application to politics of the many research guides written by sociologists. Harrison, L. (2001) Political Research: An Introduction. London: Routledge.This 180-page textbook has been written by Lisa Harrison, senior lecturer in Politics at the University of the West of England. As its title states, it has been written primarily as an introductory text for students. Marsh, D., and Stoker, G. (eds.) (2002) Theory and Methods in Political Science. Basingstoke: Palgrave. This 368-page textbook contains separate chapters by leading
About This Book academics in Politics at universities in UK, US and Australia. Its strength lies in the coverage which it offers of the main approaches and issues and their implications for research methods.
Where best to begin? A three-stage approach is recommended using the following proforma. You can copy this from www.sage.co.uk/pierce. It lists the chapter titles, topics and degrees of knowledge: 1. Assess your starting research skills. 2. In conjunction with your research supervisor, identify your research training needs. You can skip or skim those chapters and topics with which you are already familiar. 3. Concentrate on those other chapters and topics where your research training needs are greatest. This will enable you to choose and develop the research skills which best serve your research interests.
Training needs self-evaluation proforma Chapter Chapter Title
Existing Knowledge and Understanding
No.
None
Topics
0 1 2 PART I INTRODUCTION 1 Introduction Political science/ politics/political 0 1 2 studies 2 Power in Research, Ethics, Data Protection and Bias Ethics in research 0 1 2 Data protection 0 1 2 The Research Effect (Hawthorne) 0 1 2 Bias 0 1 2 3 The Philosophy and Principles of Research Positivism 0 1 2 Empiricism 0 1 2 Behaviouralism 0 1 2 Naturalism 0 1 2 Feminism 0 1 2 Marxism 0 1 2 Inductive research 0 1 2
Substantial 3
4
5
3
4
5
3 3 3 3
4 4 4 4
5 5 5 5
3 3 3 3 3 3 3
4 4 4 4 4 4 4
5 5 5 5 5 5 5 (Continued)
vii
viii
About This Book Chapter Chapter Title No.
Topics
Existing Knowledge and Understanding None 0 0 0
Substantial 1 1 1
2 2 2
3 3 3
4 4 4
5 5 5
Deductive research Grounded research PART II METHODOLOGIES 4 Qualitative versus Quantitative Methods: A Relevant Argument? Qualitative methods: claims and 0 1 2 3 4 5 criticisms Quantitative methods: claims and 0 1 2 3 4 5 criticisms Mixed methods 0 1 2 3 4 5 Research design(s) 0 1 2 3 4 5 5 Collecting Data Sets: Case Studies, Experimental, Comparative, Longitudinal and Action Research Methods Case studies 0 1 2 3 4 5 Comparative research 0 1 2 3 4 5 Longitudinal research 0 1 2 3 4 5 Action research 0 1 2 3 4 5 PART III COLLECTING INFORMATION 6 Critically Evaluating Published 0 1 2 3 4 5 Research 7 Evaluating Information: Validity, Reliability, Accuracy, Triangulation Validity, reliability and accuracy 0 1 2 3 4 5 Primary and secondary sources 0 1 2 3 4 5 Triangulation 0 1 2 3 4 5 Sampling 0 1 2 3 4 5 8 Completing a Literature Review: Accessing Published (β) Information Information search 0 1 2 3 4 5 Completing a critical literature 0 1 2 3 4 5 review 9 Asking Questions: Effective Elite Interviews, Other Interviews, Vignettes, Projective Questions, and Focus Groups Elite interviews 0 1 2 3 4 5 Group meetings 0 1 2 3 4 5 Vignettes 0 1 2 3 4 5 Focus groups 0 1 2 3 4 5 10 Questionnaire Surveys Survey sampling 0 1 2 3 4 5 Designing and coding 0 1 2 3 4 5 questionnaires Projective questions 0 1 2 3 4 5 Designing vignettes 0 1 2 3 4 5 11 Observation 0 1 2 3 4 5
About This Book
Chapter
Chapter Title
Existing Knowledge and Understanding
No.
Topics
None
Substantial
0 1 2 3 4 PART IV DATA ANALYSIS 12 Analysing Research Data: The 0 1 2 3 4 Process IVA Quantitative Analysis 13 Calculating and Interpreting Descriptive Statistics MS Excel 0 1 2 3 4 SPSS 0 1 2 3 4 Descriptive statistics Mean, median, mode, outliers, 0 1 2 3 4 range, deviance Grouped frequency distribution 0 1 2 3 4 Standard deviation 0 1 2 3 4 14 Using and Understanding Inferential Statistics Standard error of the mean 0 1 2 3 4 Confidence limits 0 1 2 3 4 15 Testing for Association Correlation 0 1 2 3 4 Significance 0 1 2 3 4 Linear regression analysis 0 1 2 3 4 16 Applying Factor Analysis and Other Advanced Techniques Factor analysis 0 1 2 3 4 Bernouli distribution 0 1 2 3 4 Time series analysis 0 1 2 3 4 IVB Qualitative Analysis 17 Analysing Qualitative Information: Classifying, Coding and Interpreting Information Coding text 0 1 2 3 4 18 Using Content Analysis 0 1 2 3 4 19 Understanding and Adopting Discourse and Narrative Analysis 0 1 2 3 4 PART V COMMUNICATING RESEARCH 20 Writing-up 0 1 2 3 4
5 5
5 5 5 5 5 5 5 5 5 5 5 5 5
5 5 5 5
ix
Preface
This book encourages researchers to begin their reports with a personal statement in the form of a preface. This serves several purposes. First, it tells the readers why the researcher has (really) chosen the research topic: what it means to the author and their identity. Second, it discloses (the inevitable) starting biases, prejudices and hunches – the intellectual and emotional baggage that the researcher carries. Third, the preface acknowledges those other people who have influenced the work. Writing the preface is a difficult task. It requires reflexivity (intellectual selfawareness) and honesty. Overall, the personal statement should clarify matters at the beginning for both the researcher and the reader. In particular, it underlines the personal – the identity of the researcher that, in the discipline of Politics and other social sciences, exerts a major influence on the choice of research, the findings and, crucially, their interpretation. The researcher is therefore, in the language of research, an independent variable (a causal factor or driver which affects the outcome). In this case, the textbook has been written to ‘close the chapter’ on my tenyear career as a mature graduate, doctoral student and, latterly, teacher of research methods in the Department of Politics at the University of York. It seeks to bring together in a single volume materials developed from many sources and academic specialities over many years. In my previous work in urban regeneration, I helped shape new environments and policies. They provide enduring evidence of effort and success – or otherwise. Hopefully, this book serves a similar purpose. Si monumentum requiris, etc.8 My baggage is essentially that of a practitioner as distinct from an academic.9 So my approach seeks to be eminently practical: to show readers how to carry out different types of research and techniques to high standards of scholarship. But it is not a-theoretical: the various methodologies are grounded in theoretical principles that must be properly understood if the tools are to be applied correctly. After all, There is nothing as useful as a good theory.10
My particular research interests are: power, deference and complicity and the explanations of Gramsci, Lukes, Bourdieu and Foucault. This textbook has been shaped by many other people. Special thanks are due to: my former mentor, Professor Mark Evans; to Dr Adrian Leftwich who provided the original brief; to Dr Roger MacGinty and Dr Simon Parker who encouraged me to develop this text from my module guides. To Liz Harrison who proof-read the drafts;
Preface
and, to Patrick Brindle of Sage who encouraged me to submit a proposal. However, the greatest influence has been exerted by the consumers: the students who provided formal course evaluation and informal comments; the graduate teaching assistants11 ; and the research subjects (people) with whom (rather than on whom) the research methods were developed. However, any errors or omissions are entirely my own. Dr Roger Pierce York
Notes 1 Panegyric, public speech. 2 Pasquinade, lampoon, libel, satire. 3 Politics is used throughout this book to refer to the subject otherwise called ‘political science’, ‘political studies’, ‘politics and government’ etc. A capital P is used throughout to distinguish the discipline and study of Politics from its more general use to describe everyday activity by politicians and electors. 4 For example, Harrison, L. (2001) Political Research: An Introduction. London: Routledge. 5 For example: Pennings, P., Keman, H., and Kleinnijhuis, J. (2006) Doing Research in Political Science: An Introduction to Comparative Methods and Statistics. London: Sage. 6 ‘footnotes [and endnotes] are the humanists’ rough equivalent of the scientist’s report on data: they offer empirical support for the stories told and arguments presented . . .’ Grafton, A. (1997) The Footnote. London: Faber. p.vii. 7 From pedagogy, ‘the science of teaching’. 8 Si monumentum requiris circumspice, ‘If you seek his monument, see around you’ inscription in St Paul’s Cathedral London to its architect, Sir Christopher Wren. 9 Dictionary definitions of ‘academic’ include: ‘of no practical use’. 10 Kurt Lewin (1890–1947), US-naturalised German psychologist, founder of ‘action research’ and mentor to Carl Festinger (‘cognitive dissonance’). 11 Especially Fiona Aspinall, Christine Hamieh and Sharleene Bibbings.
xi
The Researcher
‘He will have views and prospects to himself perpetually soliciting his eye, which he can no more help standing still to look at than he can fly; he will moreover have various … Accounts to reconcile; Anecdotes to pick up; Inscriptions to make out; Stories to weave in; Traditions to sift Personages to call upon; Panegyrics1 to paste up at his door; Pasquinades2 at that…’ Rev. Laurence Sterne (1996/1760) Tristram Shandy. Ware: Herts., Wordsworth., p. 27.
Part I Introduction
Chapter 1
Introduction
This chapter begins, like the others that follow, with a clear statement of the teaching and learning objectives (or purpose). They are shown below: Teaching and learning objectives:
1. To answer the question: What is research? 2. To identify key features of research specific to Politics. 3. To set out the structure and contents of the textbook.
What is research? Research is essentially a process of systematic inquiry. Its core activities are: • • • •
goal orientated and purposeful inquisitive – searching for answers to specific questions – especially ‘why?’ and ‘how?’ careful, systematic and methodical original.
Additionally, we can claim that academic research can be distinguished from other research by •
its central concern for theory involving either testing or extending existing theory (deductive research), or developing new theory (inductive research).
Academic research essentially involves a systematic process. It begins with a research question. This is followed by a literature review, the collection or discovery of information, analysis, interpretation and conclusions. It can be both inventive and creative in terms of designing the research process and framing new theory. However, research may also involve serendipity – the happy knack of making discoveries by accident. For example, penicillin was ‘discovered’ in 1928 when Fleming noted that a petri dish had been contaminated by mould. However, rather
4
Research Methods in Politics
than throwing out the dish, he first examined the contents and discovered that the mould had prevented the formation of Staphylococcus bacilli. The discovery thereby evidenced Pasteur’s earlier remark that ‘chance only favours the prepared mind’ (Greenfield et al, 2001: 302).1 Today, specialist ‘serendipity software’ is available to search and compare data sets to identify potential associations for subsequent investigation. Similarly, research on research has shown that intellectual curiosity is the main driver of effective research which, in turn, stimulates and sustains the essential concentration and motivation (Mace, 1962: 29).2 In a post-modern world, it may also be argued that scepticism (the doctrine of the Philosophic School of Sceptics that real knowledge of the world is unattainable) is also an essential component. There is therefore a new readiness to challenge accepted theories, ‘truth’ and ‘facts’. At a time when so much is already ‘known’, the question has to be asked: why undertake research? In particular, what real contributions can a single-semester, undergraduate research project make to the knowledge and understanding of our world? In reply, it can be argued that the real benefit of undergraduate, graduate and doctoral research lies in the contribution the research activity makes to your intellectual and personal development in terms of: • • • • •
testing the applicability and relevance of theory to new contexts promoting a better understanding of theoretical concepts developing analytical and interpretative skills learning how best to design investigative processes and manage projects uniquely, the opportunity given via undergraduate group research projects to gain experience of effective team-working – the mantra of modern management.
In other words, research provides an opportunity and arena for education for both academic or non-academic careers. But, what is Politics? Does the discipline require or pre-suppose a unique approach to research training that favours specialist, in-house, faculty-wide, generic research training? Politics is a relatively new academic discipline in the family of social sciences. It has roots in philosophy, history, law, geography, economics, sociology, psychology and, in the sub-field of voting behaviour, quantitative (statistical) analysis and mathematical modelling. It has therefore been described as: ‘the junction subject of the social sciences’ (Burnham et al, 2004: 8).3 Alternatively, it can be seen as a ‘mongrel subject’ or ‘crossover discipline’ that draws on others promiscuously. It can be both prescriptive (normative) and descriptive. Two broad and six constituent approaches have been identified in the ‘diverse and … cosmopolitan’ discipline: the formal operation of politics in government (and other arenas) embracing: behaviouralism; rational (public) choice theory and (new) institutional analysis; and, politics as a social process pervading all levels of
Introduction
society (associated with feminism); interpretative theory (anti-foundationalism); and, Marxism (Marsh & Stoker, 2002: 3).4 While there is no official definition of Politics, an official view was negotiated by a committee of academics to provide a ‘benchmark statement’: [Politics is concerned with] … developing a knowledge and understanding of government and society. The interaction of people, ideas and institutions provides the focus to understand how values are allocated and resources distributed at many levels, from the local to the sectoral, national, regional and global. The analyses of ‘who gets what, when, how, why and where’ are central, and pertain to related questions of power, justice, order, conflict, legitimacy, accountability, obligation, sovereignty and decision-making. Politics encompasses philosophical, theoretical, institutional and issue-based concerns relating to governance. (QAA, 2000: 2)5
This overarching view of Politics might, at first sight, be seen to imply that all available research approaches and techniques can be adopted in Politics and that the researcher’s tool bag is vast. But this is not the case. In particular, power (which many commentators regard as central to the discipline) is not readily quantifiable or measurable.6 It as another ‘essentially contested concept’ (Lukes, 1974: 9).7 Indeed, power may not be observable (Bachrach & Baratz, 1962).8 Similarly, conflict need
Illustration 1 Ask yourself, Father, what do we really mean by ‘power’?
5
6
Research Methods in Politics
not necessarily be overt. And historic events cannot be re-run. So, those laboratorybased, scientific research approaches and techniques which rely on the repeated observation of phenomena under controlled conditions are largely inappropriate to Politics. This textbook concentrates on those research methods which seek to discern and interpret the underlying meanings, causes and consequences of conflict and power at the level of supra-state, state, government, party, class and other identity groups, and people in either the past or present-day. So a special characteristic of research in Politics is that it collects information from both archives (historical records) and fieldwork (field research).
Textbook structure and contents The textbook is structured in five parts and their associated chapters which, generally speaking, follow the research process and degrees of complexity. Part I (Introduction) begins with this scoping chapter. It is followed, in Chapter 2, by a review of power in research, ethics, data protection and the research effect. It concludes, in Chapter 3, with a review of the underlying philosophy and principles of research, including the concept of causality, provided by the main schools of positivism, empiricism, behaviouralism, naturalism and feminism newly mentioned and distinguishes between inductive, deductive and grounded research. Part II (Methodologies) identifies the underlying principles and theories from which various methods have been developed. It begins, in Chapter 4, with a review of the dualism of qualitative and quantitative research, the claims and counter-claims of their disciples and the role of mixed methods. This concludes by identifying the main approaches to research design. This is followed, in Chapter 5, by a critical review of case studies, experimental, comparative, longitudinal and action research. Part III (Collecting information) begins, in Chapter 6, with advice on how to critically evaluate published research. This is followed by guidance on how to test the quality of information in terms of the gold standards of validity, reliability and accuracy, the distinction between primary and secondary sources, and the use of triangulation and sampling in Chapter 7. The next chapter (8) concentrates on accessing secondary sources and completing an effective literature review. This is followed, in Chapter 9, by guidance on how to organise and complete interviews with political elites and other individuals, group meetings and focus groups, projective questions and vignettes. Chapter 10 provides practical advice on designing and conducting questionnaire surveys. Part III concludes with a critical review of other observational methods including the role of the new popular science of body language in Chapter 11.
Introduction
Part IV introduces methods for analysing and interpreting the information collected. It begins, in Chapter 12, by reviewing the generic process of analysis, introducing a hierarchy of analysis and distinguishing between predominantly quantitative and qualitative methods of analysis. Quantitative methods start, in Chapter 13, by reviewing the main descriptive statistics (mean, standard deviation, etc.) and their use. It is followed, in Chapter 14 by a review of the main inferential statistics (standard error of the mean, etc.) in which conclusions about populations are inferred from samples. The next chapter introduces the concepts of association, correlation, significance and regression analysis. The quantitative methods conclude, in Chapter 16, by introducing cluster and factor analysis (which enable underlying factors to be identified and labelled), time series analysis and the binomial distribution. The formulae built into MS Excel are used in the early chapters to calculate descriptive and inferential statistics. SPSS statistical software is used for the more sophisticated calculations in the later chapters. Qualitative methods begins by offering guidance on how to transcribe and code ‘talk and text’ in Chapter 17. Chapter 18 introduces new approaches to content analysis. The section ends by introducing discourse analysis and narrative analysis and demonstrating their use in Chapter 19. Finally, Part V provides advice on communicating research. In particular, it considers, in Chapter 20 the writing-up process from the initial research proposal to the final research report. Despite its (traditional) location at the end of the book, it argues that research begins by writing the research proposal and continues in parallel with the research process. So writing-up is both the ‘alpha’ (beginning) and ‘omega’ (end) of research. The text is supported by three appendices: a glossary of terms, statistical formulae and bibliography.
Questions for discussion or assignments
1. What is Politics? Is ‘political science’ more appropriate, a misnomer or an oxymoron? 2. What is research? 3. What is the current research agenda of Politics? What topics would you wish to see added? Why? 4. Is power central to Politics? 5. Which research methods are generic to social science? Which others, if any, are specific to Politics? 6. What are the implications of post-modernism for academic research? Is there a distinction between ‘truth’ and ‘facts’?
7
8
Research Methods in Politics FURTHER READING Burnham, P., Grillard, K., Grant, W. and Layton-Henry, Z. (2004) Research Methods in Politics. Basingstoke: Palgrave Macmillan. pp. 1–29. This very readable introduction identifies and discusses the origins of the discipline of Politics, the dominant paradigms and their main methodological implications. Held, D. and Leftwich, A. (1984) Chapter 8: A Discipline of Politics? In Leftwich, A. (ed.) What is Politics? Its Activity and Study. Oxford: Blackwell. pp. 139–159. This book provides an excellent collection of essays providing different perspectives on Politics. In the final chapter, Leftwich assesses these interpretations with particular reference to the centrality of conflict. Leftwich, A. (2004) Thinking Politically: On the politics of Politics. In Leftwich, A. (ed.) (2004) What is Politics? Cambridge: Polity Press. pp. 1–22. This textbook offers new and additional essays on Politics. Leftwich introduces the debate with a scoping review which concludes that: ‘what unites political analysts is a concern for the provenance, forms, distribution, use, control, consequences and analysis of political power. What separates them is the difference of focus and the levels and frameworks of analysis …’ p. 20. Marsh, D. and Stoker, G. (eds.) (2002) Theory and Methods in Political Science. Basingstoke: Palgrave. pp. 1–16. This short introduction by the editors seeks ‘to get readers into the foothills of understanding the political science range’, p. 16. Table 1 provides a very good summary of the characteristics of the main approaches, pp. 6–7.
Notes 1 Greenfield, S., Singh, S., Tallack, P. et al (2001) The Science Book. London: Cassell & Co. p. 302. 2 Mace, C.A. (1962) The Psychology of Study. Harmondsworth: Penguin. p. 29. 3 Burnham, P., Grillard, K., Grant, W. and Layton-Henry, Z. (2004) Research Methods in Politics. Basingstoke: Palgrave Macmillan. p. 8. 4 Marsh, D. and Stoker, G. (eds.) (2002) Theory and Methods in Political Science. Basingstoke: Palgrave. pp. 3–11. 5 QAA (Quality Assurance Agency for UK Higher Education) (2000) Politics and International Relations’ Benchmark Statement. QAA, Gloucester cited in Leftwich, A. (ed.) (2004) What is Politics? Cambridge: Polity Press. p. 20. Note how this constipated ‘official view’ draws centrally on Lasswell, H. (1958) Who Gets What, When, How. New York: Meridian. 6 For example, Robertson, D. (1993) Politics. London: Penguin. 7 Lukes, S. (1974) Power: A Radical View. London: Macmillan. p. 9. 8 See Bachrach, P., and Baratz, M. (1962) Two Faces of Power. In American Political Science Review, 56: 947–52, and Lukes, S. (1974) Power: A Radical View. London: Macmillan.
Chapter 2
Power in Research, Ethics, Data Protection and Bias
‘The researchers take, hit, and run. They intrude into their subjects’ privacy, disrupt their perceptions, utilise false pretences, manipulate the relationships, and give little in return.’ (Reinharrz, 1984: 95)1 Teaching and learning objectives:
1. To develop an understanding of the asymmetric power relations between researcher and those researched and the scope for harm. 2. To recognise the types of potential harm. 3. To understand the context and scope of the Nuremberg Code. 4. To identify the implications for researchers of data protection legislation. 5. To gain knowledge of the different types of ‘research effects’. 6. To understand how the researcher’s own values can be a source of bias requiring ‘reflexivity’.
Asymmetric power relations in research The researcher (you) choose the topic, research question, design and means of data collection and its interpretation. You will choose sources selectively for accessibility, validity, reliability and accuracy. You will seek to exploit the resources and, for that matter, the prestige and goodwill of the university. Some feminists likened traditional research to a ‘rape model’. As you will see, the major resources in political research are archives and people. The people may be agents, actors or bystanders. You seek their collaboration on the basis of the benefit to society of the research and the benefits of the interviewees (the subjects). But within the survey or interview situation, power relations between the researcher and those researched are unequal. Any process of question-and-answer favours the questioner. In particular, you are likely to be more experienced and can leave the field at the end of the interview or
10
Research Methods in Politics
research project. So the potential for causing real harm, however unintentional or as agent provocateur, is very great.
Types of potential harm Harm may be physical, financial, social and psychological Physical harm can be caused where you reveal the subject’s identity to repressive groups. They may be agents of the state, criminals, terrorists or relatives. You may name subjects or enable others to identify them through details of your journey, fieldwork notes, photographs, or apparently casual, chance conversation with other people. Alternatively, you may promote a new recognition by subjects of circumstances or institutions which they previously accepted as ‘natural’. This new consciousness may lead to conflict between and within groups. Financial harm can be caused where the subject’s identification leads to loss of their job, home, support or reputation. Social harm can be caused where the subject’s position or role within the family and wider social networks is affected adversely. Psychological harm can be caused where you unlock painful memories or feelings of guilt, or create a ‘false memory’. Raising false expectations, altering identities, or imposing western, middle class values can cause great harm. But, conversely, you can also be harmed or exploited (particularly by elites). You may also face dilemmas when, for example, you uncover instances of child abuse, sexual exploitation or conspiracies to carry out acts of criminality or terrorism. What should you do – especially where you have given the subjects undertakings of confidentiality? One dubious practice is to use the ‘don’t ask, don’t tell’ caveat where you warn them that you do not wish to hear of, or see, any illegal activity which you are obliged to report to the authorities. But, in this way, you may allow their abuse of others to go undetected. Research can therefore pose important ethical challenges especially where the researcher and the subjects are members of different cultural groups. Some universal framework and local institutions are desirable.
The Nuremberg Code (1947) The trial of the Nazi war leaders at Nuremberg in 1947 was followed by the trial of scientists, doctors and administrators who had carried out experiments on prisoners in the concentration camps. They were charged with ‘crimes against humanity’. Their defence was threefold: first, that they had not undertaken acts which were contrary to German law at the time; second, that the practices were consistent with the prevailing national ideology that Jews, Gypsies and Slavs were anti-state groups; and, third, that the experiments were justified by their scientific value to mankind.
Power in Research, Ethics, Data Protection and Bias
One example cited in their defence was the immersion of prisoners in cold water to determine the effect of lack of heat, body mass, age and gender on survival times. The defences were dismissed. The argument of scientific value was thrown out on the grounds that the prisoners were not representative of the population: they had been systematically starved and beaten to the point that their very will to survive had been extinguished. The defendants were executed or imprisoned. During the trial, a group of US philosophers, doctors and lawyers was tasked to provide a universal ethical code for medical experiments. The Nuremberg Code was adopted by the Great Powers in 1947. It is reproduced in Box 2.1. The code has since
BOX 2.1
The Nuremberg Code
The Nuremberg Code (1947) Permissible Medical Experiments The great weight of the evidence before us to effect that certain types of medical experiments on human beings, when kept within reasonably well-defined bounds, conform to the ethics of the medical profession generally. The protagonists of the practice of human experimentation justify their views on the basis that such experiments yield results for the good of society that are unprocurable by other methods or means of study. All agree, however, that certain basic principles must be observed in order to satisfy moral, ethical and legal concepts: 1. The voluntary consent of the human subject is absolutely essential . This means that the person involved should have the legal capacity to give consent; should be so situated as to be able to exercise free power of choice, without intervention of any element of force, fraud, deceit, duress, overreaching, or other ulterior form of constraint or coercion; and should have sufficient knowledge and comprehension of the elements of the subject matter involved as to enable him to make an understanding and enlightened decision. This latter element requires that before the acceptance of an affirmative decision by the experimental subject there should be made known to him the nature, duration, and purpose of the experiment; the method and means by which it is to be conducted; all inconveniences and hazards reasonably to be expected; and the effects upon his health or person which may possibly come from participation in the experiment. The duty and responsibility for obtaining the quality of consent rests upon each individual who initiates, directs, or engages in the experiment. It is a personal duty and responsibility which may not be delegated to others with impunity. (Continued )
11
12
Research Methods in Politics
2. The experiment should be such as to yield fruitful results for the good of society , unprocurable by other methods or means of study, and not random and unnecessary in nature. 3. The experiment should be so designed and based on the results of animal experimentation and a knowledge of the natural history of disease or other problem 4. 5.
6. 7. 8.
9.
under study that the anticipated results justify the performance of the experiment. The experiment should be so conducted as to avoid all unnecessary physical and mental suffering and injury . No experiment may be conducted where there is an a priori reason to believe that death or disabling injury will occur; except perhaps in those experiments where the experimental physicians also serve as subjects. The degree of risk should never exceed that determined by the humanitarian importance of the problem to be solved by the experiment Proper preparations should be made and adequate facilities provided to protect the experimental subject against even the remote possibility of injury, disability or death. The experiment should be conducted only by scientifically qualified persons. The highest degree of skill and care should be required through all stages of the experiment of those who conduct or engage in the experiment. During the course of the experiment the human subject should be at liberty to bring the experiment to an end if he has reached the physical and mental state where continuation of the experiment seems to him to be impossible.
10. During the course of the experiment the scientist must be prepared to terminate the experiment at any stage, if he has probable cause to believe, in the exercise of good faith, superior skill and careful judgement required of him, that a continuation of the experiment is likely to result in injury, disability, or death to the experimental subject.
been strengthened by other codes, e.g., the Helsinki Code, to provide additional guidance to scientists. It established four principles for research for whatever purpose: voluntary informed consent; benefit to society; protection from unnecessary harm of subjects; and, the right of subject to end experiments. Today, UK’s ESRC2 and other research funders ask for research proposals to include a statement identifying the specific ethical problems likely to be encountered and how they will be resolved. Ideally, the applicant’s statement should cite the university and department’s family of ethical research codes, and the value of frequent supervision.
Power in Research, Ethics, Data Protection and Bias
Data protection legislation and academic research The experiences of World War II and the post-war fears of totalitarianism (captured in George Orwell’s Animal Farm and 1984) led to the United Nations Declaration of Human Rights and the European Convention of Human Rights & Fundamental Freedoms to protect the individual from government.3 However, fears were heightened again by the increasing use of computers by government and business from the early 1960s and their capacity to hold vast amounts of data on individuals. There was widespread concern for the accuracy and security of that information (especially HIV and other medical records) and its use for purposes for which it had never been intended. Growing concerns for political freedom and privacy led to the convergence of human rights and data protection protocols at supra-national level which were brought into effect as national legislation. The use of personal data for research was the subject of Article 108 of the Treaty of Rome, 1957 and EEC Directive 95/46. Article 6 of the Directive provided derogations for processing and long-term storage of data for ‘historical, statistical or scientific purposes’ subject to ‘compatibility with the original purpose of collection’. Specific issues that member states were asked to resolve were, on one hand, the right of access by individuals to information held on them, and, on the other, restrictions on their use of the data, considerations of national security, and the use of data generated by medical research by private organisations. In Britain, the directive was given effect by the Data Protection Act, 1998. This required ‘all persons who process or use personal data’ in electronic, paper, tape or video formats to follow eight data protection principles. The principles were that all data must be: • • • • • • • •
obtained and processed lawfully obtained for a lawful purpose adequate, relevant and not excessive accurate and kept-up-to-date not kept longer than is necessary processed in accordance with the subject’s rights kept safe from unauthorised access, loss or destruction not transferred outside the European Economic Area (i.e. EU and EFTA) unless safeguards are as adequate.4
S.33 of the Act gives exemptions for personal data kept for ‘research purposes’. S.33(5) allows ‘permitted disclosures’ to others for ‘research purposes’, to the subject or their agents, or at the subject’s request. But data need not be disclosed where it would require ‘disproportionate effort’ or ‘where the risk to the rights and freedoms of subjects is low’. The implications for academic research were considered by the
13
14
Research Methods in Politics
Association of Colleges in 1999 who advised members that: Data collected for the purpose of one piece of research can be used for other research without breaching the regulations on incompatible purposes and can be kept indefinitely. This means that staff and students involved in research can keep records of questionnaires and contacts so that the research can be revisited at a later date, or so that a research project looking into an associated area, could reanalyse the data. However, in order to avoid breaking subject access rules, researchers must ensure that the final results of the research do not identify the individual.
British universities must register with the regulating authority: the Data Protection Registrar. A senior university official must be designated as the Data Protection Co-ordinator and submit annual returns to the Registrar. Generally speaking, the head of department is designated as Data Protection Officer. They may delegate administrative responsibility to a Data Protection Manager. Researchers should notify the Data Protection Co-ordinator of: • • • • • • • •
the title of the research project name and contact address of the researcher data to be collected purpose of the research with whom the data will be shared content format data security (including anonymity).
The implications for the UK-based researcher are that: •
• •
•
data must be held securely. This means that data on subjects and interview transcripts must be stored separately in lockable cabinets or, if held on PCs or laptops etc. passwordprotected personal data on subjects should only be held where essential individuals should be anonymous or not otherwise identifiable; strategies include: – ‘anonymisation’, e.g., Mr. A said … – positional descriptions, e.g., minister A said … – deception, e.g. Ms A said … (where sex is not relevant) – replacing place names by generic pseudonyms, e.g. Northtown – changing descriptions slightly personal data should not be disclosed, transferred or copied (including by email), especially outside the EEA, without prior agreement by the recipient to confidentiality and authorisation
Power in Research, Ethics, Data Protection and Bias •
potentially reusable data should be archived centrally, e.g. at ESRC’s archive at The University of Essex.
The penalties for negligence may include fines and the withdrawal of funding support for the research. The greatest penalty will be the loss of the researcher’s, department’s and university’s reputation and loss of future research opportunities as subjects refuse to co-operate. The ‘research effects’ You will inevitably change your behaviour in the presence of power. You are unlikely to maintain the same tone of voice and general manner in all situations, for example, teachers do not speak to their head teacher in the same way that they address schoolchildren. Your actual language and tone will vary according to whom, when and where you are speaking. In the same way, researchers and their subjects will also change their behaviour, especially in face-to-face situations. This change of behaviour is termed the‘research effect’. The main types of research effect are the Hawthorne effect, the Placebo effect, the John Henry effect, the Halo effect, Experimental effects, the Pygmalion effect and the Peacock effect. Hawthorne effect The Hawthorne effect takes its name from the Hawthorne Plant of the Western Electric Company in Cicero, Illinois. There, in the relay assembly room, engineers experimented in 1927 with the effects on productivity on the introduction of improved lighting, mechanical ventilation and humidity control. Separate workspaces and groups were designated as experimental or control areas,where the lighting conditions etc. were varied in the experimental areas and unchanged in the control areas. The engineers were delighted when productivity in the experimental areas increased as lighting and ventilation were improved. However, when sceptical colleagues challenged the experimenters to reduce lighting and ventilation, they found that productivity remained high. And it had also increased in the control area. The help of the Harvard Business School was sought. The senior investigator, Elton Mayo wrote that: A highly competent group of Western Electric engineers refused to accept defeat when experiments to demonstrate the effect of illumination on work seemed to lead nowhere. The conditions of scientific experiment had apparently been fulfilled – experimental room; control room; changes introduced one at a time; all other conditions held steady. And the results were complexing: Roethlisberger5 gave two instances – lighting improved in the experimental room, production went up; but it also rose in the control room.
15
16
Research Methods in Politics The opposite of this: lighting diminished from 10 to 3 foot-candles in the experimental room and production went up again; simultaneously in the control room, with illumination constant, production also rose.6 (Mayo, 1949: 60)
The explanation given by Mayo was that productivity had increased in both the experimental and control workspaces because the workers there were the focus of the researchers’ attention. The research was subsequently extended until 1932. Mayo and his colleagues also argued that the research showed that aptitude is only weakly related to performance, and that the workplace is a social system in which informal organisation of work-groups and their attitudes are critical to production. In effect, the work-group decided what was a ‘fair-day’s work’. Mayo termed this behaviour change as the ‘research effect’. It has become better known as the Hawthorne effect. However, his explanation has been challenged. Some industrial psychologists argue that Mayo’s findings were wishful thinking: Mayo and his colleagues were biased in their interpretation by the researchers’ promotion of the (softer) human relations school of industrial relations (which favoured humane treatment of worker and workplace democracy) in the face of the (harsher) scientific management prevailing in the US.7 While the studies were extensive, the samples were relatively small and not representative. The generalisability of their findings could therefore be challenged. By re-analysing the data, critics argued that changes in performance should better be attributed to the Depression, the substitution of two particular workers and the threat of disciplinary action.8 Whether or not this challenge to the Hawthorne folklore can be sustained, the episode does illustrate the potential impact of the researcher’s own values on the research proposal, the collection of data and its analysis (which is considered later in this chapter). Placebo effect The Placebo effect pre-dates Hawthorne. The term is derived from the Latin word ‘placebo’ meaning ‘I shall be pleasing’ which was used by physicians to describe inert remedies to which some patients responded by meeting their psychological needs for medical attention. The term has been extended to describe the practice in medical drug trials, and other experiments, of giving placebos to the control group so that the real efficacy of the trial drug can be calculated. In these cases, random, double-blind trials are conducted in which the patients and medical observers are unaware who is receiving the trial drug and who is receiving the placebo. John Henry effect The John Henry effect is the converse of the Hawthorne effect. It happens when the control group compete with the experimental group and achieve greater changes in behaviour or outcomes. It can be seen as a super-placebo effect.
Power in Research, Ethics, Data Protection and Bias
Halo effect The Halo effect is the response of subjects to novelty where the press and other media (including advertising) have created an expectation. Experimental effect The Experimental effect is similar. Here expectations are raised by the researcher. Pygmalion effect The Pygmalion effect is a sub-type of expectancy change. It can be a form of ‘selffulfilling prophecy’. It is most often cited in education where the teacher’s expressed expectations of pupils, whether high or low, can greatly affect their performance. It may also occur where, in an interview, the questioner implies that specific responses are to be expected because they reflect prevailing norms, i.e. male, white, middle-class values. Peacock effect The Peacock effect is the name given to the behaviour of some male birds among females when they display their plumage or attributes. Some young men may behave similarly in the presence of young women. How can the impact of Hawthorne and other research effects on research be reduced? Four strategies can be adopted: • • • •
anticipating and discounting it reducing expectations adopting a longer timescale (so the subjects become inured to observation) adopting covert observation (however, covert observation is contrary to the principles of voluntary, informed consent and control by the subject). This raises ethical dilemmas.
Research effects will inevitably bias the research. Bias is essentially a predisposition or prejudice for or against a theory, person, group or institution which may distort or skew cognisance and interpretation of phenomena. However, your greatest source of bias may be your very own researcher’s bias.
Researcher’s bias You are unlikely to begin your with an open mind. Your choice of research topic, question and starting hypothesis will reflect deep-seated values and prejudices.
17
18
Research Methods in Politics
The greatest danger is of ‘wishful thinking’ or ‘self-fulfilling prophecies’ in which you ‘see what you want to see’ and infer conclusions. But, on the other hand, your biases can also motivate and sustain the research. Remember Beatrice Webb’s advice that: Finally, [the researcher] must realise that he is biased, and somehow or other he must manage to discover this bias … ‘Know thyself’ is the maxim uniquely imperative on the investigator of social institutions. For the greatest obstacle to the advancement of knowledge – is an obstacle in the mind of the student, the presence of biases. (Webb, B., and Webb, S., 1975: 31 and 44)9
Minimising bias requires reflexivity – intellectual self-awareness through selfexamination. You should record the outcome of this examination at the beginning of the research report so that the reader can learn from the outset ‘where you’re coming from’ and discount the bias. But then your readers will also have their own biases and pre-judge the text accordingly.
Questions for discussion or assignments
1. Identify the extent of your own prejudices using the Table 2.1 below. You can copy the full table from the web-site www.sagepub.co.uk/pierce. Add your own categories. Distinguish between your immediate (private) feelings and your (public) thoughts. Try to identify their source. Are they derived from others (e.g. parents or friends), intellectual inquiry or experience? How have they changed over the years? Why? 2. You have been asked to undertake a survey of electors to find out whether nonvoters in general elections differ in their (other) characteristics from voters and whether they are serial abstainers. What ethical difficulties do you anticipate? How would you tackle them? What are your own starting prejudices? 3. In the course of your fieldwork, you discover a continuing case of child sexual abuse. You have given an undertaking of confidentiality. What should you do? 4. Obtain a copy of your own department’s code of research ethics. What criticisms would you make? How should it be improved? OR, if a code is ‘pending’, write a draft.
Power in Research, Ethics, Data Protection and Bias Table 2.1 Prejudices: a self-completion proforma To what extent are you prejudiced in favour or against the groups listed below? University students University teachers University administrators Social scientists Scientists Doctors Police officers Members of the armed forces Unemployed people Disabled people Conservative party Liberal-Democrat party New Labour Old Labour Green party British National Party Communist party Republican party Democrat party President George W Bush Prime Minister Tony Blair Prime Minister Gordon Brown Bill Clinton Hillary Clinton Iraq war/invasion UN USA EU UK Scots people Irish people English people The British Empire Imperialism Freedom
Strongly Against
Against
Neutral
In Favour
Strongly in Favour
−2
−1
0
+1
+2
(Continued )
19
20
Research Methods in Politics Table 2.1 Cont’d To what extent are you prejudiced in favour or against the groups listed below?
Strongly Against
Against
Neutral
In Favour
Strongly in Favour
−2
−1
0
+1
+2
Democracy Liberty Terrorism/armed struggle Atheism Agnosticism Deism Roman Catholicism Protestantism Islam Buddhism Hinduism Secularism Nationalism Patriotism Conflict Competition Markets Regulation Trade unions Children Middle-aged people Older people Young adults Marriage Divorce Other people Yourself
FURTHER READING Berg, B. (2001) Chapter 3: Ethical Issues. In Qualitative Research for the Social Sciences. London: Allyn & Bacon. pp. 39–65. Berg reviews the historical development of ethical concerns in research from a US perspective with particular emphasis on the role of Institutional Review Boards (IRBs). He identifies how and why disadvantaged groups are more accessible to researchers than privileged elites. Burnham, P., Grillard, K., Grant, W. and Layton-Henry, Z. (2004) Chapter 11: Ethics and Political Research. In Research Methods in Politics.
Power in Research, Ethics, Data Protection and Bias Basingstoke: Palgrave Macmillan. pp. 250–69. The authors provide a synoptic review of the specific ethical issues in Politics research including the role of gatekeepers, fraud and professional codes of conduct. They compare the codes of the Social Research Association, American Political Science Association and the UK Political Science Association (PSA). They also raise the ethical issue of whether an institution should impose its own code on researchers. De Vaus, D.A. (2001) Chapter 19: Ethics in survey research. In Surveys in Social Research. London: Routledge. pp. 330–50. The author reviews the principles of ethical research with particular emphasis on the decisions required by ‘informed consent’ and duties and responsibilities to sponsors, funders and research colleagues.
Notes 1 Reinharrz (1984) p. 95 cited by Oakley, A. (2000) Experiments in Knowing: Gender and Methods in the Social Sciences. Cambridge: Polity Press. pp. 37–8. 2 The Economic and Social Research Council (ESRC) funds research and training in social and economic issues. It claims an international reputation both for providing high-quality research on issues of importance to business, the public sector and government and for commitment to training excellence, which produces world-class social scientists. ESRC is an independent organisation, established by Royal Charter, but receives most funding through the Government’s Office of Science and Innovation. Its budget of £181 million (2007/2008) funds over 2,500 researchers in academic institutions and policy research institutes throughout the UK. 3 Significantly, whilst the British (Labour) government supported the application of The Convention to recently-liberated, continental states, it resisted its application to the UK on the grounds that history had shown that it was unnecessary and that it might impede the government’s programme of post-war regeneration including large-scale nationalisation and economic planning. The Convention was finally only adopted by the Human Rights Act, 1998. Jacobs, F.G. (1978) European Convention on Human Rights. Oxford; Clarendon Press. p. 214. 4 EFTA: European Free Trade Area: formerly a seven-state free trade area now reduced to the non-EU states of Iceland, Liechtenstein, Norway and Switzerland. 5 Roethlisberger, F.J. and Dickson, W.J. were Mayo’s principal associates. 6 Mayo, E. (1949) Hawthorne and the Western Electric Company. London: Routledge.:p. 60 7 Scientific management was originated in the 1890s by Taylor, W.F. who argued that productivity of workers could be increased by the marriage of science, engineering and classical economics to achieve greater organisation, specialisation, de-skilling of tasks and the use of bonuses. It is associated with ‘time-and-motion’ regimes. 8 Franke, R.H. and Kaul, J.D. (1978) The Hawthorne experiments: first statistical interpretation, American Sociological Review, 43: 623–43. 9 Webb, S. and B. (1932/1975) Methods of Social Study. LSE: Cambridge University Press. The Webbs were founder members of the Fabian Society and the London School of Economics.
21
Chapter 3
The Philosophy and Principles of Research
Teaching and learning objectives: 1. To develop an understanding of the main philosophic and theoretical principles underlying research approaches including positivism, empiricism, behaviouralism, naturalism, feminism and postmodernism. 2. To introduce the concept of ‘causality’. 3. To introduce and distinguish between inductive, deductive and grounded research.
Introduction Qualitative and quantitative methods are regarded by many practitioners as simply alternative sets of designs and techniques for carrying out research. However, both qualitative and quantitative methods stem from fundamentally divergent ontological and epistemological paradigms having their roots in philosophy. Ontology is, simply expressed, the branch of philosophy (‘thinking about thinking’) devoted to the nature of being. It considers such questions as: do minds exist? In contrast, epistemology is the branch concerned with theories of knowledge. Theory is, simply stated, a statement of general principles of the underlying relationships in phenomena or events. Theory may be expressed as laws, propositions, arguments or hypotheses (tentative explanations). Historically, theory has generally been descriptive in terms of describing and, therefore, explaining relationships. Some descriptive theories have been elevated to the status of laws (for example, Newton’s law of universal gravitation) when they are accepted as having been universally verified by observation. Newton’s law was subsequently superseded by Einstein’s theory of relativity. The title ‘law’ is therefore rarely given to theories. Alternatively, theory may be normative where it proposes what ought to be the relationship. For example, the theory of egalitarianism is that all people should be treated equally regardless of origin or circumstances. A proposition is an unproven, generalised explanation.
The Philosophy and Principles of Research
An hypothesis is a specific proposition which is presented for testing by research. It can be derived from other theory or a researcher’s ‘hunch’ (informed guess based on theoretical insights or observation). Arguably, the essential operational distinction between quantitative and qualitative research is whether the researcher is a neutral, objective observer studying a person as an object, or as an independent variable engaging subjectively the person as a subject or client. Positivism and naturalism are prototypical of the competing ontologies. Positivism Until the Age of Enlightenment (Eighteenth century), significant events were attributed to God, his miracles, or magic. So Hobbes and other philosophers were forced to invoke God and king to legitimate their writings. While David Hume challenged this practice, the French philosopher, Auguste Comte (1798–1859) while developing the ideas of Saint-Simon is credited with ‘secularising philosophy’ and developing a ‘religion of humanity’.1 He sought to place the French Revolution – whose Terror undermined British support for the Enlightenment – into an historical context as an intermediate phase of the development of knowledge. He argued that all societies were fated to move from a theological stage of ‘fictitious knowledge’ (in which all otherwise inexplicable phenomena were attributed to spiritual forces) via an intermediate, metaphysical stage to a positive stage. The underlying beliefs of the theological stage were medieval faith and custom in which the family was the social base.2 Those of the metaphysical stage were Enlightenment philosophy, the ‘scientific revolution’ and the nation state. The state of knowledge of the positivist stage was scientific when rational logic and humanity prevailed. When these purer forms of explanations were reached, there would be no miracles or magic: phenomena would be explained by physical causes. Explanation would be confined to verifiable and measurable correlations between phenomena which revealed universal laws. The essence of positivism is the application of natural science methods (i.e. those used in astronomy, maths, physics, chemistry and quantitative biology) to social sciences.3 Comte advocated a value-free form of enquiry which would eliminate any bias. However, his work has been criticised for accepting the dominance of white society and providing opportunities for social engineering.4 Positivism would therefore enable Man to control all other forms of nature. In its time, positivism was ‘progressive’. It became the dominant paradigm in the social sciences until the 1960s when it became increasingly a pejorative term (i.e. a term of abuse).5 Comte’s work was developed by John Stuart Mill, Emile Durkheim, Ernst Mach (whose name is commemorated in aerodynamics) and the interwar ‘Vienna Circle’ of ‘logical positivists’ including Wittgenstein and Popper (author of The Poverty of Historicism) and other, mainly Jewish, mathematician-philosophers. These positivists
23
24
Research Methods in Politics
sought to apply scientific measurements and objectivity to generate causal laws. They relied on specific assumptions about social reality. Social reality was ordered rather than random: cosmos rather than chaos. This ordered system was based on the rationality of self-interested individuals who, ceteris paribus (‘other things being equal’), behaved in the same way. The reasoning capacity of individuals (i.e. their rationality) enabled them to distinguish fact from fiction. So positivist explanations could only be entirely logical and derived from the facts observed. Observable facts were entirely different and a better foundation for understanding behaviour than abstract theories, ideas and feelings. Positivist researchers are entirely objective and without political or religious bias. It was the application of positivism which enabled the study of social life to claim the title of social science and the study of politics to legitimate nineteenth century claims to the title ‘Political Science’.6 Popper’s major contribution to positivism was to replace the test of verifiability (an early, central doctrine of the Vienna Circle) by falsifiability, falsification or fallibilism. No theoretical generalisation could be finally verified as truthful or certain: a single unexpected outcome could challenge an hypothesis. Mere prima facie (‘at first sight’) proof was insufficient: absolute certainty was essential. This could never be attained. A researcher could never claim or show that they could not be wrong. Furthermore, an hypothesis could not be accepted until the ‘alternative hypothesis’ were dismissed. After initial claims to inductivism, positivism became essentially deductive (moving from the general to the particular), hypothesis-testing research which tended to rely on numeric data to make objective claims. This change reflected the recognition that researchers could never collect the universe of information potentially available. Furthermore, selecting a research question or topic implied a theoretical starting position; the researcher’s mind was rarely a ‘tabula rasa’ (Locke’s empty slate). Subsequently, positivism has been criticised for adopting a ‘mechanical model’ of man. More recently, it has also become criticised for being heavily gendered, i.e. perpetuating the historic dominance of men and the subordination of women. Types of positivism include logical empiricism, behaviourism and behaviouralism.
Empiricism Empiricism is, simply stated (and at its most extreme), the doctrine that the only source of real knowledge (‘the facts’) is experience gained by the senses through observation and experiment. Only the observable can be observed and measured. Emile Durkheim (1858–1917) is widely regarded as a foremost empiricist in his advocacy of the scientific study of social facts (objects of analysis particular to a social science discipline). The empirical research for which he is best known is his comparative study of suicide to determine its causes (Suicide, 1897). He regarded suicide as the ultimate outcome, an event unique to the individual. Using official records and by analysing these for ‘empiric regularities’ (patterns and correlations),
The Philosophy and Principles of Research
he sought to replace the traditional explanations of moral weakness or sin, by societal explanations. He identified three common factors as potentially causal: religious denomination; membership of rural or urban society; and existence of dependents. These explained four main types of suicide: altruistic; egotistic; anomic; and fatalistic. He later developed a functionalist view of religion whereby ministry reproduced institutions, e.g. marriage. Behaviouralism must be distinguished from Behaviourism. Behaviourism Behaviourism is a school of psychology associated with Harvard professor B.F. Skinner (1904–90). He was the architect of the controllable ‘Skinner box’ and ‘programmed learning’. Its key tenet is that only observable behaviour may be scientifically studied. So, while behaviour may include speech, intentions and mental processes must be disregarded as ‘unobservable’. Behaviourism is associated with animal experiments and the application of causal relations, e.g. Pavlov’s dogs, to operant conditioning and behaviour modification. This emphasis on experimentation and statistical analysis led some empiricists to subscribe to the ideas of eugenics (the science of heredity associated with proposals for scientific breeding to reproduce ‘higher’ intellectual characteristics and to exclude genetic weaknesses) promoted by Darwin’s cousin, Galton (‘whenever you can, count’). Indeed, the UCL professor, Professor Spearman, who devised the widely used Spearman’s coefficient of correlation, r, was a leading eugenicist (and early socialist). Today, Skinner’s contribution has become superseded by greater support for ‘cognitive learning’. Behaviouralism Behaviouralism (note special spelling) was the term adopted by social and political scientists for the application of positivism and empiricism to test and extend explanatory theory at the level of state, party or other group, or individual. In particular, the new approach offered a progressive approach to those academics critical of the discipline’s obsession with the ‘old institutionalism’ of the study of the structures of government. It became the dominant approach of politics – especially in the US – in the post-war period. The approach is characterised by its greater use of: theoretically derived hypotheses; sampling; early use of computers for data collection and analysis; statistics; and belief in replication. Its principal exponent remains Robert Dahl whose 1950s study of New Haven, Connecticut (described in Who Governs?) remains the classic example of behaviouralist research. It demonstrates the strengths and weaknesses of the approach.7 Dahl sought to explore two, rival hypotheses: the elite theory view (‘of European origin’) that a unified oligarchy governed US cities (as argued by Hunter from his study of Atlanta and C. Wright Mills’ Power Elite); and the (alternative) pluralist
25
26
Research Methods in Politics
theory that urban political systems were polyarchic. In his introduction, Dahl noted the extreme variations of income and educational attainment, property ownership and material environments in the US and asked: ‘Given the existence of inequalities like these, who actually governs in a democracy?’. The research problem which Dahl faced was how to observe and measure power and influence. He chose what he called an ‘eclectic approach’ which spread the risk of unreliability of data over six separate sources of information and methods of analysis, namely: 1. To study changes in the socio-economic characteristics of incumbents in city offices in order to determine whether any rather large historical changes may have occurred in the sources of leadership; 2. To isolate a particular socio-economic category and then determine the nature and extent of participation in local affairs by persons in this category; 3. To examine a set of ‘decisions’ in different ‘issue-areas’ in order to determine what kinds of persons were the most influential according to one operational measure of relative influence, and to determine patterns of influence; 4. To survey random samples of participants in different sub-areas in order to determine their characteristics; 5. To survey random samples of registered voters in order to determine the characteristics of those who participate in varying degrees and in varying ways in local affairs; 6. To study changes in patterns of voting among different strata in the community. (Dahl, 1969: 331)8
The cornerstones of his research were observation of ‘what actually happened’ in terms of the decisions made at meetings, and interviews with business and political leaders. Dahl concluded that different, separate groups dominated different policy areas. Furthermore, power, in terms of decision making, was openly exercised by elected officials who were accountable and thereby sensitive to public opinion. Therefore political power was not organised into a single pyramid with a ruling elite at its apex. Instead, political power in New Haven was marked by pluralism, openness and competition. But Dahl’s conclusions were confounded by the riots which destroyed inner areas of New Haven in 1965. This raises wider questions of research: are answers the results of the methodology used? Dahl’s approach was famously criticised by Bachrach and Baratz (after Schattschneider) who argued that there was a second, unobservable dimension of power: the power to control the agenda (Bachrach and Baratz, 1962).9 So, Dahl’s observation that the officer’s recommendations to the City Planning Commission were generally approved overlooked the possibility that other reports were excluded from the
The Philosophy and Principles of Research
agenda. Critics argued that potentially-controversial reports were withdrawn by non-elected officials at their private, agenda-setting meeting with the senior officials. Alternatively, other recommendations were never aired because they would be likely to be rejected and harm relations between commissioners and advisers.10 Stephen Lukes argued (drawing on Gramsci’s writings on hegemony) that there was a ‘third dimension of power’: the most supreme and insidious use of power which prevents grievances by shaping people’s perceptions, cognitions and preferences in such a way that they accept their role in the order of things. (Lukes, 1974: 24).11
Furthermore, the ‘counterfactual’ (what you would have expected to happen but didn’t) was - like Sherlock Holmes’ ‘dog that didn’t bark’ in Conan Doyle’s Silver Blaze – evidence of the exercise unseen of third-dimensional power.12 But the de-bunking of Dahl’s empiricist, behavioural model was not accomplished alone by new, theoretical argument. As a noted pluralist and supporter of the ‘American dream’, he has been criticised for wishful thinking and ‘seeing what he wanted to see’. More recently, a systematic re-analysis of Dahl’s data has suggested serious methodological and interpretative weaknesses. These were that: the survey samples were not representative; Yale University was a much greater power-player than (its professor) reported; he too readily accepted the ‘truth’ of interview statements by the mayor and others which other documentary evidence contradicts; and he was not sufficiently sceptical (Domhoff, 2005).13 Behaviouralism has become widely criticised in recent years. Three main criticisms have been made. First, the emphasis on deduction meant that research became ‘boxed-in’ from developing new theory. Second, it tended to concentrate on more readily measurable phenomena rather than other, more relevant topics (Sanders, 2002: 52).14 And, third, important theoretical concerns and nuances were unexplored or overlooked.
Naturalism Naturalism has two, almost uniquely, contradictory conceptualisations. The first, the minority view held by Giddens, is that naturalism refers to the adoption in the social sciences of models of inquiry from the natural sciences. So naturalism is effectively a new, non-pejorative name for positivism. But the dominant view is that naturalism rejects positivism in favour of a free-standing method of inquiry which is essentially humanistic and hermeneutic (after Hermes, interpretative) and concerned with social meanings, actor’s beliefs, motives, purposes and reasons, which lead
27
28
Research Methods in Politics
to ‘social actions’, rather than frequency. While positivism emphasises scientific, controlled, replicable experimentation, naturalism seeks to study everyday life in naturally-occurring situations. Naturalism is experiential: researchers seek to see the world through the eyes of the ‘subject’ and to understand what people feel, interpret and do. Moods, ideas, identity and beliefs are the stuff of naturalistic research. People have a capacity for language and use it to construct and reproduce their social worlds. Society, unlike inanimate material, is always changing. But individuals are neither bound by universal rules nor independent of their contexts; they are structured by institutions. Because there can be no universal organising principles, then there can be no research hypotheses. Naturalistic research is primarily inductive. The five main principles of naturalistic research are therefore: •
meeting subjects in their own habitats of home or work, etc
• • •
asking them their views and meanings
•
asking them in such a way that they can reply in their own language, dialect and words achieving depth of inquiry addressing their social context and situation.
Feminism Feminism offers another, additional critique of positivism: ‘that the concern with measurement and control underlying quantitative, experimental ways of knowing developed is part of the same social process which enables men to exercise power over women’ (Oakley, 2000: 16).15 Feminists point to the historical association between quantification and masculinity. Social science defined society in terms of male values which reflected historic power relations in society where women were simplistically categorised. Feminists argued that the relation between researchers and researched reproduced the power relations between men over women. They argued that relations between researchers and subjects should be consensual and that greater weight should be given to ethical considerations. The subjects should be empowered by the research process. Research should therefore be an emancipatory project: ‘research for women not about them’. Feminist perspectives of research were strengthened by increasing public opposition to experimental research, especially involving animals, to which women were the main opponents. But the eminence of qualitative methodology has been marked paradoxically by the divergence between those traditional and new feminists. The traditionalists continue to promote exclusively qualitative method. The new feminists (and others) argue that the real conditions of women and other exploited, powerless groups can be more effectively exposed by greater use of quantitative data, especially by harder, objective data on the distribution of low pay, poverty and unwaged work (especially parenthood).
The Philosophy and Principles of Research
Prototypical examples of new feminist research in Politics are the published works of Norris (1995, 1999) and Lovenduski (1995).16
Post-modernism Post-modernism initially developed in the 1970s from a critique of modernism and positivism by intellectuals in the arts and architecture disenchanted by the failure of modernity. This critique was given a stronger intellectual core by social scientists, drawing on existentialism, nihilism and anarchism, who argued that the founding concepts of the Modern Movement, industrialisation, urbanisation and the individual as producer, were fast becoming obsolete. They saw instead a new, post-industrial age in which the old divisions between economic classes had become eroded by new lifestyles in the ‘information societies’ when the individual had become a consumer able to exercise choice in the market. Post-modernism challenged universal truths and the meta-narratives of Marxism, liberalism and other utopian ideas. There is no certainty: any knowledge is local, contingent and partial. The scientific researcher is unable to be objective. The individual is not rational. Language is not neutral: meaning is modulated by dominant interests. There is no division between the arts and social sciences. The complexity of the social world cannot be captured by research or causality deduced. Intuition, emotion and imagination should not be discouraged. Instead, research should engage, stimulate and evoke (Neuman, 2003: 89).17 On the other hand, post-modernism in the discipline of Politics has been criticised as relativist, pessimistic and anti-progressive (Heywood, 2001: 101–03).18 The significance of postmodernism for research in Politics has been the impetus given to the closer study of ‘talk and text’ and ‘interpretative social research’ with its emergent disciplines of hermeneutics, ethnography and qualitative sociology. It therefore favours participative fieldwork and real involvement of researchers with their subjects. It revives the tradition of ‘trying to see the world through the subjects’ eyes’ by verstehen (empathetic understanding) advocated by Max Weber (1864–1920).
Causality Post-modernism does not entirely dismiss the concept of causality (‘cause and effect’ or causation) as a basis for explanatory theory: it argues that the world is chaotic, rather than a closed system, in which causes and consequences cannot be identified. Many post-modernists therefore adopt functional, structural, historical or interpretative explanations where, for example, a functional explanation of poverty might be because it ‘works’. But causality is central to all the main approaches used
29
30
Research Methods in Politics
in Politics research where causes are sought to explain effects. In emergent, everyday English, causes are often referred to a ‘drivers’. But, in academic research, causes and effects are generally called variables where: • •
causes are termed independent variables or causal events effects are termed dependent variables, consequent events, or outcomes.
Cause and effect appears straightforward. For example, from the report or observation: A knocks over B, then you can claim that A is the cause of B’s falling over. But how do you know that A is not responding to B’s behaviour or seeking to protect C? The social world and political world are incredibly complex. Unlike, say, nuclear physics, the subjects of Politics research are sentient human beings exercising some agency (personal autonomy) within a structure (rules of the game). The degree of agency will vary with the individual context. Assume, for example, that a university teacher, A, appears increasingly exasperated by student B. A raises a clenched fist directly to B’s nose. What will happen? Well, you don’t know what will happen. You can only speculate. A may or may not hit B. B may adopt a passive response and accept ‘whatever happens next’. Or B may take pre-emptive action by either moving or striking first. They may strike back if hit. But the choices of A and B are both contingently structured by the teacher-student power relation, gender relations, academic regulations and criminal law.19 There is therefore a substantial risk that the researcher may identify, incorrectly, causes merely because they appear to immediately precede apparent effects. This fallacy is termed post hoc, ergo propter hoc (‘after this, therefore because of this’). This problem was considered by the Scottish Enlightenment philosopher, David Hume (1711–76). He dismissed the claim that causes necessitate the events that are their effects. He also refuted the argument that cause could be determined by reason alone. The belief that flames are hot can only be based on sensory experience, custom and practice. He developed eight ‘general rules’ which must be met before ‘probable causation’ could be claimed. 1. The cause and effect must be contiguous in space and time. 2. The cause must be prior to the effect. 3. There must be a constant union betwixt cause and effect. ’Tis chiefly this quality, that constitutes the relation. 4. The same cause always produces the same effect, and the same effect never arises but from the same cause … 5. Where several different objects produce the same effect, it must be by means of some quality which we discover to be common amongst them. 6. The difference in the effects of two resembling objects must proceed from that particular, in which they differ … 7. Beware not to draw such a conclusion from a few experiments …
The Philosophy and Principles of Research 8. An object which exists for any time in its full perfection without an effect, is not the full cause of that effect …20
After Hume, five precise conditions have been specified for causation: covariation; constancy of association; cause must take place before the effect; independent and dependent variables must be discrete (rather than continuous); and non-spuriousness.21 Covariation describes the phenomenon where two variables appear to change at the same time, for example, unemployment and poverty appear to increase or decrease at the same time. The covariation can be plotted graphically and tested for association (the degree of consistent relationship) by the statistical test of correlation. However, ‘association does not necessarily equal causation’.22 And association does not identify the independent and dependent variable. There are several ways that variables may change at the same time. In social science, the cause is normally indicated as x (from the Greek xenon meaning unknown) and the effect y. The causal effect of independent variable x, on the dependent variable y, can be represented as shown by the diagram: x→y However, another, independent variable, x2 may also be affecting y by the process termed addivity: x1 → y ↑ x2 Alternatively, x2 may be affecting x1 directly by the process of intervention: x1 → y ↑ x2 Or x2 may act as a type of catalyst on x1 and y. This is termed interaction: x1 → y ↑ x2 Finally, the association between x1 and y may be a spurious relation where they are both being affected by x2 : x1
y
x2
31
32
Research Methods in Politics
In practice, few causal relations in Politics research involve only one cause. Most effects have many causes; and these causes will affect many effects. There is also the argument that, even so, causes are only the tangible manifestation and agent of the underlying distribution of power within society: The flaw in the pluralist heaven is that the heavenly chorus sings with a strong upperclass accent. (Schattschneider, 1961: 35).23
Hume explained cause and effect in terms of billiard balls in which moving ball A hits static ball B which is then propelled along the ‘green baize’. However, today we would claim that, in Politics, the baize is not level and smooth but furrowed and pitted, structured by institutions that affect the trajectory of the ball and frustrate the trajectories of A and B.
Inductive, deductive and grounded research These methodologies generally follow the explanatory theory adopted. Inductive research has been defined as: ‘the inference from the particular to the general’ (Honderich, 1995: 403).24 It is used to ‘build’ theory and is traditionally the methodology used in the natural sciences where observation of phenomena is followed by a search for new explanatory theory, e.g. the behaviour of superconductors at low temperatures. Inductive research can also be adopted to find answers and then explanations for questions of ‘what if ?’ For many years, ‘the problem of induction’ attracted the attention of philosophers like Hume who argued that all inferences from observation relied on the premise of the future always resembling the past. But the roots of modern-day induction lie in the philosophy of idealism which sees the mind as the only true source of ideas. The methodology is widely used by sociology researchers where they are trying to understand the social meanings (beliefs, attitudes, and reasons) of their subjects exhibited through their behaviour. These researchers adopt Weber’s verstehen and interpretative methods. The idealised inductive research process may therefore be illustrated as: 1. 2. 3. 4. 5. 6. 7.
select the topic choose the research question collect data (through repeated observation of phenomenon) interpret the data develop a theoretical explanation of data collect modified data to test initial explanation reiterate (repeat) until conclusions can be made which best meet the test of falsifiability.
The Philosophy and Principles of Research
Conversely, deductive research can be explained as seeking inferences from the general to the particular. It is the methodology adopted by positivist, empiricist and behaviouralist researchers. While this methodology is associated with quantitative enquiry, it is widely used in Politics research (especially by undergraduates, and ‘taught’ graduates) to apply and test the application of theoretical models to new (often overseas) contexts. Examples include the application of policy transfer theory to the transfer between developing states, and Stone’s urban regime theory to UK cities. The idealised model is therefore: 1. select the theory, e.g. urban regime theory 2. frame the research question: e.g. ‘can the regeneration of UK cities be attributed to urban regimes?’ 3. state the hypothesis: (This might be:‘No: in practice many regimes have been concocted of the “usual suspects” to meet government criteria for aid; other factors are more significant’.) 4. collect selected data relevant to the research question: e.g. urban regimes in successful and unsuccessful urban regeneration 5. analyse and interpret the data 6. confirm or infirm (disprove) the hypothesis.
However, in practice, the distinctions between deductive and inductive research are blurred. In particular, the claim by inductive social researchers that their research is a-theoretically grounded may be questioned. Can social scientists ever really ‘rid their minds’ of the grand theories that they acquired in their earlier years? If they really had no preconceptions or hunches, then how or why did they come to select the phenomena for study? Can a young, white, male, middle-class, western social researcher ever really ‘see the world through the eyes’ of an old, white, workingclass western woman let alone a Third World subject speaking an entirely different language? Grounded research was proposed by Barney Glaser and Anselm Strauss of Columbia University in the 1960s as a new ‘third way’ derived from their critique of the dominant tradition of deductive research in US sociology (Glaser and Strauss, 1995).21 In their judgement, US social researchers were pre-occupied with testing hypotheses derived from the ‘grand masters’ of sociology: Marx, Weber, Tonnies and Durkheim. Consequently, new researchers were not developing new theory to explain the new social dynamics of a fast-changing, urban society. Most research added only detail or embellishment. Social research had become insular and was effectively overlooking new concerns. They were also critical of some of the claims of a-theorocity of inductive researchers. At the same time, they were critical of the cost and resource demands of large-sample quantitative data. They wanted to
33
34
Research Methods in Politics
develop a new approach which harnessed the skills and other resources of social researchers to generate new, exciting, relevant and imaginative research. The solution they devised was grounded research where the findings would be entirely grounded in the data collected. The essence of grounded theory lay in the selection by researchers of almost semi-randomised variables (for example, by examining the contents of the ‘returned book trolley’ in the university library). Rather than hypothesise specific causal relations, the researchers should first seek out evidence of any relation. The initial sample and data would be limited. Their findings would be grounded in that data, tested and re-formulated in further samples until theoretical saturation were reached, when the additional data failed to reveal any further significant findings. Importantly, ‘less was more’. It would no longer be necessary to collect, at vast financial and opportunity costs, all the data before analysis were attempted (with the temptation to overstate the findings to justify the costs). One noted variant of grounded research is its application to deductive research by adopting small, iterative samples to test hypotheses. However, despite the appealing propositions put forward by Glaser and Strauss for ground-breaking ‘quick and dirty’ research, the criticism remains that, in practice, the selection of variables for scrutiny invariably reflects the values and theoretical biases of the researchers. Additionally, how can the researcher claim that the process is a-theoretical without reviewing existing theory and the associated literature? How can they know that their research is new and original? The idealised grounded research model is: 1. 2. 3. 4. 5. 6.
select the variables collect data from a small sample advance an explanation test against larger samples revise the initial explanations reiterate until theoretical saturation is reached.
Grounded research has great appeal to most British Politics researchers because it avoids number-crunching and can be cited to justify a ‘thin’ theoretical review and limited immersion in the field. Most researchers develop variants appropriate to their research question.
Questions for discussion or assignments
1. What is positivism? What are its strengths and weaknesses? 2. Critically compare positivism and naturalism.
The Philosophy and Principles of Research 3. Explain Popper’s concept of ‘falsifiability’. In what circumstances might it be employed in Politics research? 4. ‘Grounded research’: another useless American import? Defend. 5. You have been asked to explore the relationship between social class, party identification and turnout in the UK. Which research methods would you adopt? Why? 6. What is verstehen? Is it ever possible? In what circumstances is it most likely and least likely to be achieved?
FURTHER READING Dahl, R. (1969) Who Governs? Democracy and Power in an American City. New Haven: Yale University Press. pp. A new edition was published in 2005, ISBN 03001 03921. This is a well-written and detailed account of Dahl’s research method and his interpretation of its findings. It remains a seminal work and model for present-day researchers. De Vaus, D.A. (2001) Chapter 2: Theory and Social Research. In Surveys in Social Research. London: Routledge. pp. 11–26. This short account summarises the interaction of theory and research, the process of theory building and testing, and the sources of theory. Glaser, B.G. and Strauss, A.L. (1967/1995) The Discovery of Grounded Theory: Strategies for Qualitative Research. New York: Aldine de Gruyter. This very pacy account provides the critique of contemporary research in the early 1960s from which grounded theory was developed. As the American Sociological Review claims: ‘The authors successfully transmit the sense of adventure, air of excitement and of positive apprehension over what is discovered as one tracks down clues and sorts [them] among attractive alternatives’. Gubrium, J.F. and Holstein, J.A. (1997) The New Language of Qualitative Method. Oxford: Oxford University Press. pp. 19–94. This 244-page textbook provides incisive chapters on naturalism, ethnomethodology, emotionalism and post-modernism including extracts from many case studies. Marsh, D. and Furlong, D. (2002) Chapter 1: A Skin is Not a Sweater: Ontology and Epistemology in Political Science. In Marsh, D. and Stoker, G. (eds.) Theory and Methods in Political Science. Basingstoke: Palgrave Macmillan. pp. 17–41. This chapter reviews the centrality of ontology and epistemology and the divergent perspectives of positivism, interpretivism, and constructivism, and demonstrates their application to case studies of globalisation and multi-level governance.
35
36
Research Methods in Politics Neuman, W.L. (2003) Chapter 4: The Meanings of Methodology. In Social Research Methods: Qualitative and Quantitative Approaches. London: Allyn and Bacon. pp. 68–94. This chapter describes the historical development of methodologies in social science. The author identifies the differences between the main approaches by asking eight questions. They include: what is the basic nature of human beings? What is the relationship between science and common sense? The answers are tabulated on Table 4.1 on p. 91. This will be of particular values to researchers new to Politics and seeking to understand the complex approaches. Smith, M.J. (2003) Chapter 3: The Emergence of the Social Science. In Social Science in Question. London: Sage. pp. 75–116. This is a core text of the Open University’s social science modules. It describes and evaluates the origins of positivism and its contribution to emergent social sciences and the development of empiricism. A section is devoted to explaining Popper’s ‘falsificationist solution’ to the ‘problem of induction’. The chapter ends with a rich collection of readings from Comte, Bentham and Eysenck.
Notes 1 Lacey, A. (1995) In Honderich,T. The Oxford Companion to Philosophy. Oxford: Oxford University Press. pp. 145, 705–6. 2 Smith, M.J. (2003) Social Science in Question. London: Sage. p. 79. 3 The ‘natural sciences’ are generally considered to be astronomy, physics, maths, chemistry and quantitative biology. 4 Smith, M.J. (2003) Social Science in Question. London: Sage. p. 81–2. 5 Williams, R. (1983) Keywords: A Vocabulary of Culture and Society. London: Fontana. pp. 238–9: ‘[positivism] becomes a swear word, by which nobody is swearing’. 6 The first ‘school of political science’ was established at Columbia in 1880: Burnham, P., Gilland, K., Grant, W. and Layton-Henry, Z. (2004) Research Methods in Politics. Basingstoke: Palgrave Macmillan. p. 12. 7 Dahl, R. (1969) Who Governs? Democracy and Power in an American City. New Haven: Yale University Press. A new edition was published in 2005, ISBN 03001 03921. 8 Dahl, R. (1969) Who Governs? Democracy and Power in an American City. New Haven: Yale University Press. p. 331. 9 Bachrach, P. and Baratz, M. (1962) The two faces of power. In American Political Science Review, 56. ‘Dahl thereby excludes the possibility that in the community in question there is a group capable of preventing contests from arising on issues of importance to it … [Dahl is] unable adequately to differentiate between a ‘key’ and ‘routine’ political decision’. 10 This author’s own experience supports the claim that, in UK local government, the real decisions on planning policy (but not planning applications) are made at the ‘pre-agenda’ meeting between the (elected) executive members and the ‘officers’ (unelected officials). 11 Lukes, S. (1974) Power: A Radical View. London: Macmillan. 12 I experienced a counterfactual in my research on hegemony and social exclusion in rural England where I note that the estate villages appeared to lack the war memorials that were characteristic
The Philosophy and Principles of Research
13 14 15 16
17
18 19 20 21 22 23 24
of the greens or crossroads of villages elsewhere. I found that the memorials had actually been located inside the village churches – despite most of the casualties having been members of Methodist chapels. Domhoff, D.W. (2005) Power in America: Who really ruled in Dahl’s New Haven. http:/sociology.ucsc.edu/whorulesamerica/power/new_haven.html (20.06.2006). Sanders, D. (2002) Chapter 2: Behaviouralism. In Marsh, D. and Stoker, G. (eds.) Theory and Methods in Political Science. Basingstoke: Palgrave Macmillan. p. 52. Oakley, A. (2000) Experiments in Knowing: Gender and Methods in the Social Sciences. Cambridge: Polity Press. p. 16. Evans, G. and Norris, P. (1999) Critical Elections. London: Sage, and Norris, P. and, Lovenduski, J. (1995) Political Recruitment: Gender, Race and Class in the British Parliament. Cambridge: Cambridge University Press. Neuman, W.L. (2003) Social Research Methods. Boston: Pearson Education. p. 89. Neumann develops an alternative classification of social science into three paradigms: positivist, critical social science and interpretative social science. He explains the latter as: ‘the systematic analysis of socially meaningful action through the direct detailed observation of people in natural settings in order to arrive at understandings and interpretations of how people create and maintain their social worlds’ (p. 71). Heywood, A. (2000) Key Concepts in Politics. Basingstoke: Palgrave Macmillan. pp. 101–3. In UK criminal law, assault in ‘assault and battery’ refers to the perceived threat of battery by the ‘victim’. Hume, D. (1739) A Treatise on Human Nature. 1: 467–8. Glaser, B.G. and Strauss, A.L. (1967/1995) The Discovery of Grounded Theory: Strategies for Qualitative Research. New York: Aldine de Gruyter. Voight, P. (1999) Dictionary of Statistics and Methodology. London: Sage. p. 58 Schattschneider, E.E. (1961) The Semisovereign People: A Realists’s View of Democracy in America. New York: Holt, Rinehart and Winston. p. 35. Honderich, T. (ed.) (1995) The Oxford Companion to Philosophy. Oxford: Clarendon Press. pp. 405–6.
37
Part II Methodologies
Chapter 4
Qualitative Versus Quantitative Methods: A Relevant Argument?
‘Quantitative research is hard and reliable … qualitative research is deep and rich’ (Bryman, 1996:94)1 ‘Qualitative and quantitative methods are more than just differences between research strategies and data collection procedures. These approaches represent fundamentally different epistemological frameworks for conceptualising the nature of knowing, social reality, and procedures for comprehending those phenomena’. (Filstead, 1979:45)2 Teaching and learning objectives:
1. To understand the origins, merits, strengths and weaknesses, claims and counter-claims of qualitative and quantitative research. 2. To consider whether ‘mixed methods’ are incompatible with best research scholarship. 3. To enable each student to reach and defend their own preferences.
Introduction Qualitative and quantitative methods have historically been portrayed as mutually exclusive research approaches promoted by competing specialists appealing to the hearts and minds of new students. The arguments and antagonisms have become very tired over the years as each new cohort of students is asked to critically compare the competing claims. However, the traditional divergence of opinion has been challenged by the increasing interest expressed by Politics researchers in ‘mixed methods’ and the availability of sophisticated CAQDAS (Computer-Assisted Qualitative Data Analysis System) software. This uses quantitative methods to analyse qualitative data.3 These include programs which systematically analyse ‘talk and text’. However, they depend
42
Research Methods in Politics
on systems of coding designed and applied by the researcher. So the analysis is not therefore automatic. They also rely on assumptions, especially content analysis software, about the significance of the frequency and regularity of words which are essentially positivist. The full, definitive discussion of the claims and counter-claims of qualitative and quantitative methods is provided by Alan Bryman’s Quantity and Quality in Social Research, London, Routledge, 1996/88. Read it. This chapter seeks to summarise briefly the origins, characteristics, merits, strengths and weaknesses, claims and counter-claims of qualitative and quantitative methods. You will already be aware of the particular pitfalls and incoherence of comparing two or more concepts. The conventional method of comparative, descriptive narrative can become easily weighed down with repeated use of ‘on one hand’ and ‘on the other hand’. A weakness of text is its linear, one-dimensional presentation of information. This makes ready comparison difficult and timeconsuming. An alternative approach, which this book advocates for writing research reports, is to tabulate (set out in a table) for ready comparison the core characteristics of the concepts. Tabulation enables specific aspects to be extracted for detailed discussion. This approach is used in Table 4.1 to highlight the essential differences between the two methods. The strengths and weaknesses of the two research methods are not mirror-images of each other and are best considered separately.
Quantitative methods: strengths The greatest strength of quantitative method lies in its general acceptance by others as being rational, logical, planned and systematic. The findings are regarded as credible. This method is therefore particularly favoured by public, research-funding bodies keen to justify their investments to a sceptical public. It is also preferred by news media whose audiences, they claim, regard percentages as ‘hard news’. Therefore quantitative method, however complex the statistical methods employed, is regarded as being straightforward and providing the facts. The researcher is seen as dispassionate, objective and, therefore, trustworthy. It employs very large samples designed to reflect and be representative of the population being studied. The use of questionnaires ensures that every member of the sample is asked the same question in the same manner. Supporters argue that attitudes can be measured by using scales, e.g. the Likert scale of strongly agree, agree, etc. (see Chapter 9). Face-to-face contact is not necessary and may contaminate the data. Research objects are best kept at arm’s length. Geographically remote or immobile people can be surveyed by postal survey, telephone or internet. Quantitative method makes best use of computers and other new technology. So the time-scale for data analysis and the publication of findings can be relatively short. Furthermore, the data can be re-examined, audited
Qualitative Versus Quantitative Methods: A Relevant Argument? Table 4.1 Characteristics of qualitative and quantitative methods Characteristics
Quantitative Research
Qualitative Research
Origins Philosophic roots Research design
Natural science Positivism Deductive Systematic Theory testing Seeks explanation and universal laws Observer, outsider Objective Objects Research centre or laboratory Random Large Focus on universes The individual
Social science Naturalism, feminism Inductive Flexible Theory building Seeks explanation and solutions
Contribution to theory
Researcher Researcher’s attitude People are regarded as: Location Samples or cases
Focus Records Data Data collection Represented by Analysis Generalisability Costs Findings
Frequency Numeric concept indicators Surveys and structured questionnaires Tables and charts Statistical High High Nomothetic (general laws)
Participant, insider Subjective, empathetic Subjects Field based Non-random Small Focus on minorities The group (family, clan, peer group, work group) Social meanings Non-numeric concepts Personal interviews and unstructured interviews Transcripts Non-statistical None claimed Low Ideographic (specific to historical and cultural context)
and re-analysed or used for other purposes. The process of analysis is overt in which the scope for bias by the researcher is deliberately minimised. The method enables research projects to be carried out by teams in which specialist talents can be properly exploited and work sub-contracted to agencies. Quantitative research effectively conforms to the modern-day business model. In terms of academic research, quantitative method is particularly suited to the development of grand, meta and micro theory by testing logical hypotheses. Applying Popper’s tests of falsifiability ensures that the research hypothesis can only be accepted after all alternative hypotheses have been disproved. It offers the best means of identifying and comparing the distribution between people, places and times of phenomena such as party membership, voting, income, poverty, housing conditions, and changing attitudes. It can also be used to identify clusters of relatively small-scale phenomena and to analyse these statistically to identify whether
43
44
Research Methods in Politics
particular concentrations may be attributed to chance or potentially localised causes. Quantitative method can mine and analyse existing data banks, for example, census data and voting records. It is particularly useful in electoral research which focuses increasingly on the problem in western states of declining turnout in what commentators have termed the ‘period of post-democracy’ (Crouch, 2004).4 One notable study used the data available from the British Election Study (BES) and local government records to explain potential causes of the drop in UK turnout in the 1997 general election (when, after 18 years in opposition, Labour replaced the Conservative party in government). The drop, from 77.9% (in 1992) to 71.6%, could be attributed mainly to the combination of the closeness of party ideology and the extent to which the outcome was in no doubt (Heath and Taylor, 1999).5 The apparent certainty of the result was reported by news media using the evidence of opinion polls – another quantitative device. In this way, ‘hard news’ became self-fulfilling.
Weaknesses and criticisms of quantitative methods These are mainly fivefold. First, the underlying doctrine of positivism is, as already stated in Chapter 3, contestable in its application to the social world. Second, it is too detached, remote and clinical to really understand and explore the complex social and political world. Quantitative research is amoral. Third, its use by the social sciences does not meet the high standards of the natural sciences in which its reputation and claims lie. It is ‘bad science’. The samples, despite claims of randomness, are not statistically reliable. There are too many variables for causality ever to be determined, especially when the measures are superficial. Measures of strength of attitude are fallible: is ‘strongly object’ twice the value of ‘object’? Fourth, quantitative research in Politics research relies on the ability to express concepts as measurable indicators. But, as already discussed, how can power be measured? Furthermore, official data is often ‘doctored’ to serve government interests. To what extent are records of, say, civil disturbances and arrests really representative of underlying dissent? Can we really weigh an iceberg by measuring its tip? The necessity for measurable conceptindicators means that quantitative research is confined to researching measurable variables rather than more important issues. Lastly, the reliance on observation limits the range and depth of research to what is both observable and measurable. The stereotypical quantitative researcher is a clean-shaven, male, white-coated, ascetic, Times-reading Dr Strangelove working in an ‘established university’.
Defence and counter-claims Advocates of quantitative research rebut the criticisms by defending the applicability of positivism and empiricism to most aspects of the social world. It is less imperfect than qualitative research. In purist terms, it is the evil of two lessers. But samples
Qualitative Versus Quantitative Methods: A Relevant Argument?
can be devised and statistical analysis applied to produce findings within specified confidence limits. Questionnaires can be designed with sufficient checks and controls to identify bias by respondents and questioners. Most concepts are capable of being represented by measurable indicators. Subconscious feelings and motivations can be identified by using projective questioning (see Chapter 9). Quantitative method accepts its own fallibility which its users consciously seek to minimise. Most variables are capable of observation by trained researchers using modern technology. At the end of the day, quantitative research is reliable and concrete and has ‘put the science’ into Politics to create political science.
Qualitative methods: strengths This is the dominant approach adopted in UK Politics research. This choice reflects in part the lack of confidence of many UK researchers in their quantitative skills. In the main, qualitative method is preferred because it is considered best suited to the study, understanding and explanation of the complexities of social and political life. The strength of this method lies in its unique capacity, through in-depth interviewing and observation, to learn and understand the underlying values of individuals and groups. It better enables theory to be created by induction. By learning the social meanings that the subjects apply to their world, researchers are better able to ‘see the world through the subject’s eyes’. So researchers can identify and understand the interpretative lens that subjects adopt, and, therefore, the dominant powers and institutions that frame the view and tint the lenses. Essentially, politics takes more than one person: it is an intra- and inter-group activity. Qualitative method enables the focus to be shifted from the individual to the group(s) and to learn (following the theories of Goffman and others) how meanings are negotiated between members and the group dynamics involved.6 It allows comparisons and distinctions to be drawn between what the individual says in the privacy of a personal interview and what they say, or don’t say, in a group. Furthermore, the varied opportunities for participant observation, along a continuum between thresholder and full member, allow degrees of access to the group. The method also enables minorities to be researched who would otherwise be missed by sample surveys because they are either small in number or might be unwilling to identify themselves. Covert groups include homosexuals, drug-users, gang-members, and terrorists and their supporters. The method avoids distance and objectivity by seeking verstehen (empathetic understanding). Many of the researchers are also (critical) activists. They seek to expose exploitation and to improve the lives of their subjects through policy change. They want to assist their subjects to see their circumstances through other perspectives and lenses, and to develop their own interpretations. They seek to empower their subjects through the research process. There is therefore a normative dimension to qualitative method that quantitative
45
46
Research Methods in Politics
method lacks and, for that matter, avoids. The method also offers a greater focus on verbal and other communications and the application of linguistics to the analysis. Sophisticated (digital) recording devices enable the data to be shared with other researchers. New, miniaturised technology allows fieldwork to be carried out in the homes and workplaces of their subjects in a more surreptitious way. So the subjects are not inhibited by the equipment and contingent variables are less affected.
Criticisms and weaknesses of qualitative methods The archetypal criticism of qualitative methods is that the data collected is largely anecdotal or exaggerated. The method is also ‘soft’: it lacks the intellectual and operational rigour of quantitative method. The researcher is likely to ‘go native’ (over-identify with the subjects) or, alternatively suffer, from observer drift (where obvious deviances become accepted norms that are under-recorded). So the data and findings are highly unreliable. The claims to induction are spurious: the research question and choice of issues and concepts must be influenced by prior learning. The researcher is weighed down by bias. By participating with their subjects, the researcher will inevitably contaminate the social field and determine outcomes, or at least influence them in some way, rather than observe them. Furthermore, despite their distaste for concept-indicators, qualitative researchers inevitably adopt concepts which involve some form of proxy to make them intelligible to the subjects. Verstehen is elusive. How can a young, white, middle-class male researcher ‘see the world through the eyes’ of a female with the same background let alone a person from a wholly different culture? But, even if the data is sound, there is the insurmountable problem of interpretation. The researcher must interpret in turn the interpretation by the subject of their world. The scope for misinterpretation is huge. How can the researcher know or show that their interpretation is inaccurate? There is also the ethical danger that, by participating closely in oppressed minorities, the researcher may act as an agent provocateur who leads the group into harm’s way and then retires to the sanctuary of their university. The stereotypical qualitative researcher is seen by critics as long-haired, earringwearing, leather and denim-clad, Guardian-reading, smoking, Dr Howard Kirk (The History Man) in a ‘new university’. Or, worse still, he is a woman.
Defence and counter-claims on behalf of qualitative research Advocates reject the criticisms made as wholly exaggerated. The social sciences are not a soft option. Qualitative research is intellectually demanding and rigorous. Researchers are fully aware of the weaknesses and dangers of close identification between subject and researcher. Protocols and close supervision are required. Data is not anecdotal. It is no longer accepted on trust: the researcher is required to make
Qualitative Versus Quantitative Methods: A Relevant Argument?
Illustration 2 The great divide: Quantitative versus Qualitative Research. The stereotypes
available full transcripts and other records of interviews and group meetings for corroboration. There is a full paper trail available for audit purposes and external examination. The research method provides the only means by which overlooked or concealed minorities can be studied. In this way, their needs for special recognition can be addressed by policy-makers before continued rejection leads to outright dissent. The emphasis on studying subjects in their own habitat means that contingent factors can be identified and the research effect (see Chapter 2) minimised. It recognises that language is never neutral: ‘every word is a bias’ (Nietzsche). The normative opportunities should be welcomed rather than criticised: research should improve the circumstances of its subjects. The method acknowledges the central role of the researcher in the research rather than pretending that this can be eliminated. Quantitative method does not readily enable group behaviour to be studied. Overall, qualitative research provides high quality data and findings, and deep, meaningful insights into underlying values, fears and motivations of agents and actors in the political world.
Mixed methods Given the separate strengths and weaknesses of quantitative and qualitative methods, then using them together – mixed methods – would appear to offer the best of
47
48
Research Methods in Politics
both worlds. But, given their opposing ontology and epistemology (see Chapter 3), the criticism could be made of the lack of scholarship and the inevitable theoretical weaknesses of seeking to mix oil and water. Alternatively, the criticism could be rebutted as being too esoteric and academic and of overlooking the advantages, especially of time and other costs, of bringing the two methods together when the opportunities arise. The justifications cited for a mixed approach could include synergy, corroboration of sources and data triangulation (see Chapter 7). The received view among research practitioners tends to be that the differences between the methods have been exaggerated. However, there remains strong objection to a ‘mix and match’ approach. Instead, the use of combined methods can be better accepted as appropriate where one method dominates and the other is used in a secondary, supportive way. Two examples of appropriate combined methods can be given. Both concern research into voting behaviour.
EXAMPLE 1: Election turnout UK election turnout has fallen since 1951.The research question is: why has turnout fallen? It adopts a positivist, deductive approach in which Crouch’s theory of post-democracy is used and tested. The research seeks to test Crouch’s argument that turnout has fallen because globalisation and the decline of industrial production have loosened the previous sense of antagonistic social identity and class. So the appeal of unions and political parties has weakened and reduced support for active participation. Instead participation has once again become an elite activity by ‘self-referential political class more concerned with forging links with wealthy business interests’ (Crouch, 2004).7 In this case, the proxy-indicators of active participation selected are: membership of parties and unions; attendance at political etc. meetings; holding office; and, voting. The data will be collected from national published records of voting and party, etc., membership since 1951. This population data will be augmented by a national, postal questionnaire sample of a stratified sample of say 2,000 electors (following a pilot survey). It may offer hard, reliable evidence of changes in voting behaviour, class consciousness and party identity and participation. Analysis is likely to show a strong correlation between the decline in participation and loosening of political identity. But it will lack compelling evidence of causal relations. In this case, the data could be supported by interviews with a small sample of activists and non-activists to learn of their personal experiences and to tease out their changing motivations. This qualitative data could either be sought before the questionnaire survey to help frame the questions and alternative answers, or afterwards to explore particular findings. In either event, ‘quotable quotes’ could be incorporated into the research report to illustrate the findings and to create interest. The report is likely to conclude with a discussion of the implications of the findings for ‘democracy’ and electoral reform.
Qualitative Versus Quantitative Methods: A Relevant Argument?
EXAMPLE 2: Increasing abstention The second example also concerns the fall in UK turnout since 1951. The research question remains the same although expressed in a slightly different way: why are more people abstaining from voting? But this time, the research is inductive: there is no hypothesis to be tested. The researcher is not seeking one explanation but a number of reasons applicable only to the persons concerned. In this case, the researcher will interview a sample of people in depth to learn what participation and voting means to them. The sample will not seek to be representative of the population. Whilst the researcher will hope to build explanations there is no expectation that a law of electoral behaviour will emerge. The researcher will also interview and observe meetings of groups of people to learn how and if group norms are negotiated and peer pressure exerted. In this case, these interviews could be supported by evidence of the extent of abstention and classifications drawn from official records and the British Election Survey. This quantitative data will quantify the incidence and extent of the behaviour rather than to give explanations.This research is unlikely to stop at the explanations gleaned. It will consider the implications for the individual and groups and to suggest how the processes of participation and articulation should be improved to benefit abstainers.
Questions for discussion or assignments
1. What do you understand by quantitative and qualitative methods? 2. Summarise the main claims and counter-claims of quantitative and qualitative methods. Which do you regard as more appropriate to research in Politics? Why? 3. You are asked to carry out research to test Galbraith’s ‘culture of contentment’ thesis.8 Which method would you use? Why? 4. Mixed methods: can they be logically consistent? Are they ever justified? Choose and defend two examples which support your position.
FURTHER READING Bryman, A. (1988/96) Quantity and Quality in Social Research. London: Routledge. This text provides the most authoritative review of qualitative and quantitative methods, their claims and counter-claims and learned discussion on mixed methods. Harrison, L. (2001) Political Research: An Introduction. London: Routledge. pp. 74–7. This book includes a very concise analysis of the research methods in
49
50
Research Methods in Politics terms of whether they produce different results and answers to research questions. Reed, M. and Marsh, D. (2002) Chapter 11: Combining Quantitative and Qualitative Method. In Marsh, D. and Stoker, G. (eds.) Theory and Methods in Political Science. Basingstoke: Macmillan, pp. 231–48. This book follows separate critical accounts of qualitative methods (pp. 197–215) and quantitative methods (pp. 216–30) with an authoritative discussion of mixed methods (‘a false dichotomy?’) including two case studies. Seale, C. (1999) The Quality of Qualitative Research. London: Sage.
Notes 1 Brymam, A. (1998/96) Quantity and Quality in Social Research. London: Routledge. 2 Filstead, W.J. (1979) Qualitative Methods: a needed perspective in evaluation research. In Cooke, T.D. and Reichardt, C.S. (eds.) Qualitative and Quantitative Methods in Evaluation Research. Beverly Hills: Sage. p. 45. 3 Popular software includes Nud*st and ATLAS.ti. 6 Erving Goffman (1922–82) Canadian-US sociologist who specialised in studying face-to-face interaction within the social interactionist tradition in which he identified the significance of ritual in everyday activity 7 Crouch, C. (2004) Post-Democracy. Cambridge: Polity. 8 Galbraith, J.K. (1992) The Culture of Contentment. London: Penguin. Galbraith argued that US voters were predominantly socially and economically advantaged electors who voted in favour of groups which offered to reduce taxation and therefore welfare provision for others in need of intervention.
Chapter 5
Collecting Data Sets: Case Studies, Experimental, Comparative, Longitudinal and Action Research Methods
Teaching and learning objectives: 1. To consider the distinctive characteristics of case studies, experimental, comparative, longitudinal and action research methods. 2. To identify their strengths and weaknesses. 3. To learn where each is best used.
Introduction Case studies, experimental, comparative, longitudinal and comparative research are all different methods that you can use to gather data. This data may either be existing, documented information, or new information which is required to test your (deductive) hypothesis, or develop (inductively) new theoretical explanations. Simply described, a case study method is a ‘sample of one’ event, instance, state or sub-unit at one point in time. Experimental method involves a ‘sample of two’: a control sample and an experimental sample drawn (ideally randomly) from the population under study. The experimental sample is made subject to a change in some independent variable whilst conditions are held constant in the control sample. For example, the independent variable may be new, additional forms of news information, financial incentives or cost reductions by way of, say, postal voting. Comparative method involves obtaining data from a population or sample of equivalent states or sub-units, sectors or groups, at the same points in time. It is the principal method used in comparative politics and international relations. Longitudinal method seeks data from the observation of a cohort, a group of people, states or organisations, etc. sharing one or more common characteristics, over an extensive period of time. The common characteristics may include age, education, place or
52
Research Methods in Politics
specific condition. Action research method essentially involves the researcher obtaining data through direct participation in a group of actors experiencing particular conditions. The method therefore allows you to share and experience the institutional and other barriers the group faces in the struggle towards transformation.
Choice of method The choice of research method will generally be determined by five factors: 1. 2. 3. 4. 5.
academic or other institutional requirements the time and other resources available to the researcher the sub-discipline of Politics accessibility of the data sets whether (mainly) quantitative or qualitative data is sought.
Generally speaking, university regulations for final-year, single-term research projects by undergraduates and ‘taught’ graduates will be satisfied by a single case study. This will provide sufficient opportunity for students to complete original research which demonstrates their theoretical understanding and their ability to obtain and analyse high-quality, in-depth data and offer profound insights. However, doctoral research generally requires data to be sought from a number of sources which combine both depth and extent. In this way, the set of multiple sources should demonstrate whether the particular findings are unique or, hopefully, universal and the impact of contextual variables. A single-term or single-semester timescale will normally limit the method employed to a single, in-depth case study, or to a comparative study of either a limited number of variables from a large sample, or a larger number of data from a smaller sample. At the other end of the scale, longitudinal research will require the necessary time to track and observe members of the sample over the longest possible periods. Different data collection methods are better suited to particular sub-disciplines of Politics. Comparative politics, international relations and development studies tend to use comparative method. Gender studies are more likely to use action research. Students of conflict studies are more likely to adopt an in-depth case study of a particular geographical area. Accessibility will also influence the choice of method. It means both access to the group or set under study and to the relevant information about them. Groups may be inaccessible because they are physically remote, concealed, nomadic, or lack sufficient identity amongst themselves. Even when the groups are readily contactable it may be difficult to access information about them. For example, information on income, health, participation, exclusion and illegal activities is difficult to access.
Collecting Data Sets
Additionally, the specific phenomenon or indicator may not be readily observable or measurable. Pre-disposition to quantitative or qualitative method will also influence, if not entirely determine, the choice of method for collecting data. The limited availability of numeric information from a single case and the requirements for generalisability (the wider applicability of its conclusions) will generally require a large sample of comparable cases to be used to achieve the law of large numbers.1 Conversely, the requirements for ‘deep and rich’ data in qualitative method are more likely to be met by a single case study or, if resources permit, a small group of cases selected for their apparent similarity or difference.
The single case study The single case study is the essential ‘building block’ of empiric research. It must provide the basic minimum of information to enable the research question to be answered and the research hypothesis to be tested. It is a sample of one. Historical research may define the case study and its scope. For example, the research question ‘why did the TUC halt the British general strike of 1926’, will concentrate on the specific events leading to the general strike and its conduct. The difficulty here will be to decide when the ‘events leading to’ actually began and the main actors involved over this timescale. The law of diminishing returns may be evidenced when the marginal cost of further interviews becomes greater than their benefit to the research, and where opportunity costs are high. On the other hand, research into, for example the causes of strikes in the UK car industry post 1997 may focus, in the first instance, on a single company or a plant within that company. Self-containment and typicality are important requirements for an effective case study. Self-containment means that the variables being researched should be clearly distinguishable and be relatively unaffected by complex external factors. The case study should also be typical of the range of other potential case studies. So conscious choice – rather than blind or random selection – is required. It is therefore important that the initial preferred case study should itself be carefully ‘researched’ before the choice is confirmed. This initial examination should show that the case is not affected by other, exceptional factors. It should also show whether the necessary access to key individuals and other sources of data is likely to be achievable. You should also find out whether the group or location has been researched before: frequent prior research may have ‘poisoned the well’ of information and left people uncertain of their ‘social reality’. Insertion and immersion (i.e. prolonged exposure) in the proposed case study early in the research process is therefore essential if disappointment, or even failure, is to be avoided. However good the fieldwork, the quality of data and its analysis, the great weakness of a single case study remains that its generalisability – and therefore its
53
54
Research Methods in Politics
significance – remains unproven. The explanation may be unique to that case. A solution may be to adopt a thick/thin approach or case study and a quarter. Here, the findings of an in-depth case study are tested against another typical group or situation. For example, suppose that a case study of ‘large-scale voluntary transfer’ of council houses in a northern, English, industrial town had concluded that the high support by tenants there could be attributed to the involvement of a national housing charity as a mediating body.2 In reaching this conclusion, the researchers had established that other factors such as rents, repairs backlogs, key personalities and support by tenants for the council, were much less significant. The findings of this study could then be tested by a ‘thin’ study of another, similar case where a mediator had been appointed to find out whether their involvement was regarded as critical to the outcome. Alternatively, the findings could be tested against a similar ‘thin’ case where a mediator had not been appointed to see whether support was lower. The use of this solution may be particularly effective for ‘taught masters’ students.
Experimental research Experimental research is rarely used in politics research. Here, the necessary observation of an experimental group and control group under laboratory-type conditions – let alone the gold-standard of random double-blindness – is difficult to achieve. To be successful, large-scale institutional support may be required. One particular type of experimental research is the pilot project where a change of policy is tested on a particular group or area before the initiative is ‘rolled-out’ (extended). A good example of pilot testing is the research carried out by the UK Electoral Commission in 2002–4 into the effect of introducing all-postal voting to replace previous forms of ballots. This initiative was introduced to counter the decline of electoral turnout since 1951, especially in local government elections where this had fallen to less than a third of electors. Low turnout was considered to undermine the legitimacy of councillors in government/local government relations. The traditional form of voting required electors to vote in person at the specified, local polling station on a single date unless the elector could justify the necessity for a proxy vote (on grounds of disability) or postal vote (on the grounds of absence). All-postal voting meant that the ballot paper was delivered direct to the elector well in advance of the election and could be returned by post before the polling day or delivered to the polling station. Following a selection process, 32 local authorities were chosen to test the practicality of all-postal voting and its effect on turnout. Each was the subject of a statutory report by the Commission. The reports show that, overall, the level of turnout in the pilot areas increased to 50%. For example, turnout in the City of Newcastle upon Tyne increased from 32% (in 2003) to 49.9%.3,4 Similar increases were reported in the other pilot areas. On this evidence, the (unstated)
Collecting Data Sets
hypothesis – that the change from traditional to all-postal balloting would increase turnout – appeared confirmed. The outcome supported the rational choice interpretation of voting behaviour (in which reducing barriers and costs of voting would increase participation). However, other factors may have contributed to the increase in turnout. In particular, the local authorities undertook substantial publicity campaigns in the local press, radio, television, billboards and additional mail-outs to electors. So significant Hawthorne and Pygmalion effects cannot be discounted. I also suspect that pilot tests generally tend to realise higher improvements beyond those likely to be attained when the initiatives are extended nationally. This pilot effect may be attributed to the original selection of the pilots for their greater suitability and the additional resources made available including the greater involvement of leading officials. Subsequently, complaints were made to the police that applications for postal voting in marginal wards had risen by 500%. The police investigations led to trials and convictions in a small number of cases. Subsequently, in August 2004, the Commission recommended that the system of all-postal voting should be discontinued because, in the absence of the reforms it had recommended for voter registration, postal voting was vulnerable to abuse.5 Yet its earlier statutory reports had stated that ‘it was unaware of any allegations of fraud and malpractice …’.
Comparative research Comparative research, or comparative method, is the approach most widely adopted by comparative politics, international relations, public policy and developmental politics. Indeed, in North America, comparative politics is regarded as essentially the study of ‘foreign states’. The underlying principle of comparative research is that, by comparing two or more cases, researchers can identify causal variables which could not be deduced from a single case. It is essentially positivistic and relies on the system of analysis afforded by J S Mill’s methods of difference and similarity (Mill, 1843).6 But whilst comparative research is positivistic, it need not necessarily be quantitative. A good example of early comparative research is Engel’s The Condition of the Working Class in England (1845).7 After reviewing the contemporary development of the main industrial sectors and creation of the proletariat, he examined in turn the ‘great towns’. He described conditions in the ‘working men’s housing areas’ in, for example, London, Dublin, Edinburgh, Glasgow, Birmingham, Salford and Manchester, Nottingham, Leeds and the smaller industrial towns of Yorkshire and Lancashire. He concluded that: To sum up the facts thus cited. The great towns are chiefly inhabited by working people, since in the best case there is one bourgeois for two workers, often for three …; these workers have no property of their own, and live wholly for wages, which usually go from
55
56
Research Methods in Politics hand to mouth … every working man is therefore constantly exposed to loss of work and food … The dwellings of the workers are everywhere badly planned, badly built, and kept in the worse condition, badly ventilated, damp, and unwholesome … The average is nearer the worst case than the best. (Engels, 1993/1845: 85–6)
His comparison of the ‘great towns’ enabled him to conclude that the conditions were universal within British industrial towns and cities. The causes were therefore not local, specific to different sectors of industry or otherwise contextual. He concluded that the cause of these common conditions were state-wide: competition between workers and Irish immigration. He then investigated these causes in turn, extending his research to the ‘mining proletariat’ and the ‘agricultural proletariat’ and concluded that ‘The war of the poor against the rich … will become direct and universal [i.e. not confined to Britain]’ (Engels, 1993: 302). Engels’ research demonstrates many of the purposes (reasons) for undertaking comparative politics research: 1. to provide rich, contextual descriptions which identify clearly the observed similarities and differences between cases and places 2. to identify and develop systems of classification and typologies which generate data sets appropriate to the research question 3. to distinguish independent variables (causes) from other variables 4. to test hypotheses and, thus, the validity of explanatory theories 5. to develop predictive capacity where, for example, there is evidence of stages of development.
The selection of cases for comparative research depends on the number of variables to be examined, the availability and access to related, comparable data, data format (numeric or non-numeric) and the means of analysis to be adopted. Elementary algebra has taught you that the value of two variables cannot be found from a single equation: two, simultaneous equations are required. At least one case is required per variable. Similarly, algebra and logic has taught you that you cannot compare apples and pears: the data to be compared must be measures or descriptions of the same variable. They must also be equivalent to the variables being examined. The cases to be selected must offer the same data sets over the same periods. But data need not be numeric or quantifiable. It can equally well be binary (e.g. yes/no, democracy/non-democracy, etc.) or nominal (e.g. the name of the state). The method of comparison to be adopted will determine the cases to be selected, and the size of the sample relative to the size of the population. The three most widely-used methods of comparison are Mills’ method of agreement, method of difference and method of concomitant variations.
Collecting Data Sets
Where the method of agreement is to be used, different cases which have the same outcome (result, effect) are selected. The potential independent variables (causes, contributory factors) are sought and listed. Where one (or more) variable is common to each case, then the researcher will conclude that this is the independent variable. This can be demonstrated below: Case 1 2 3
Variables a b c a c d a b d
Outcome y y y
In this group of cases, only a is common to all cases where the outcome is y. So a appears to be the cause/independent variable. The method of difference is the converse of the method of agreement. It is generally regarded as more practicable (easier). Here, similar cases are chosen for their different outcomes, e.g. single-party or coalition governments. The potential variables are sought and listed. Where the absence or presence of one variable is associated with specific outcomes, then that variable may be concluded to be causal: Case 1 2 3 4
Variables a b a b a b p b
c c c c
d d d d
Outcome y y y z
In this example, variables b, c and d can be seen not to effect the outcome whereas variable a appears to be the cause of outcome y and variable p appears to cause outcome z. The method of concomitant variations is used where the magnitude of the potential cause appears associated with the magnitude of the outcome. For example: Case 1 2 3
Variables a b 2a b 3a b
c c c
d d d
Outcome y 2y 3y
In this group of cases, the magnitude of a appears to influence the magnitude of y. In all these examples, the findings are qualified by saying ‘appears to be’. Firm conclusions can only be drawn when other explanations – e.g. spurious correlations or chance – have been eliminated. The individual cases must also be comparable. So samples of states may be drawn from the EU (or EEA if comparison between EU and other European states is sought), OECD, G8, states sharing a geographical area (e.g. South America), or states having a similar history (e.g. former British colonies). Or alternatively, a comparison
57
58
Research Methods in Politics
can be made between offices (e.g. foreign ministers), events (e.g. devaluations, civil wars), policies (e.g. privatisation), organisations (e.g. NGOs) and ideologies. The size of sample will also have a bearing on the degree of certainty to be attached to the findings. Ideally, the entire population of, say, OECD states should be chosen. Population-wide data is now more readily available especially via the internet. Where a sample of the population is used, then there may be a suspicion that cases which do not support the hypothesis have been deliberately excluded. For example, the comparative research carried out by Przeworski et al (to answer the research question ‘what makes democracies endure?’) used a large sample of 135 states over the period between 1950 and 1990 (Przeworski et al, 1996).8 But a comparatively small number of UN member states were omitted. They included the oil-producing Gulf states, where, with western support, feudalism continued to prevail. The necessary, compelling defence against criticisms of selection bias was not made. The potential samples can be categorised as small-n or large-n, i.e. a small or large sample of the population, N. As previously noted, the size of sample must be no less than the number of potential independent variables to be investigated. A small-n sample is more appropriate for qualitative research where a more intensive level of investigation is proposed, generating rich, thick descriptions, and where the focus of analysis is differences or similarities of outcomes. The small sample enables Mills’ method of similarity or difference to be used. These methods are termed most similar systems design (MSSD) or most different systems design (MDSD). Comparative research using small samples is called case-orientated analysis. Generally speaking, the optimum size for small-n samples is 15. In contrast, a large-n sample is used to enable a limited number of potential independent variables to be identified by statistical methods (including regression analysis and discriminant analysis) from a very large number of cases. It is termed variable-orientated analysis. The decision whether to adopt small or large samples for comparative politics research will, as ever, depend upon time and other resources and access to reliable data. Each has its strengths and weaknesses. Large samples enable computer-driven statistical techniques to be adopted and levels of confidence to be determined. A higher level of abstraction is permitted. Unusual cases are more likely to be readily identified for further examination. However, the level of abstraction may be too high and superficial and the available data may not properly represent the variables under consideration. Data collection standards may also vary between cases. A more profound understanding of advanced statistical analysis may be required. At the end of the day, the acceptance of the research findings will depend on the reader’s readiness to accept statistical methods. By comparison, small samples offer thick description, a lower level of abstraction, intensity and analysis of differences and similarities which the reader is more likely to follow and accept. But the suspicion of selection bias may be difficult to refute. One solution may be to combine both methods in which, say, a large sample is first used to identify apparently key variables, which are then
Collecting Data Sets Table 5.1 Selected average annual rates of growth of GNP per capita, 1965–90 Democratic State Jamaica Venezuela Senegal Trinidad Costa Rica India Sri Lanka Mauritius Malaysia Singapore Botswana
Change GNP per Head (%) −1.3 −1.0 −0.6 0.0 1.4 1.9 2.9 3.2 4.0 6.5 8.4
Non-democratic Regime
GNP Change (%)
Libya Zaire Zambia Nigeria Algeria Brazil Thailand Indonesia China Taiwan South Korea
−3.0 −2.2 −1.9 0.1 2.4 3.3 4.4 4.5 5.8 7.0 7.1
Source: Council for Economic planning and Development (1992); World Bank (1992) cited by Leftwich, 2000: 132i9
tested against a smaller sample from which the influence of the variables may be more properly understood. One topic of intensive comparative politics research is the relationship between democracy and development. This is illustrated in Table 5.1.9 The democracy-development link is the holy grail of western, comparative politics and development research. The findings thus far vary. For example, Moore’s comparative qualitative study (1966) of the paths to modernisation of Britain, USA, France, India, Japan and China concluded that, where the new bourgeoisie and progressive landowners combined, then democracy was more likely (Britain and USA).10 But where the traditional elites were unable to meet the peasants’ demands, then communism would prevail (China). Finally, where traditional elites were able to frustrate democratic institutions, then fascism was possible (Japan). In the post-war period, the prevailing view, especially among colonial powers, was that democracy would hinder development. Democracy (decolonisation) would best be the child of development only when higher levels of literacy, civil society and class formations had been achieved. Indeed as Table 5.1 shows, states achieving higher levels of economic growth 1965–90 included China, Taiwan and South Korea. A critical, comprehensive review of the published comparative research is provided by Landman (2003).11 He distinguishes between large-n and small-n samples and qualitative and quantitative inquiry. He shows that the large-n studies carried out by such luminaries as Lipset, Dahl and Przeworski support both ‘weak’ and ‘strong’ conclusions where, respectively, democracy is associated with development and development causes democracy. Wealthy democracies are more likely to endure. Small-n, quantitative research either supports the ‘weak version’ (i.e. association) or
59
60
Research Methods in Politics
no relationship. Small-n, qualitative research generally concludes that democracy is the product of discrete historical events. Single-country studies conclude that: ‘casespecific factors, particularly political culture, condition the relationship’ (Landman, 2003: 91, Table 4.5). So, the variance among the conclusions reached would suggest that the type of comparative research undertaken can influence the findings. However, Landman rejects this view. He argues that the variations can be attributed to the absence of a contemporaneous trajectory of development, selection bias and selection. The different comparative methods should be regarded as complementary: inferences from large samples should be investigated by an analysis of a smaller sample of states. Deviant cases should then be examined separately by single-case studies (Landman, 2003: 90–2). But Landman’s prescription is a counsel of perfection (unattainably high standards) which is unlikely to be achieved by a single, doctoral researcher working alone.
Longitudinal research Longitudinal research – also called panel survey research or tracking research – enables researchers to monitor changes in behaviour or opinion over time. Statistical time series analysis can be used on quantitative longitudinal data, e.g. unemployment data, to distinguish between long-term trends, seasonal variations and irregular events (see Chapter 16). The magnitude of irregular events enables the impact of policy change (or other intervention) and the time-lag involved (between announcement, introduction and effect) to be identified, e.g. the impact of New Deal-type job creation initiatives on unemployment. Longitudinal research essentially involves obtaining data over time. It may take the form of personal life histories or narratives when an individual is asked to describe life events from earliest memory. Whilst this can be a rich source of data, access can be difficult and necessary confidence and empathy (rapport) difficult to develop. It can be very time-consuming especially when recordings have to be transcribed. The data may suffer from faulty memory, selective memory, constructed memory and the tendency for subjects to select unrepresentative, exceptional or dramatic events to maintain the researcher’s interest. They may also exaggerate their own role. Constructed memory occurs when you describe events which you cannot remember but of which you have been frequently reminded by parents, e.g. your first steps. Alternatively, personal life histories can be augmented or rely on personal diaries, letters or other records. Longitudinal research may be retrospective or prospective. Retrospective research looks back over previous periods of time whereas prospective research studies changes from the present-day onwards. The ideal model of longitudinal research involves the cross-sectional interviewing or surveying of a cohort or panel of subjects. A cohort is a group of subjects sharing some common characteristic, e.g.
Collecting Data Sets
date of birth, or membership of a particular school class. In this type of survey, each of the subjects is interviewed separately for every year or period of years. Those who leave (by death or migration) are not replaced. You will find it very difficult to complete successfully in urban areas where the residents frequently move homes and jobs, or change family names on marriage or divorce. The most widely-used longitudinal research data is collected by panel surveys. A panel is a set of subjects chosen to be representative of the population under study. For example, the government-sponsored British Household Panel Survey (BHPS) – begun in 1991 – involves an annual survey of all the adults within 5,000 randomly selected British households.12 However, the term panel can be misleading: unlike common usage, the members of the panel do not meet together. Each is interviewed separately using a highly-structured questionnaire. When children in the selected households reach 16 they are added to the panel to replace those who are lost through death, illness or disappearance (which may include either withdrawing consent or moving contact address without telling the researchers). BHPS is carried out by the Institute for Social and Economic Research incorporating the ESRC Research Centre on Micro-Social Change at Essex University. Each cross-sectional survey is termed a wave. The BHP survey for 2006 was called ‘Wave 14’. Ideally, a panel survey should seek to ask the panel members the same core questions at every wave to enable comparison to be made of changes of attitude or behaviour over time. However, new questions can be added. The panel may also be increased to ensure better representation of minorities. For example, in Wave 9, the membership was increased in Wales to enable attitudes towards the Welsh language and national identity to be monitored. The data gathered by BHPS is available for inspection and use at the ESRC Data Archive at Essex University.13 Another authoritative source of longitudinal research data is the British Election Survey (BES) funded by ESRC and the Electoral Commission, and carried out at each British general election. A new panel is recruited by random sampling at every general election. Members of the panel are interviewed before and after the election. BES 2005 interviewed 4,700 British electors. It gave special attention to the issue of (falling) turnout and (increasing) abstention.14 Panel survey data is also collected – on a near-monthly basis - by the EU commission and published in the Eurobarometer. New panels are recruited by stratified, simple, random sampling (see Chapter 7). Eurobarometer 65.2, published in Spring 2006, included data from a British panel of 1,312 electors.15 A particular concern for the EU is tracking the movement of proand anti-EU sentiment among nationalities in the 25 member states. Elsewhere, the Panel Study of Income Dynamics (PSID) provides a comparable example of US longitudinal research. Longitudinal Politics research can be time-consuming and costly. Sponsorship by some government research agency is therefore likely to prove essential in order to provide a panel large enough to produce good, representative data. The great
61
62
Research Methods in Politics
advantage is that this data is available for re-analysis and use by the Politics researcher. Why collect new information when good data is already available?
Action research Action research is derived from critical social theory whose central aim is the emancipation of society (rather than mere description or explanation). The researcher focuses on the myths and contradictions, e.g. ‘women’s work’. A common maxim is: ‘reports don’t change policies … it is people who will change them’. Action researchers therefore tend to seek out underprivileged groups and work with and among them to enable them to recognise and overcome the institutional barriers which prevent them from self-actualising (achieving self-fulfilment). The groups may be Travellers, migrant families, illegal immigrants, gays and lesbians, former prisoners, and other, pariah groups. The researcher is a passionate, committed activist. Researching the group means sharing their lives, experiences and indignities. It can be a high-risk undertaking for you and for your co-participants. The risk is particularly great for the participants when they challenge the institutions which have served dominant groups. The danger is that you may lead the group into harm’s way or raise expectations which cannot be achieved. The involvement of an experienced supervisor is therefore essential if some form of emancipation, rather than repression, is really to be achieved. The task facing the action researcher is difficult. You will face problems obtaining access to the subject group and establishing mutual trust. Funding is problematic. The UK Community Development Programmes lost funding when they criticised the government’s funding ministry and its local allies. Your task is to enable the subject group to ‘think (and discuss) the unthinkable’, to identify potential actions and to implement change.The arena of action research is the group meeting. Your role here is to facilitate (enable) group identity to form and a new consciousness of their social reality to emerge. In particular, you should gently prod the group to re-examine institutions which have become regarded as ‘natural’ or ‘for the best’. Examples may include the disproportionate burden of childcare borne by women, restricted opportunities to education, or the classification and exclusion of groups from mainstream programmes. Another role for you will be to carry out those tasks which the group is unable, as yet, to accomplish. They may include researching similar groups and activities, arranging meetings, taking notes, printing and distributing leaflets, lobbying elites and representing the group at meetings with dominant others. You should also train the group to carry on these activities for themselves as part of the essential capacity-building exercise. This should form part of your exit strategy which should also include a celebration of the group’s activities. This event, like other social ritual, marks and announces to all the transfer of responsibility to the group.
Collecting Data Sets
Because action research focuses on outcomes rather than process, there are relatively few research reports. Publication is an anathema to many social action researchers who would regard their own preferment as further exploitation of those they seek to assist. However, writing-up and publishing action research projects is justifiable for the insights they provide into the why’s and how’s of domination and the success (or otherwise) of action to dismantle barriers. The increasing emphasis in western polities on participation has led to a widening of research interest in and funding for quasi-action research. A recent publication, Effective participation in anti-poverty and regeneration work and research (Beresford and Hoban, 2005), assesses the lessons drawn from UK initiatives to involve people with direct experience of poverty in anti-poverty programmes.16 The study concluded that: ‘Powerlessness is central to people’s experiences of poverty and disadvantage … capacity building to develop people’s confidence, self-esteem and understanding supports their empowerment and participation. It is not the same as skill development …’
But there remains a strong suspicion that increasing western government interest in public involvement seeks to smooth the process of policy change by co-opting and thus neutralising potential losers.
Questions for discussion or assignments
1. You are researching the causes of local government corruption in England. Your case study has identified new causes which are not included in the current literature. What steps should you take to establish whether these causes were unique to your case study or more widespread? 2. Can experimental research be wholly discounted in Politics? How might the impact of various electoral innovations on turnout be evaluated? 3. You wish to carry out comparative research on the relationship between democratic reform, economic policy change and changes in GNP per capita in the former Warsaw Pact states in Eastern Europe after 1991. What sample and method of analysis would you use? Why? 4. Gypsies in your locality are widely criticised for their exploitation of welfare services. You regard the basis for this criticism and related hostility as a myth. A national charity offers funding for social action research to improve the conditions of Gypsies and other Travellers. Draft a research proposal setting out how you propose to access this group, what roles you expect to pay, indicators of success and your exit strategy.
63
64
Research Methods in Politics FURTHER READING
Comparative research Hopkin, J. (2002) Chapter 12: Comparative Methods. In Marsh, D. and Stoker, J. (eds.) Theory and Methods in Political Science. Basingstoke: Palgrave Macmillan. pp. 249–67. This chapter provides a good introduction to the development of comparative politics, the comparative advantages of small-n and large-n samples and the competing claims of quantitative and qualitative comparative research. Landman, T. (2003) Issues and Methods in Comparative Politics: An Introduction. London: Routledge. This 292-page textbook is devoted to comparative research in Politics. It begins by identifying and examining the reasons for using comparisons, theory and method, units of analysis, and levels of analysis. This is followed by comparing comparisons where the comparative research on a range of significant topics is compared and evaluated. The topics are: economic development and democracy, violent political dissent and social revolution, non-violent political dissent and social movements, transitions from democracy, institutional design and democratic performance, and human rights. It ends by identifying the new challenges facing the sub-discipline. Leftwich, A. (2000) States of Development: On the Primacy of Politics in Development. Cambridge: Polity. pp. 127–51. This excellent, rich chapter uses comparative data and published comparative research to abstract and examine five main conditions for democratic consolidation: legitimacy; adherence to rules-of-the-game; policy restraint by winning parties; poverty as an obstacle; and ethnic, cultural or religious cleavages as constraints. He concludes controversially by arguing that: ‘democratic politics is seldom the politics of radical economic change’ (p. 150). Mayer, L.C. (1989) Redefining Comparative Politics: Promise versus Performance. London: Sage. pp. 1–46. The author identifies the epistemology of social science and its implications for comparative research. The role of comparative research is to suggest explanation of differences and similarities at state level. He argues that the unique role of comparative research lies in its capacity to ‘identify two distinct objects for analysis: the attributes of individuals and the attributes of whole systems’ (p. 42). Pennings, P., Keman, H. and Kleinnijenhuis, J. (2006) Doing Research in Political Science: An Introduction to Comparative Methods and Statistics. London: Sage. pp. 1–51. Pennings et al is a challenging text. However, it uniquely provides hard, technical explanation of the concepts behind comparative methods within the discipline of Politics.
Collecting Data Sets Przeworski, A., Alvarez, M., Cheibub, J.A. and Limongi, F. (1996) What makes democracies endure? In Journal of Democracy, 7(1): 35–55. This is a very good example of the application of comparative methods to international data sets. However, as previously noted, the exclusion of Gulf-states from the large-n sample compromises the generalisability of the conclusions.
Longitudinal research Gilbert, N. (2003) Researching Social Life. London: Sage. pp. 275–9. This short extract provides an authoritative explanation of panel, longitudinal and cohort studies and an introduction to the main data sets available from US, UK and EU sources. Pennings, P., Keman, H. and Kleinnijhuis, J. (2006) Doing Research in Political Science: An Introduction to Comparative Methods and Statistics. London: Sage. pp. 166–79. This extract provides a higher-level introduction to the mathematics of time series analysis and ways of diagnosing and reducing the potential problems of autocorrelation and heteroscedasticity. Robson, C. (1993) Real World Research. Oxford: Blackwell. pp. 50–1. This very brief extract outlines the design of effective longitudinal research.
Action research Harvey, L. (1990) Critical Social Research. London: Unwin Hyman. pp. 19–32. Lee, R.L. (1995) Dangerous Fieldwork. London: Sage. pp. 1–13. In this introduction, Lee scopes the types of danger that researchers face when working with ‘outlaw bikers’, youth gangs, people infected with HIV, and informants in inherently dangerous occupations. He argues that proper risk analysis can effectively minimise the dangers facing researchers and prevent them from being inhibited – or even paralysed – by their fears in the research setting. Stringer, E. (1996) Action Research: A Handbook for Practitioners. London: Sage. pp. 142–60. Stringer has become a foremost authority on participatory or community-based action research especially in the discipline of education. In this extract, he sets out strategies for effective action research to practitioners.
Notes 1 The ‘Law of Large Numbers’ states that the probability that a sample shares the characteristics of the population increases as the size of sample (i.e. number of cases) increases. 2 The Thatcher government introduced ‘large-scale voluntary transfer of housing stock’ in 1989 as an extension of its ‘right-to-buy’ policy of assisting tenants of council houses (municipal social
65
66
Research Methods in Politics
3 4
5 6 7 8
9 10 11 12 13 14 15 16
housing) to buy their own homes (at a discount). This policy was retained by the New Labour government. www.electoralcommission.org.uk/about-us/statutoryreports.cfm 25/07/06 Elections staff at the city council reported that, at the all-seats all-postal voting local council and EU elections in May 2004, turnout was 46.8%. The results ended the Labour party’s hegemony over the city council when the Liberal-Democrats took control. ‘Delivery democracy? Postal voting. Executive summary. www.electoralcommission.org.uk/files/dms/Deliveringdemocracy 4473-10934 E S W.pdf Mill, J.S. (1843) A System of Logic. London: Longman. Engels, F. (1845/1993) The Condition of the Working Class in England. Oxford: Oxford University Press. Przeworski, A., Alvarez, M., Cheibub, J.A. and Limongi, F. (1996) What makes democracies endure? In Journal of Democracy, 7 (1) pp. 35–55. The research concluded that democratic government is more likely to endure where per capita income is high, income inequalities moderate or falling, economies maintained or increased levels of growth, and there is no recent history of democratic government being overthrown. Furthermore, parliamentary democratic governments are more likely to endure than presidential systems (where deadlock between president and parliament can lead to paralysis and a coup d’etat by the defence forces). Leftwich, A. (2000) States of Development; On the Primacy of Politics in Development. Cambridge: Polity. Moore, B. (1966) Social Origins of Dictatorship and Democracy. Boston: Beacon Press. Landman, T. (2003) Issues and Methods in Comparative Politics: An Introduction. London: Routledge. For more information on the British Household Panel Survey, see www.esds.ac.uk www.data-archive.ac.uk The survey questions and data can be inspected at www.essex.ac.uk Eurobarometer data can be accessed at http://ec.eurpoa.eu Beresford, P. and Hoban, M. (2005) Effective participation in anti-poverty and regeneration work and research. York: Joseph Rowntree Foundation. The full report can be accessed at www.jrf.org.uk/knowledge/findings/social policy/0395.asp
Part III Collecting Information
Chapter 6
Critically Evaluating Published Research
Teaching and learning objectives: 1. To learn how to critically evaluate published research using research component analysis. 2. To apply Rose’s ABCDE model (1982) for deciphering research.1
Introduction As a child and student, you are likely to have developed and improved your own writing skills – consciously or otherwise – by reading books and other publications recommended to you. Similarly, reading others’ research reports will help you to develop your own research report-writing skills. However, going further, by consciously, critically evaluating others’ research reports, you can draw valuable lessons into effective research design, data collection, analysis and interpretation, and how best to communicate your findings. Critical capacity is an essential part of academic scholarship (which owes its origin to its original function of training priests to identify heresies). In this way, you can therefore appreciate better the successes or otherwise of other researchers before you begin your own research. In any event, the literature review of texts and research reports is an essential first step in research (which is considered later in Chapter 8). Some overseas students are inhibited from criticising published works – especially by great authors – because they have been taught to respect authority and the expert opinion contained within university libraries. However, in the UK and other western universities, you must offer justified criticism of the literature relevant to your topic. But, as an overseas student, you may also be concerned that your criticisms of your own government’s policy or accepted textbooks may lead to you or your family being punished. In this special case, you can ask for those criticisms to be excluded from public copies of your dissertation. This chapter introduces two ways of evaluating published research: research component analysis (RCA); and, Rose’s ABCDE validity analysis.
70
Research Methods in Politics
Research component analysis Research component analysis dissects the published research into 15 or more components which are assessed separately. The absence or weakness of any one is likely to be significant but not necessarily fatal. The analytical components are: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.
when where who abstract research question and rationale theory and literature review hypothesis methodology data collection: what, when, how data presentation
11. 12. 13. 14. 15.
data analysis, interpretation and discussion conclusions implications style referencing and bibliography.
When concerns the data of publication. The assumption is that recently-published research is more likely to incorporate the latest data, thinking and state of the art than earlier research, say, published ten years ago. Research with a publication date pre-2000 is now ‘last century’. Ceteris paribus (all other things being equal), most recent research can be considered more valuable than older publications. But where can be more significant than when. Research published in a refereed, academic journal is likely to be of greater quality and authority than research, say, published as a departmental working paper. The process of refereeing means that, before the research is published, it will have been sent to two or more academic referees for their assessment. They will have been chosen for their reputation as commentators or researchers in the field of research. Their anonymous evaluations will have been forwarded by the editor to the researcher who will be told whether the research is accepted for publication, refused or whether changes are sought in the light of the referees’ observations. Refereed UK journals of political science include: British Journal of Politics & International Relations, British Journal of Political Science, Comparative Politics, European Journal of Political Research, Journal of Public Policy, Party Politics, Political Quarterly, Political Research Quarterly, Political Science Quarterly, Political Studies, Political Studies Review, Governance, Politics, Politics and Policy, Public Administration, Journal of Theoretical Politics, Public Administration and Development. PhD
Critically Evaluating Published Research
dissertations will have been subject to a similar process of examination by leading academic specialists and modification; although unpublished, they will be a rich, authoritative source. You can inspect these at the university libraries where they are held, request copies or microfilm, or, in some cases, read on-line. Who refers to the provenance and attribution of the research, i.e. claimed authors. Many journal articles are attributed to more than one author. They can be a leading academic researcher and a research associate (normally named last). In those circumstances, then it is likely that the leader won the funding whilst the associate undertook and wrote-up the research. The source of funding may also be significant. Funding by government research agencies – ESRC, AHRB, MRC2 – signifies that the research topic is considered of national significance. Similarly, EU or UN funding is indicative of higher status research. Funding by these bodies means that the researchers are likely to have been given access to decision-makers and other privileged sources of information that may not have been available to others. The academic department may also be significant. Research undertaken within a UK department given a 5* rating in the most recent RAE is likely to be of a higher standard than that of another with a lower rating.3 The abstract provides a synopsis of the research undertaken and its conclusions. It should ideally attract your attention and stimulate your interest. Are the topic and conclusions relevant to your proposed research? Research question and rationale. The centrality of the research question distinguishes and elevates research from essays or other scholastic activities. The research question should be clearly identifiable and unambiguously stated. The rationale – the reason(s) for undertaking the study should also be given. They should show that the research question relates to a topic of acknowledged importance which should ideally be topical and a significant problem, i.e. a barrier to policy objectives. The rationale should also set out the context. The theory and literature review should demonstrate that the researcher has taken the necessary steps to cite and critically evaluate the relevant explanatory theories and the main literature. The review should pick out contradictions, overlaps and gaps. The review should justify the selection of any theory which is to provide the conceptual lens of inquiry and identify any uncertainties which the research should address. The hypothesis – the researcher’s initial answer to the research question – must be given where a deductive methodology is to be adopted. It should cite the causal, independent variables inferred from the theoretical and literature review and their relationship with the outcome. The hypothesis may be single or doubletailed. A single-tailed hypothesis will specify the directional relationship of the independent variables x, and the outcome y, e.g. as x increases, y will also increase. A ‘double-tailed’ hypothesis will merely assert that, as x increases, y will change. The methodology should state and show consistency between the underlying philosophy (positivistic or otherwise), inductive or deductive approach, quantitative
71
72
Research Methods in Politics
or qualitative method, and method of data collection (case study, comparative study, etc.). Data collection should identify what data was collected, why, by whom and when (date and duration). The selection of data is critical. Where the variables and concepts are not directly measurable, then the selection of the indicators used must be justified. A number of indicators may be adopted to represent the concept under scrutiny. For example, overseas aid may be expressed either as the donor’s budget allocation, locally purchased goods and services, technical assistance, etc. This may show that, whilst state A allocated most funds to overseas aid, state B invested a higher volume of aid locally. Indicators will also have been chosen to reflect local changes, e.g. changes in infant mortality. No single measure is likely to be sufficient. The description of the data collection process should also describe and justify the choice of samples made, questionnaires designed and elites interviewed. Data presentation concerns how key data is presented in the research report and how it is illustrated and supported. Sufficient data must be given to enable the reader to share and have confidence in the researcher’s analysis and conclusions. Graphs and diagrams are particularly helpful. ‘Quotable quotes’ from interviewees are especially useful. Data analysis, discussion and interpretation should be considered separate from its presentation. The method of analysis should be described and its selection justified. Where appropriate, the confidence limits should be made clear. The discussion should review the potential relations and explanations between variables leading to their interpretation by the researcher. The conclusions should summarise the research and state whether the initial hypothesis was confirmed, qualified or infirmed. Other important findings should be given. Research that entirely confirms the starting hypothesis must be treated cautiously. The implications should say ‘what the research means’, the lessons learned and advances claimed, overall significance and the scope for follow-on research projects. Style refers to the general ‘readability’ of the research. Does the report do justice to the research undertaken? In plain English, is it boring? How does the author maintain (or lose) pace and interest? Does the researcher exaggerate the significance of the research or their role? Referencing and bibliography should provide proper acknowledgement of others’ earlier works and a ‘paper trail’ of texts which others can follow. The type of referencing system adopted – Harvard or footnotes etc. – is less important than its consistency of application. Finally, re-read the abstract. Does it fairly reflect the contents and importance of the research report? Research component analysis can be criticised for its ‘checklist’ approach and its greater concern for the individual components than their linkages and overall
Critically Evaluating Published Research
coherence. By comparison, Rose’s ABCDE deciphering method offers a more holistic approach.
Rose’s deciphering model (1982)
A
Theory: an explanatory statement about social phenomena.
↓ B
← ↑
Theoretical Proposition: specific propositions to be
←
investigated in the study.
↑
↓ C
↓
↑ Operationalisation: decisions made on how to carry out
←
empirical work; techniques of data collection;
↑
sampling; concepts and indicators; variables; units.
↑ ↑
D
Fieldwork: collecting data; practical problems of
↑
implementing Stage C decisions.
↑
↓ E
↑ Results: data analysis leads to findings; interpretation leads back
↑
to C, B, and A
Figure 6.1 Rose’s ‘model for the research process’ Source: (Rose, 1982: 14)4
Despite its longevity (1982), Rose’s deciphering model has not been bettered. It is premised on a Weber-type ideal model of empiric research. This model has five stages and elements named – somewhat prosaically in an era before memorable acronyms became widely adopted – A, B, C, D, E. The model for theory-testing research is: A is the theory which is being tested. The propositions in B are the hypotheses in the form of testable statements about the causal relationships between the independent variables and the outcome. Rose classifies stage C as the ‘pivotal stage in the research process’. Here, the theoretical concepts are transformed into measurable
73
74
Research Methods in Politics
indicators (concept indicators) on which data is collected in Stage D and analysed in Stage E The concept-indicator link is highly problematic. For example, Downs applies rational choice theory in his Economic Theory of Democracy (1957) to develop three hypotheses.5 His second hypothesis is that: every citizen rationally attempts to maximise his utility income, including that portion of it derived from government activity. (Downs, 1957: 297).
From this hypothesis, he derives 15 ‘testable propositions’. They include the relatively straightforward proposition (19) that: the percentage of low-income citizens who abstain in elections is higher than the percentage of high-income citizens who abstain (ceteris paribus). (1957: 299).
This proposition can be tested at micro- and meta-levels. A large sample of voters could be randomly selected and asked details of their income and voting. But, income data can be a very sensitive area. The prospect of a high level of response is poor. A more acceptable means of asking people’s income might be to adopt thresholds of, say, less than £12,000 for low income and more than £35,000 for high incomes. In those cases, the thresholds have become indicators of income. The meta-level of analysis would compare the (readily available) data on ward turnout with socio-economic data from the census. In this case, turnout (or its absence) could be used as an indicator of abstention whilst census data on house tenure, social class, education and occupation, etc. would be used as indicators of income. Using multiple regression analysis, the relationship between turnout and socio-economic variables, and the power of the equation can be calculated. Rose uses his model of the research process to provide a three-stage framework for critically evaluating research: Central to this method of evaluation is the concept of validity. Derived from the Latin, validus meaning ‘strong’, validity in the context of research means: the extent to which a measure, indicator or method of data collection possesses the quality of being sound or true as far as can be judged. … in the social sciences generally, the relationship between indicators and measures and the underlying concepts they are taken to measure is often contested. (Jary & Jary, 1995: 714)6
Critically Evaluating Published Research
Relationship to other theory and research External validity A
Theory:
↓ B
↑ Theoretical Proposition
↓ C
Operationalisation
←
← ↑
Fieldwork
↓ E
Internal theoretical validity
↑
↓ D
←
Internal empirical validity
↑ ↑
Results
↑
Figure 6.2 Research Evaluation Source: Rose, 1982: 32, Figure 2.2
Rose suggests that the first stage involves evaluating the internal empirical validity by assessing the extent to which the conclusions are fairly drawn from the data collected. The second stage involves assessing the internal theoretical validity. This examines whether the theoretical propositions have been fairly drawn from the chosen theory and whether the stated concepts are fairly represented by the indicators chosen. The final stage examines the external validity – the strength of the relationship between the research, the wider literature and the ‘real world’. A criticism of this method of evaluation is that, by working backwards through the research process, it ‘puts the cart before the horse’, i.e. if the research does not relate well to the literature (or the indicators to the concepts, etc.) then the internal empirical validity doesn’t really matter. Rose develops separate ‘strategies’ for evaluating research reports based on quantitative and qualitative data. The strategy for evaluating theory-testing, quantitative research has six steps: 1. summarising the research into ABCDE stages 2. assessing the operationalisation in C: concept-indicators, samples, units and variables
75
76
Research Methods in Politics 3. adding further descriptive details of: the researcher’s theoretical exposition and theoryevidence linkages; and tables of data presented, reliability and accuracy, and interpretation 4. assessing the internal empirical validity by examining the ‘fit’ between the data and its interpretation 5. assessing the internal theoretical validity, i.e. the relationship between A (theory ) and B (hypothesis) 6. concluding with a general assessment which consolidates the separate assessments made and makes an assessment of the overall contribution of the research to the discipline. (Rose, 1982: 104–5)
The strategy for evaluating theory-building research utilising qualitative data is very different. It can be reduced to three steps: 1. summarising the research report into its ABCDE stages stating: its natural history (origins and development); the data and methods of collection; sampling; method of analysis; and presentation of results 2. evaluating in turn the validity of: the concept-indicator links; theory; and sampling and generalisation 3. concluding by reviewing the consistency (or otherwise) of the various validities and other factors and assessing of the overall contribution of the new theory to the discipline. (Rose, 1982: 130–2)
Rose can be criticised for the assumptions made about the neat dichotomy of theorytesting/quantitative and theory-building/qualitative research and for unnecessarily complicating the evaluation strategy. Both research component analysis and Rose’s ABCDE provide good, highlystructured methods for analysing research which are particularly useful for first-time or inexperienced researchers. With practice, you will learn how to home directly on the strengths and weaknesses of published research and to answer the question: does it do what it says on the tin?
Questions for discussion and assignments
1. Consider the concepts of: conflict; freedom; liberty; hegemony; and consciousness of identity. What concept-indicators would you suggest for these? Why? What data would you collect?
Critically Evaluating Published Research
BOX 6.1
UWUK Department of Politics University of Watersea Dr Adrian Helvetica Editor New Politics Digest [date] Dear Arial [title of draft research report] Thank you for inviting me to referee this paper. Overall, my view is that, whilst this excellent paper breaks new ground, there is a number of areas where minor revisions and additions would be helpful. Let me begin by summarising what I believe are the research question, theoretical framework, the hypothesis and the key concepts involved. The research question etc., is …. [200–300 words]. The strengths of the paper are … [300–400 words]. The weaknesses are … [300–400 words]. I would also like to comment on the style of the paper … [200–300 words]. The paper is, of course, suitable for publication in your journal in its present form. However, I would suggest that a number of small changes are made. They are … [200–300 words]. I look forward to meeting you at the Washington conference. Best wishes, Prof [your exam number]
77
78
Research Methods in Politics 2. [For undergraduates] You are a distinguished academic. You have been asked by the editor of a refereed journal to act as a referee for a draft research report. Select one of the two published reports: Bevir, M. and Rhodes, R.A.W. (2006) Prime Ministers, Presidentialism and Westminster Smokescreens, Political Studies, 54 (4): 671–90 Chaney, P. (2006) Critical Mass, Deliberation and the Substantive Representation of Women: Evidence from the UK’s Devolution Programme, Political Studies, 54 (4): 691–714. Write your assessment using the letter (above): 3. [For graduates] Select a research report on a subject that interests you from a refereed journal in your university library. Evaluate it critically using research component analysis or Rose’s strategies. Attach a copy of the research paper to your assignment showing what you concluded were the various analytical components. How could the research and the report be improved?
FURTHER READING Rose, G. (1982) Deciphering Sociological Research. Basingstoke: Macmillan. pp. 155.
Notes 1 Rose, G. (1982) Deciphering Sociological Research. London: Macmillan. 2 ESRC, AHRB and MRC are the acronyms of the government-funded Economics & Social Research Council, Arts & History Research Council and Medical Research Council. 3 RAE: Research Assessment Exercise: a process of academic review which concentrates on research performance and which provides ratings which determine allocations of government funding. 5* is the highest rating. 4 Rose, G. (1982) Deciphering Sociological Research. London: Macmillan. Figure 2.1, p.14. 5 Downs, A. (1957) An Economic Theory of Democracy, New York: Harper & Brothers. 6 Jary, D. and Jary, J. (1995) Sociology, Glasgow: HarperCollins.
Chapter 7
Evaluating Information: Validity, Reliability, Accuracy, Triangulation
‘Do not feel absolutely certain of anything.’ Bertrand Russell, 19511 Teaching and learning objectives:
1. To consider why information should be assessed 2. To understand the distinction between ‘primary’ and ‘secondary sources’ of information 3. To learn what is meant by the validity, reliability, and accuracy of information 4. To consider some warnings about ‘official data’ 5. To consider further the distinction between ‘facts’ and ‘truth’ 6. To understand the origin of triangulation and its application to research 7. To consider methods of sampling which can be used to collect data.
Introduction In Chapter 6, you read how published research reports can be assessed. The research component analysis and Rose’s ABCDE model examined the completeness and coherence of the research process adopted. They also considered the validity or otherwise of the relationships between theory and hypothesis, concepts and indicators, empiric data and analysis, and conclusions. Research essentially involves the gathering or collection of data that addresses the research question and enables theory to be tested or developed. So the data from which answers to the research question are to be drawn must be appropriate in terms of its relevance and efficacy – ‘fitness for purpose’. Much of this information will be drawn from published sources
80
Research Methods in Politics
that will be supplemented as necessary by new information specially collected for the research project. So this chapter therefore suggests how best you can assess existing data and seek additional material. Many textbooks use information and data interchangeably. Some complicate matters by treating ‘data’ as a plural noun and therefore writing ‘the data are …’. While this is grammatically correct (for data is indeed the plural of the datum), it can sound odd to students untrained in Latin conjugation. The author, Kingsley Amis notably described such Latin correctness as the practice of ‘wankers’ as opposed to ‘berks’ who used slipshod English (1977).2 So this book follows everyday practice of treating data as singular. Politicians also tend to use the word ‘evidence’ to describe what they would wish us to regard as ‘conclusive, compelling information’ which either proves or, in its absence, disproves allegation. But is there a real difference between data, information and evidence? Certainly, the dictionary meanings are similar. But some distinction is useful. Researchers tend to speak of data as the mass of disordered, raw material from which information (knowledge) is abstracted to provide evidence to support argument and conclusions. (Information technologists adopt a similar distinction by defining information as processed data sets attaining meaning.) Information informs. Evidence supports conclusions. So it is helpful to conceive of research as involving three stages. First, the raw data is gathered. Second, the data is organised and distilled into information. Third, evidence is abstracted from the information through processes of analysis and testing. But neither information nor evidence is self-evident: the material seldom ‘speaks for itself’. Some interpretation is required. However, when interpretation is reinterpreted, some distortion of the original is inevitable. So some distinctions, criteria and tests are useful to weed out distortions and ‘untruths’. The distinctions adopted are between primary and secondary sources of information. The criteria used are validity, reliability and accuracy. The main test adopted is triangulation.
Primary and secondary sources The value of this distinction depends on which of the different definitions of primary and secondary is adopted. Some authorities adopt the definition that primary information is data generated specifically for the research project whilst secondary information is data collected for other research. But, in this book, the ‘majority view’ prevails: that data is distinguished at the outset by its provenance (source). Primary data is original, unedited and ‘first-hand’ whilst secondary data is ‘second-hand’, edited and interpreted material. However, the distinction between the information that you generate in the course of our research and that which you have abstracted from other sources is valuable. I will therefore term this (after Huxley) α data
Evaluating Information: Validity, Reliability, Accuracy, Triangulation
and β data.3 Wherever possible, Politics researchers prefer to use primary, eyewitness data recorded at the time by participants or privileged observers. The main sources of primary data used by Politics researchers are fourfold: 1. contemporary documentary (written) records including minutes, letters, emails and diaries 2. your interviews with key individuals, ‘agents’ and ‘actors’ 3. numerical records, e.g. election results, census data 4. your own observation and records of interviews, etc. and other events.
Other sources are popular songs, poems, paintings and cartoons, photographs, graffiti, murals (e.g. N. Ireland), T-shirts and videos. But beware, all records, however ‘primary’ incorporate some degree of bias, perception, interpretation, and editing, whether contextual, cultural, curatorial or deliberate. Written primary records include accounts of meetings, minutes, diaries, letters, reports, telephone transcripts, telegrams, emails, and newspaper reports, etc. But how reliable, accurate and truthful are they? Who actually prepared them and why? Arguably, all accounts are partial because they are functional, i.e. designed to fulfil a purpose. But whose purpose? Most public records reflect the interpretation of those holding power. Foucault argued that the victors write history. Alternatively, how reliable are the diaries of (former British Labour Cabinet ministers) Richard Crossman, Barbara Castle or Tony Benn? Did their cabinet colleagues know that they were keeping diaries and change their behaviour accordingly? Crossman offers telling insights into official records: Thursday, 28 July 1966 One of the disconcerting features of the recent crisis has been the Cabinet Secretariat’s habit of suppressing whole sections of the minutes on the grounds that they are too secret to circulate. But this morning they didn’t do that.The section on prices and incomes was reported at enormous length and most of what we said has been very adequately summarised. Of course, this means that the Cabinet Secretariat regards the whole subject as fraught with danger and was careful to record the arguments of the opponents. Cabinet minutes are highly political and the way they are written has enormous effect. By eliminating whole sections from the discussion and reporting other sections in full, the Secretariat can greatly affect the way a decision is interpreted in Whitehall. (Crossman, 1976: 590)4
But the BBC’s former Political Editor, John Cole wrote of Crossman that: … [Crossman] had a brilliant mind, was a great polemicist, and a subtle – though sometimes self-defeating – operator. But if you were Constable Plod seeking a reliable
81
82
Research Methods in Politics witness, he would not be your first choice. I sometimes wondered if he knew how to distinguish what he said to the Prime Minister from what the Prime Minister said to him. (Cole, 1996: 64)5
But even PC Plod can be an unreliable witness: Churchill’s bodyguard, Detective Inspector Walter Thompson was criticised by Churchill’s biographer, Roy Jenkins for exaggerating his importance in events (Jenkins, 2001: 552).6 Inspector Thompson in his two volumes of reminiscences is good at capturing the heart of the matter but less reliable on exact dates, times and places than might have been hoped for a meticulous detective. (Jenkins, 2001: 562)
As a general principle, all primary information in the form of records – other than those that you make through your own observations – should be treated with caution. A ‘health warning’ is necessary. You should always ask yourself: 1. 2. 3. 4. 5. 6.
who prepared the record? why? for whom was it prepared? for whom was it intended? for what purpose was it made? who would have ‘corrected’ or otherwise altered the record before it was finalised?
A common misconception is to believe that numeric information is more trustworthy than other formats because it is less vulnerable to ‘spin’. But, because numeric records are generally regarded as trustworthy, they attract manipulation. For example, the TUC and ILO accused the Thatcher government of changing the definition of unemployment 23 times (between 1979 and 1991) to reduce the headline figure and therefore conceal the true extent of unemployment. The government responded to the criticism by saying that each new definition distinguished further between genuinely unemployed people and others claiming to be unemployed to obtain benefits. A similar charge was levied later against the New Labour government that the lower levels of unemployment recorded and reported had been achieved by accepting more readily claims (on mainly health grounds) for the (higher) incapacity benefit. You should therefore check numerical records for any changes of definition and any selective use of periods to enable worst records to be omitted. By implication, data that is not primary must be secondary – after the event, secondhand. But it should not be discarded. Secondary information will include records
Evaluating Information: Validity, Reliability, Accuracy, Triangulation
gathered from a number of separate, primary sources and may contain authoritative commentary and analysis. The source’s interpretations and bias are important – especially of evidence of how events were interpreted at the time and later, and the moral relativism of value-judgements.
Validity, reliability and accuracy As you learned in Chapter 6, social science research confers a special meaning to validity: the extent to which a measure, indicator or method of data collection possesses the quality of being sound or true as far as can be judged. … in the social sciences generally, the relationship between indicators and measures and the underlying concepts they are taken to measure is often contested. (Jary & Jary, 1995: 714)7
In effect, the validity of information is its relevance and appropriateness to your research question and the directness and strength of its association with the concepts under scrutiny. Often you will have to use best available information whose validity may be weak. For example, to what extent, if any, does the decline in ‘sectarian violence’ in N. Ireland post-2001 reflect a lessening of antagonisms between conflicting groups? Does the election of an opposition party reflect popular support for its manifesto or criticism of the outgoing government? Do declining rates of party membership reflect a lessening of interest in health and education? One measure that intrigues Politics researchers is the counterfactual – events that don’t happen – as evidence of hegemonic domination.8 But how can researchers be confident that the absence of an event can be attributed to the omnipresence of another? One solution to this particular problem of problematic validity is for you to adopt a wider range of measures to reduce dependence on any one. Reliability is, literally, the extent to which you can rely on the source of the data and, therefore, the data itself. Reliable data is dependable, trustworthy, unfailing, sure, authentic, genuine, reputable. Consistency is the main measure of reliability. So, in literary accounts, the reputation of the source is critical. In John Cole’s view, Richard Crossman was not a reliable diarist. Indicators of reliability will include proximity to events, (whether the writer was a participant or observer), likely impartiality, and whether, as the police say, the record was really contemporaneous or an eventide reflection on the day’s events. Very few politicians admit to real failings: all too often, their own agenda appears to justify their actions or to criticise others. Tony Benn’s diaries seek to portray the inner workings of
83
84
Research Methods in Politics
cabinet government. But Dennis Healey claimed (playfully) that Tony Benn ‘always seemed to be on the toilet every time a difficult decision had to be made’ (BBC2 interview). Accounts may have been ‘sexed up’ to promote sales. Biographies may be hagiographic. For example, Michael Foot’s biography of Aneurin Bevan uncritically portrays the Welshman as a wholly heroic figure, whereas my father – a fellow native of Blaenau-Gwent – told me how, after 1948, some local trade unionists called the Ebbw Vale MP ‘Urinal Bevan’. This epithet followed Bevan’s assertion that: it is for the [Party] Conference to lay down the policies of the Parliamentary Party, and for the Parliamentary Party to interpret those policies in the light of the parliamentary system. (Foot, 1973: 236)9
In contrast, Grigg’s biography of another, Welsh hero, Lloyd George, provided a ‘warts and all’ portrait (Grigg, 1978).10 Numeric data need not necessarily be reliable. The source – even official statistics – may not be wholly impartial. Populations may be undercounted (e.g. 2001 census). The samples used may be insufficient or not randomly selected. Confidence limits (margin of error) may be omitted. The rate of non-responses to questionnaires may be disguised. Respondents may not have been wholly truthful in their replies. For example, on the basis of replies to their questions, most opinion polls (wrongly) predicted a Labour victory in the 1993 general election. Inappropriate statistical techniques may have been used. But reliable witnesses may also be inaccurate on occasions. Andrew Marr, John Cole’s successor as the BBC’s Political Editor and a former editor of The Independent is very sceptical of the reliability of modern-day news reporting by the newspapers and TV news services (Marr, 2004). He blames this on the competition to drive down costs, consequent reductions in the number of journalists, and their being confined to their desks where they must too readily accept the stories ‘fed’ them by professional press officers. He recommends readers (and researchers) to: Know [which newspaper] you’re buying. Reporting is so contaminated by bias and campaigning, and general mischief, that no reader can hope to get a picture of what is happening without first knowing who owns the paper, and who it is being published for. The Mirror defines its politics as the opposite of the Sun’s, which in turn is defined by the geo-politics of Rupert Murdoch – hostile to European federation and the euro … It is ferociously against Tony Blair, this is because Number Ten has been passing good stories to the Sun. (Marr, 2004: 251)
Evaluating Information: Validity, Reliability, Accuracy, Triangulation
He also warns against news of research from: hundreds of dodgy academic departments put out … to impress busy newspaper people and to win themselves cheap publicity which can in turn be used in their next funding applications. (2004: 254)
Similarly, Marr explains that TV news editors are: biased towards exciting or unusual pictures; news that is refreshing or odd; and news that bears some relation to viewers’ lives. (2004: 291)
So anything that looks dull, ‘stories about northern European countries, about buses, about old people, about infrastructure, banking, manufacturing, Whitehall and regeneration,’ is unlikely to be televised. Marr argues that a task of TV news is to increase viewing figures – which means also retaining the viewers of the preceding programme – usually popular light entertainment of the ‘soap’ or ‘chat show’ genre. Accuracy is sensitivity to change – especially of detail, e.g. dates, numbers, persons present, etc. Remember that some biographers deliberately add false detailed information to trap and sue plagiarisers.
Facts and truth Once again, you will find that adopting a critical distinction between facts and truth is useful. Facts are the available data. They present incomplete snapshots of events. Truth is the reality behind the facts. Sometimes the facts may obscure the truth – perhaps deliberately so. A good example was provided to me by a leading academic. He privately described how he had critically reviewed a bestselling account of British rural life where the author had misrepresented the facts by combining material from a number of interviews to represent a composite figure. The author had replied to the effect that his critic was unable to distinguish between the facts and truth.
Interviews Interviews with political elites provide a major source of information in Politics research. They may be undertaken by the researcher or, where personal access is not
85
86
Research Methods in Politics
possible, by watching video recordings of interviews in TV news and documentaries. But you must never assume that what you are told or hear is reliable and accurate. For example, a former Prime Minister told me that he strongly supported a specific White Paper. However, the Minister of State who claimed to have instigated the new policies told me that the Prime Minister had opposed the White Paper. You must always question (implicitly) the answers to your questions and look for signs of deception or self-deception by informants, e.g. the coping strategy of long-term prisoners who are guilty but believe that they are innocent, i.e. in denial. TV interviews (i.e. secondary sources) are highly edited – especially field interviews where a single camera is used or where the interviewee has been granted some editorial control. Triangulation is the means adopted by researchers to secure effective corroboration. However, before this method is described, consider the case study below:
Case Study Harold Nicholson provides a detailed narrative of the fall of the second Labour government in 1931 and its replacement by a National Government which was to last effectively until 1945 (Nicholson, 1953: 453–469).11 The Labour Prime Minister, Ramsay MacDonald, became the leader of the National Government in what became named by Labour Party members as the ‘great betrayal’. Nicholson describes the relevant background as the rapidly deteriorating public finances caused during the worst years of the Depression when the demand for public expenditure on unemployment benefit etc. grew whilst income from taxation fell. In response to demands by the Conservative and Liberal parties (amplified by the Tory press), the Government formed an independent committee under Lord May. On 31 July 1931, May recommended substantial cuts of up to 20% in public sector salaries, 20% cut in unemployment benefit and reduction in the pay of the armed services to 1925 levels. But two of the six May members issued a minority report dissenting from May’s recommendations on the basis that the costs of the cuts would fall mainly on the working classes. Nicholson wrote that ‘The rank and file of the Labour party agreed whole heartedly with [the Minority Report]; MacDonald and Snowden [Chancellor of the Exchequer] did not’ (1953: 455). Nicholson reports how, later that day, MacDonald formed a five-man, special, Cabinet Economy Committee to consider how May could be implemented. The ‘Big Five’ consisted of MacDonald, Snowden (Chancellor), J. H. Thomas, Arthur Henderson and William Graham.The likely continuing withdrawal of deposits held in London meant that the government would be unable to fund the public sector deficit without support from bankers in Paris and New York. The bankers were unwilling to lend the
Evaluating Information: Validity, Reliability, Accuracy, Triangulation
money unless and until firm proposals were made to balance the UK budget. On 19 August, the Cabinet’s Economy Committee proposed cuts similar to the May report. Half the deficit would be met by reductions in unemployment benefit and public sector pay and the other by increasing taxation etc. According to Nicholson, the proposals were reluctantly approved by a majority of the Cabinet with the exception of transitional unemployment pay (1953: 457).They suggested an additional revenue tariff which MacDonald told them would not be accepted by the Liberals.The TUC – which had created and funded the Labour party – met MacDonald on 20 August. The Council refused to accept cuts in unemployment benefit or public sector pay. On 22 August, MacDonald proposed a modified version of the programme – including a 10% reduction in unemployment pay – to the Cabinet. He obtained Cabinet support to ‘enquire’ of the Opposition leaders whether the revised proposals were acceptable. The Opposition leaders responded that the overseas bankers’ support was critical to wider support for the package. The King (George V) had been kept informed by MacDonald of the increasing crisis. He returned from Balmoral to London on 23 August when he was told by MacDonald that leading members of the Cabinet would not support the latest proposals (being considered by the bankers). The King decided that the ‘correct constitutional course’ would be to meet the leaders of the three main parties: MacDonald (Labour), Baldwin (Conservative) and Samuel (Liberal, as Lloyd George was in hospital). Nicholson describes how the King stated his preference for MacDonald and the Labour government to stay in office and to implement the cuts. If that were not practicable, then the best alternative would involve the formation of a National Government – headed by MacDonald – with a Cabinet drawn from three parties and commanding a sufficient majority in the House of Commons to approve the necessary legislation. The King met the three leaders separately who agreed to join a National Government if necessary. Once the crisis had been resolved, then new elections should be held. Later that day, the Labour Cabinet met. After a long adjournment, they were advised by telegram from the Government’s agents (the bankers, J. P. Morgan) that the necessary US public support for a public loan would be problematic until Parliament had approved the proposals. They suggested a short-term treasury loan. Finally, they enquired whether they were correct in assuming that the package proposed by the Cabinet had the support of The Treasury and City. But the Cabinet had not and would not approve the programme. MacDonald told them that he would report the divisions (eleven in favour: eight opposed) to the King whom he would ask to convene a meeting of the three party leaders. He would tell the King that the Cabinet had placed their resignations in his hands. He immediately reported to the King. Acting unilaterally, MacDonald began that night to plan the new Government Continued
87
88
Research Methods in Politics
with what Snowden described ‘an enthusiasm which showed that the adventure was highly agreeable to him’ (1953: 465). On 24 August, the King met the three party leaders who agreed to form a National Government. MacDonald tendered his resignation. He was then invited to lead the National Government. He asked them to prepare a communiqué saying that the formation of the new Government was being considered. The party leaders agreed that the new Government would not be a coalition but a ‘co-operation of individuals’. MacDonald described the proposals to his Cabinet and invited them to join a new ‘Cabinet of Individuals’. With the exception of J. H. Thomas, Lord Sankey and Philip Snowden, they declined. Following the resignation of the Labour Cabinet, the new Cabinet was formed on 26 August with Baldwin as Vice-Premier.
There are a number of other accounts of this episode. One central dispute among them is whether the King or MacDonald first raised the proposal for a National Government, i.e. whether MacDonald accepted the King’s proposal to lead a national government out of patriotism and loyalty, or, proposed the arrangement as a means of continuing in office and increasing his power whilst appearing to follow the constitutional requirements of the King. For example, the celebrated Labour historian, G. D. H. Cole wrote that: The exact method of the split is vehemently disputed. The Labour Cabinet was still discussing the outrush of gold and the ‘threat to the pound’ under the influence of the deliberately exaggerated menaces of Philip Snowden, when it came. They had agreed to enormous concessions but jibbed (it is stated) at penalising the unemployed. Then it was put to them that arrangements had already been made, with the King’s consent but clearly on MacDonald’s initiative for the formation of a ‘National Government’ of Labour, Conservatives and Liberals. (Cole, 1938/66: 593)12
So how much importance should the researcher apply to Nicholson’s account where it differs from others in critical respects? Obviously, the researcher will seek corroboration from other sources and assiduously compare the various clues. But just how reliable is Nicholson’s account. Who is Nicholson? What was his book about? Why did he write it? What documents and witnesses did he have access to? The answers are that Sir Harold Nicholson (1886–1968) was a distinguished diplomat, historian and biographer. He was educated at Wellington and Balliol
Evaluating Information: Validity, Reliability, Accuracy, Triangulation
College. He attended the Paris Peace Conference (1919) as a diplomat. He retired from the service to become a writer. In 1931, he stood as an MP for Harold Mosley’s New Party but left when Mosley formed the British Union of Fascists. He became a National Labour MP in 1935 but was defeated in 1945. He became a governor of the BBC. He was a ‘man of independent means’ who married Vita Sackville-West. They lived at Sissinghurst Castle. Both were bi-sexual and practised an ‘open marriage’. However, their life-style was not unusual amongst the upper classes. We can therefore regard Nicholson as an Establishment figure albeit on its arts and literary wing. His book was entitled King George V and published in 1953, i.e. 27 years after the King’s death in 1936. He had been appointed by the Royal Family to write an ‘authorised biography’. So Nicholson had access to the King’s diaries and the official papers kept by his secretaries. He also interviewed the King’s secretary, Lord Samuel, Herbert Morrison and other participants. However, we are unaware of the editorial control exercised by the Royal Family. Clearly, the overall purpose of the biography was to make public King George’s hand in the making of history. But Nicholson could not be accused of presenting a wholly flattering picture of the King. For example, he observed that George was neither very clever nor witty: he was a relatively dull man who was therefore entirely representative of the British people. Overall, we can probably conclude that Nicholson’s account is probably very accurate in terms of the detail. However, we cannot be entirely sure about its reliability: the biography is more likely to portray King George V as a central figure, imposing wise, constitutional solutions, rather than acquiescing to what other commentators (like Cole) saw as MacDonald’s devious scheme. No single account can ever be regarded as wholly reliable or accurate. Other sources must be sought and used. However, the number of competing sources may be huge. The most widely used method of selecting sources and materials from the range available is termed triangulation.
Triangulation Triangulation is a method developed over the centuries for navigation and surveying. It provides the basis for satellite navigation. Its origins lie in geometry. A point can be precisely defined in space by the angle it subtends to a line joining two other points by the application of the Law of Sines. Any geographical area could be mapped by first selecting two ‘trig points’ (triangulation points) that are a measured distance apart and then recording the position of any other point in terms of the angle subtended. In this way, a third trig point could be established and the exercise extended. In navigation, triangulation is used to establish a ship’s precise position by taking bearings on three or more known landmarks. The position is
89
90
Research Methods in Politics
most accurately determined when the three points are equally located around the ship, namely
Lighthouse A
Lighthouse B
Ship ⇑
Lighthouse C
The same method is used in Politics research to obtain an efficient corroboration of any crucial account. Triangulation involves seeking accounts from three or more perspectives. So, for example, a researcher investigating the General Strike of 1926 would seek to obtain accounts from the TUC, the Government and at least one source independent of the two adversaries. Politics researchers face a special challenge: the two main parties and their perspectives are often diametrically opposed to each so that any, third-party, independent views are gained from only one side. To overcome this difficulty, they seek as many independent sources as possible. Furthermore, Politics researchers seek to triangulate at each level of data media. So you should seek to triangulate between contemporary written records and news reports, autobiographies, personal interviews with participants, and other research narratives. You should also try to find new angles. However, given that each party will adopt different perspectives, then the ‘truth of the matter’ may be unique to each participant. You may be able to repudiate some accounts but you may find that you are unable to offer a definitive version of events. Indeed, the participants may be unsure of their real motivations or involvement. At the end of your triangulation, you may well know more about and understand better the particular event than the participants because you will have accessed records unavailable to them. But remember the adage that, whilst success has many parents, failure is an orphan. However, by demonstrating the application of triangulation, you will be able to show the reader the process by which corroboration has been sought. You should also be able to pinpoint both gaps in and inconsistencies between the accounts.
Evaluating Information: Validity, Reliability, Accuracy, Triangulation
Sampling You will quickly find that, even when you adopt triangulation, the volume and potential sources of data in terms of people and records can still remain vast. So a selective approach is essential. You simply cannot interview every member of a union or examine every council minute. The age-old solution to this particular problem is to concentrate your activity on a sample of the population. These everyday words have special meanings in academic research which warrant explanation. The population is the universe of all the subjects or cases under study. You must define your population. It may be all the members of a union, the residents of a city or town, states of Africa, a particular ethnic or age group and so on. The population is a set of individuals, cases, states, etc., which share a common characteristic. The sample is a selection of individuals, cases, states, etc., made from the population. The sample is, therefore, a subset of the population. You must define your research population in a sample frame. Your sample frame is a list or schedule of the population from which the sample will be drawn. It may be a membership list or a directory of engineering firms. You can use either probability or non-probability samples.
Non-probability samples Non-probability samples are samples where members of the population do not have an equal chance of being selected. They are not statistically reliable. They cannot generate generalisable data. You select the members of the sample. Nonprobability samples include nomination, snowballs, volunteers, case studies and theoretical samples (used in grounded research). Non-probability samples will not generate generalisable information. You have already been introduced to case studies and theoretical samples in Chapter 5. Non-probability samples are very small. They are used for qualitative research. Nomination is the most widely-used form of recruiting a non-probability sample. Essentially, you ask a local social gatekeeper or intermediary to nominate (name) a group of people who meet your requirements for a research sample. A social gatekeeper exercises control over who enters a community. They may be head teachers, village headmen, employers or heads of family (if you want to interview children). Essentially, you search them out, write to them explaining who you are and the nature of your research, and ask them to nominate a cross-section of local people to interview. Often, they will offer to arrange the meetings. But this offer of assistance is incompatible with the ethical ideal of voluntary consent. The people nominated may feel as if they have been commanded to attend. This will inhibit your meeting: they will feel obliged to say what the gatekeeper would like them to say. It is better if you contact the named people individually (preferably in writing). Again you should introduce yourself, explain the nature of your research and its
91
92
Research Methods in Politics
benefits to them. You should say why you want to speak to them individually or at a (focus) group meeting, and add that X (the local social gatekeeper) has suggested that they might be able to help. Intermediaries are not social gatekeepers but trusted outsiders or thresholders who are respected by the community. Typically, they are ministers of religion, teachers, nurses or local leaders of voluntary agencies or NGOs (Non-Governmental Organisations) like Oxfam. Snowballs grow larger as they are rolled across snowfields. The term snowball sample is therefore used to describe samples which become larger as each contact suggests more people to contact. They are particularly useful for researching groups whose identity is concealed. For example, say you wish to research the likely impact on crime of a greater liberalisation of laws against illicit ‘hard’ drug use (crack, meth, etc.). You would be unlikely to make contact with drug-using criminals through a probability sample of, say, 1000 from the UK population. First, the incidence of drug-using criminals is relatively small. And, second, they would be unlikely to reveal themselves to you. The two problems you face here are access and trust. These can be overcome by using a snowball sample. In the example above, you could begin by contacting a person who is publicly known to have had some contact with these criminals. They might be a specialist doctor, a prison visitor, shelter manager or journalist. Your first task would be to meet them to establish your bona fides (Latin, meaning ‘good faith’) as an academic researcher and, therefore, trustworthy. Several meetings might be required. You would ask them to refer you to other people who might be able to assist you. They might refer you on to recovering addicts, their relatives, or organisers of self-aid groups. In turn, you would hope that they would find you sufficiently genuine and trustworthy to provide further contacts. You would hope that your research trail would end in clandestine meetings with practising addicts who funded their drug purchases through petty crime or wider involvement in criminal networks. At the end of your research, you will not be able to claim that your findings applied to addicts beyond those you had met: however, you can claim that they apply to all the subjects in your sample. A volunteer sample is one where members of the research population volunteer to take part in your research. You are most likely to seek volunteers where your sample is likely to undergo a period of discomfort, pain or financial cost. You will have received emails from university departments – especially Psychology – asking for volunteers for lab-tests. They may offer a small cash payment to volunteers. For example, you might seek volunteers from colleagues if you wished to pre-test and compare the possible impact of positive and negative political advertising in the UK. But appeals for volunteers need not necessarily be made only to people. You can also ask for volunteers from organisations. For example, to pursue research on political networks, you could write directly to each unitary authority asking them to take part in your research. The great advantage of seeking volunteer samples is that they can be relied upon to co-operate fully. Conversely, because the volunteers
Evaluating Information: Validity, Reliability, Accuracy, Triangulation
are self-selecting, then they are more likely to be especially interested in the topic and therefore may be less likely to be representative of the population as a whole.
Probability Samples Probability samples are samples in which every member of the defined population has an equal likelihood of being selected for inclusion. So, in a population of 1,000, each person has a 1/1000 probability (also expressed as 0.1% or p = 0.001) of being selected. Probability samples are statistically reliable. This means that they are capable of generating data which is representative of the population. In other words, if the average age of members of a probability sample is 30, then you can be confident that the average age of the population as a whole is also 30. This capacity to generate accurate representation is called generalisability. Probability samples are large. They are used in all quantitative research where the population is very large. The reliability of the data obtained from a sample will increase as the sample increases in size towards that of the whole population. But it is the size of the sample which determines its accuracy: the size of the population is less relevant. But doubling the size of the sample will not double the reliability of the information. Accuracy is proportional to the square root of the sample size. So, to double the accuracy, the sample size must be increased fourfold – which will greatly increase the cost of the sample survey. There is, therefore, a trade-off between cost and reliability – and time. This is an example of the so-called law of diminishing returns (or diminishing marginal utility). It explains why most samples are relatively small, e.g. national opinion polls rarely use more than 1,500 people, whilst even national, life-or-death medical surveys rarely exceed 60,000, i.e. a 0.1% sample. There are many types of probability samples: simple random samples (entirely random); systematic samples (every nth person); stratified samples (e.g. 50:50, men: women, etc.); multi-stage cluster samples, and, probability proportionate to size (PPS) samples. Essentially, stratified, multi-stage cluster and PPS involve pre-designing the sample to reflect the known characteristics of the population under study, e.g. by gender, age group, ethnicity, social group, residence. They are also used to make the sample more readily contactable and thereby reduce costs. For example, if you were proposing to carry out a face-to-face survey of 10,000 people in England, then you could choose 1,000 electoral wards randomly and then 10 addresses within each (or 200 wards and 50 addresses in each). A further refinement would be to select randomly 125 wards within each of the 8 standard regions and 10 residents in each ward. But some regions have larger populations than others. So you could vary the number of wards pro rata so the likelihood of any ward being chosen was more equal. But wards also vary significantly in size (depending on whether they elect one, two, or three members). So the list of wards from which the sample is to be chosen should reproduce multi-member wards twice or three
93
94
Research Methods in Politics
times, and so on. In this way, you are constructing a sample with a probability proportionate to size. In this case, you would have constructed your sample frame in three layers: region, ward, population. These layers are termed strata. The more strata employed, the smaller the size of each group and the lower the reliability of the data. One statistical survey rule – rarely followed – is that the smallest group should not be lower than 1,000. In practice, a ‘rule-of-thumb’ is adopted in which the group should be at least 50–100 (Hoinville et al, 1977: 61).13 There is a particular problem with very small minorities, e.g. non-whites in UK rural county towns. A group of, say, 100 educated males 26–35 may well include less than five members of minority ethnic communities. But five people are unlikely to be representative of the many minority ethnic communities. In this case, you should seek a higher number of participants from these communities and scale down the data accordingly. Alternatively, if ethnicity is a critical variable, then a very much higher stratified sample should be sought. Tables have been developed which relate sample size to the degrees of acceptable sampling error and levels of confidence (CL). Most Politics research adopts levels of confidence of 95%. This means that you are confident that, in 95 out of every 100 cases, the characteristic (e.g. party preference) shown by the sample will be shared by the research population. The sampling error is the inaccuracy arising from the use of a sample. So, as Table 7.1 shows, if you are willing to accept a sampling error of 5% either way, you can use a sample of 400. But, if you insist on a sample error as low as 1% either way, then you must use a sample of 10,000. For example, say you have adopted a random sample of 2,500 people whether they support or oppose NATO forces, involvement in Afghanistan. Of these, 36% say they are supportive. What the table above tells you is that, in this case, you can be confident that, in 95 out of every 100 members of the population, 36% will be supportive ± 2%. So, at 95 confidence levels, support will lie between 34% and 38%.
Table 7.1 Sampling errors/sample size of random samples at 95% confidence levels. Sampling Error %
Sample Size
1 2 3 4 5 6 7 8 9
10,000 2,500 1,100 625 400 277 204 156 123
Abstracted from De Vaus, 2001: 71, Table 5.4
Evaluating Information: Validity, Reliability, Accuracy, Triangulation
Therefore, if you want to reduce the sampling error by half (to ± 1%), then you will have to increase your sample to 10,000. The greatest difficulty in using probability samples is designing the sample frame. The sample frame is the list of the population from which the sample will be drawn. Say, for example, you want to carry out questionnaire research of residents of a city. Twenty years ago, you could have used the electoral roll or telephone directory. However, to protect the privacy of the public and to encourage more people to register and therefore to vote, the full electoral roll is no longer publicly available. The telephone directory is now much less representative than previously now that a third of ‘subscribers’ choose to go ‘ex-directory’. In any event, the telephone directory lists heads of household and not their partners. Additionally, young people are more likely to use only mobile telephones which are unlisted. Large-scale probability sampling is very expensive. One practicable means available to (fully-funded) researchers to obtain very large samples is to buy into one of the very large (100,000) sample omnibus questionnaire surveys undertaken by market researchers acting on behalf of commercial clients – notably superstore grocers and financial services. They use random samples obtained from a sample frame of a national gazetteer of postal addresses. The disadvantage is the relatively low completion rates and the bias arising from the use of prize incentives. For these and other reasons (especially costs and the lack of life-or-death consequences for the population), Politics researchers rarely use probability samples. Instead, like most private firms, you are most likely to use quota samples.
Quota sampling Despite being the most widely-used, quantitative sampling technique, quota sampling is non-probabilistic. However, for calculation purposes, custom allows you to analyse the data using the same statistical techniques as if it had been obtained using simple random sampling. A quota sample is a sample of the population which is predesigned to be representative. So, for example, if you know that 69% of your population of UK electors voted in the last general election, then you will design your sample to have a quota of 69% voters. How do you know whether a person voted? You ask them. Quota samples are usually recruited in town centres by researchers who select passers-by to complete their quota of interviews. It is non-probabilistic because the interviewers select individuals to meet their quota. So each passer-by does not have the same probability of being selected. Furthermore, because the weekday city centre contains higher proportions of some groups of the population and less of others, then the probability of each member of the research population being selected is unequal. This also explains why you may not have been interviewed by the ever-present pollsters in your city centre. Their quota of people like you had already been filled. Or, alternatively, your type of person did not form part of the sample frame of, say, pensioners. Or, importantly, they may have felt uncomfortable
95
96
Research Methods in Politics
by your appearance. You will find out when you carry out your own quota sampling that you are likely to select people with whom you believe that you are likely to develop rapport. You will find more detailed guidance on administering (carrying out) an on-street survey in Chapter 8.
Designing a quota sample You will find that the simplest way to design and use a quota sample is to start by setting up a 10 by 10 matrix of 100 cells. For example, suppose you require a representative sample of the research population in which age, gender and socio-economic group are considered relevant – as potential independent variables – to your research question. You find from published sources that: 1. the breakdown of population between males and females is 50:50% 2. between 18–34, 35–59, 60+ years old, the relative distribution is 20:50:30% 3. as a proxy indicator of socio-economic group, the percentage ratio of owner-occupations to tenant is 60:40%.
Then you can sub-divide the 100-cell matrix into columns for sex and socioeconomic group (assuming the owner/tenant distribution is uniform between genders and ages) and into rows for age bands, namely: In this matrix, the highest, right-hand cell (marked ‘X’) will be a woman, aged 18–34 and living in rented accommodation whereas ‘P’ will be a man, aged 35–49 who is an owner-occupier. You can then ‘scale-up’ the matrix to provide the optimum sample size on the basis of weighing the advantages of accuracy, reliability and representativeness, against the resource costs. You can, of course, add further sub-divisions, for example, of ethnic origin. But remember that, each time you subdivide the sample further, the sub-groups become smaller and potentially less representative. So armed with your quota matrix and questionnaire, you can begin your quota sampling. As you recruit each member of the quota sub-group, you ‘tick them off’ from the matrix. Beginning is easy. However, the technique becomes more difficult as the number of vacant cells in the matrix reduces.
Questions for discussion or assignments
1. What do you understand by the essential difference between validity, reliability and accuracy, in terms of data? Why are the distinctions useful?
Evaluating Information: Validity, Reliability, Accuracy, Triangulation Table 7.2 Quota sample matrix (100 cells) Men
Women
18–34
X
35–59
P
60+
Tenants
Home-owners
Tenants
2. Discuss the case study of the formation of the National Government in 1931, drawing on other accounts which, together with Nicholson’s biography of George V, enable an appropriate triangulation to be achieved. 3. Consider your university’s prospectus. Identify examples where numeric or other authoritative data may have been used selectively to create a best-case presentation to potential students. 4. Arthur Scargill attracted both strong supporters and critics for his role as leader of the NUM (National Union of Mineworkers) during the miners’ strike of 1983–4. You have obtained documentary sources from the NUM and the autobiographies of government ministers and advisers of the day. What other sources would you seek out to achieve triangulation? 5. Design a quota sample of 400 adults to represent your research population of adult residents of your university city, or town.You wish to test the potential causal
97
98
Research Methods in Politics relationship between ‘green practices’ and age, class and sex. You have learned that 60% of households separate and sort recyclables from their household refuse. Find the data on age, class and sex from the census data for your city or town. What are the advantages and disadvantages of this method?
FURTHER READING Harrison, L. (2001)Political Research: An Introduction. London: Routledge. pp. 25–9, 106–12. In the first extract, Harrison discusses and distinguishes between validity and reliability, and provides additional material on types of validity including construct validity and content validity. In the second extract, she examines and discusses alternative sources on existing political data including the ‘mass media’, party resources, biographical, autobiographical and political memoirs and the internet. Neuman, W.L. (2003) Social Research Methods. London: Pearson. pp. 178–87, 137–8. The first extract discusses reliability and validity in the separate contexts of quantitative and qualitative research. Table 7.1 on p. 183 summarises the measurement reliability and validity types identified. The second extract provides a very readable discussion of triangulation of which he provides an example of four types in Box 6.1 on p. 138. De Vaus, D.A. (2001) Surveys in Social Research. London: Routledge. pp. 54–79. This extract begins with a discussion of reliability and validity. The author introduces three means of assessing validity: criterion validity; content validity; and construct validity. He also discusses the special problem of how people may interpret indicators in different ways. The second part of the extract provides good, practical advice on the separate types of probability and non-probability samples.
Notes 1 Russell, B. (1971) A Liberal Decalogue. In The Autobiography of Bertrand Russell, 3. 1944–67, London: George Allen & Unwin. pp. 60–01. 2 Amis, K. (1997) The King’s English – A Guide to Modern Usage, cited by Mount, H. (2006) Amo, Amas, Amat and All That: How to Become a Latin Love, London: Short Books. p. 40. 3 Huxley, A. (1932) Brave New World. 4 Crossman, R.H. (1976) The Diaries of a Cabinet Minister 1. Minister of Housing 1964–66. London: Hamish Hamilton and Jonathan Cape. p. 590. 5 Cole, J. (1996) As It Seemed to Me: Political Memoirs. London. Weidenfeld & Nicolson. p. 64. 6 Jenkins, R. (2001) Churchill: A Biography. London: Macmillan.
Evaluating Information: Validity, Reliability, Accuracy, Triangulation 7 8 9 10 11 12 13
Jary, D. and Jary, J. (1995) Sociology. Glasgow: HarperCollins. p. 714. Lukes, S. (1974) Power: A Radical View. London: Macmillan. p. 51. Foot, M. (1973) Aneurin Bevan: A Biography, 2. 1945–60, London: Davis-Poynter. Grigg, J. (1978) Lloyd-George: The People’s Champion. London: Eyre Methuen. Nicolson, N.(1953) King George V. London: Routledge. Chapter XXVII. pp 453–69 Cole, G.D.H. (1938/66) The Common People. London: Methuen. Hoinville, G., Jowell, R. et al (1977) Survey Research Practic, London: Heinemann. p.61, cited in De Vaus, D.A. (2001) Surveys in Social Research. London: Routledge. p. 73
99
Chapter 8
Completing a Literature Review: Accessing Published (β) Information
Teaching and learning objectives: 1. To understand why researchers undertake literature reviews. 2. To learn how to structure and present a literature review. 3. To consider the various search processes to access published (β) information.
Introduction The literature review is a major part of any research. But many literature reviews are done badly. Poor reviews often consist of one description of a text after another. They lack any original criticism or evidence that the student has actually pondered what they have read. There remains a suspicion that some students haven’t actually read the texts they cite, but merely summarised the abstracts or dust-covers. They are also very boring – especially to the external examiners who will have been chosen because they are authorities on the research topic. Perhaps the fault lies in the expectation by higher education that students will have been taught how to do a literature review at secondary school and vice versa. There is also the view that doing a literature review is ‘common sense’ which requires no preliminary guidance. I strongly disagree with these views.
Why do a literature review? The review is not undertaken for its own sake, or as a kind of academic penance (although it may feel that way at times). The primary purpose of the literature review is to establish the state of current ‘knowledge’ – or argument – about your research topic. Knowledge includes views, concepts, theories, understanding, evidence, schisms
Completing a Literature Review: Accessing Published (β) Information
and schools, and the main authors. In the first place, it involves you finding out what has already been written (or said). It also involves finding out what you don’t know and possible unknowns. As Donald Rumsfeld famously said: Reports that say that something hasn’t happened are always interesting to me because, as we know, there are known knowns; there are things that we know we know. We also know that there are known unknowns; that is to say we know that there are some things that we do not know. But there are also unknown unknowns – the one’s we don’t know that we don’t know. (Donald Rumsfeld, US Secretary of State for Defense, 2003)
Reaction against Rumsfeld’s part in the Iraq war has meant that his statement was widely ridiculed, whereas, in hindsight, it is worthy of Foucault. Reviewing the literature prevents you from ‘reinventing the wheel’ and enables you to contribute to new knowledge or understanding. However, in Politics, there are few uncontested, ‘knowledge claims’. Instead, there are a number of – often competing – explanations of phenomena which reflect opposing paradigms, information and periods of time. But those explanations are not beyond criticism. Your task as a Politics researcher is, therefore, to identify: who are the main theorists, commentators and texts; how the arguments have developed over time; the areas of agreement and disagreement; the overlaps and the ‘gaps in the literature’. The literature review should also summarise and evaluate published criticism and, most importantly, offer fresh, original criticism. Your criticism should identify loopholes, illogical deductions and conclusions, which are not supported by the information from which they are allegedly derived. A literature review is not merely a chronicle of who wrote what and when, but a forensic examination of texts to identify one or more critical elements where current understanding is unclear or contested and which the new research can address. After all, it is this lack of clarity or information that is the justification for you undertaking the research and for the reader spending their time. The explosion of text, in the form of books, journals and the Internet, makes literature reviews more difficult than before. Conversely, the Internet has made searching for publications very much easier. But you will be unable to read every available text. A selective approach is essential in terms of what texts – and which parts of those texts – you should read and what records you should make. The ‘law of diminishing returns’ applies to literature. And, conversely, reading ever more texts can become a displacement activity to justify putting off the start of fieldwork. A good supervisor can spot when you are blaming the literature review for general lack of progress (or effort). As a rule-of-thumb, the literature review is likely to occupy between a quarter and a third of the time available for the research project. A quarter of the research report is likely to be set aside for the theory and literature
101
102
Research Methods in Politics
review. Traditionally, the first year of a three-year PhD research project was devoted to ‘reading the literature’. Whilst the proportion of time remains the same, current practice is to integrate the literature review with the search for empiric data in a type of grounded research with a near-continual process of writing-up-as-you-go that begins with the original research proposal. The objective of the review is to identify, criticise and synthesise the most recent, relevant, authoritative texts. Authority is critical. It generally means those texts that are widely accepted by the academic community as works of considerable scholarship. Finding and accessing these texts requires a systematic search of literary sources.
Searching the literature The literature search necessarily precedes the literature review. So, where should you begin to search for authoritative texts? A starting point is your course modules, your core texts and the bibliographies provided by their authors. Then you must (not ‘should’) move on to the bibliographic databases of journals (and their predecessors the printed bibliographies) provided by the university library. As already noted, journals are much shorter and more concise than books, and will have generally been vetted before publication by referees selected from the leading academic specialists in the topic field. Bibliographic databases allow you to search for articles which include specific words in the title or abstract or, alternatively, written by a particular author. You can limit your search to a specific range of years, a particular set of journals, and so on. The database is likely to have been designed to enable you to follow a link directly from the database entry to the article itself, where the journal is available in electronic format. Useful databases for UK-based Politics researchers include (listed alphabetically): ASSIA (Applied Social Sciences Index and Abstracts) www.csa.com/factsheets/ assia-set-c.php BIDS (Bath Information and Data Services) www.bids.ac.uk British Humanities Index www.csa1.co.uk/factsheets/supplements/bhi.php IBSS (International Bibliography of the Social Sciences) www.ibss.ac.uk PAIS International (Public Affairs Information Service) www.pais.org Social Sciences Citation Index (SSCI) www.isinet.com
Each journal article will also contain its own bibliography. The Social Sciences Citation Index allows you to move from the details of one article to all the other articles that have been cited and also to identify which articles are cited most often. You must log in to use these databases. Many have been acquired by the US information firm, CSA based in Maryland, or other commercial providers. But researchers based in UK
Completing a Literature Review: Accessing Published (β) Information
universities can normally access the databases free-of-charge by registering at their university with ATHENS which will provide the necessary password etc. Others can use the databases by taking advantage of the free, short-term trial membership provided by CSA and others. Other relevant material on your topic may have been published on the Internet without having been submitted to an academic or professional journal. When using a commercial search engine like Google or Yahoo, it can be difficult to identify which web sites carry authoritative texts or other information. A general health warning is necessary. To overcome this problem, some UK academic institutions have established their own searchable directories of web sites that meet acceptable standards of scholarship. These include RDN (Research Discovery Network, www.rdn.ac.uk), Intute (www.intute.ac.uk) which includes an excellent ‘virtual seminar’ on how to make electronic searches, and its social science sub-set SOSIG (Social Sciences Information Gateway, www.sosig.ac.uk). Masters and doctoral dissertations should also be revealed in RDN, etc. searches. But, they are unlikely to be made available on-line. Some universities will – on request – send your university library a microfilm copy (on short-term loan) which you will be able to print at your university library using a special photocopying machine. Where microfilm is not available, then you will have to visit the university library where the dissertation is held and where you may be allowed to photocopy it.
Access to official records Government and local government maintain extensive records of periodic surveys – including the decennial census and electoral rolls – and records of policies and decisions. Original census returns for 1841–1901 are now available for inspection and copying at the relevant local reference libraries. The 1901 census returns can now be examined on-line at www.1901census.nationalarchives.gov.uk/. For 1911 onwards, individual census returns are not yet available. However, aggregate data is available from 1911 onwards at: county; local authority; electoral ward; parish and enumeration district (ED). ED’s are the basic building block of the UK census and consist of around 200 households. Whilst county, district and ward boundaries change, ED boundaries can endure and therefore provide a basis for historical comparison. Census data includes: households, household sizes, age of residents, housing and facilities, socio-economic class, economic activity and, from 2001, ethnicity and health. The census is supplemented by a 10% household survey that includes information on type of employment. The 2001 census can be examined on-line at www.statistics.gov.uk/census. The Freedom of Information Act, 2000 enables UK residents – regardless of nationality – to seek specific information from public bodies including: central government; local government; NHS trusts; police authorities and advisory bodies.
103
104
Research Methods in Politics
You must make the request in writing (which may include email) to the organisation which you believe holds the information. The organisation is required to answer in 20 working days (i.e. four weeks). A small charge may be made for photocopying. However, where the cost of providing the information is more than £600 to central government or £450 for other bodies, then the request can be refused. Some records will be protected from scrutiny on grounds of necessary secrecy, or the Data Protection Act. Where a request is refused, then you can appeal to the independent Information Commissioner. The onus remains on the body holding the information to justify not making it available. For further details, see: www.dca.gov.uk/index.htm. Local councils, NHS trusts and Police authorities are required to open their meetings to members of the public. So you can attend and note what happens. However, cameras and tape-recorders, etc. are not allowed. But you should be issued with a copy of the agenda papers (if not, ask the committee clerk/democracy officer). Reports will identify the relevant ‘background papers’ which you can ask to inspect. But you will be excluded from hearing those ‘exempt items’ concerning commercially sensitive or personal information. The UK’s national archives are held at the National Archives for England, Wales and the United Kingdom (formerly the Public Records Office) at Kew, London, where they are available for public inspection. The records held cover the British Isles, former British Empire colonies and Commonwealth states. The National Archives includes the Family Records Centre (at Islington). A new search engine at www.nationalarchives.gov.uk/search enables you to search all official and private archives in the UK. Researchers do not have to register or obtain permits before they visit Kew or Islington. All other states maintain comparable systems. The US national archives are held by the National Archives and Records Administration in Washington (www.archives/gov) which also maintains regional archives. US states and city authorities have archives. Alliances and supra-state groups also maintain records. For example, EU archives can be accessed via ec.europa.eu/historical_archives/ UK county councils (and some unitary authorities) keep extensive local archives. These are a valuable source of information. The professional archivists know their records and are generally very helpful. It may be worth making an appointment before a visit to arrange for the specialist member of staff and relevant records to be made available. Trade unions, large companies, the churches, etc. all maintain large archives. The county councils and unitary authorities also have specialist ‘research and information units’ who may be able to run cross-tabulations of census data etc. using SASPAC and other software. The Land Registry is also available for public inspection. This holds records of the titles, extent and ownership of all properties that have been sold. A small charge (currently £3.00) is made for each title search. The Land Registry remains the only current authoritative record of land ownership in the UK. However, information on
Completing a Literature Review: Accessing Published (β) Information
land that has never been sold – for example, the core estates of the great aristocratic landowners – is not available. Companies House in Cardiff holds information on all current and some past public companies, including their annual reports and accounts. Records can be obtained on-line at www.dupont.co.uk/companies-house.htm. A small charge (currently £1.99) is made for each search.
Reading the literature Whether your search has revealed a very large number of texts or otherwise, you will have to be selective in your reading and note taking. Consider the advice given to historian William Woodruff in his first tutorial in Oxford. He was asked to write an essay on Inclosure. He asked his tutor which books he should read. Eight volumes were recommended. When he asked whether it was really necessary to read all of them, his tutor replied: Gracious me no, Woodruff. You’d be mad to. You don’t read books; you gut them; it’s the gist you’re after. If you feel that an author has nothing important to say, drop him. You’ll get the nub of things pretty quickly. Anyway, you won’t find half the books I’ve given you. There are other students preparing essays, you know … Even looking for a book you can’t find will teach you something … what you are after is the gist, remember. (Woodruff, 2003: 130–1, my emphasis)1
You ‘gut’ the literature by reading the abstract first. If this is relevant, then read the introduction and conclusion and – only if necessary – individual chapters. Learn to skim the text to identify relevant passages for slow, careful reading. You will need to make a rolling bibliography of texts that you have read and those that you intend to read. You will also need to make a record of those texts to be cited in your theory and literature review. A traditional method is to record the relevant information for each publication – author, year of publication, title, publisher’s address and name and a summary of the main arguments and page references – on a standard 150 × 100 mm (6’ × 4’) record card. The cards can be kept in an alphabetical index where they can be abstracted and grouped into themes, topics or approaches. But there is a limit to the information that can be held on a single record card. An alternative approach is to keep an electronic record of each text in a separate file within a bibliography folder and a hard copy. These records are, therefore, not limited by space to bibliographic citation and key arguments. They can include key passages identified by their page numbers. In this way, a key text can be reduced to four pages (or less) of quotations (which, incidentally, can be traded with colleagues
105
106
Research Methods in Politics
for similar records of their summaries). The passages can then be copied and pasted directly into the research report or discussion paper.
Writing the literature review Having reduced the literature to a series of records, abstracts and summaries, you can now begin the literature review. The trend among inexperienced students is to write-up the literature as a series of descriptions of what each author wrote in chronological order. Such approaches are over-long; at best overly descriptive and, at worst, obvious plagiarism, confusing and boring. In particular, they lose the interest of the writer and reader, and tend to give least attention to the most recent texts. Instead, a critical synthesis is required which focuses on the ideas and arguments rather than the authors. Indeed, I would argue that, with the exception of the ‘grand theorists’ – Marx, Weber, etc. – the names of the (lesser) authors should generally be omitted from the text, and given in Harvard references or footnotes. You must give references to acknowledge the sources of information and to show where readers can find them. They should be clear, consistent and follow an ordered structure. A guide is given in Box 8.1.
BOX 8.1
Referencing: a guide
A reference consists of two parts: a notation or insertion in the text; and a full description of the source. There are two systems for notations and insertions: the footnote system and author-date (Harvard ). The footnote system uses a supertext notation in conjunction with footnote or endnote, e.g.1 (In MS Word you would use the Insert/Reference/Footnote routine.) You can either insert the footnote number immediately after the source or quotation to be referenced or at the end of the sentence (after the full stop). The full description of the source should be given in the footnote or endnote. When you use the number system of referencing, the full description of the text in the footnote or endnote should follow the convention of: Author, Initials (Year of Publication used) Title of Book. Place of Publication: Publisher. Pages. The Harvard system is more widely used than the numbering system. It consists of: (Name of Author, Year of Publication used: page number) e.g. (Smith, 2007: 57). Harvard is particularly useful where you want to acknowledge several texts and authors, e.g. (Smith, 2007: 57; Waterman, 2007: 132–5).
Completing a Literature Review: Accessing Published (β) Information
In all systems of referencing, you must provide a full description of the text or other source in the Bibliography (list of sources) at the end of your research report.The convention is Author name, Initials. (Date of Publication) Title and sub-title. Place of Publication: Publisher, e.g. Burnham, P., Gilland, K., Grant, W. and Layton-Henry, Z. (2004) Research Methods in Politics. Basingstoke: Palgrave Macmillan. Where you draw your reference from an edited textbook, then the full description given should be: Budge, I. (1997) Chapter 1: Party Policy and Ideology: reversing the 1950s? In Evans, G., and Norris, P. (eds.) Critical Elections: British Parties and Voters in Long-Term Perspective. London: Sage. Where a journal is cited, then the reference should be: Author of Article, Initials (Year of Publication) Title of Article, In Title of Journal, Volume, Number: Pages, e.g.: Przeworski, A., Alvarez, M., Cheibub, J.A., and Limongi, F. (1996) ‘What makes democracies endure?’ In Journal of Democracy, 7 (1): 35–55. Where you cite an Internet source then the reference given should be: Retrieval Method, e.g. http://, host and domain name, Path and File Name (Date of Visit).
Example In 2003, I completed a short literature review on power. The first pages are reproduced at the end of the chapter. Ideally, the literature should be brought together in distinct groups or typologies. One very effective way of communicating these is to tabulate the arguments by schools or sub-topics. In this way, you can clearly distinguish the major differences between groups of texts. The review can therefore concentrate on the main arguments between the schools on key topics that you can then make the subject of detailed scrutiny. In this way, you can achieve real depth – especially in terms of original criticism – and show your mastery of the material. But remember that you must justify your criticism: to argue that a text is ‘fundamentally flawed’, you must demonstrate how. Where the synthesis reveals that the arguments have developed over the years between and within schools of thought, then this relationship can be mapped diagrammatically. Table 8.2 in the Questions for discussion or Assignments shows how I was able to map the development of western philosophies on property. Remember that tables and diagrams (and photographs) drastically reduce the word count and add interest.
107
108
Research Methods in Politics
The literature review can be the most difficult part of the research project and the most likely to be re-written as you seek to reduce its length. Three drafts are likely. At the end of the first draft, ask yourself: •
Has the literature search revealed all the main sources? How do you know?
•
Has the search identified those texts that are the most relevant, authoritative and
• • • • • • •
recent? How? Have you ‘gutted’ the most important texts and uncovered their ‘gists’? In writing the literature review, have you been able to organise the texts into distinct schools or approaches? Have you clearly identified the agreements between different schools, their disagreements, overlaps and, crucially, gaps in the literature? Have you identified and evaluated the key criticism already made by commentators? Have you provided original, penetrating and pungent criticism? In your criticism, have you clearly identified what is ‘known’, contested and ‘unknown’? Have you clearly identified the theoretical perspective to be adopted, the contested area or gap in the literature to be addressed in the fieldwork, and a refined hypothesis to test?
If the answer to any of these questions is no, then you must revise – or wholly rewrite – your literature review. Where the research is taking place over several years, then you will have to revisit your literature review to identify any new texts and amend your review accordingly. Your bibliography will also need revising.
Plagiarism Plagiarism is failing to acknowledge the source of material used in your research report or assignment, and therefore taking credit for other people’s work. It is a form of cheating and can attract heavy penalties. Wherever you include part of another textbook or other source, you must use single quotation marks at the beginning and end of the passage and a full footnote or Harvard reference. Where the passage is greater than thirty words, then it should be placed in an indented paragraph and use a smaller font. In both cases, you may italicise the text. Plagiarism can be detected readily by supervisors primarily because the language is very different in style and vocabulary, strikingly error-free and is likely to be of a higher standard than your previous work. There are many other signs. Most universities now ask that written work is also submitted in electronic format so that it may be searched by plagiarism-detecting software. A similar offence is to include references to textbooks and sources which you haven’t actually read. Once again, this is readily identifiable. Don’t do it.
Table 8.1 Property ownership: the evolving arguments Common Ownership
State
God & Church
God & king
Private Ownership Plato 375BC (Republic)
Plato 360BC (Laws) Aristotle (348–322BC) Cicero (106–43BC) Seneca (48) Early Christian Church
St Augustine (354–430)
Early Christian Church Early Christian Church Albert Magnus (1206–80)
Rufinus (1158) St Thomas Aquinas (1224–74) William of Ockham (1285–1347) Melachton (1521)
Moore (1516) Erasmus (1516)
Fortesque (1470)
Luther (1535) Calvin (1559) Ponet (1556) Grotius (1625)
Levellers (1646)
Filmer (1680)
Hobbes (1651) Locke (1690)
Rousseau 1755
Hume (1739)
Paine (1796)
Smith (1776)
Jacobins (1797)
Burke (1790)
Proudhon (1840) Owen (1840)
Hegel (1821)
Marx/Engels (1848) Mill 1873
Mill (1848) Maine (1873)
George (1880) Spencer (1884) Communism Fascism
Libertarianism Capitalism ‘Free Markets’ ‘Property-Owning Democracy’
110
Research Methods in Politics
Questions for discussion or assignments
1. You have been asked to undertake a literature review on ‘social exclusion in the UK’ (or another topic set by your teacher). Describe how you would undertake the literature search and the priority to be given to particular sources. How would you ‘read’ the relevant texts? What records would you make of the key texts and how? Describe the structure of your literature review. 2. Select a literature review from a journal article. Review it critically. How would you improve how it was written and presented?
FURTHER READING Hart, C. (1998) Doing a Literature Review: Releasing the Social Science Research Imagination. London: Sage. This 230-page textbook remains the premier guide for Politics researchers seeking to take a disciplined approach to their literature reviews. It provides practical guidance on how to: search out existing literature on a topic; effectively analyse arguments and ideas; and map out arguments. Hart advocates the use of network diagrams to map out the development of ideas and philosophy by different schools and authors. He provides a good example of DNA development on p. 169. I followed his advice to demonstrate how different and competing concepts of property had developed in western society since Biblical times. This is shown in Table 8.1.
BOX 8.1
Example of a literature review
Power Introduction Power is a universal experience. Sometimes it is recognisable, tangible, noisy. At other times, it is silent, barely discernible, scarcely felt. So, what is power ? Lukes famously described power as an ‘essentially contested concept’(Lukes, 1974: 9). But power might better be called an ‘essentially contestable concept’. It is not ‘under-theorised’: there are many theories and theorists. They include Hobbes, Hume, Machiavelli, Marx
Completing a Literature Review: Accessing Published (β) Information
and Gramsci. More recent theorists include Bourdieu, De Jouvenel, Foucault, Clegg, Mann, Wrong and the authors of what have been called the three dimensions of power : Dahl; Schattschneider; Bachrach and Baratz; and Lukes. But whilst there are many theories, most are either partial or offer views from different perspectives which reflect the normative values or disciplines of the authors. Given the proliferation of theories and perspectives, then exceptionally in this dissertation, their authors are identified in the text. Arguably, the perspectives are also culturally and historically eurocentric (Foucault, 1973). But many of the conceptualisations are supportive or compatible. For example, most support the view that power is relational and relative. There is no absolute power. There is usually some resistance – conscious or unconscious, overt or covert – to the exercise of power. There is also complicity. Power is contingent. It is contextual. It is also fluid. It is generally a means to objectives rather than an end in itself (which raises the question whether all means of attaining objectives are forms of power). Power is a generalising medium, a characteristic akin to – but different from – a resource and money (Parsons, 1963: 232–62). But the argument that power, like money, is a ‘circulating medium’ (Pareto, 1935; Parsons, 1963; Habermas, 1987) is not widely supported. Power can be held and exercised by and between individuals, groups of individuals, by organisations (‘embodied institutions’), and states (geopolitics). Power can be either extensive, comprehensive or intensive (De Jouvenel, 1958: 159–69) where, as extensiveness increases, comprehensiveness and intensity fall (Arendt, 1951). Power is exercised within realms or scopes (Wrong, 1997: 10). Society is stratified by power where the strata reflect the distribution of power. Power may be distributive (conflictual – involving zero-sum outcomes), or collective (co-operative, non-zero-sum outcomes), (Parsons, 1963). Distributive and collective powers are dialectical : organisation is the manifestation of collective power which is inherently oligarchic, where leaders exercise distributive power over members (Michels, 1949; Lowi, 1971; Mann, 1998). One consequence of collective power is that, paradoxically, individuals become subordinated by the delegation they extend to officials (Bourdieu, 1999: 107–16). However, the individual rarely belongs to only one organisation (Truman, 1951: 508). Similarly, organisations rarely act alone: organisations are ‘functionally promiscuous’: ‘organisational outflanking’ is a common strategy contributing to the turbulence of power relations (Mann, 1998: 7). Collective power is the product of size (membership), solidarity and organisation (Wrong, 1998: 237). Solidarity derives from mutual identity, consciousness of kind and rituals of belonging. But, as membership increases, intensity of common identity may diminish and ‘organisational diseconomies of co-ordination’ may multiply. The organisations created are ‘living machines’, sui generis characterised by their own objectives and internal conflicts among members and officials (Weber, 1947: 140). Political mobilisation is the ultimate form of collective power – where the organisation seeks control of the state (Wrong, 1998: 144). Alternatively, parties or unions are seen as ‘class persuaders’ providing the structural cement Continued
111
112
Research Methods in Politics
of ‘class reality’ rather than as an organisational expression, class or other groupings (Sartori, 1969 in Wrong, 1998: 167). By most accounts, power over is a special (distributive) type of power to (of which power against is a defensive type of power to). There is also general agreement on the bases and forms of power. The bases – sources or domains (Dahl, 1961), power resources (Wrong, 1997) – include ‘collectivity’ (the greatest), wealth, reputation, fear, respect, success, nobility and excellence (according to Hobbes, 1651/1996: 58–59). Others have added legitimate position derived from office, personal appeal, skill or specialised knowledge (Dahl, 1961). Additional power (re)sources include property, class, credentials, capacity for reasoning, information and knowledge, control of discourses, historiography and discursive media (Foucault, 1978), ‘systematic luck’ (Dowding, 1996: 170), speech, bearing, accent, clothes, how the mouth is pursed (Bourdieu, 1999: 17) and legitimacy – moral, traditional or rational-legal (Weber, 1986). Thus power can be ‘original ’ (intrinsic) or ‘instrumental ’ (acquired) (Hobbes, 1651/1996: 58). However, postmodernists argue that knowledge, rather than being a source of power, is its product (Foucault, 1978).The argument appears compromised by the secrecy of the modern state. Alternatively, the counterargument can be made that the state gathers, screens and guards the data from which knowledge is fabricated to reinforce its dominant discourse; the ‘stem’ of the word statistics is state. Authority is problematic. The essential characteristic of authority is the general approval and acceptance of those over whom it is exercised (Weldon, 1953 in Friedrich, 1958: 31). But some argue that authority can only be derived from credentials or received wisdom which confer ‘evidence of a capacity to reason, weigh alternatives and exercise delegation’: authority is a source of power (Friedrich, 1958: 31; De Jouvenel, 1958: 159–69). They believe that the quality of communication is the ‘essence of authority’ (Friedrich, 1958: 34–7). Thus authority can be exercised without power, e.g. Roman Senate, or vice versa, e.g. Nero (Friedrich, 1958). Context may be instrumental (Bourdieu, 1999). Most regard authority as a special form of power (Wrong, 1997) inseparable from legitimacy (Weber, 1986). Others regard legitimacy as consent to authority in which authority is a legitimate use of power and therefore different from power (Giddens, 1998: 339). Power is a universal experience. Most people exercise power over others at some time – especially as parents – and are also the subject of others’ power (although they may not recognise its source or influence). Perhaps this universality explains the broad compatibility of distinguished definitions: ‘an intuitive idea of power, is … A has power over B to the extent that he can get B to do something that he would otherwise not do’ (Dahl, 1957: 201–05); ‘the probability that an actor within a social relationship will be in a position to carry out his own will despite resistance, regardless of the basis on which this probability rests’ (Weber, in Lukes, 1986: 28–36); ‘the capacity of persons or collectivities to “get things done”’ (Parsons, 1963: 232); ‘the production of intended effects’ (Russell, in Lukes, 1986: 19–21 ); ‘the capacity to achieve outcomes … to make a difference’ (Giddens, 1984
Completing a Literature Review: Accessing Published (β) Information
in Clegg, 1984: 138 ); ‘the capacity to produce intended and foreseen effects in others’ (Wrong, 1997: 2);‘the ability to pursue and attain goals through mastery of one’s environment’ (Mann, 1998). Power is, therefore, widely held to be primarily a capacity. Power can thus be dispositional (capacity) or episodic (exercise of power) (Ryle, 1949, In Clegg, 1998: 73). Intentionality is considered by many commentators to be a requisite of the exercise of power (Russell, 1986: 19–21; Wrong, 1997, In Clegg, 1998: 73). Others argue that a ‘law of anticipated reactions’ is evident where subjects anticipate and comply-in-advance which makes redundant the exercise of power (Friedrich, 1937). This phenomenon challenges the classical notion of power as necessarily observable in the sequence of stimulusdelay response – ‘push-and-shove’ (Hobbes, 1691; Dahl, 1957). Intentionality is therefore contested. Power can be negative (prohibitory) or positive (especially Foucault) or both. But the issue of overt and covert power is contested. Behaviouralists argue that only the observable exercise of power is verifiable and substantive (Dahl, 1957). Others criticise this view as ‘one-dimensional’. They argue that power has ‘two faces’ and can also be exercised unseen by non-decisions and by preventing issues from reaching agenda (Bachrach and Baratz, 1962: 947–52 after Schattschneider, 1961). Those with power determine both the rulesof-the-game and which game is played (Schattschneider, 1961). Some argue that power can be exercised unseen and unknown through hegemony - the cultural domination of civil society by discourses which lead subjects unquestioningly to carry out activities which are contrary to their interests (Gramsci, 1935 In Forgacs, 1988: 192; Lukes, 1974; Foucault, 1978). Lukes termed this: ‘three-dimensional power . . . the most insidious use of power . . . by influencing, shaping and determining the very wants’ of those over which it was exercised (Lukes, 1974: 23). But others argue that three-dimensional power is unachievable (‘How is it done? With mirrors?’ Clegg, 1998: 165). This challenges a major tenet of psychology that the major determinants of personality, identity, attitudes and behaviour are the family, the state and the economy. The (arrested) development and resource-dependence of children on the institutions of parents and school makes them subject to exceptionally long periods of control. However, ‘brain-washing’ has no enduring efficacy and may be a ‘coping strategy’ (Brown, 1983: 24). Foucault alternatively argues that control of thought and attitudes is first achieved through control of physical behaviour, e.g. children being seated at desks, soldiers marching, etc. (So, forms of dress which surround the body – ties and rings – are, after Saussure, signs of others’ control.) Supporters of three-dimensional power argue that this can be evidenced – paradoxically – through counter-factuals (Lukes, 1974) or counter-actuals (Dowding, 1996), e.g. Conan Doyle’s ‘dog that didn’t bark’ in Silver Blaze. In any event, the consequences of the intended exercise of power will generally have unforeseen or counter-intended consequences (Friedrich, 1937; Foucault, 1978). Continued
113
114
Research Methods in Politics
The dissension concerning hegemonic or three-dimensional power arises from opposing views on the existence or otherwise of objective interests or moral relativism and the incidence of internalisation of (other) norms (Gramsci, 1971; Lukes, 1974). Critics argue that there are no real or objective interests but only alternative interests or discourses (Clegg, 1998; Foucault, 1978). Alternatively, hegemony is regarded as more akin to business leadership than cultural domination (Whitt, 1979: 81–100). The paradox of emancipation describes how the oppressed are so propagandised as to be incapable of any emancipation (Benton, 1981: 161–84). These arguments challenge the classical view that interests are derived primarily from needs (after Smith, 1776). They are universal and absolute in a hierarchy extending from survival to self-actualisation (Maslöw, 1954: 80–106). Alternatively, the argument is made that needs and, therefore, interests reflect the subject’s existential perception of dominant norms and experience which are themselves the product of dominant discourses (Bohin, 2002). Hence, self-sacrifice – rather than self-preservation – can counterintuitively become the highest form of self-actualisation (e.g. Sartre’s The Roads to Freedom, 1950). Hence also the prisoner’s complicity in his own incarceration because of his learned helplessness. Prisoners and guards are incarcerated alike by Bentham’s panotpicon and by themselves (Foucault, 1980: 156). There is more agreement on the forms of power. These are generally accepted to include force, persuasion, manipulation and (among those who do not see it as a source of power) authority (Wrong, 1998). Force may be physical or psychic (ritual degradation) and violent or non-violent. Manipulation is ‘hidden persuasion’. The power-holder’s intentions are concealed or disguised; the subjects believe that they are exercising a free choice. In contrast, persuasion is overt involving appeals and arguments. It is less likely than force to provoke resistance by the subjects. Whilst persuasion involves the tested acceptance of the power-holder’s judgement, authority involves the untested acceptance of the power-holder’s views: source and reputation are significant determinants. The outcome is identification – which is likely to be ephemeral – rather than enduring internalisation. Authority is power ex cathedra. In this account, authority rests on the subject’s acceptance of the power-holder’s claim to legitimacy (Wrong, 1998; Weber, 1986). The sub-types of authority (as power) are: coercive authority, induced authority, legitimate authority, competent authority and personal authority. Coercive authority is the most extensive form of power. It depends on the subject’s conviction of the power-holder’s capability and readiness to use force. It is a major component of political power and of state institutions who have (only comparatively recently) monopolised control of the forces of coercion. Both the Mafia and terrorists challenge the state’s monopoly of violence. States emphasise the growth of organised crime to justify the growth of (their) statewide forces of coercion (Foucault, 1999). But coercion stimulates both opposition – real or imagined – and development of systems of surveillance. Induced authority involves offering inducements to subjects. Whilst there is little or no resistance initially, diminishing marginal utility (‘law of diminishing returns’) progressively undermines the effectiveness
Completing a Literature Review: Accessing Published (β) Information
of the inducement. Indeed, the practice may provoke resistance if the value of the inducement is not raised and especially when inducements are withdrawn. Hence the combination of coercive and inducive authority used in the form of ‘throffers’ (Dowding, 1996: 56–8). Legitimate authority requires a bilateral acceptance by power-holder and subject of the right-to-command and duty-to-obey, whereas competent authority (by this account) arises from the subject’s belief in the superior knowledge or decision-making of the power-holder, e.g. Socrates’ patient-doctor relationship. Personal power is unique to the power-holder and subject; its bases are love and admiration or predispositions to dominate or submit. A special type of extensive personal power is charismatic authority (Weber, 1986). Arguably, all forms of authority are overlain by personal authority which power-holders craft to bolster their power. Each form of authority is unstable and subject to decay or de-mystification. Powerholders must develop strategies of metamorphosis and diversification.The conqueror’s initial coercive authority metamorphoses into legitimate authority as the conquered adjust to the new power-holder. Arguably, the adage that ‘political power grows out of the barrel of a gun’ (Mao, 1938/72: 61) refers principally to the long-term consequences of coercion. However, acceptance by subjects of the power-holder’s legitimate use of force is greatest when they are likely to be unaffected by its use and where its use will be directed against ‘others’. The professor who gives an incomprehensible lecture increases his academic authority (Foucault, 1999). Power can be considered as a type of influence or, alternatively, all influences can be considered as power. Understanding is obscured by the ‘polyvalence’ of words (De Jouvenel, 1958). Power has no transitive verb. Whilst its ultimate causality – agency or structure – may be contested, its ultimate purpose must be the achievement of the power-seeker’s mastery of the present and future environment – in its widest sense – social, economic, political, technical, physical. Arguably, the ultimate power is social power – ‘mastery over other people’ and, thus, their resources (Mann, 1998: 6). The ultimate means is through control of the state which territorialises, institutionalises and regulates its population (Mann, 1998) (although others have seen the state as the instrument of international capitalism (Marx) and globalisation (Cerny, 1999). Power, unlike money, is hierarchical (Parsons, 1963). The state stands at the intersections of ideological, economic and political systems and has sole control of coercive military power (which has internal and external application). Until the late eighteenth century, ideological (especially religious) and military power were dominant in Europe (Mann, 1998). Now economic and political systems of power prevail. States have both despotic power and, increasingly, infrastructural power – especially through state education (Mann, 1998). Who exercises the power of the state is contested. Pluralists argue that power is either widely diffused within the systems or, where elites develop, no single elite dominates (e.g. Dahl. 1961/89:). Elitists now emphasise the distributive power of a centralised elite Continued
115
116
Research Methods in Politics
and the geopolitics between states. Institutional statists argue that states – not elites – institutionalise social conflict in organisations which develop autonomous interests which structure future conflicts (e.g. Skocpol, 1979 in Clegg, 1998: 259). The institutions provide ‘regulatory passage points’ (e.g. Parliament) in ‘fields of force’ (Lockwood, 1964: 244–57). Alternatively, these views are rejected because they fail to recognise the disunity of state institutions, confederalism, the ‘empty centre’, globalisation and ‘polymorphism’ (the way states crystallise at different centres in power networks). Instead a cock-up/ foul-up, ‘organisational materialist’ model of the state is argued (Mann, 1998: 41). However, these views embrace a variety of normative and descriptive viewpoints of a range of divergent states. These views also incorporate polarised perspectives of where, ‘in the final analysis’, power lies and who initiates power. Modernists, after Hume and Hobbes and extending to Dahl and Lukes, have developed primarily mechanistic concepts of power and causality. They emphasise individual agency and autocracy and accept the centralising power of the state through the social contract to achieve protective social order. Power is ‘zero-sum’; ‘the negation of power of others’ (Clegg, 1998). The alternative view is that structure prevails. The baize over which Hume’s billiard balls move is already marked by folds which channel them along pre-determined pathways. The modern individual is ‘institution-ridden’ and ‘institution-supported ’ (De Jouvenel, 1958: 164). There was no original social contract. Marxists argue that the interests of the dominant mode of production will prevail. The lives of people are ‘written’ by those who frame the dominant discourses and exploit the social sciences to create ‘bio-power’ and systems of surveillance (Foucault, 1978, 1995). Self-surveillance is the most effective. ‘They’ construct nodal points which fix meanings essential for political articulation (Laclau and Mouffé, 1985). Individuals are also caged by the past: ‘Men make their only history … under circumstances directly encountered, given and transmitted by the past’ (Marx, 1851/1977: 300). They are also caged by whoever constructs that history: ‘Who controls the past controls the future; who controls the present controls the past’ (Orwell, 1949/1989: 37). Alternatively, agency and structure ‘interpenetrate’ to form a ‘duality of structure’ in which power can be analysed as ‘relations of autonomy and dependence between actors in which they draw upon and reproduce structural powers of domination’ (Giddens, 1981: 29). Within organisations, power relations reflect (mutual) resource-dependency, e.g. work-pay (Mintzberg, 1983: 172). Whilst these and other power relations are asymmetric, subordinates can extract benefits by emphasising the reciprocal obligations of power relations, e.g. the ‘deferential (farm) worker’ (Newby, 1977: 381–432).
Note 1 Woodruff, W. (2003) Beyond Nab End, London, Abacus.
Chapter 9
Asking Questions: Effective Elite Interviews, Other Interviews, Vignettes, Projective Questions, and Focus Groups
Teaching and learning objectives: 1. To distinguish between the different types of questions and structures of interviews. 2. To understand who are ‘political elites’, why we interview them and how to secure an appointment. 3. To consider how the interview should be carried out and recorded. 4. To learn how interviews with other people differ from those with political elites. 5. To learn what is meant by ‘projective questions’ and when they can be used. 6. To consider why, when and where ‘vignettes’ can be used and how they can be framed. 7. To understand the value of focus groups and other group meetings, and how to organise and facilitate them.
Questions Questions are a means to an end: obtaining information in the form of answers. The answers are rarely straightforward. The subject (interviewee, respondent) may be evasive. You may also want to question the answers which you have received. Asking questions is central to both qualitative and quantitative research methods. They are used to corroborate background and secondary sources, and to collect new primary information. While face-to-face questioning is, of course, ideal, it is not necessarily essential. Indeed, physical access and security considerations may prevent on the spot interviews being held. Many overseas students on MA courses are unable to access directly elites or others in their own countries to provide data for their dissertations. So, in order to replace or supplement face-to-face questioning,
118
Research Methods in Politics
increasing use is made of the indirect telecommunications media. Traditionally, researchers have used surface mail and, to a lesser extent, telephone.1 Now, email or voice-over-internet protocol (VOIP) is widely used. In practice, a combination of media is employed. They all begin with the researcher establishing their bona fides2 as an academic researcher (rather than say, a ‘muck-raking’ journalist posing as a university researcher) and setting out fully the objectives of the research, its significance and the contribution that the subject (also termed ‘interviewee’ and ‘respondent’) can make. Questions can be categorised across a continuum with closed questions and open questions at each pole. Closed questions essentially seek – or receive – closed answers. Closed answers are generally short and confined to yes, no or don’t know or specific answers, e.g. date of birth. Open questions – seek open, lengthy answers. They are concerned with why and how, beliefs, opinions, forecasts and narratives (i.e. stories, biographies, etc.). Another similar categorisation of questions concerns the structure of the interview. Sets of questions constitute interviews. They can be classified on a continuum of highly-structured to unstructured encounters. In highly-structured interviews, you must ‘follow’ a tight pre-designed schedule of questions. Where comparisons are necessary, then you must ask the same questions to different subjects in an identical way in terms of wording, inflexion and other aspects of delivery. At the other extreme, the entirely unstructured interview will follow the course of general conversation in which the next question follows from the preceding answer, and where the roles of questioner and respondent interchange. In Politics research, the most widely-used type of interview – especially in elite interviews – is the semi-structured interview. Here you use a schedule of a limited number of topic-related questions and, pre-determined, alternative supplementary questions (which question further aspects of the answer received). For example, you may ask: ‘and how did you feel about this?’ The format of the semi-structured interview is essentially one of question-and-discussion. It is the type of interview adopted in news reviews where both interviewer and subject have their own agendas. In practice, most productive interviews follow the pattern of good conversations in which rapport (mutual empathy and confidence) develops. You will find that, where rapport is achieved, your subject will share their intimate thoughts and feelings. They therefore make admissions which they might otherwise withhold. Such interviews will generally range across the continua described. Generally speaking, qualitative researchers tend to ask open questions in semistructured interviews of a small number of people. They may ask their questions of individuals or groups. In contrast, quantitative researchers are more likely to ask closed questions in highly-structured interviews of large numbers of respondents one-at-a-time. The most common type of quantitative interview uses questionnaires which may be administered (carried out) directly (face-to-face), indirectly (voice-tovoice) or remotely (by self-completion questionnaire via post or email). Advice on questionnaire design is provided in Chapter 10.
Asking Questions
Elite interviews Interviews are generally defined as: ‘conversations with a purpose’ (anon). Interviewing members of the political elite – often shortened to ‘political elites’ – as a routine activity is unique to Politics research. But it can prove difficult, especially for first-time or young researchers to gain access to the most appropriate elite or useful information from the encounter. Whilst gaining access to the most senior elites may better serve your research interests, you may well be overawed and inhibited by the combination of their personal presence, the presence of their ‘minders’ and the surroundings (which are often designed to create a sense of power and prestige). Effective interviewing requires sound preparation, planning and confidence. So class-based simulation and individual practice can be very valuable to prepare you for the interview. Remember that the best way to learn is often from other people’s mistakes in class exercises. The whole process of arranging and conducting the interview is bound by institutions (traditions, customs and rules) that vary significantly between states and cultures. For example, officials in the People’s Republic of China will rarely agree to be interviewed unless their superior has given formal approval. Even so, they are obliged to re-state official policy. So, before you seek to interview elites from a different state or culture to your own, find out – from a fellow student from that country or from your diplomatic service – what the institutions are, how they have developed and what purposes they perform. The advice given below is written for students seeking to interview UK political elites. However, much of the advice will apply to students seeking to interview elites in other western states.
Who are the members of UK political elites? Arguably, political elites are people who exercise disproportionately high influence on the outcome of events or policies in your research area. They may be ministers, MPs, senior civil servants, business leaders, union leaders, members of think tanks or financial institutions, learned commentators, journalists, local councillors, chief executives, ‘gatekeepers’ etc. They may influence outcomes without becoming directly involved through what Friedrich (1937) termed the law of anticipated reactions.3
Why do we interview them? • • • • •
to confirm (our understanding of) documentary material to fill gaps or clarify grey areas to check ‘Have I got the story right?’ to try to understand their perceptions, beliefs, mindsets, i.e. underlying psychology to obtain ‘quotable quotes’
119
120
Research Methods in Politics • • • •
to help identify other actors involved to identify networks to facilitate: to open the way to interviews with others to triangulate, i.e. to corroborate (check, test) accounts from other interviews
Triangulation is often best achieved by interviewing the subject’s ‘opposite number’, ‘shadow’ or ‘parliamentary pair’4 (who is likely to know the subject well and may provide a penetrating analysis of personalities and precipitating events), a commentator (e.g. journalist) and a subordinate. Retired individuals are likely to be more accessible and open but not necessarily more honest: self-justification – often sincere – is ever-present. Remember what Enoch Powell said: ‘all political careers end in failure’. You will find that few political elites appear to admit their failures.
Why do they agree to be interviewed? • • • • • • •
because the potential benefits are attractive and outweigh the costs (indeed, you can ask them why they agreed to be interviewed) because you have persuaded them of the importance of your research and their potential contribution because it-goes-with-the-territory and underlines their status because their family and colleagues, etc. have ‘heard it all before’ because they ‘want to set the record straight’ because they welcome the opportunity to be reflective the rules-of-the-game have been agreed in advance.
The rules of the game for interviews cover whether the interview will be on a oneto-one basis, whether the replies will be treated in confidence, attribution, circulation of drafts, publication and recording. Subjects may ask for advance (written) notice of questions to be asked. Paradoxically, there is some confusion as to what the various terms ‘in confidence’ etc. actually mean. You are recommended to adopt a conservative interpretation unless the subject agrees otherwise. Here, in confidence means what it says: that whatever is said remains a secret between the interviewer and subject. Off-the-record similarly means that no recording or note is kept of the interview and that the information is provided to enable the researcher to understand the context and background better. On-the-record means that the comments can be recorded, used and attributed to the speaker. Subjects may switch between ‘off the record’ and ‘on the record’ comments. Attribution means to whom the comments are attributed. Unattributable means that the source is not identified. However, agreement may be sought to attribute comments to a ‘former minister’ to underline the authority of the source. Alternatively, sources may be
Asking Questions
anonymised, e.g. ‘a northern Labour MP, A argued that Minister B’s comments had been reported out of context’.
How should you first approach them to arrange an interview? The approach will depend upon the cultural context. Always follow local conventions and protocol. In the UK, the procedure is generally: 1. Select the ‘target’ political elites on the basis of their recorded involvement in your research topic. Do not rely on any one elite providing you with an interview or all the information: adopt a ‘multi-track strategy’. Aim high rather than low by approaching the ‘A-list’ elites rather than lesser elites whom you may consider to be more approachable. 2. Access personal details from the Internet or parliamentary etc., directory. Telephone their PA (personal assistant or secretary) to confirm their full title and preferred form of address (e.g. Dr, Mr, Mrs, Ms, Minister, etc.). Then write a personal letter on your Departmental University letterhead. Ask for their ‘help’: ‘help’ is a so-called ‘hook word’ to which people have become conditioned to respond positively. Describe the importance of your research and their potential contribution. Ask for an interview. Emphasise that the research is for academic purposes. Reassure them that their comments can be treated entirely in confidence, off-the-record or, alternatively, not individually attributed if they wish. Ask them for an interview preferably before … Add how grateful you will be and end by saying how much you look forward to meeting them. Offer a contact and reference (your research supervisor) so that their staff can authenticate your identity and status. Say that you will telephone their PA on a specific date to agree a time and date for an interview. Describing your intention to call on a specified date means that the PA – who will normally screen and ‘triage’ their mailbox into ‘immediate’, acknowledge/pending or ‘acknowledge/bin’ – will have to classify your letter as ‘immediate’ to seek instructions (or give advice) whether the interview is to be granted or not. Retain the initiative: don’t expect them to contact you first. When you telephone, you may be told that the elite is unable to see you. In this case, ask why in a casual way: the PA may let slip information that may be useful, e.g. ‘Oh, he wants to forget about that: it wasn’t his finest hour’. Paradoxically, a non-interview can be a good source of useful material. An example of an initial letter is shown in Box 9.1. 3. When you speak to the PA, try to obtain a late-morning appointment. This will allow your elite interviewee to extend a ‘good’ interview by inviting you to lunch where you are likely to be introduced to other contacts. Confirm the appointment in writing (email will suffice). 4. If you are unable to speak to the elite or their PA after several attempts, write again. Attach a copy of your earlier letter. Be gently persistent. Exploit contacts and leverage. For example, if the subject is an MP, attend their constituency surgery. Or ask your own
121
122
Research Methods in Politics
BOX 9.1
Example of introductory letter UWUK Department of Politics University of Watersea Watersea WA3 7HU
Rt Hon Tina Atkins MP, PC House of Commons London SWIA 0AA Dear Ms Atkins I’m writing to ask for your help. I’m a local government practitioner studying for an MA in Urban Policy at the University of Watersea. The subject of my dissertation is policy change theory. I’ve chosen ‘public participation in the National Health Service’ for my case study. My special interest is the apparent contradiction between the public’s readiness to comment as patients on their own experiences of NHS services, and unwillingness to become involved as members of the public in broader consultation exercises. I know that, when you were Minister of State at the Department of Health (2005–6), this was a specific area of your responsibility. I understand that, despite the successful introduction of YCTP, your wider ambitions were frustrated by (what the Guardian described, 22 August) as ‘the opposition to reform from the medical profession and leading foundation hospital trusts’. I would therefore welcome an opportunity to meet you in London to learn at first hand of your experiences and insights of promoting policy change. Your contribution will be very significant. I can assure you that information gained will be used for academic purposes only – although I hope that my research can be published in Policy Research Quarterly.Your contribution can – if you wish – be entirely ‘off the record’ or ‘unattributable’. I will phone your PA on Wednesday to – hopefully – arrange a date, time and place where we can meet. Ideally, I would like to see you in June–July and, in any event, before 12 August. I look forward to meeting you. Yours sincerely Ted Smith Email address:
[email protected] Mob: 07790432111 Authentication (if required): Dr Mona Lott (supervisor) Tel: 01903 366 042 (direct line)
Asking Questions MP to arrange an interview. Ask your MP or local councillor to help set up meetings with reluctant civil servants or local government officers. Remember that serving Ministers, civil servants and local government officers are likely to be less willing to be interviewed than others because of the heavy demands on their time and the risk of loss-of-office from unanticipated disclosure. They will want to ensure deniability of any material that inadvertently reaches the public domain. So only one researcher will be allowed to attend the interview. Retired ministers, MPs and officials are likely to have more time on their hands and may welcome the renewed attention and the opportunity to set the record straight. But, whilst they may be less discreet – especially about former colleagues – they will not necessarily be more truthful.
How should you get the best out of the interview? •
•
•
•
•
•
Prepare well in advance. Research the subject using the latest edition of Who’s Who in the university or public reference library (or local equivalents), Internet search or seek a CV from the subject’s PA. Look for points of common interest (e.g. birthplace, school, university or support for a particular football team) and publications for prior reading. Read the opening chapter and conclusion of their latest publication. Also use the Internet to search for recent press coverage. Find a photograph of the elite so that you recognise them at first sight. Having written to say how much you admire their contribution to your topic, it can be very embarrassing to be unable to identify them in a crowded room. Look and dress the part, i.e. conform to the setting. Observe the setting closely. What can we learn from the subject’s choice of books or CDs? Which books look well read? Who are in the photographs? Why does C have photographs on their desk of pets rather than the customary partner-and-children? What image is the subject trying to convey by their habitat: gravitas or person-of-the-people? Conspicuous consumption or thrift? Come prepared with a semi-structured interview proforma, an audio-recorder and clipboard. Politicians generally welcome a recorder as a symbolic acceptance of their authoritative, recordable voice. It will also enable eye contact – to transfix you with their gaze – to be maintained. But civil servants and local government officers will decline (refuse) to be recorded: they want to retain deniability. If the interview is particularly important, use two recorders. Try to avoid an eyeball-to-eyeball arrangement – which is more likely to become an adversarial interrogation. Instead, seek a more relaxed setting where you and the subject are seated at an angle (see Figure 9.1 below). Ideally, place the recorder to one side so that you and the subject can ignore it. Accept the hospitality provided. The offer of coffee is a social convention in an office or home environment. Don’t insist on Fairtrade decaffeinated. Adopt normal good manners: expect and reply to questions about the weather and travel. ‘How are you?’ is a form
123
124
Research Methods in Politics
Avoid if possible
Seek . . .
Figure 9.1 Alternative interview seating
•
of greeting to which the expected reply is ‘fine, thank you’ (rather than a recital of your current medical conditions). Follow Elton Mayo’s advice: 1. Give your whole attention to the person interviewed, and make it evident that you are doing so. 2. Listen – don’t talk. 3. Never argue; never give advice. 4. Listen to: (a) What he wants to say. (b) What he does not want to say. (c) What he cannot say without help. 5. As you listen, plot out tentatively and for subsequent correction the pattern (personal) that is being set out before you. To test this, from time to time summarise what has been said and present for comment (e.g. Is this what you are telling me?) Always do this with the greatest caution, that is, clarify but do not add or distort. (Mayo, 1949: 64)5
• •
Seek to develop and maintain rapport. Respect silence. What you are seeking is profound disclosures. Use the psychologist’s prompts to help the subject cross barriers, e.g. ‘go on . . . ’, ‘can you tell me more?’, ‘how did you feel?’, etc. Act as a mirror. Don’t answer the subject’s questions, reflect them. Ask questions like the psychiatrist Professor Anthony Clare did – not a sneering Jeremy Paxman (BBC2 Newsnight’s feared interrogator). So don’t retort (like Paxman): ‘Surely you don’t expect me to believe that, do you?’ But don’t be over-eager to ingratiate yourself.
Asking Questions And don’t agree with (and thereby reinforce or lend credence to) irrational prejudices, •
fantasies or self-beliefs. Try to maintain eye-contact. Minimise note-taking. If you take verbatim notes then you will be unable to study the subject. They are also likely to slow down their speech so that it
• •
•
•
becomes too well-considered. Don’t be intimidated. Be gently assertive. Watch the subject’s body and look out for differences between what the subject is saying and what is being communicated in body language. Touching the face around the mouth or averting gaze can indicate a less-than-truthful response. Conversely, much waving of open palms – a sign of openness and honesty – may be more evident of a learned attempt to simulate sincerity (see Chapter 11). Retain control (which is difficult with authority figures).You must terminate the interview – generally ten minutes before the hour likely to have been set aside. This early end is likely to lead to apparently unimportant comments by the interviewee which may be very significant. (GP’s [UK family doctors] are told that the most important remarks by patients are made when they are opening the door to leave the consulting room.) Conclude by asking if there is anything that you should have asked but missed out and what would have been the subject’s reply. Write up your field-notes as soon as possible after the interview. Accurate recall fades quickly (which is why wartime bomber crews were ‘de-briefed’ immediately after they landed, even after very long flights).
The structure of the elite interview Generally speaking, a successful elite interview will last 50 minutes and consist of five open questions and five supplementary questions. Your aim is to obtain depth – to reach the subject’s innermost and private thoughts (of which they may perhaps be unaware). I call this ‘level 5’. The interview can therefore be likened to a funnel. So the sequence of questions should ideally begin from the inoffensive ‘level 1’ question through levels 2, 3, 4 before level 5 is reached. What you are trying to do is to win the subject’s trust and interest before asking the most intrusive questions. But you may never reach level 5. It may be necessary to return to a less intrusive question when you sense resistance. By the same token, you may find that the obvious rapport established early in your meeting enables you to ask more difficult questions from the start. Consider the example of the interview with former minister, Tina Atkins about her involvement with the government’s programme for greater public participation in the NHS (National Health Service). What you are really trying to find out is whether the government’s apparent commitment to greater public participation is genuine, or (as you suspect) a stratagem to counter the power of the medical professions and hospital managers who oppose further reform. But, if you ask
125
126
Research Methods in Politics
that question first, she’s likely to repeat government policy about greater public involvement in shaping government policy. So a possible sequence of questions and supplementaries might be: 1. Minister, your (Labour) party founded the NHS 60 years ago. It has been described as the ‘jewel in the crown’ of your party’s record in government. More recently, your government has substantially increased the funds provided for health services. What do you consider has been the NHS’s greatest success? This is a non-adversarial question which is designed to flatter, to place the government minister at ease and to give her the opportunity to be enthusiastic and, thus, open. Her answer may well cite the success of NHS services for children where mortality rates have been drastically reduced. Your supplementary question may therefore be: Why do you think that the NHS has been so successful with children’s services? 2. Many GPs and hospital consultants objected to the founding of the NHS. The minister, Aneurin Bevan famously said that he had obtained the consultants’ agreement by ‘stuffing their mouths with gold’. Despite very generous new contracts, many doctors remain critical of the NHS. How best can the government win their support? Or – as Denis Healey said – is acquiescence sufficient? (Possible supplementary) What lessons have you learned from the ‘new contracts’? 3. Demand by patients for emergency and other health services continues to grow as do the costs of new drugs and equipment. Commentators have referred to a ‘crunch time’ when either services will have to be rationed, or taxes or charges increased. What’s your view of the future? (Possible supplementary) How can your own MPs’ historic opposition to increased charges be overcome? 4. The government favours greater delegation of decision-making to local NHS trusts and greater public participation in the process. But this can lead to different practices being adopted in different areas which the news media describe as a ‘postcode lottery’. The government also sets national policies in the form of ‘national frameworks’ specifying standards of care. Given this complexity, what therefore is the real scope for public participation? (Possible supplementary) I think I understand. But then why did the government scrap the Community Health Councils and the replacement Forums for Patient Involvement? 5. A final question: some critics claim that the government’s new support for greater participation by the public, is merely a cynical ploy to disguise its necessary strategy of wholesale reform of the NHS. They say that you enlist the support of the public only when it suits you, and then as a counter-balance in a ‘force-field’ to the opposition by the medical professions, hospital managers and unions. I understand that you had some
Asking Questions reservations to the high priority given by your colleagues to ‘Patient Choice’. What do you think? (Possible supplementary) Your constituency has two district hospitals providing the full range of services. The government now favours a mix of larger, ‘locality hospitals’ offering full services, and more localised walk-in and GP surgeries. How will you ‘square the circle’ of defending both government policy and your constituents? 6. Minister, thank you for your time. Are there any questions which you believe that I should have asked but didn’t. Can you recommend other people I should speak to?
Remember to remain flexible: don’t continue to ask your pre-arranged questions regardless. Be prepared to use your quick intelligence, charm, humour, guile and cunning to get the most from the interview. Note that you must show technical mastery of your subject if you are to be regarded as a credible interviewer and if you are to achieve any depth. So don’t waste interview time by asking for information that the elite will have expected you to have obtained before the interview. One contentious subject is whether you should mention your own hypothesis during the course of the interview and, if so, when. You must weigh the balance of advantage of disclosure. My experience is that, where rapport is very good, then there are advantages in giving the elite the opportunity to dissuade you from your hypothesis. Post-interview • • • • • •
transcribe the recording and apply the format of commentary-transcript-analysis separate out ‘quotable quotes’ summarise the answers to your original questions triangulate with other sources follow-up leads write thanking the subject and forward a copy of your abstract at a later date. You may want to interview your source again for other research. Use this opportunity to seek confirmation of unexpected replies to important questions.
Interviews with other people You are also likely to want to interview other people – ‘non-elites’ – to provide perspectives from users or practitioners. For example, if you are researching social exclusion, then you will need to interview a cross-section of sociallyexcluded people and those responsible for implementing government policy. They may include homeless people, unemployed people, single-parents, older people, JobCentrePlus staff, social workers, voluntary agencies and council officers. Unlike
127
128
Research Methods in Politics
political elites, they are more likely to be: • •
inaccessible or not readily identifiable apathetic, wary or hostile.They may suspect that you are a police or council officer, benefit inspector, journalist, salesman, crook or confidence trickster, or (worse of all) a ‘university
• •
type’ exploiting their dire circumstances to gain a degree or academic reputation unused to expressing their innermost beliefs and personal experiences – especially to outsiders not able to fully understand what you are saying – and vice versa – either because you speak a different language, or because their use of your common language is highly nuanced.
The issue of communicability is critical to understanding. Schramm’s model of communication is useful here (Schramm, 1961: 5–6).6 He argued that individuals have fields of experience which include language, culture, conventions, institutions and life experience. So people can only communicate effectively in those parts of their fields which overlapped:
A
B
Figure 9.2 Schramm’s ‘Fields of Experience’ (1961: 5–6)
So, to really see the world through the subject’s eyes, you must, first, identify your cultural and communications overlap and, second, expand your own field of experience. Living in the locality and carrying out pilot interviews can be very useful to develop the local vocabulary and to understand local discourses. You should be able to access the research subjects using either the snowball or nomination methods described in Chapter 8. The interview should follow the same semi-structured interview schedule of open-ended questions and supplementary questions used for elite interviews. However, the general style will be gentler and softer. You are more likely to engage at a closer, personal level and gain enduring friendships. Indeed, interviewing members of the general public can be the most enjoyable and rewarding part of the research process. Members of the public are more likely to make admissions and confessions to complete outsiders than to members of their own family. The interview may prove a liberating experience for them.
Asking Questions
BOX 9.2
Twelve commandments for interviewing non-elites
1. Don’t start ‘cold’ (Berg, 2001, 99–100)7 2. Don’t talk too much (Smith, 1993: 40–1)8 3. Use open-ended questions; vary voice and gestures (Smith, 1993) 4. Don’t use stress tactics (but try to avoid feel-good interviews) (Smith, 1993) 5. Don’t emphasise status differences (Smith, 1993) 6. Interview in a comfortable place (comfortable to subject, e.g. their office or home) (Berg, 2001) 7. Use comments (‘active listening’), summaries and transitions (giving opportunities for interviewee to correct) but do not state initial analysis which will otherwise bias responses to following questions (Smith, 1993) 8. Use ‘mirroring’ to copy the subject’s body language and, more importantly, use their words and terminology to build a ‘key vocabulary’ (Bergman, 2003) 9. Be respectful, cordial and appreciative (Berg, 2001) 10. Respect silence: the subject is likely to break-in with important admissions (Bergman, 2003)9 11. Remember the likelihood that subject will also make significant admissions at the end of the interview in ‘hand on the door’ comments once the tape-recorder is switched off. When this occurs, seek a follow-up interview (Bergman, 2003) 12. Take notes (or appear to take notes) (Smith, 1993)
Various guidelines are offered by leading commentators. These can be consolidated into ‘twelve commandments’ for researchers (set out in Box 9.2 above). Payment In the UK, payment is not expected for interviews. Indeed, its offer may offend. But, in other countries, some ‘reciprocity’ is expected. You will therefore need to find out local custom and practice. In the UK and elsewhere, the offer of a gift of a small memento in the form of a university key-ring or letter-opener at the end of the interview is likely to be appreciated. Where a subject has been especially helpful, then sending a relevant paperback book may be appropriate. Recording interviews Interviews should generally be audio-recorded which should be supplemented by notes of key non-verbal events or observations. Ideally, the audio-records should be
129
130
Research Methods in Politics Table 9.1 Three-column transcription of interview Interview A01 Notes
Questions, Responses and Line Number
Analysis
Q.001 And what was your 002 reaction to the news?
4 sec pause, rubs chin
7 sec pause looks up to see reaction telephone rings in outer office
A. [coughs] 003 I rushed to the House 004 immediately to . .
wary of expressing real intention? support ? Was he really a strong supporter of Ken or did he regard him as most likely winner and wish to see what advantage he could gain? Or did he lie low until the outcome became clearer?
005 offer my support to . . 006 … Ken
transcribed into text using the three-column format shown in Table 9.1. Note how you should give each subject and interview a unique reference and add line numbers to enable accurate references to be given, e.g. (A01: 003). A 50-minute interview generally takes a whole day to transcribe. Audio-tapes can be transcribed using a pedal-operated player. Digital recordings can be transcribed direct to your PC. Voice transcription software is now available (Dragon Naturally Speaking Professional 9 from Nuance Communications) which claims to be able to transcribe one voice without extensive calibration. This may be very useful for transcribing narratives. However, voice-transcription software is not yet able to automatically transcribe several voices at the same time (although this appears likely in the future). Full transcription is therefore generally confined to selected parts of elite interviews where a detailed textual analysis is proposed. A supplementary or alternative approach (which I prefer) is to write-up an ethnographic account which combines observations, events and quotable quotes. An example is given in the box (9.3) overleaf.
Difficult questions and subjects Every community has its own pariah or difficult topics where special care is required in introducing, framing and expressing questions. British people have many. One explanation of why British people discuss the weather at their first meeting is that it is the only common, safe topic in a repressed society where religion, politics, sex
Asking Questions
BOX 9.3
Ethnographic account of an interview
The Landowner (2001) The ‘big house’ was a modest, weathered, stone-built, seventeenth century long-house separated and screened from the village below. The only access was a narrow metalled road which terminated in the shadow of the main building. Access was limited to a single archway separating the house from its former coach-house.Two general-purpose family cars stood inside the small yard. A green, woollen sportsman’s tie twisted in the wind from the long washing-line. The main frontage of the house had been rendered and painted white. It looked down a private vista of banked, grazing land bordered by copses of birch. Three, black, Labrador gun-dogs guarded the small child playing outside on the terrace in the unseasonable February sunshine. They barked furiously until stayed by the neat, small woman who introduced herself as the landowner’s wife. He was waiting in the office at the west end of the house which he shared with a teenage secretary. He was short, stocky, boundlessly energetic and, contrary to local reports, very likeable. He wore a green woollen jumper, checked-shirt and light-green moleskin trousers. He led the way into the small, painted dining-room so that he could snatch a lunch whilst he answered the questions. The whole style of the interior of the house was well-maintained, ordered simplicity. He explained that the estate consisted of 7,000 acres of moorland, 300 acres of forestry, 1,500 acres of enclosed land – let as agricultural land to ten tenants – and eight houses … etc. He’d ‘come into’ the estate 15 years ago on the death of his aunt. Since then he had always ‘sought to square the circle … on a daily basis of traditional obligations and responsibilities with the present-day needs of modern estate management. It’s a matter of getting the balance right’.When he took over, he had inherited a workforce consisting of a house-keeper, gardener, two foresters and a game-keeper. He couldn’t justify a house-keeper or gardener and the wages paid to the foresters greatly exceeded the net value of timber produced - for which prices had remained static for 20 years. Asked whether his style of estate-management was traditional-stewardship-paternalistic or modern-exploitative-impersonal, he answered: ‘both … getting the estate economically viable is the overriding consideration … if something isn’t viable, then it must be changed … putting it off will only let everyone down’. His approach was ‘to make the assets sweat … to realise their full potential and to provide only good jobs where the men can recognise their own self-worth and contribution’. Similarly, he granted tenancies, not on the basis of sentiment, but to whoever offered the highest price – subject to references – to ensure that ‘the land would be kept in good heart’. He was therefore opposed to subsidy or any other ‘distortion of the market’ which was likely to prove unsustainable and would prejudice rather than protect people’s futures, etc.
131
132
Research Methods in Politics
and even sport are regarded as essentially difficult and therefore private matters. The pariah topics in western society have been categorised as: • •
illness and disability – especially sexually transmitted diseases and mental illness illegal and contra-normative behaviour including crime, tax evasion, drinking and sexual
•
practices financial status including income and savings. (Foddy, 1993: 115)10
There will also be some people who are unwilling or unable – consciously or subconsciously – to answer your questions. In all these circumstances, projective techniques or vignettes may be appropriate.
Projective techniques and vignettes Projective techniques are designed to prompt answers. The main types are: • •
Quick-fire word association, e.g. night/day, tea/biscuits, etc. Blair/? Quick–fire ‘yes/yes/yes’ a set of questions followed an inadvertent ‘yes’ which are designed to undermine the arrogance or self-confidence of hostile interviewees: e.g. • ‘Is your name XX? yes • You’re a student at Watersea? yes • In Politics? yes • You stole the photocopier? yes, … er, I mean no … • Well, is it yes or no? It’s no • You’re sure? [pause] yes • Well, although you’re unclear about simple matters, let us move onto more complex issues’. This technique is used by barristers and police. • ‘Nothing unusual’ questions, e.g. ‘Have you ever smoked cannabis? after all, nothing unusual, most students do one time or another’. • Substituting difficult ‘if ’ questions by ‘when’ enquiries. For example, asking the question: ‘Have you ever had unprotected sex?’ implies that unprotected sex is ‘bad’. So the subject is likely to answer ‘no’. However, a truthful admission is more likely when you re-phrase the question as: ‘When did you last have unprotected sex?’ •
Asking comments on list or photographs, e.g. ‘What sort of person do you think wrote this list?’ • Using unexpected noises. • Using film or video, e.g. ‘What do you think happened next? Why?’ • ‘Desert Island Discs’ questions, e.g. ‘If you were to be castaway on a desert island which eight music recordings or books would you like to take with you? Why?’
Asking Questions
You will appreciate that not all types of projective techniques are appropriate to all subjects and circumstances. Tact is essential. But you will find that many subjects find some of these types of question to be enjoyable and act as ‘ice-breakers’ in otherwise tense situations. A vignette is a more subtle form of question. Vignettes are: … Short stories generated by the researcher and focussing on hypothetical characters in particular situations. Interviewees are asked what they would do in these circumstances or, alternatively, how they think that a third party might react. The latter mode of question specifically distances the interviewer from the issues being studied and, in this sense, is impersonal and less threatening. (Arksey & Knight, 1999: 94–5)11
Vignettes are premised on empiric evidence that what people identify as their beliefs are often compromised by their actions. For example, when people are asked whether they are honest, they will generally apply affirmatively. Similarly, they are likely to denounce racial prejudice. But actions – what people actually do – provide better evidence of underlying belief systems. Indeed, small changes of behaviour can prompt fundamental changes of belief. Good vignettes pose some form of moral dilemma. They may also ask what the subject should do and what they would be most likely to do. For example, most people, if asked, will say that they strongly oppose cruelty to children or racial segregation. They will also say that they are honest. In these circumstances, vignettes may be helpful. Consider the three examples: 1. A family has moved into the terraced house next door to you. After a few months of heated arguments, the husband quits the house leaving the mother alone to care for two children under six years old. The boy is a ‘bit of a handful’. But you hear him being slapped and being told that he’s ‘stupid’ and ‘just like his dad’. What should you do: A Speak to the mother? B Speak to the authorities? C Do nothing: mind your own business? What would you actually do? Why? 2. You join the queue at the bus stop. When your bus arrives, it’s nearly full. You’re last on board. There are only two seats vacant. They’re next to people of your same age and sex. One is the same race as you. The other is a different race. They both remove their bags. You have three options: A Accept the place next to the person of your same race? B Accept the place next to the person of a different race? C Stand?
133
134
Research Methods in Politics What should you do? What would you actually do? Why ? 3. You approach the vending machine in the refectory to buy a coffee. You are about to insert a 50p coin when you notice that there is already one lying in the change slot. What should you do? A Hand the 50p coin into the porter’s lodge as ‘lost property’? B Use the coin to get your drink? C Put the coin in the charity box in the bar? What would you actually do? Why ?
Good vignettes are difficult to write and should be pre-tested before finalising. One useful adjunct to vignettes is to ask respondents – at the end of the interview – what they actually did in memorable circumstances to calibrate their general responses. Examples might include whether they’ve always voted for the same political party. One set of questions that I’ve found useful is to ask older subjects their support (on a scale of one-to-ten) for the mineworkers in the ‘three-day-week’ (national strike which led to the fall of the Conservative government) of 1974 and, after they have answered, their support for the mineworkers in 1982–3. In many circumstances, partners may ask if they can stay during the interview. You must always give parents or carers of children this option. This can inhibit the subject. But it can also provide or deny corroboration. I well remember that, when a husband replied that he was ‘fully behind the miners in 1983’, his wife added: ‘Oh, no you weren’t. You said that Arthur Scargill should be strung up!’12 Focus groups Focus groups were initially developed largely by market researchers as a means of finding out the potential market for new products and services and, subsequently, ‘blind tasting’ the prototype product against competitors. This type of focus group is also known as a ‘hall trial’, ‘consumer panel’ or ‘citizen jury’. Focus groups have now become widely used in a similar way by political parties to learn how potential leaders, topics and policies are regarded by cross-sections of the general public. More recently, focus groups have been adopted by academic researchers in medicine, sociology and Politics to investigate the dynamics of interpersonal relationships, inter-group conflict and how consensus (or dissensus) emerges. A focus group is not a ‘group interview’ in which the subjects are asked for their replies to the same question. Instead, the focus group seeks to emulate the spontaneous discussion of people who broadly share the same circumstances or identity. In particular, the focus group should ‘draw upon the respondents’ attitudes, feelings, beliefs, experiences and reactions in a way that would not be feasible by other methods,
Asking Questions
for example, observation, one-to-one interviewing or questionnaire surveys (Gibbs, 1997).13 Your role as the researcher is essentially as a facilitator who: • • • • •
invites the participants arranges the venue, refreshments and ‘gifts’ initiates the session moderates the proceedings arranges and transposes the recording of the encounter.
The optimum number of participants is usually between five and eight. This will ensure a variety of views and an opportunity for each person to speak. The participants should, in the first place, provide a cross-section of people sharing a particular identity and location. They can be recruited by word-of-mouth, public notice or via a social gatekeeper intermediary. Where a social gatekeeper is available then they should be used, first, because they are likely to be able to recruit a willing cross-section more easily and, second, because if their help is not sought, then they may be offended and oppose the research. They may be clergy, teachers, councillors or the secretaries of local groups. They will advise you whether and what kind of ‘incentives’ (i.e. a small gift) should be offered to the members of the group. However, you should personally write to or contact the people suggested to you to invite them to the meeting and to outline your research, the benefits to them, and the rules-of-the-game. The venue should ideally be a neutral, central location in which the group can sit comfortably for as long as two hours. ‘Hospitality’ should be offered in accordance with local conventions. So, in the UK, tea or coffee and biscuits should be provided. Occasionally, wine may be offered. The seating should be set out in a circle. But many researchers find that people feel more confident when they are sitting in a circle of easy chairs or, if these are not available, around a defensive perimeter of tables. You should sit down first and allow the members of the group to sit where they wish. The seating pattern is likely to reflect their assessment of their relative social position. So, the members considering themselves most important and capable of speaking out are likely to sit facing you. Conversely, the least confident members are likely to sit at each side of you. You should start the session by explaining the nature of the meeting and restating the rules-of-the-game. Where you have a central topic, then you should specifically invite the views of the group. Alternatively (and often better), you should ask the group which issues are most important to them. Ideally, the session should then proceed entirely driven by its members. But, in practice, some intervention – moderation – by the researcher may be necessary. This may include (tactfully) silencing dominant members (by a finger to the lips), inviting those who have remained silent to state their views, stopping rows emerging and moving on when
135
136
Research Methods in Politics
a topic has been exhausted. So moderating requires good interpersonal, leadership and managerial skills. In addition to learning of individuals’ beliefs and experiences and how group views emerge, the focus group should provide evidence of the language that they use in everyday conversation and local institutions. Focus groups can also provide important ‘forums for change’ where the disparate individuals become empowered by their new collective identity and develop solutions of their own (Race, 1994: 738).14 In these circumstances, you must be prepared to offer to help the group by seeking information and writing letters and reports, etc. for them. Some practitioners strongly argue that the focus group should always be supplemented by meetings with individual members. This enables the researcher to learn individual’s views and how these relate to the group consensus, serious misgivings or changes of belief as a consequence of taking part in the discussions. But the difficulty here is determining what really were the individual’s initial views before the focus group was held. So, some researchers argue the case for a split-half strategy to be adopted in which half the group are interviewed individually before the focus group, and the others afterwards. In this way, the role of the group in developing new perceptions or understandings can be identified better. But holding individual interviews as well as a focus group compromises an important benefit of focus groups: their great efficiency in terms of your time. The focus group should be audio-recorded and the recording transcribed. However, recording a long meeting and transcribing the record can be difficult – particularly where voices overlap, or where it is difficult to attribute contributions. It is also very difficult for you to make notes and moderate the group at the same time. So a further refinement is to use a back-up researcher who will sit outside the group, organise the recording and make notes of key contributions and any off-tape events. The writing-up of the focus group follow the format of an annotated, threecolumn transcription ideally supported by an ethnographic account of observations and ‘quotable quotes’.
Questions for discussion or assignments
1. Draft letters to each of the following seeking an interview to learn their views on the use of military options to counter terrorism: George W. Bush Tony Blair President Chirac Martin MacGuiness
Asking Questions 2. The (Labour) Leader of the city council in the north of England has agreed to be interviewed on the subject of corruption in local government. Your research hypothesis is that ‘standards of public office’ are locally contingent and emergent rather than absolute. A recent press report alleged that the leader had favoured a cousin in a construction contract. Draft a semi-structured interview schedule. Complete and video record an in-class simulation in which two students separately interview the Leader of the council (who is best played by the course teacher or a mature postgraduate). Contrast and compare the interviews. 3. A bacon factory in a nearby market town slaughters and packs pork in a continuous, industrialised process. It employs over 2000 workers. The majority are young women who are bussed in from villages in the upland areas. Union membership is low and staff turnover high. The management recognises USDAW for pay (etc.) negotiations.Your hypothesis is that membership is low because of a combination of a rural culture of deference, the portrayal by managers of the union as a ‘big city thing’ and the domination of union offices by men. The HR staff and union have nominated a cross-section of female workers for you to interview in paid time as a group and singly. Draft a semi-structured interview schedule to be used in individual meetings with the workers. How would you organise and structure a focus group of the workers? Simulate, contrast and compare the interview and meeting. 4. You have arranged to meet a newly-retired infantry major who took part in the invasion of Iraq. He strenuously denies that torture is used by British soldiers. Design a series of vignettes to test your hypothesis that undue pressure was exerted by some soldiers on prisoners to obtain information with the knowledge of their officers who turned a ‘blind eye’ to these practices.
FURTHER READING Arksey, H. and Knight, G. (1999) Interviewing for Social Scientists. London: Sage. This textbook provides a rich variety of theoretically-based, practical techniques for all social science researchers. Berg, B.L. (2001) Quantitative Research Methods for the Social Sciences. Boston: Allyn & Bacon. pp. 66--99. Chapter 4 adopts a ‘dramaturgical look’ at interviewing as variously an art, skill, science, game and face-to-face interaction. It examines the separate roles of the interviewer as an actor, director and choreographer. This text provides very good advice for the inexperienced interviewer. Burnham, P., Killand, K., Grant, W., and Layton-Henry, Z. (2004) Research Methods in Politics. Basingstoke: Macmillan. pp. 205–20. Chapter 9 of this textbook is devoted to interviewing political elites.
137
138
Research Methods in Politics Foddy, W. (1993) Constructing Questions for Interviews and Questionnaires. Cambridge: Cambridge University Press. This well-established text provides sound advice on wording and phrasing of questions and sensitive issues. Harrison, L. (2001) Political Research. London: Routledge. pp. 89–104. Chapter 6 is devoted to conducting interviews in Politics research with particular regard to practical and ethical concerns. McEvoy, J. (2006) Elite Interviewing in a Divided society: lessons from Northern Island. In Politics, 26 (3) 184–91. Joanne McEvoy discusses the particular difficulties raised when the researcher is interviewing elites from ideological extremes. She argues that researchers must consider how their identity may effect the outcome of interviews, bias, trust and how they might seek to ‘probe beyond ethnic party positions based on mistrust of the other side’. Neuman, W.L. (2003) Social Research Methods. London: Pearson. pp. 390–401. The author distinguishes between ‘typical survey interviews’ and ‘typical field interviews’ where the former are standardised to enable comparison and the latter tailored to the subject and circumstances. He also discusses the role and value of ‘informants’ and the contribution of focus groups. Richards, D. (1996) Elite interviewing: approaches and pitfalls. In Politics, 16(3) 199–204. This journal article provides good analytical and perceptive advice on the significance of elite interviewing to Politics research.
Notes 1 Note that, in the UK and many other countries, it is illegal to record telephone conversations without the consent of the respondent. 2 bona fides, Latin, ‘in good faith’. 3 Friedrich, C.J. (1937) Constitutional Government and Politics. New York: Harper & Brothers. 4 An elite’s ‘opposite number’ will normally be the member of the opposition party or group nominated to lead the opposition to a particular policy area for which a minister holds responsibility. They may be called the ‘shadow spokesperson’. A ‘parliamentary pair’ is the term used to describe a grouping of a government and opposition MPs in which, when one is unable to vote in Parliament for unavoidable personal or parliamentary reasons, the other will abstain. 5 Mayo, E. (1949) Hawthorne and the Western Electric Company. London: Routledge. 6 Wilbur Schramm, How Communication Works. The Process and Effects of Mass Communication ed. Wilbur Schramm (Urbana, Ill.: The University of Illinois Press, 1961) pp. 5–6. 7 Berg, b.l.(2001) In Quantitative Research Methods for the Social Sciences. Boston: Allyn & Bacon. p. 99–100. 8 Smith, M. (1993) Personal Interviewing: A Mini-Training Workshop In Breakwell, G.M.; Foot, H. and Gilmour, R. (eds.) Doing Social Psychology: Laboratory Exercises. Leicester: BPS. pp. 23–31, 40–1. 9 Dr Max Bergman, lecture to Essex Summer School as part of the course ‘Qualitative Data Analysis: Interpenetrative Methodologies for Analysing Text and Talk’.
Asking Questions 10 Foddy, W. (1993) Constructing Questions for Interviews and Questionnaires. Cambridge: Cambridge University Press. p. 118. 11 Arksey, H. and Knight, G. (1999) Interviewing for Social Scientists. London: Sage. pp. 94–5. 12 Arthur Scargill was the controversial, left-wing General Secretary of the National Union of Mineworkers during this ‘last, great’ miners’ strike. 13 Gibbs, A. (1997) Focus Groups. In Social Research, 19. Guildford: University of Surrey. 14 Race, K.E., Hotch, D.F. and Parker, T. (1994) Rehabilitation programme evaluation: use of focus groups to empower clients. In Evaluation Review, 18 (6) 730–40.
139
Chapter 10
Questionnaire Surveys
Our researchers with Public Opinion are content That he held the proper opinions for the time of the year; When there was peace, he was for peace; When there was war, he went W.H. Auden (1940) The Unknown Citizen Teaching and learning objectives:
1. To understand the special characteristics of questionnaires and where they are best used. 2. To consider how questionnaires should be structured, designed, tested and coded. 3. To discuss how questionnaires can best be ‘administered’. 4. To discuss issues of personal safety.
Questionnaires Questionnaires are essentially pre-designed lists of closed questions with predesignated, alternative answers. They are used to collect data from a sample of individual subjects (respondents, interviewees). The data can then be aggregated (combined) to create a representative profile of the sample and cross-tabulated to explore the relationships between classifiable variables, e.g. income and education. You can draw inferences about the research population from questionnaire responses of representative samples. You can also use them to measure and compare attitudes (although the accuracy of this data is contested). The essential differences between questionnaire surveys and interviews (or focus groups) are that questionnaire surveys: • •
use very large samples (e.g., opinion polls use samples of 1,500 people, medical research can use samples of 60,000 or more) use only closed questions
Questionnaire Surveys • • • •
use pre-determined answers – including Don’t Know (DK) and No Response (NR) take less time for the subject to complete seek to use representative samples can be administered (carried out) by intermediaries
•
similarly, the tasks of questionnaire design, administration, coding and analysis can be carried out by separate individuals or teams
•
can be administered either: • face-to-face with the subject • remotely by telephone • self-administered by the subject in their own home (or workplace, etc) using selfcompletion questionnaires delivered and returned by mail • email. are a common tool of quantitative method are designed and coded to be readily analysed by computers necessarily involve a trade-off between cost, reliability and time.
• • •
But another important characteristic of modern questionnaire research is declining response rates. 50% is now considered high for commercial market research in faceto-face surveys. Market researchers reconcile low response rates with information claims by adopting the assumption that non-respondents or non-responses (NRs) – including ‘don’t knows’ (DKs) – are not inherently different than respondents. So, the information continues to be typical and representative. However, this assumption can be challenged on the basis that non-respondents may well be atypical. Furthermore, they may share some common characteristics which makes them unwilling to take part or complete the questionnaire. Good response rates require good access to potential respondents, the ability to motivate them, and effective questionnaire design to maintain their interest and co-operation.
Access Access concerns the means by which the subject is contacted. It requires a sample frame (a list of the research population from which the sample can be drawn), a means by which a representative sample can be selected and some form of address or arena where the questionnaire can be administered to them. In the UK, the two most common forms of lists were the electoral register and the telephone directory. So, if you wanted to research the general adult population of a city or part of a city using a 10% sample, then you would use the electoral register to select every tenth person. The register would also give their address where the researcher could call on them, post the questionnaire to or, after looking up their number in the telephone directory, phone them. However, the introduction of the ‘community
141
142
Research Methods in Politics
charge’ (‘poll tax’) in 1990 led to a widespread avoidance of registration. There were also problems in urban areas where population turnover was high. Despite the introduction of a ‘rolling register’, the electoral list remains incomplete. Many students are not included (although, exceptionally, they are permitted to register both at the university and home address and to vote in both localities in local government elections). But, access to the electoral register has become restricted as a consequence of its misuse by stalkers and criminals to locate victims. Now, the full register can only be inspected at local authority headquarters. Whilst a register continues to be published, this is confined to those electors who have agreed to their names and addresses being published. Although the telephone directory continues to be published, a third of (mainly higher income) subscribers choose to be ‘ex-directory’. At the same time, increasing numbers of people – especially young people – use mobile phones in preference to land-lines. Market researchers have responded to these changes by buying lists of names and addresses from commercial agencies, by using ‘random walks’ and by using on-street questionnaire surveys. Even so, some potential subjects remain difficult to access. They include: housebound people, people working away from home, and urban nomads who are not registered with NHS, electoral registrar or national insurance. Homeless people, by definition, lack a contact address. So accessing the necessary full sample frame is very difficult. The absence of free, published lists and the limited resources of most Politics researchers, mean that you are likely to need to use other effective, low-cost means of accessing subjects appropriate to the research. Where you are researching a small geographical area, then the simplest way to achieve a cross-section of the population is to use the nth address technique. In this method, you deliver a questionnaire to (or seek to conduct a doorstep questionnaire survey at) every, say, tenth, house in every street. However, rather than starting at No. 1, you should choose a number between 1 and 9 at random. If that number is 6, then the house numbers will be 6, 16, 26, 36 etc. Where the size of the geographical area is much greater, then you can use the random walk technique. You begin at the centre calling at every nth house in the first street, turning left at the first intersection, right at the second and so on. Where the morphology of the urban area consists of the classic, concentric rings of inner city, inner and outer suburbs it is likely that a crude representative sample of households will be achieved. But in much Politics research, questionnaires are confined to specific groups who may be councillors, trade unionists, party members, etc. In this case, the best method – if a list of members is not available – is to ask the ‘secretary’ or other social gatekeepers to forward questionnaires to every member or nth member. Where a shorter, ‘vox pop’ (‘voice of the people’) questionnaire survey is useful to find out, say, voting behaviour or support for particular parties, then an on-street, face-to-face questionnaire survey of a quota sample is probably the most effective means of accessing a representative sample. Advice on the design of a quota sample is given in Chapter 7.
Questionnaire Surveys
Pilots Having designed the quota sample, then the next stage is to pilot your questionnaire survey, i.e. test it on a small sample. This is most readily done by trialling the draft on non-teaching staff at your university. They will tell you if they find the questions difficult to understand or answer. They may suggest more appropriate (to the local population) ways to ask your questions. The pilot should also help you identify or amend your list of potential answers.
Administering the questionnaire In this context, administering means ‘taking the questionnaire to the people’. The full survey will generally be carried out in the local shopping centre(s) at the weekend. You will therefore look for people who appear to correspond to each of your quota boxes. It is relatively easy to identify potential respondents by sex and age. The most difficult characteristic to identify is socio-economic class, where ‘working’ and ‘leisure dress’ styles have become more universal. Trial, error and intuition (learned experience) become important. One street pollster told me that she had found the quality of shoes to be the best visible indicator. Otherwise, as the survey proceeds, you will have greater difficulty in ‘filling your quota’. The general advice is to avoid people who are busy or otherwise fully occupied. So, avoid parents with young children, people laden with shopping bags or rushing about. Instead, approach those people who have time on their hands. People sitting on benches are ideal. They may be waiting for a partner to finish their shopping and may welcome a conversation with you. You should approach them by ‘catching their eye’ (making eye contact), smiling gently and greeting them. Your initial remarks might be: Hello, what a lovely/cold/awful day.You look as if your enjoying the rest … Can you help? I’m a student at the university. We’re trying to find out what local people think about the EU and why …. May I sit down? It won’t take long … Do you live in Watersea?
Motivation Response rates and co-operation are likely to be much higher when the potential subjects feel motivated to assist. Motivation means providing the necessary incentives for respondents to give their time, views and personal details. This involves appeals to altruism (concern for others’ welfare), public or personal benefits, or small measures of ‘reciprocity’, e.g. a pen, token, lottery ticket or donation to charity. Motivation also involves dispelling doubts about your identity, real purpose
143
144
Research Methods in Politics
and confidentiality. Modern urban relations are characterised by distrust between strangers. They may fear that you are really trying to get them to sign up for doubleglazing, or trying to steal their identity. So motivating them begins by you identifying who you are: the importance of the research to them and how the information they give will be used. You should display your name and university or college on a photo-ID card and carry a letter of authorisation. Your dress and manner should not be intimidating. In terms of privacy and confidentiality, you should be able to reassure them from the start that you do not need or want to know their names or addresses, and that personal views and details will be treated in confidence. They don’t have to answer a question if they don’t want to. However, if they make a very ‘quotable quote’ in the course of the interview, you should ask if you can repeat it as an anonymous comment.
Effective questionnaire design Ideally, a questionnaire should optimise information collected and time involved to enable a large sample to be questioned and representativeness maximised. But good questionnaires are very difficult to design. The most common faults are that they are too long, too pedantic in their use of language, difficult to understand, or ask obvious or stupid questions, e.g. are you male/female or don’t know? The general guidance is: • • • • • • • • • • • • • •
make the language simple make the language short avoid ambiguous or double-barrelled questions avoid leading questions: be neutral avoid negative questions: especially double-negatives avoid questions which assume subjects have perfect knowledge always allow for N/R or D/Ks and continue politely be aware that many words have different meanings to different people: e.g. ‘cool’, ‘gay’ distinguish and separate factual from non-factual questions and answers avoid ‘prestige bias’ (when you imply that certain forms of behaviour associated with high status groups are necessarily better) clarify the time frame, e.g. weekly or monthly income place sensitive questions in context don’t ask very detailed or objectionable questions don’t ask too many questions: try to keep to 12 or less.
The main problem for students is framing questions in plain conversational language. There is a tendency to use academic or technical language and to be precise
Questionnaire Surveys
to the point of pedantry. Excessive qualification of questions tends to produce gobbledygook (literally, ‘turkey-talk’). Verbal and written questions require very different styles of question. Preamble to questions or examples can be helpful to subjects. Remember that, to generate comparable data, each subject must be asked the same question in the same way: avoid the temptation to interpret or personalise the question. The sequence (order) of questions is critical. Begin by checking that the potential subject is within your sample frame. For example, if you are doing a street-survey of support for political parties in the city centre, first establish that the person is, in fact, entitled to vote in UK elections. But it is difficult to ask someone their nationality. So, instead, begin by asking Do you live locally?. They are likely then to say either Yes, or No, I come from Leeds, or No, I’m a US citizen etc. Start with questions which are easy (and, if possible, enjoyable) and which will put the respondent at ease and lead to good rapport (open, friendly, communicative relationship). Leave sensitive or contentious questions to the end. Conclude by seeking key classification data for possible determination of independent variables or cross-tabulation. Classification data are: gender, age group, socio-economic group, education, housing tenure and dependants. Some can be guessed by careful observation but should be checked with the subject. Sensitive questions should be left until nearer the end of the questionnaire, very carefully designed and presented or omitted entirely. In the UK, people – especially middle-aged women – are reluctant to tell you their age. But, older people tend to be proud of their age. But, paradoxically, all are likely to tell you when they were born if asked direct: they’re used to being asked the question to confirm their identity. Income and class can also be sensitive. Political or religious affiliation is now less problematic on mainland UK. Be wary of asking questions of children without the consent of parents or teachers, etc. Find out what questions are sensitive to UK minorities or in countries overseas by speaking to colleagues or cultural groups first. Sensitive questions can be asked by using: • • •
proxy-indicators show-cards post-codes.
Proxy-indicators One way of asking sensitive questions is to use proxy-indicators where precision is unimportant. A proxy-indicator is an alternative means of measuring or describing a variable, e.g. using housing tenure (whether subjects own or rent their homes and the number of bedrooms) as a proxy measure for income. The indicator is effectively an approximation. Alternatively, you can use established classification schemes. The social classification scheme most widely adopted by government and
145
146
Research Methods in Politics
private organisations is the A, B, C1, C2, D, E. The broad definitions and percentages of the population of England given in the 2001 census are shown in Box 10.1:
BOX 10.1 Socio-economic classification scheme, UK Census 2001 A B C1 C2 D E
Higher managerial/administrative/professional Intermediate managerial/ administrative/professional Supervisory, clerical, junior managerial/administrative/professional Skilled manual workers Semi-skilled and unskilled manual workers On state benefit, unemployed, lowest grade workers
22% 30% 15% 17% 16%
This ‘ABC’ classification is a composite of income and status developed to reflect purchasing power. University lecturers are included in A whereas students are included in E together with others who are ‘not economically active’. Adults are classified either by their present or last employment or occupation, or collectively as ‘pensioners’. Children are classified by their parents’ occupation. It is, of course, difficult to ask people which social class they belong to. Asking them whether they are middle class or working class can be difficult and offers only a crude twofold classification. In these circumstances, you can ask questions about subjects’ occupation, newspaper readership, education and housing tenure. Typical questions might be: • ‘What do you do for a living?’ (If they reply that they are retired or not working, then ask: ‘What did you do last’?) • ‘Which daily newspaper do you normally read’? • ‘When did you leave school: at 16 or 18?’ (if they reply ‘18’, you can then ask: ‘Did you go on to university?’) • ‘Do you rent or own your home?’
Note how the questions are phrased using conversational English. Note also how they avoid the prestige bias of implying that the respondents should go to university and should own their own home. The issue of newspaper readership is interesting. Currently, UK newspapers rely on commercial advertising for over three-quarters of their income. To survive, they must therefore sell their readerships to advertisers. The National Readership Survey audits sales and readership using the ABC classification. Newspaper groups seek to ensure maximum market penetration by publishing a range of ‘titles’
Questionnaire Surveys
(newspapers) which are designed to appeal to all the major market segments. Examples are: Times, Guardian, Independent, Telegraph (AB) Mail, Express (C1,C2) Sun, Mirror, Star (C2/D)
So, asking people which newspaper they read can be an easy, inoffensive way of gaining information into their lifestyle group and social class. In the UK, there is also a relationship between newspaper readership and political affiliation. The Telegraph, Times, Mail and Express are traditionally associated with the Conservative Party and The Mirror with Labour. The Guardian and Independent are more sympathetic to Labour and the Lib-Dems whilst the (best-selling) Sun is regarded as supporting whichever party is most likely to win the next election. Therefore, if a subject tells you that they read The Telegraph, then, rather than asking which party they support, you can ask (with a slight smile) whether they support the Conservative Party.
Show-cards Another ways of seeking information on class, socio-economic group or other sensitive issues is to use show-cards. Here you ask the subject to look at a card and to say ‘which letter corresponds to your income’. The card might read:
BOX 10.2
Sample show-card
Q.11 Income per year (before tax) A
£0
5 000
B
£5 000
10 000
C
£10 000
15 000
D
£15 000
20 000
E etc J
£20 000
25 000
L
Don’t know
M
Won’t say
N
No response
£100 000 or more
147
148
Research Methods in Politics
Postcodes A third method is to ask subjects for their (home) postcode (similar to the US zipcode). The housing market in the UK tends to bring together people of similar age, income and socio-economic class. Groups of around 50 homes are classified as enumeration districts (EDs) which form the basic building block of population census data. The firm CACI has matched small area census data and postcodes to create ACORN : a classification of residential neighbourhoods. ACORN categorises all 1.9 million UK postcodes, which have been described using over 125 demographic statistics within England, Scotland, Wales and Northern Ireland, and 287 lifestyle variables, making it the most powerful discriminator, giving a clearer understanding of clients and prospects. (www.caci.co.uk)
So, a simple way of finding out social class, etc. is to ask people their postcode. The weakness of this approach is twofold: first, its assumption that all people sharing the same postcode are very similar; and, second, the cost of subscribing to CACI. However, you can avoid the subscription fee by using a map of post-codes (likely to be held in your university library) to cross-refer to a map of enumeration areas and the census data. Scaling attitudes A major claim of opinion pollsters is the ability of questionnaire surveys to find out people’s attitudes and beliefs by measuring these using scales. Scales are linear representations of measurement. Three techniques are widely used: the semantic differential, Likert scale and rating-scale. Semantic differential The semantic differential was developed by Good in 1957. It uses a five- or seven-point scale to assess respondents’ assessment of conditions or abilities, etc. between pairs of words with diametrically opposite meanings. For example: ‘[In your opinion] how honest is the Prime Minster?’ Extremely Very Quite Neither/Nor 2 2 2 2 Honest
Quite
Very
Extremely
2
2
2
Dishonest
Alternatively, you can give the respondent a list of semantic differentials for comment, e.g. trustworthy/untrustworthy, sincere/insincere. The difficulty is to find generally-acceptable differentials for some words like ‘aloof’ etc.
Questionnaire Surveys
Likert scale The Likert scale (Likert, 1932) puts the question in the form of a statement with which the respondent is asked to scale their agreement or otherwise. For example: ‘How strongly do you agree or disagree with the following statements: the Prime Minister is essentially honest’, etc. Strongly agree
agree
neither agree nor disagree
disagree
strongly disagree
2
2
2
2
2
One variation of the Likert scale is to use wingdings to indicate agreement or disagreement. For example:
☺
☺
☺
Guttman scale A weakness of both the semantic differential and Likert scale is the marked tendency for many respondents to select the central, neither/nor answer. The Guttman scale seeks to overcome this problem by asking a series of related questions which will determine the intensity of the respondent’s attitudes to a specific issue. It can therefore be likened to limbo-dancing or high-jumping. It seeks to build a continuum in which the respondent’s answer to one question is likely to be repeated. The process begins with you drafting a series of statements referring to the chosen topic. Ideally, you should draft as many as 100. You should ask your research colleagues to separately consider the drafts and to say whether or not each statement directly relates to the topic or not. The ten statements receiving the highest ‘yes’ vote are selected. A pilot study of the sample population is carried out in which the respondents are asked whether they agree or not with the statements. The answers are then tabulated into a scalogram in which the questions are ranked by the highest number of positive responses received. The ranking constitutes the scale. An example of the Guttman scale is the Bogardus Social Distance Scale which seeks to measure respondents’ attitudes towards social groups in terms of the degrees of social acceptance. For example, a Guttman scale question might ask each subject:
Would you accept Afro-Caribbean immigrants as: • •
close relatives by marriage (score 7.00) personal friends (6.00)
149
150
Research Methods in Politics • • • • •
neighbours (5.00) work-colleagues (4.00) UK citizens (3.00) visitors (2.00) would exclude from my country (1.00).
Note that agreement with the first statement indicates likely agreement with the following statements until the last. So respondents most favouring immigrants would achieve the highest score, 28 (i.e. 7 + 6 + 5 + 4 + 3 + 2) whereas those most hostile would score only 1. Another example would be attitudes to capital punishment, e.g.
Tell me whether you agree or not with the following statements: • •
• •
There can never be any justification for the death penalty (score 0.00) The death penalty can only be justified in exceptional circumstances following a trial where evidence had shown beyond a shadow of doubt that the accused are guilty of: • crimes against humanity including genocide (1.00) • a murder following a previous conviction for murder (2.00) • the murder of two or more children (3.00) • murder and rape (4.00) • murder in the course of other criminal activity (5.00) • any murder (6.00) lethal injection is not painful enough (7.00) executions should be carried out in public (8.00)
In this case, abolitionists would achieve a score of 0.00 whilst ‘hangers and floggers’ would score 36.00. Note that, except for the first statement, agreement with other statements implies agreement with all preceding statements. The responses can then be displayed in a frequency table and chart which show the distribution and intensity of opposition and support for the capital punishment (see Fig 10.1). Ideally, the statements should be scrambled before they are presented to the respondent. But, in practice, it is much easier for the researcher and subject if the questions are asked in accordance with the scale. Scale-rating Scale-rating is a new scale developed largely by the entertainment media and widely used by younger people. It asks people to ‘rate’ a particular individual or other entity on a scale of 1–10 (where 10 is ‘well cool’).
Questionnaire Surveys Table 10.1 Guttman scale: summary of respondents and scores Proportion of Population (%)
Support for Death Penalty (Maximum 36) 0 1 4 8 13 19 26 35
Intensity of support for death penalty
33 18 14 10 9 7 5 4
40 35 30 25 20 15
Support for death penalty (max 36)
10 5 0 0
10
20
30
40
Respondents (%) Figure 10.1 Chart of Guttman responses
So, a question might be:
Q. ‘On a scale of 1 to 10, how did you rate John Kerry as a Democratic Presidential Candidate?’ But whilst a question might be simple and straightforward for ‘media-savvy youngsters’, it might not be readily understandable to older people.
Ranking Subjects can also be asked to rank the importance or significance they give to particular characteristics, e.g.
151
152
Research Methods in Politics
Q. Which of these characteristics do you consider the most important for a councillor?
A
honesty
B C D E F G H
intelligence local knowledge party loyalty shrewdness business experience personal wealth involvement in the voluntary sector
Q. Which do you consider to be the least important?
Whilst this technique has certain advantages, it can irritate subjects when they are asked to rank a complete list of criteria. It is therefore best used sparingly and with a show-card and pencil to hand where conditions or the short-term memories of respondents prove difficult.
Personal safety Face-to-face interviewing poses potential risks to you. These can be reduced by observing the following do’s and don’t’s:
DO • • • • • •
carry ID and a letter of authorisation from your course leader or tutor written on departmental letterhead and providing a series of contact telephone numbers carry a mobile phone always carry out your questionnaire surveys within sight and earshot of another student carry out your surveys in daylight in a public place consider borrowing or buying a personal alarm comply with any instructions from a police officer to ‘move on’
Questionnaire Surveys
•
where the survey is being carried out as part of an assignment, ask the course leader to write to the local police commander and local authority notifying them that you will be carrying out the survey, when and where.
DON’T • • •
don’t harass or hinder members of the public don’t stand in doorways don’t get involved in arguments.
Overseas students often worry that they will have greater difficulty in recruiting members of the public to help with their surveys. However, experience in York has been that overseas students have rather less difficulty than ‘home’ students because the public are curious and want to find out what they think of the UK.
Coding An important feature of questionnaire design is the coding of questions and alternative responses to enable manipulation of data and analysis by computers. A code is the representation of any group of words (or numbers) by a unique set of numbers or letters. The initial pilot questionnaire should establish the range of most frequent responses (scoping). Advances in software have enabled the coding to be simplified. A simple generic form of coding can be used which combines the number of the question and a number allocated to each potential response. For example, if Question 7 asks whether the respondent left school at 16 or 18 then the coding (see brackets) might be: Left at 16 (07.1) : Left at 18 (07.2) : Don’t know (07.3) : No reply (07.4) :
In this example, the coding is entirely nominal: it has no mathematical significance. However, giving a numeric code to a semantic differential or Likert scale has mathematical significance and allows the ‘average attitude’ to a particular topic to be calculated simply. For example, the Likert ‘strongly agree’ can be coded 2, ‘agree’
153
154
Research Methods in Politics Table 10.2 Schedule of coded responses to questionnaire survey Respondent Ref. A00001 A00002 A00003 A00004 A00005 A00006 A00007 A00008 A00009
Q01
Q02
Q03
Q04
Q05
1 2 1 3 1 1 2 3 4
4 1 3 2 2 2 1 4 2
2 5 4 3 2 2 3 5 2
1 2 1 2 3 2 2 2 1
3 4 4 3 2
etc
Notes interrupted
rushed 5 2 2
coded 1, ‘neither agree nor disagree’ 0, ‘disagree’ −1, and ‘strongly disagree’ −2. So, if the average is 0.25 then the average view of the public is only marginally in agreement with the statement. But there is a flaw: the assumption that levels of agreement etc, are linear and equal, so that one person’s ‘strongly agree’ is equal to two people’s ‘agree’. This assumption is highly questionable. The coding will enable the answers received to be recorded directly into a table or spreadsheet: you do not need a separate questionnaire for each face-to-face interview. It is usual to list the respondents in the left-hand column and the questions in the top row. Each cell in the table will contain the sub-code for the answer given (see Table 10.2). Each respondent should be given a unique reference which, where a team of researchers is used, should identify who administered the questionnaire. This can be extended to denote where and when the survey was carried out. The best way to learn how to design a questionnaire is to complete one. An example of a questionnaire is therefore given below as a demonstration.
Questionnaire You have been set the assignment of answering the research question: ‘Why are the British so anti-EU’?
Examination of the Eurobarometer site has shown that support by the UK public for the EU has fluctuated. But support is greater in Scotland and Wales. Overall UK support reached its lowest following French and German refusal to join the invasion of Iraq and widespread opposition to the proposed new constitution. The UK government has deferred any decision on ‘joining the Euro’ (European Monetary System) and any new constitution which would involve majority voting.
Questionnaire Surveys
However, the UK government continues to support further ‘enlargement’ – possibly to reduce Franco-German dominance. Your theoretical and literature review leads you to argue that UK people are indifferent towards the EU and adopt a rational choice perspective of economic self-interest. So, when the UK economy is performing better than continental Europe, people become less enthusiastic about the EU. You therefore reject the argument that hostility to the EU reflects an increasing nationalism shaped by the anti-EU Murdoch press (News International Corporation) whose front page headline was ‘Up yours Delors’ (when Jacques Delors was head of the EU commission and favouring greater EU integration at sub-national level to create a ‘Europe of the Regions’). Your assignment requires you to design a questionnaire to administer to a quota sample of 100 adults in Watersea (which happens to offer a microcosm of the UK population). Your initial design is: Hello, can you help me? I’m a student at the university. We’ve been asked to find out why British people are so anti-EU. I’d like to ask you ten questions. It will take five minutes. I will not ask you your name or address, so whatever you tell me can’t be traced back to you.
Q1. Do you live locally? • Yes • No • Don’t know • No response
Code
1.1 1.2 1.3 1.4
[This is a filter question. If the answer is no or no response, then you should thank them for their time and move away. If they reply that they don’t know, then show them a map of the Watersea administrative area to discuss where they live and whether or not it forms part of the city.]
Q2. How would you describe your overall attitude to the EU? • Strongly supportive • Supportive • Opposed • Strongly opposed
2.1 2.2 2.3 2.4
155
156
Research Methods in Politics • Neither support nor oppose
2.5
• Don’t know • No response
2.6 2.7
Q3. So what would you prefer the British government to do in the future? • Become more involved with the EU, for example, by joining the Euro at a date in the future • Leave the EU • Remain the same • Don’t know • No response
3.1 3.2 3.3 3.4 3.5
Q4. Supporters of the EU claim the following benefits of membership: • A larger ‘common market’ for goods • Greater protection for workers’ rights • Freedom for British people to work abroad • Greater anti-pollution controls • Protection from ‘dumping’ by other countries • Counter-balance to USA [Pass show-card] Which do you consider the two most important of these benefits? • Don’t know • No response
4.1 4.2 4.3 4.4 4.5 4.6
4.7 4.8
Q5. On the other hand, critics of the EU say that the costs are considerable. They argue that the disadvantages are: • Loss of UK sovereignty • Harm to Third World caused by tariff walls • High price of EU food • Common Agricultural Policy, wine lakes and butter mountains
5.1 5.2 5.3 5.4
Questionnaire Surveys • Freedom for Poles and other EU people to enter UK to seek work
5.5
• The ‘Brussels bureaucracy’ [Pass show-card]
5.6
Which do you think are the two biggest disadvantages on this list? • Don’t know
5.7
• No response
5.8
Q6. I’d like to ask you how patriotic you feel. How proud are you to be a resident of
The city (Watersea) The county (Watershire) The region (Yorkshire and Humberside) England United Kingdom EU Don’t know No response [Classifier data]
A lot
A little
Not at all
26.1 26.4 26.7 26.10 26.13 26.16 6.19 6.20
26.2 26.5 26.8 26.11 26.14 26.17
26.3 26.6 29.9 26.12 26.15 26.18
Some people claim that age, sex and background affect people’s attitude to the EU. Q7. When were you born? Or, how old are you? • Before 1939 • 1940–49 • 1950–59 • 1960–69 • 1970–79 • 1980–89 • No response
68+ 58−68 48−57 38−47 28−37 18−27 7.7
7.1 7.2 7.3 7.4 7.5 7.6
Q8. Which daily newspaper do you normally read? • The Times • The Telegraph
8.1 8.2
157
158
Research Methods in Politics • The Independent
8.3
• The Guardian • The Mail
8.4 8.5
• The Daily Express
8.6
• The Sun
8.7
• The Mirror
8.8
• The Star • The Yorkshire Post • Other • None • Don’t know • No response
8.9 8.10 8.11 8.12 8.13 8.14
Q9. What do/did you do for a living? [ask them for their occupation which you allocate to the most appropriate group below] A B C1 C2 D E G G G Male Female
Higher managerial/administrative/professional Intermediate managerial/administrative/professional Supervisory, clerical, junior managerial/administrative/ professional Skilled manual workers Semi-skilled and unskilled manual workers On state benefit, unemployed, lowest grade workers (and students) Not classifiable Don’t know No response
9.1 9.2 9.3 9.4 9.5 9.6 9.7 9.8 9.9 10.1 10.2
Thank you for your help. Have you any questions for me or additional comments about the EU that you’d like to make?
Note how the temptation has been avoided of asking the respondents’ knowledge of the research office by, for example, asking: ‘who is Head of the European Commission?’ A questionnaire should avoid being a quiz or asking questions which seek to verify answers: don’t be too clever.
Questionnaire Surveys
Questions for discussion or assignments
1. Pilot the questionnaire on at least two members of the public (not students). What improvements would you suggest and why? 2. You are asked to test the hypothesis that age and education strongly influence the likelihood of electors voting in general elections. You must use a questionnaire. Identify the sample frame. Devise a quota sample for an on-street survey. Design, test and revise a questionnaire of no more than 12 questions. Administer the questionnaire to ten members of the public in the main shopping street. Comment critically on your experiences as a questionnaire writer and questioner. 3. Discuss critically the claim that questionnaires can measure people’s attitudes and beliefs. 4. Most of the national opinion polls published in the lead-up to the 1992 UK general election wrongly forecast the outcome. Critically assess the various explanations given by analysts.
FURTHER READING Burnham, P., Grillard, K., Grant, W. and Layton-Henry, Z. (2004) Research Methods in Politics. Basingstoke: Palgrave Macmillan. pp. 92–105. This text provides sound, practical advice on questionnaire design and coding. It includes memorable examples of poorly-worded or ambiguous questions from national surveys. De Vaus, D.A. (2001) Surveys in Social Research. London: Routledge. pp. 80–105. This textbook differs from most others by emphasising the importance of designing the questionnaire to measure the dependent and independent variables, test variables and background measures. It also offers additional guidance on telephone interviews and approaches to pre-testing questionnaires. Gilbert, N. (2003) Researching Social Life. London: Sage. pp. 85–104, 227–51. Gilbert provides more pages of advice and examples than most other texts and offers extensive guidance on pre-coding data. Harrison, L. (2001) Political Research: An Introduction. London: Routledge. pp. 46–55. Harrison discusses the methodological issues and provides rich case studies including analysis of the errors made in the national opinion
159
160
Research Methods in Politics polls leading up to 1992 when most of the pollsters’ forecasts (that the Conservatives would lose) proved wide of the mark. Yates, S.J. (2004) Doing Social Research. London: Sage. pp. 31–64. Whilst coverage of the topic may seem slight, this book excels in the rich case studies (including the 1992 UK polls) and the discussion of model answers to the questions asked of readers.
Chapter 11
Observation
‘A politician can never [lie] flawlessly because their body language will always give them away, psychologists say’. (BBC News 5 September 2006)1 Teaching and learning objectives:
1. To consider the relevance of observation to Politics research. 2. To critically assess the claims made for a new ‘science of body language’. 3. To consider other claims which consider the origins of body language and argue that, rather than being instinctive, body language is culturally specific and often learned. 4. To consider how to systematically observe, record and interpret behaviour.
Introduction A senior nurse pursuing doctoral research in Politics at York argued that too much attention is paid by Politics researchers to what people say and write and too little to how they behave and what they do. She described the importance given in medicine to the systematic observation of patients as part of the initial process of analysis and diagnosis and, subsequently, in assessing the efficacy of any ‘intervention’ (treatment) and ‘management’ of illnesses (which are no longer regarded as ‘cured’). Particular attention is paid to how the patients initially ‘present’ themselves: degrees of consciousness; whether they exhibit cheerfulness, sadness, pain or anxiety; whether their faces are pale, perspiring or ‘florid’; whether they walk with or without difficulty; whether they are obese, well-nourished or thin; the attention they pay to their surroundings; hearing or other sensory difficulties; how they treated their carers and medical staff, and so on. These initial observations are noted and followed by measuring the ‘vital signs’ of blood pressure, pulse, temperature, breathing and heart rate. This researcher argues that Politics researchers should similarly observe, record and analyse how subjects present themselves as an integral part of the data collection and analysis process.
162
Research Methods in Politics
Chapter 9 has already commented on the value of recording the appearance, manner and surroundings of political elite or other subjects during elite interviews or focus groups. But the main purpose of recording this information is to provide richer material for your readers so that they can better share your experience. The claim that this information can provide a basis for analysis is contestable. The patient who ‘presents’ to the medical team does so because they or others believe that their condition is abnormal and heightened. In most cases, hopefully, the political elites who you interview will not normally be displaying noticeably unusual behaviour. But when they do, it is quite right and proper that you note this and consider its significance later. This is particularly important when you find yourself or others affected by their charisma (or lack of it) or loathfulness or feel intimidated by them. My own experience is that personality does count in arenas of power. Dominant personalities are often able to drive through policy change which their peers recognise as disastrous but find themselves – like rabbits frozen in the headlights of a vehicle – unable to oppose. Mrs Thatcher and community charge (poll tax) is an example of this phenomenon. Historically, lobby correspondents (Parliamentary journalists) and political opponents have suggested that observation of the mannerisms of political elites provides important clues to sincerity, truthfulness and real intentions. More recently, body language has been developed and promoted as a scientific technique for the analysis of intentions. Its disciples repeat the assertion that: over 80% of effective communication is non-verbal. (anon)
But what are the claims of body language? How can they be assessed? What is its value to Politics research?
Body language: the claims There is a large literature – especially in management science – which presents body language as a science. It claims that mastery is essential for success as a negotiator, salesperson or analyst. Duly equipped, the manager can detect liars and, as an added bonus, simplify courtship by ‘reading the signs’ (or tells – from ‘tell-tales’). There are a large number of books on the theme of Learning Body Language in a Week etc. Body language has also attracted the attention of the news media who can use their back catalogue of video-tapes to illustrate its application. Claims made are that: Tony Blair unconsciously fiddles with his little finger whenever an opponent makes him anxious … George Bush bites the inside of his cheek at anxious moments … Bill Clinton
Observation tended to bite his lip when he wanted to appear emotional … George Bush walks like a bodybuilder, hanging his palms to the rear as though laden by huge muscle, to imply that the is larger than he is’ … [However] no amount of coaching or media training can co-ordinate the hand gestures and facial expressions to fully cover up what a person knows not to be true.2 You can’t fake [a genuine smile]. It’s too difficult.3
Expert opinion Leading British commentators include Dr Desmond Morris: Oxford Fellow and former Keeper of Mammals at London Zoo. His publication The Naked Ape (1967) stimulated interest in body language and was the first of many publications. However, his reputation may have suffered from later over-exposure in the news media and the popularisation of his literature. However, unlike other ‘lite’ body language publications, he presents well-argued explanations and encourages caution in the interpretation of human behaviour. Morris’ Manwatching (1977) begins with the claim that our everyday actions – postures and movements – are performed unconsciously.4 In most cases, they are a response to a perceived stimulus. He argues that the claims that all actions are natural (innate) or all actions are nurtured (learned) are false. Blind, deaf and dumb children will smile and cry. However, empiric evidence shows that the meanings of other actions (e.g. crossed-fingers which historically reflect the crucifixion in western societies) are unique to different cultures. He therefore begins by drawing a distinction between genetic inheritance (innate) and discovered, absorbed, trained and mixed actions. Discovered actions are those which people develop as they mature. An example is the folded arms posture which Morris suggests is derived from the ready movement and hinging of the arms, and the comfort and protection provided. Absorbed actions are those that are unconsciously copied. Copying is more likely of higher than lower status groups. He cites the example of the camp posture adopted by some adolescent homosexuals. But technology and fashion can play a part. He attributes the informal posture of teenagers to the jeans that they have adopted. So he argues that the same people wearing suits will adopt more formal postures to avoid damaging the less-forgiving material. But he also attributes the new informality to changes in philosophy and teaching which emphasise more open styles and less overt deference, and the desexualisation and sexualisation of dress. Trained actions are those that require some teaching, e.g. winking, clicking fingers and hand-shaking (although the distinction between absorbed and trained actions seems perhaps over-stated).
163
164
Research Methods in Politics
Mixed actions have, as the term implies, a combination of origins. For example, innate actions like crying are modified by social pressure to an occasional tear or sniff in western society and wailing in other cultures. Another mixed action is the realisation of the comfort derived from sitting cross-legged. But US culture regards cross-leggedness as effeminate: hence, perhaps, the US practice of men sitting behind the backs of the chairs to achieve manly postures. Social etiquette also affects the precise form of actions. Gestures are a sub-group of actions: they are adopted to visually transmit a signal to the observer. Morris introduces a convenient classification of gestures as either incidental or primary. Primary gestures are those which you perform alone. They are non-social and convey no meaning. You rub an itch because it relieves the discomfort. Winking, on the other hand, is an incidental gesture which provides mood signals (in this case, collusion). Sometimes you deliberately use primary gestures as stylised incidental gestures to convey mood, e.g. when you deliberately slump or yawn to communicate your boredom and disrespect for the teacher. There are six primary gestures of which five are unique to Homo erectus (‘Man’). The exception is the expressive gesture. This is made by the face or hands. But, as we mature, they can become modulated by cultural and gender influences. Incidental – purposeful, communicating – gestures include mimic gestures. They can be social mimicry (putting on a good face), theatrical mimicry (which imitates specific actions or seeks to emulate the imagined mood of the part), partial mimicry (where we imitate an entirely different species like a bird or cat) and what Morris terms vacuum mimicry – which today you would call air mimicry like air-guitar-playing – where, for example, you emulate drinking to indicate your wish to drink. Other types of signal gestures include schematic, symbolic, technical and coded gestures. Schematic gestures are abbreviated (e.g. using two fingers to emulate cigarette smoking). Symbolic signals are essentially local social conventions: for example, tapping the head and pointing in western society can mean mad, whereas in Saudi Arabia this signal is given by lowering the eyelid. Technical gestures are made by specialist minorities where voice communication is not reliable in potentially dangerous situations. They include, for example, the hand-signals given by construction workers to crane operators, media directors to presenters, and those given by bat-men to pilots of aircraft on the taxiway. Although specialised, these gestures are intuitive. Coded gestures are the formal language of sign language (of which there are several types), semaphore and the bookmaker’s assistant (tick-tack man) at race courses. Some gestures may be relics where they imitate an action outdated by technical change (e.g. indicating a bad smell by holding the nose and imitating the action of pulling-the-chain to the historic overhead cistern). Morris argues that the near-universal head-shake – meaning ‘no’ – is derived from babies’ actions when they have had enough milk and push the breast way. Similarly, the cigarette is an adult baby-comforter (dummy). Baton signals have been called: ‘beating time to the rhythm of words’ (1977:56). They are widely used by political elites. They include the open palm, the air grasp,
Observation
the hand-chop, the air punch, the raised fist and the palm back. The raised fist is widely used to add vehemence (Martin Luther King) or aggressive intent (Mohammed Ali), whereas the lowered, tight fist transmits unshakeable determination (Nixon). Guide signs are used to indicate direction. They can indicate ‘go there’ (extended hand) or ‘come here’ (beckoning, hinged hand). Gaze behaviour (eyeballing) is complex: when two people meet and make eye contact, they find themselves in an immediate state of conflict. They want to look at each other and at the same time they want to look away. The result is a complicated series of eye movements … (1977: 71).
Gaze can also show dominance or submission. Passive dominance or passive submission will involve looking away whereas active aggression or fear involves intense looking (even glaring) at the other person. You are also likely to look away or down when speaking to prevent other people from catching your eye and disrupting you. So you indicate turn-taking (the alternation of dialogue between speakers) by looking up and at the other person to show that you have finished and to give them the opportunity to speak. But people can also lower their gaze when they are concentrating. So a parent’s appeal to a child ‘to look at me when I’m talking to you’ may be seeking the child’s submission rather than their attention. Postural echo refers to the social mimicry that is demonstrated by people at ease with one another. You will probably deduce from two people whose body language is mirroring each other’s that they are developing a relationship. Body-contact tie-signs refer to the degree of exaggeration of normal greetings like shaking hands. The bone-crunching hand shake, the grasping of the other person’s hands with two hands or gripping their arms can all be signs of special friendship or dominance (but do close friends shake hands?). In contrast, auto-contact behaviour is the term used to describe our actions when we touch ourselves. Common forms include the jaw-support, chin support, hair clasp, cheek support, mouth touch and temple support. Some – like touching the mouth – are popularly believed to be signs of telling lies. Morris includes lying as a type of non-verbal leakage: clues that you give away unwittingly your real feelings or intentions. Like everyone else, you will lie in your everyday life: you say that you are OK when you may be unhappy. Morris argues that you best lie with those parts of your body with which you are most familiar. So, because of your knowledge of your face from mirrors, you can lie more easily by adopting a combination of words and facial expressions of, say, happiness (putting on a brave face), etc. But you are less aware of the movements of our lower limbs. So Morris advises that: ‘if you have to lie, do it over the telephone or peering over a wall’. He disparages research that seeks to show that, when people lie, they make more frequent hand movements, hand-to-face contacts, mouth-covers
165
166
Research Methods in Politics
and nose touches. Instead, he argues that these are physiological auto-responses to pressure. These include the increase in the sensitivity of the nasal lining which nosetouching reduces. So, these tells should never be regarded as conclusive. Similarly, barrier signals – for example, touching shirt-cuffs or crossing limbs across the body – are also more likely to be responses to a sense of nervousness. Examples of activities stimulated by nervousness – displacement activities – include: jingling coins, clicking biros and re-checking boarding passes (reportedly ten times more common among air than rail passengers). The most reliable indicator of mood appears to be eye-pupils. These react spontaneously to changes of light and emotion. They cannot be controlled. Your pupils will automatically narrow in brighter light or when you see an unpleasant object. They will widen when you see something you like, for example, a baby or a kitten. Pupil signals are both unconsciously given and received. (Hence, perhaps, the appeal of dark glasses to some insecure political elites and poker players.) But the difficulty remains actually being able to detect the width of pupils beyond very close range and without appearing over-curious (which presumably would affect the pupils under our observation). In any event, medical sources point out that pupil dilation can also be a symptom of illicit drugs or amphetamines, drug overdose, certain medications, eye drops, brain injury, stroke, brain tumour or, in extreme cases, death.5 Morris concludes by discussing threat signals, dress, body adornments, infantile signals, sexual signals and play patterns, etc. He also discusses (and appears to support) the claim that, as human beings are the only female primates to have both hemispherical breasts and buttocks, the female breast shape has evolved as a mimic of the other primates’ buttocks which are the principal body parts used for sexual display In two later books, Morris explores the distribution of signs and gestures in a large sample of European and North Africa states (Morris et al, 1979).6 They show that, whilst the same signs are common throughout Europe, they may have entirely different and opposed meanings. For example, the ring sign means OK in the UK, Ireland, Spain, Germany and southern Italy. But in France and Tunisia, the ring sign means zero. Yet in Greece, Malta and Sardinia, it means orifice. In Tunisia, it means threat (1979: 99–103). The influence of trade and cultural trade links is obvious from the mapping. Morris extends his study of body language worldwide by trying to identify signs and gestures which are universal and thus more probably innate (Morris, 1994).7 Nearly 600 informal gestures are studied. Only 50 are worldwide. The gestures (G) (and their meanings) are: • • •
arms akimbo: (keep away from me!) arms reach: (I offer you my embrace) belly rub: (hunger)
Observation • •
body leant forward: (I am paying attention) body leant forward with hands gripping chair: (I am about to leave)
•
chest tap: (me!)
•
chin jut: (threat)
•
chin lift: (I am above such things!)
•
chin rub: (I don’t believe you: subconscious)
•
chin withdraw: (fear)
•
ear cup: (speak up!)
•
ear rub: (I don’t wish to hear this: subconscious)
•
ear scratch: (I am confused)
•
ears block or cover: (stop the noise!)
•
eyebrows flash: (greeting)
•
eyes side-glance: (bold shyness or coyness)
•
eyes stare: (threat)
•
eyes weep: (distress)
•
fist punch: (forceful emphasis)
•
fist shake: (threat)
•
footlock: (discomfort)
•
finger beat: (moderate threat)
•
forefinger point: (threat)
•
hand chop: (I cut through argument)
•
hand jab: (I insist)
•
hand purse: (I am hungry)
•
hands scissor: (that is finished)
•
hands wring: (please, help me)
•
head nod: (yes!)
•
head support: (boredom)
•
knees clasp: (I am about to leave)
•
knees crossed: (I am very relaxed)
•
ankles crossed: (I am politely relaxed)
•
lips kiss: (love: possible relic of food passing)
•
mouth smile: (pleasure)
•
neck clamp: (I am angry)
•
nose flare: (anger)
•
nose tap: (I am – unconsciously – hiding something)
•
nose up: (superiority)
•
nose wrinkle: (disgust)
•
palm up cupped: (please give)
•
palm up flat: (pay up!)
•
palms back: (I embrace you)
167
168
Research Methods in Politics • • • • • •
palms down: (calm down) pupils dilate: (widen) (I like what I see) shoulders shrug: (I don’t know) stomach clasp: (I am hungry) tongue protrude: (insult) tongue protrude – slight: (I am concentrating)
Note that, in this book, Morris contradicts his earlier argument that nose-touching could better be explained as a response to changes in the nasal lining caused by nervousness. Conversely, some gestures that are acceptable in some cultures may be unacceptable and insulting in others. For example, in western society, the knees-cross is an indication of relaxation and confidence. But the extended kneescross will inevitably show the sole of the raised foot to onlookers (observers). In Saudi Arabia, Egypt, Singapore and Thailand, this gesture is interpreted by onlookers as showing that that they are considered to be the lowest of the low and dirt-ridden. It is therefore especially offensive and can provoke extreme violence (Morris, 1994: 77).
Recording and analysing body language You will find many public meetings of many councils and health trusts, etc. and court proceedings very dull. A useful way of spending your time is to record the body language, and frequency and tenor of speeches. You can do this by drawing either ‘matchstick figures’ or, if you are more ambitious, sketching profiles. (You are not allowed to photograph or record meetings of UK public bodies.) But drawing in this way enables you to analyse what you would otherwise overlook in a photograph. So it uses your time productively. Pay particular attention to posture and the attitude of heads and hands. My experience is that people who sit close to the table and remain still are primarily lending support rather than taking part in the deliberations. They are usually backbenchers or officials. Most of the main protagonists will adopt – literally – a laid-back approach resting back in their seats. They may appear bored. Those leaning forward are likely to be aspiring councillors or others keen to demonstrate their attack and defence skills to their leaders. Note how – in a new tradition derived from the US – the most senior party member present will often take off their jackets and sit in shirt-sleeves. Note the exaggerated, theatrical gestures and how they appear to lose their tempers. Watch the young male councillor who leans back with his hands behind his head indicating his self-satisfaction. Note how, when party leaders say something even slightly funny, the others laugh loudly. This courtier-like behaviour is a classic sign of dominant power. Compare this to when a junior councillor says something really amusing and no one laughs until the leader
Observation
Figure 11.1(a) Simple matchstick drawing
leads or halts the laughter. Look at the mutual dependency between the senior councillors and the senior officers. Record who speaks to whom and the tenor (volume, pitch, rhetorical style, etc.) of their voices. Show these as thick or thin arrows in your record. Note that, despite the apparent intense arguments between the parties, they readily enjoy one another’s company after the meeting. What you have seen is the elaborate role play of the ritual of democracy: the real decisions are often taken privately elsewhere. Examples of matchstick figures and more representative drawings and annotation are in Figure 11.1(a)-(d). These drawings and analysis were made from sketches made at a meeting of a local council meeting. The same techniques can also be applied to
169
170
Research Methods in Politics
Figure 11.1(b) Simple representative drawing
Figure 11.1(c) Detailed representative drawing
Observation
Figure 11.1(d) Communication chart
focus groups although the initial sketches should best be made from memory after the meeting.
Conclusions The principal conclusion must be that – perversely – body language is an inconclusive indicator of underlying behaviour. As a famous US movie star remarked: ‘there is less to this than meets the eye’.8 There are relatively few innate gestures. But those which are innate can be consciously emulated for effect. Similarly, many of the indicators of insincerity – like nose- or mouth-touching – may be more readily attributed to nervousness. They can also be restrained. It can also be argued that, in our televisual age, body language has become learned and exploited by political elites to enable them to communicate their own ‘spin’ on events more readily. Similarly, the news media are able to extract seconds of tape from hours of film to illustrate any point. So the body language of non-elites and others untrained in the artistry of body language may be more revealing. In any event, body language is a classic case where the observations should be recorded separately and before any analysis is attempted or conclusions claimed. Perhaps the most important conclusion that we should draw is of the potential for our own repertoire of gestures to antagonise people from other cultures during fieldwork. So, in the final analysis, self-awareness and restraint of your own body language during fieldwork may be more important than your observation of others’ behaviour.
171
172
Research Methods in Politics
Questions for discussion or assignments
1. What significance should be attached to the observation and interpretation of body language in Politics research. 2. Review the outcome of a search on the Internet for ‘body language pictures’. Using a data projector, display and discuss your interpretation of the various catalogues of pictures from sources including the BBC. 3. Record, project, discuss and analyse the non-verbal communications techniques used in the most recent wave of party political broadcasts. 4. Attend a meeting of your local council or NHS trust. Note and comment on the body language displayed by the main participants when they are speaking and listening to others. 5. ‘A politician can never [lie] flawlessly because their body language will always give them away’ (Collett, 2006). Discuss using examples and illustrations.
FURTHER READING Morris, D. (1977) Manwatching; A Field Guide to Human Behaviour. London: Jonathan Cape. Morris, D., Collett, P., Marsh, P. and O’ Shaughnessey, M. (1979) Gestures: Their Origins and Distribution. London: Jonathan Cape. Morris, D. (1994) Bodytalk: A World Guide to Gesture. London: Jonathan Cape.
Notes 1 http://news.bbc.co.uk/1/health/5316916.stm (10 October 2006). 2 http://news.bbc.co.uk/1/health/5316916.stm (10 October 2006) Quotations attributed to Dr Peter Collett, formerly of the University of Oxford speaking to the British Association’s Science Festival in Norwich. He co-authored with Desmond Morris the book Gestures: their origins and distribution (1979). See end-note 5 below. 3 http://news.bbc.co.uk/1/health/5316916.stm (10 October 2006) quotation attributed to Prof. G. Beattie, University of Manchester. 4 Morris, D. (1977) Manwatching; A Field Guide to Human Behaviour. London: Jonathan Cape. 5 www.wrongdiagnosis.com 6 Morris, D., Collett, P., Marsh, P. and O’Shaughnessey, M. (1979) Gesture: Their Origins and Distribution. London: Jonathan Cape. Collett and Marsh were members of the Department of Experimental Psychology at Oxford. O’Shaughnessey is a language graduate. 7 Morris, D. (1994) Bodytalk: A World Guide to Gestures. London. Jonathan Cape. 8 www.centralsquare.com/quotations.html
Part IV Data Analysis
Chapter 12
Analysing Research Data: The Process
‘A single swallow doth not a summer make.’ (Greek proverb cited by Aristotle in Nichomachean Ethics, I, vii, 16, fourth century BC) ‘One swallow doth not a summer make. But two swallows might attract a research grant, and three are enough to work up a paper in a marginal journal of climatology …’ (James Meek, The Guardian, 9 November, 2000) ‘Politics researchers tend to draw summers of conclusions from single swallows of data.’ (Anon) Teaching and learning objectives:
1. To understand the weaknesses and limitations of earlier methods of analysis. 2. To consider the recommendations made by authorities. 3. To develop a general hierarchy of analysis for Politics research.
Introduction Traditionally, the analysis by Politics researchers of the information collected – especially qualitative information – has been weak if not altogether non-existent. Previously, the research has been characterised by: • •
•
a loose, implied methodology the absence of supporting evidence (i.e. critical supporting information) in the form of: • transcripts of interviews or meetings • other accounts of meetings • records of other empiric observations instead, generalisations are made which are sexed-up (exaggerated) with quotable quotes (which are not cross-referenced to sources and could be entirely fictional)
176
Research Methods in Politics • • •
analysis and discussion are given without cross-reference to the information on which they rely evidence is not compelling: acceptance is based almost entirely on trust: no opportunity or means are given to corroborate the findings much statistical analysis assumes an in-depth knowledge of the mathematical routines, formulae and symbols used, and may be impenetrable to most UK students; the rationale
• • •
and meanings of the techniques adopted go without comment, explanation or indication of confidence limits quantitative and qualitative methods of analysis have been seen as entirely different and lacking any common process research findings are rarely persuasive to those readers not otherwise subscribing to the implicit assumptions adopted consequently, much Politics research tends to confirm or qualify existing explanatory theory in an ever-decreasing circle
You will be familiar with the axiom: ‘rubbish in: rubbish out’. Five guidelines for effective analysis are suggested: 1. The data never speaks for itself: ‘all knowledge is theoretically impregnated’ (Silverman, 2001: 23)1 : your analysis and interpretation are always required. 2. Similarly, you must make the process of analysis transparent if the critical reader is to share your findings: trust and belief in your scholarship and honesty are not enough. 3. Ensure that the analysis and interpretation are proportionate in depth and extent to the information to which they are applied. 4. The means of analysis you use must be relevant and appropriate to the research question and to the information collected; you must justify using them. 5. You must identify, consider and justifiably discount (discard) other explanations before you claim firm conclusions.
You may have gathered the data in quantitative (numeric) or qualitative (nonnumeric) form. However, the former distinction between quantitative and qualitative analysis has been overtaken by technological development. Whilst numeric data can only be analysed by quantitative means, you can now analyse qualitative data by qualitative or quantitative methods.
The process of analysis What do we mean by analysis? What does it involve? Oddly enough, the authorities are largely silent – presumably, on the assumption that, after all, we all know what
Analysing Research Data: The Process
analysis means. However, the following characteristics of analysis can be inferred from the discussion they provide: •
analysis is an objective-seeking, rational process
•
in academic research, the objective is to answer the research question either by confirming or infirming the explanations set out in the hypothesis, or, in inductive research,
• • • • • • •
the relationship, if any, between the key variables analysis essentially involves deconstruction – breaking down the whole into its constituent parts analysis involves a systematic, deliberate, planned examination of these parts, and of the relationship between them analysis invariably involves some comparison between variables analysis seeks to identify patterns or rhythms of relationships analysis also involves creative interpretation testing is used to eliminate any spurious relationships the analysis concludes by inferences being drawn. These are generalisations about the research population drawn from samples or other limited information. For example, if your study of a sample of South American states has shown a causal, contributory relationship between, say, levels of interpersonal trust and economic performance, then how likely are you to find this same relationship in other countries? These inferences must be followed by discussion before firm conclusions can be drawn.
The greatly increased interest in analysis has spawned a very good literature, including Miles and Huberman (1994), and Ritchie and Lewis (2003).2,3 However, this emerging literature tends to concentrate exclusively on either qualitative or quantitative data analysis and implies that they adopt divergent approaches. Exceptionally, Neuman (2003) provides a valuable review of similarities and differences which have been tabulated in Table 12.1.
A hierarchy of analysis Despite the differences, it is possible and useful to suggest a general, generic model of analysis (which draws in part from Ritchie and Lewis Analytical Hierarchy, 2003: 212, Box 8.1). The model has five, iterative, component phases: raw data assembly, validation and reduction; classification, coding and sorting; testing; inference-drawing; and theory development. The process can be illustrated as a pyramid (after Maslow)4 which reflects the successive reduction and refinement of the volume of raw data into information and evidence.
177
178
Research Methods in Politics Table 12.1 Qualitative and quantitative analysis compared Similarities between Qualitative and Quantitative Analysis 1 Both involve making inferences 2 Both methods of analysis involve ‘a public [transparent] method or process’ 3 Comparison is central to all analysis 4 Researchers ‘strive to avoid errors, false conclusions, and misleading inferences’ Differences between Qualitative and Quantitative Analysis
Quantitative Analysis
Qualitative Analysis
Uses standardised, statistical techniques Delays analysis until all data collected Tends to test hypotheses Low level of abstraction
Less standardised Begins earlier before all data collected Can create new concepts and theories High level of abstraction
Source: (abstracted from Neuman, 2003: 439–40)
Δ Theory development Inference drawing from evidence
Testing
Information classification, coding and sorting
Raw data assembly, validation and reduction
Figure 12.1 Hierarchy of analysis
Raw data assembly, validation and reduction This first phase involves you bringing all the raw data together at the same place and time. The objective here is to reduce the mass and mess of data in sheets of text, photocopies, emails, scoping documents, transcripts, letters, memos, questionnaires, etc., audio and video-records and photographs. You should selectively discard all the non-relevant and duplicate data to a ‘recycle bin’, transposing the relevant data into a common format (e.g. Microsoft Office) and organising it into folders and files. The files are then validated. This involves checking the data for errors
Analysing Research Data: The Process
or omissions. It may also involve you anonomysing people and places (hiding their real identities). You should also identify and flag (mark) outlier data. Outliers are extreme and exceptional data which lie outside the expected range. For example, where a new leader has shown an increase in support of, say, 5–15% in most opinion polls, then you might classify an increase in one poll of, say, 25% as an outlier. But, you should check first whether it is exceptional because of some recording or transposition error, or may represent a significant finding. Where outliers are considered dubious, then you can discard them. By the end of this initial process, you will have reduced and organised the mass of raw data into useful information ready for coding. When this first stage is complete, you will be able to begin the processes of classification, coding, testing and inference drawing. These processes are very different for quantitative and qualitative information and are therefore described separately in the following chapters.
Questions for discussion or assignments
1. What do you understand by ‘analysis’? 2. The suggested hierarchy of analysis seeks to reduce (raw) data to (reliable) information, and this information to evidence. An operational distinction between these is suggested in Chapter 7:
Commentators and politicians tend to use the word ‘evidence’ to describe what they would wish us to regard as ‘conclusive, compelling information’ which either proves or, in its absence, disproves allegation. But is there a real difference between data, information and evidence? Certainly, the dictionary meanings are similar. But some distinction is useful. Researchers tend to speak of data as the mass of disordered, raw material from which information (knowledge) is abstracted to provide evidence to support argument and conclusions. Information informs. Evidence supports conclusions. So it is helpful to conceive of research of involving three stages. First, the raw data is gathered. Second, the data is organised and distilled into information. Third, evidence is abstracted for or by analysis from the information.
Is the suggested distinction between data, information and evidence useful, exaggerated or dangerous in the context of analysis?
179
180
Research Methods in Politics FURTHER READING Spencer, L., Ritchie, J. and O’Connor, W. (2003) Chapter 8: Analysis: Practices, Principles and Processes. In Ritchie, J. and Lewis, J. (eds.) Qualitative Research Practice: A Guide for Social Science Students and Researchers. London: Sage. pp. 199–218. Whilst this textbook is written for a readership of social scientists undertaking qualitative research, this chapter provides a very succinct account of the principles underlying analysis which is of value to Politics researchers pursuing quantitative, qualitative or mixed methods.
Notes 1 Silverman, D. (2001) Interpreting Qualitative Data: Methods of Analysing Talk, Text and Interaction. London: Sage. 2 Miles, M.B. and Huberman, A.M. (1994) Qualitative Analysis: An Expanded Sourcebook. London: Sage. 3 Ritchie, J. and Lewis, J. (2003) Qualitative Research Practice: A Guide for Social Science Students and Researchers. London: Sage. 4 Abraham Maslow’s Hierarchy of Needs (1943), see Maslow, A.H. (1970) Motivation and Personality. New York: Harper and Row.
Part IV A: Quantitative Analysis
Chapter 13
Calculating and Interpreting Descriptive Statistics
‘Lies, damned lies and statistics’, attributed to Benjamin Disraeli by Mark Twain. Teaching and learning objectives:
1. To understand what is meant by descriptive statistics. 2. To learn the separate functions of the mean, mode and median, range, interquartile range, variance and standard deviation. 3. To learn how to calculate these using MS Excel.
Introduction Descriptive statistics resolve complexity by summarising and compressing data to identify their essential characteristics to create a brief but relatively accurate impression to the observer. You already do this in everyday use by describing the total number of, say, your fellow students’ range of ages, their average age, male/female split and ethnic composition. Consider this example. In reply to a question from your parents, you tell them that your university class consists of about 20 students, half are women and six come from the Indian sub-continent. You add that most students are about the same age. (Twenty years old but there are three older students.) The oldest is 48. This creates a reasonable picture in their minds. You use this rough description or numeric shorthand to avoid having to describe each student individually. However, more accurate statistics may be required – by say, by the registrar’s staff – giving the precise average age. The ages of the class are (in ascending order): 18,18,18,18,19,19,19,19,19,19,19,19,19,19,19,19,20,20,20,36,45,48. There are actually 22 students. To calculate the average, you must, of course, add all the ages together and divide the sum by the number of students. In this
Research Methods in Politics
case, the total ages are 489. There are 22 students. So the average is 489/22 = 22.23 years. So what? Well, in this example, none of the students is aged 22. So the average is misleading. The reason for this apparent discrepancy is because the class contains much older students. This is more obvious when the students and their ages are individually plotted on a type of graph or chart termed a scattergram: In this simple example, the average has been distorted by the inclusion of three much older students. But, suppose that one of the 18-year-old students were to ‘drop out’ and an 85 year-old admitted. In this case, then the average mean would increase to 25.3 and the graph to: 60 50 40 Ages
30 20 10 0 0
10
20
30
Individual students Figure 13.1a Scattergram of students and their ages
90 80 70 60 Ages
184
50 40 30 20 10 0 0
10
20
Individual students Figure 13.1b Scattergram of students and their ages (revised class)
30
Calculating and Interpreting Descriptive Statistics
In this second case, the calculation of the average has been grossly distorted by the replacement of a more typical student by a very much older one. This is called the outlier effect – the effect of those highest or lowest terms to skew – distort – your everyday, ‘average’ mathematical description of complex information. (Incidentally, some quantitative researchers argue that qualitative researchers rely on outliers; the exceptional case is more attention-grabbing than more typical information.) To overcome this weakness, statisticians use a set of simple statistical measures to better describe, say, a student class. These are the arithmetic mean, median, mode, range, variance and the standard deviation. These are termed measures of central tendency (or measures of dispersion). As the terms imply, they describe numerically the extent to which the individual terms cluster around the ‘centre’. The class is termed the population: the total group of people or events being described or under study. The population is represented by the Roman capital, N. The individual measurements of data from the population are called terms. A group of terms which measure the same characteristic at the same moment in time is called a series. The arithmetic mean is what is generally called the average, i.e. the sum of the terms divided by the number of terms. In statistics, the terms are called X and the number of terms N . ‘Sum of’ is represented by the Greek symbol (pronounced sigma). The arithmetic mean is called X bar and represented nowadays by the symbol X . So X = X N The median is the middle term when the series is ranked (normally in ascending order), e.g. 4,10,2,8,6 becomes ranked into 2,4,6,8,10. The median is the (N+1)th term/2. So, the series 2,4,6,8,10 has five terms. Therefore the median is the (5+1)/2th term = 3rd term. The third term in the series is 6. If the series were reduced to four terms – 2,4,6,8 – then the median would be the (4+1)/2th term = 2.5. In this case, the median is calculated as lying midway between the second term (4) and the third term (6). So the median is 5. The mode is the most frequent term in the series, e.g. in the series 1,3,6,5,1,2 then the mode is 1. This is rarely used. Returning to the example of the initial class of 22 students, then, whilst the arithmetic mean may be 22.3, the median is the (22+1)/2th term, i.e. the 11.5th term or half-way between the 11th and 12th term. In this case, the 11th and 12th terms are both 19 so the median is 19. The mode (the most common term) is also 19. So, in this example, the median and the mode provide better descriptions than the mean (of 22.23). In the revised class, the median and mode remain 19 and the effect of the (85 year old) outlier is effectively discounted. In cases like these where most terms lie below the mean, then the data – or distribution – is termed positively skewed. In many universities, the final degree classification uses the median exam mark rather than average mark. In this way, exceptionally good or bad exam marks are discounted. Another statistical measure of the data is the range – the difference between the highest and lowest term. However, as noted earlier, the range can be distorted by
185
186
Research Methods in Politics
exceptionally high or low outliers, e.g. the very mature student. In the revised class, the range would be 67 (i.e. 85–18). So statisticians developed the interquartile range. This is the difference between the first and third quartiles when the terms of the series are placed in ranked order. So in a series of 99 terms, the interquartile range is the difference between the 25th and 75th term. The first and third quartiles are identified in a similar way to the median by calculating (N + 1) 3(N + 1) and 4 4 For example, in our initial class of 22 students ranked by age, then the interquartile range is the difference between the ages of the 5.75th and 17.25th students, i.e. 19 and 20 = 1. The greatest use of median measurements probably lies in representing unequal distributions especially in terms of resources. Income is a prime example where a small number of people may have vast wealth and, at the other end of the scale, a large number virtually nothing. In these cases, the inequality is shown by calculating and comparing the income of the 10th and 90th ‘percentiles’. So, in a fair society where incomes are equal, then the 10th and 90th percentiles will be the same whereas, in less equal countries, the ratio of 10th to 90th percentiles may be as high as 100. A more sophisticated descriptive statistic for measuring unequal distribution of income and wealth is provided by the Gini coefficient. This is the ratio between the areas above and below the Lorenz curve (of cumulative incomes) and the area of equal distribution. The coefficient varies between less than 0.25 (Greenland) and 0.6 (Namibia). No coefficients are calculable for Sub-Saharan Africa. The Gini coefficient for the UK is 0.35–0.39. In the UK, the official (upper) level of poverty is calculated as 60% of median household income. ‘Deep poverty’ is calculated as 40% of the median household income. The definitions reflect a rejection of the historic concept of poverty being calculable as an absolute level in favour of relative measures. Complete income equality
Lorenz curve for a typical income distribution Cumulative income share (%)
Cumulative income share (%)
0
0 0
A
100 Population share (%)
0
B
100 Population share (%)
Figure 13.2 Charts of income distribution (from www.statistics.gov.uk)
Calculating and Interpreting Descriptive Statistics
However, the median itself can be misleading. Take for example, data obtained from applying the Likert scale to the question: Q. To what extent do you agree with the following statement: The Anglo-American invasion of Iraq has actually increased the threat of terrorism in both countries. Do you strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree? You code the possible answers 1–5 (where 1 is strongly agree, etc.). Let us say that you ask three samples of ten people of three different age groups (in practice, you would use much greater samples). The results (ranked in descending order) are: Young people Middle-aged people Older people
5 4 4
5 4 3
5 3 3
4 3 3
3 3 3
2 2 3
1 2 3
1 1 3
1 1 2
In each of these three groups, the median is 3 (neither agree nor disagree) although the intensity of agreement or disagreement is significantly higher among the sample of younger people. To overcome this problem, mathematicians developed the variance.
The variance The variance is a descriptive statistic which measures the degree of concentration or dispersal of the terms around the mean. This is found by calculating the difference between each term and the mean, i.e. X − Xi . In some cases, the difference will be +, in others, –. (Indeed, if added together they should cancel each other out, leaving an answer of 0.) So, to overcome this difficulty, the differences are squared – thus eliminating the minus quantities. The variance is then found by adding all these squared differences and dividing by the number of terms. The formula for the variance is: (X − Xj )2 N
where j is the number of terms in the series
This sounds more complicated than it is. Consider the example above of the Likert scores for the sample of ten, younger people: Young people, X Arithmetic mean, X = Difference between term and mean (X −X ) = Difference 2 (X −X )2 =
5 3 (5−3) +2 4
5 3 (5−3) +2 4
5 3 (5−3) +2 4
4 3 (4−3) +1 1
3 3 (3−3) 0 0
2 3 (2−3) −1 1
1 3 (1−3) −2 4
1 3 (1−3) −2 4
1 3 (1−3) −2 4
187
188
Research Methods in Politics
Sum of difference2 (X − X )2 = 26 Variance
(X − Xj )2 26 = = 2.6 N 10
Standard deviation A further development of the variance is the standard deviation, SD or σ which is simply the square root of the variance. So the formula is: (X − Xj )2 where the value of σ is ± N
N and n You will note that, in most scientific research publications, it is the number of terms and standard deviation which is normally given to describe experimental data, e.g. n = 1600, σ n−1 = ±12.1. However, the symbol n rather than N is given. Why? The answer is that the symbol N is used to denote the number of terms – people or events, etc – in the population: n is used to denote the number of terms in the sample. Similarly individual terms in the sample are denoted by x rather than X . The symbol for the standard deviation of the sample is also written in lower case as σ n−1 . But why is the n − 1 notation added? The reason is that, where the series is a sample of the population, then the number of terms may be comparatively small. To compensate for the small number of terms and the potential experimental error, the number of terms is notionally reduced by subtracting 1 to give a larger standard deviation. To distinguish a sample from a population, the symbols x, n and σ n−1 are used. So the formula for the standard deviation of a sample of n terms is: (x − xj )2 (n − 1)
Grouped frequency distribution Data – whether populations or samples – may contain a large number of terms. To simplify their representation and analysis, the terms are translated into what is called a grouped frequency distribution. For example, a small sample of eight
Calculating and Interpreting Descriptive Statistics
families may show the following numbers of children in each: 2,1,2,3,2,1,0,4, i.e. a total of 15 children. These can then be tabulated in a grouped distribution as: Table 13.1 Grouped frequency distribution table Number of Children in Each Family
Number of Families
Total Children
x 0 1 2 3
f 1 2 3 1 Total families, n = 8
fx 0 2 6 3 Total children = 15
The measure in which data is grouped is called an interval. In the example above, the interval used is the number of children. The most commonly used age intervals in Politics research are years, i.e. 1998, 1999, etc., age groups, e.g. 0–17, 18–28, 29–38, 39–48, 49–58, 59 +. , and incomes, e.g. $0– $999, $1000 – $1999, etc. Grouping data by frequency distribution also enables the data to be displayed as a graph, e.g.
Number of families (Frequency)
3.5 3 2.5 2 1.5 1 0.5 0 0
1
2
3
4
5
Number of children Figure 13.3 Chart of Grouped frequency distribution table: Frequency of children in families
In nature, charts of grouped frequency distributions of most phenomena (e.g. weight of adults, their heights, intelligence, shoe sizes, etc.) form a normal distribution. Examine Fig 13.4. In this case, the mean (average) number of clubs etc. to which members of the sample (of 1,000) were members is 12. Almost 100 of the sample belonged to 12 clubs whilst less than 10 belonged to only one club. Note how the normal distribution (or Gaussian distribution) is characterised by a symmetrical, bell-shaped curve in which the mean, median and mode coincide at the most frequent term. The normal distribution is also asymptotic, i.e. the frequencies approach but never reach zero.
189
Research Methods in Politics Frequency (memberships)
190
100 80 60 40 20 0 0
10
20
30
Number of clubs joined Figure 13.4 A normal distribution: Membership of clubs
How to calculate descriptive statistics Most descriptive statistics can now be calculated using a good calculator, for example, the Casio fx-83 series (which can be bought for as little as £6.99). However, most students are more likely to have a PC or laptop readily available than a calculator. Unlike a calculator, a computer can save the data and calculations as a file and enable them to be pasted (copied) into a report. Whilst a wide variety of statistical software is available, most students use the MS Office software package which includes the spreadsheet Excel. Most students are also familiar with using Excel which is quite adequate to perform descriptive statistics. Before using Excel to perform statistical calculations, it is necessary to add the Analysis ToolPak – which contains a large number of specialist statistical formulae – bundled into the software. This is relatively straightforward. Open an Excel spreadsheet. On the menu bar, click on Tools. Then click on Add-Ins. Click on Analysis ToolPak and Analysis ToolPak VBA at the head of the dialogue box and click OK. You can now enter your data. Insert the term Income in cell A1. Then insert the following data in cells A2:A11 in Column A. This is a small sample from a northern benefits office of the annual household income of current single-parent claimants having three children for income support. In practice, of course, we would seek to use a much larger sample of 35 terms or more for greater accuracy. Claimants’ household income: Cell A1: Income £ Cell A2: 12,317 Cell A3: 14,861 Cell A4: 8,003 Cell A5: 11,370 Cell A6: 12,561
Calculating and Interpreting Descriptive Statistics Cell A7: 13,889 CellA8: 15,270 Cell A9: 18,241 Cell A10: 25,279 Cell A11: 19,671
Save the data as a file titled Claimants’ Household Income. We want to calculate the mean, median, tenth and ninetieth percentile, range, variance and standard deviation. The first step is to copy the data onto a nearby column, E using the Copy/Paste routine. This safeguards your original data. Your next step is to critically examine the data. This is best done by sorting the data into ascending order and by drawing a chart (graph). To do this, first highlight the data (including the title ‘income’), then click on Data and Sort. In the sort dialogue box, click OK. The data will now appear in ascending order. Leave it highlighted. Then click on Insert and Chart. Click on X–Y (Scattergram). Click on Next until the graph appears. 30,000
Income
25,000 20,000 15,000 10,000 5,000 0 0
2
4
6
8
10
12
Benefit claimants Figure 13.5 Chart of distribution of claimant’s household income
This graph shows that the distribution of incomes is relatively straightforward and that there are no ‘outliers’ or other unexceptional terms. To calculate the descriptive statistics, click on Tools, then Data Analysis, then click on Descriptive Statistics in the dialogue box and OK. The Descriptive Statistics menu will appear. In the box marked Input Range, insert E2 : E11. (Whilst there is a short-cut alternative to inserting the cell numbers, it can complicate routines at this initial stage.) Click on Grouped by columns. The Output Range is the cell where you want the statistics to appear: insert G1. Then place a tick in the box marked Summary Statistics. Finally, click OK. The table of statistics below will appear: By reading this table and rounding up to one place of decimals, we can report that: mean income median income
= £15,146.2 = £14,375
191
192
Research Methods in Politics Table 13.2 Descriptive statistics output of claimants’ household income data Income £
Column1
8,003 11,370 12,317 12,561 13,889 14,861 15,270 18,241 19,671 25,279
standard deviation variance range
Mean Standard error Median Mode Standard deviation Sample variance Kurtosis Skewness Range Minimum Maximum Sum Count
15146.2 1543.355 14375 #N/A 4880.518 23819453 1.035046 0.839166 17276 8003 25279 151462 10
= ± £4,880.5 = 23819453 = £17,246
In this case, the summary statistic for the standard deviation calculated is the standard deviation of the sample. If the data were a population, then you would have to use a different formula =STDEVP( ). Click on the cell where you want the standard deviation of the population to appear. Insert =STDEVP(A2:A11) on the task bar next to the fx notation. Press the Enter key. The answer 4630.066 will appear. To record the meaning of this data you will have to enter standard deviation of the population in the cell above or to the right. (If you use a cell to the left then the title will be partially obscured unless you increase the column width.) Note that the standard deviation of the population is smaller than the standard deviation of the sample. To calculate, the 10th and 90th percentile, click on Insert and then Function. The Insert Function box will appear. In the first box headed Search for a function, insert percentile and then click on Go. PERCENTILE will appear highlighted blue on the larger box below. Click on OK. A box marked Function Arguments will appear. In Array, insert E2 : E11. In k, insert 0.1 (meaning 10%). As soon as you insert this data, the answer will appear to the right of Formula result = 11,033. Change the k value to 0.9 (i.e. 90%) and the formula result will be 20,232. So we can report that: 10th Percentile = £11,033 90th Percentile = £20,232
Calculating and Interpreting Descriptive Statistics
You will appreciate that it makes no difference to the practical ease of completing the calculation whether the spreadsheet contains – as in our example – 10, or 10,000 terms.
Types of numbers In the example above, numbers have been used to represent incomes. However, there are two different groups and four different types of number that you can use. The two groups of numbers are categorical and cardinal.
Categorical numbers Categorical numbers involve nominal numbers and ordinal numbers (rankings). Your university number is an example of a nominal number. Your university number gives you a unique identity and enables your name to be replaced by a number. If your university number is lower than a friend’s then it does not mean that you are better or worse in any respect. Nominal numbers have no mathematical significance: they are entirely nominal. In contrast, ordinal numbers are used to indicate rankings. But their significance is confined to some ranking relative to a list of criteria, e.g. 1 may indicate the person with the highest mark.
Cardinal numbers Cardinal numbers involve interval and ratio scales. Ratio scales have an absolute zero, e.g. income, time, degrees Kelvin (degrees centigrade of Fahrenheit are interval scales). Interval scales measure the variance above or below a particular benchmark. Ratio and interval scales can be negative. For example, an annual income of −£500 means that you owe someone else £500. Accountants express this as (£500). Wherever possible, quantitative researchers seek ratio scales. Where variables are expressed as categorical or cardinal numbers they can be termed categorical variables or cardinal variables. Variables may also be discrete or continuous. Discrete variables are those where there is only a limited number of intervals in the range. For example, the numbers of children in a household are discrete because you can only have zero, one, two, etc children: you cannot have 1.4 children (although that may be the arithmetic mean of children in all households). In contrast, continuous variables are those where there are an infinite number of intervals in the range. So, for example, the ages of the children in a household are continuous: even twins will have different ages when measured in terms of minutes or seconds. Generally speaking, categorical variables are discrete whilst continuous variables are cardinal.
193
194
Research Methods in Politics
Whilst the distinctions between categorical and cardinal numbers and discrete and continuous variables may appear academic, they are important. For example, the descriptive statistics in the example above can only be calculated using cardinal numbers. As you will see in Chapter 15, the type of number will dictate which advanced statistical formulae can be used.
Questions for discussion, workshop tasks or assignments
1. Access a UN, World Bank or EU data bank on the Internet, e.g. http://unstats.un. org/unsd/cdb/cdb or http://mdgs.un.org/unsd/unsd/ Choose a table of interest to you. Copy it to Excel. Calculate the summary descriptive statistics, 10th and 90th percentile. Compare the mean and median and explain the difference. 2. Create your own file of 1,000 random numbers.To do this click in turn on Tools in the menu bar then Random Number Generation. In the Random Number Generation Dialogue box, enter: Number of Variables (columns) Number of random numbers (rows)
= 10 = 100
Distribution Mean Standard deviation Random number seed Output Range Then click ‘OK’
= normal = 50 = 15 =5 = A1
Save as (file) Normal distribution random numbers. Then calculate the summary statistics for the entire data set. Retain the file for future use as your own bank of random numbers. 3. Now analyse the random numbers above into a grouped frequency distribution. First, decide what class interval you wish to adopt. Your data extends from 0 to 100.You may decide to use a class interval of 5.Then enter 5 in cell K1 and continue to enter 10, 15, 20 etc until you reach 100 in cell K20. (You can, of course, use the Autofil routine.) Then click on Tools and Data Analysis. In the Data Analysis dialogue box, click on Histogram and OK. In the Histogram dialogue box, insert Input Range Bin Range Output Range
A1:J100 K1:K20 L1:L20
Then click on Chart Output and click OK.
Calculating and Interpreting Descriptive Statistics The output and chart will appear:
Table 13.3 Output and chart 5
Bin
Frequency
10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
12 12 20 32 42 48 76 78 79 101 92 89 83 77 53 41 32 12 9 7
120
Frequency
100 80 60 40 20
Bin Figure 13.6 Histogram
95
80
65
50
35
20
5
0
195
196
Research Methods in Politics You will note that the histogram apparently takes the shape of a uniform distribution. However, as we constructed our 1,000 cell bank of random numbers from a normal distribution then the chart is reassuring rather than surprising. But note how, despite having been calculated from 1,000 random numbers, the histogram does not follow a perfect uniform distribution: there will always be some variations from the perfect normal distribution when bins are used.
FURTHER READING Clegg, F. (2005) Simple Statistics: A Course Book for the Social Sciences. Cambridge: Cambridge University Press, pp. 1–28. This is an excellent, readily readable, authoritative account for students new to or lacking confidence in statistics. Burnham, P., Gilland, K., Grant, W., and Layton-Henry, Z. (2004) Chapter 5: Descriptive Statistics. In Research Methods in Politics. Basingstoke: Macmillan. pp. 114–42. This account provides good advice on the ‘charting’ of descriptive statistics using examples from Politics. Pennings, P., Keman, H., and Kelinnijhuis, J. (2006) Chapter 5: Explorative and descriptive statistics. In Research Methods in Politics. London: Sage. pp. 88–131. This is an advanced text better suited to experienced researchers wishing to develop their mathematical skills.
Chapter 14
Using and Understanding Inferential Statistics
Teaching and learning objectives:
1. To understand what is meant by inferential statistics. 2. To learn how the normal distribution enables generalisations to be drawn from the descriptive statistics of representative samples. 3. To learn how to calculate the confidence limits which can be attached to opinion polls. 4. To understand the meaning of the null hypothesis and how it can be applied to sample data.
Introduction Consider the example of a survey of a random sample of 100 Labour party members which reveals an average weekly net disposable income of £84.12 with a standard deviation of £21.07. Twenty-six of the sample (i.e. 26%) regularly read The Guardian newspaper. In terms of other (non-print) news media, 72% rely mainly on BBC TV and radio services for national and international news. Given this information from a small sample of people, then how can you calculate the extent to which it is shared by the general population? Furthermore, how can you tell whether or not your sample is representative in the first place? The answer is that you can use inferential statistics. These are: a branch of applied mathematics or statistics based on a random sample. They let the researcher make precise statements of the levels of confidence they have in the results of a sample being equal to the population parameter (where a parameter is a characteristic of the entire population drawn from a sample). (Neuman, 2003: 539, 541)
Research Methods in Politics
The solution to drawing inferences from sample data lies in the attributes of the normal distribution and the central limit theorem.
The normal distribution In the previous chapter, you learned how to describe a series of data by the number of terms, N , the arithmetic mean, X and the standard deviation, σ . The standard deviation probably appeared contrived and of no real practical use. However, consider the normal distribution in Figure 14.1 An important characteristic of the normal distribution is that 95% of the terms will lie within two standard deviations (actually 1.96 SD) either side of the mean; conversely, only 5% will lie outside two standard deviations. Of the terms, 99% will lie within 2.97 SD. These characteristics can be expressed as probabilities. A probability is the likelihood that an event will occur. For example, if an event is certain – like death albeit in the distant future – then the probability is 100% or 1.00. Alternatively, a remote possibility – like winning the UK national lottery next week – may be 0.001% or 0.00001. A one-in-four likelihood would be expressed as a probability of 25% or 0.25. A bookmaker would give odds of 4–1 on an event having a 25% probability. So, returning to the normal distribution, there is
100
80
Number of men
198
60
40
20
0
0
50 55 60 65 70 75 80 85 90 95 100 105 110 115 x ± 1 SD x ± 2 SD x ± 3 SD Diastolic blood pressure (mmHg)
Figure 14.1 The normal distribution
Using and Understanding Inferential Statistics
a probability that of 95% or 0.95 any individual term will lie within 1.96 standard deviations of the mean. This means that it is likely that 95 out of every 100 terms will lie within 1.96 standard deviations of the mean. We can increase the likelihood of any term lying in the distribution to 99% by increasing the span to 2.97 standard deviations. There is therefore a trade-off between probability and accuracy.
Central limit theorem The means of samples randomly taken from a population will always differ. This difference is termed the sampling error. The central limit theorem (where a theorem is a mathematical deduction accepted as a truth) effectively proposes that the arithmetic means of large samples (greater than 30 terms) will approach a normal distribution – even where the population is not normally distributed. Therefore, by referring to the distribution of terms in the normal distribution curve above, we can say that 95% of all the sample means lies within 1.96 standard deviations of the ‘mean of the means’. (For samples of less than thirty terms, the sample means will follow the Student’s T distribution. This is symmetrical and looks very similar to the normal distribution. However, the spread of the distribution reflects both the standard deviation and the sample size. The accuracy of inferences from the T distribution is therefore lower than from a normal distribution and decreases as the sample size falls.)
Standard error of the sample mean The standard error of the sample mean, SE, provides the statistical method of generalising from samples to populations. It estimates the likely variance and therefore accuracy between the sample mean and population mean. Its formula is: sd SE = √ n where sd is the standard deviation of the sample and n the number of terms in the sample (sample size) The formula which brings together the characteristics of the normal distribution, central limit theory and standard error of the sample mean is that, at 95% probability, the population mean X , lies within 1.96 standard errors SE, of the sample mean x, i.e: X = x ± 1.96SE
199
200
Research Methods in Politics
Consider again the initial example of a random sample of 100 Labour party members that reveals an average weekly net disposable income of £84.12 with a standard deviation of £21.07. So we can say that: X = 84.12 Sd = 21.0 n = 100 √ √ The standard error, SE = sd/ n = 21.07/ 100 = 21.07/± 10 = ± 2.107 You can calculate the income of the population of all Labour party members at a probability of 95% by substituting in the formula: = x¯ ± 1.96SEP X X = 84.12 ± (1.96)(2.107) = 84.12 ± 4.12972 = 79.9902 to 88.24972 Alternatively, you can complete the calculation using Microsoft Excel. Open an Excel spreadsheet. Then click on Insert and fx function. In the Insert Function dialogue box, select the category statistical and scroll down the functions until you reach CONFIDENCE. Click on OK. The Functions Argument dialogue box will appear asking you to insert values for Alpha, Standard deviation and Size. Alpha is the significance level for which 95% is the default for Politics. For a significance level of 95%, the Alpha is 0.05. The standard deviation is £21.07. The (sample) size is 100. Insert these figures. The answer (formula result) of 4.129 will appear automatically at the foot of the dialogue box. Click on OK and the answer will be entered in cell A1 (or any other cell that you’ve highlighted in advance). What the manual or Excel calculations mean is that you can infer that 95 of every 100 members of the Labour party overall will have a net weekly disposable income of between £79.99 and £88.25. These two figures are termed the upper and lower confidence limits. The variance – difference – between these two figures is termed the confidence interval or margin of error. The margin of error can either be expressed as the difference between the confidence limits or their variance from the sample mean. So, in the example above, we could report that the margin of error was either 8.26 or ± 4.13. Note how the numbers calculated have been ‘rounded up’ to two places of decimals only in the final answer: this avoids accumulated errors which can occur where initial calculations are rounded up. If you wanted to learn the likely spread of income of 99% of all party members, then we would use the figure of 2.97SEs or an Alpha of 0.01. The result will be a confidence interval of ±5.42 This initial example of sample data also showed that, of our random sample of 100 Labour Party members, twenty-six of the sample (i.e. 26%) regularly read The Guardian newspaper. In terms of other (non-print) news media, 72% rely mainly on BBC TV and radio services for national international news. How can you calculate the confidence limits of likely Guardian readership and BBC news of other party members from the sample?
Using and Understanding Inferential Statistics
In this case, you can use the standard error of the proportion, SEp where SEp =
√ {p(1 − p)/n}
Where p is the standardised proportion of the sample sharing the characteristic. So the value p for 26% is 0.26; 100% would be 1.0. To calculate the population proportion P, you can use a formula similar to that using the standard error. In this case, at the 95% confidence level: P = p ± 1.96SEp In our initial sample (% of Guardian readers): p = 0.26 (i.e. 26% expressed as a proportion of one) n = 100 Therefore, the standard error of the proportion: √ SEp = √{p(1 − p)/n} √ = {0.26(1 − 0.26)/100} = {0.26(0.74)/100} = 0.043863244 (do not round up!) Therefore, substituting in the equation: P = p ± 1.96SEp P = 0.26 ± (1.96)(0.043863244) = 0.26 ± 0.0859723118 P = 0.2161365756 to 0.348597231182 Now, you can round up to say that P = 0.216 to 0.349 or, in percentage terms, you can say that, at 95% confidence limits, between 21.6 and 34.9% of all Labour party members will read The Guardian. Similarly, you can calculate that, if 72% of the sample relied on BBC news services, then the percentage of the national members who also rely on the BBC at 95% confidence limits would lie between 63.2% and 80.8%. In this case, the confidence interval is 17.6%. This confidence limit is obviously too large – because of the small sample size – for useful inferences to be drawn. Greater predictive accuracy requires much larger samples.
National opinion polls Opinion pollsters – NOP, Gallup, etc – generally use a random sample of 1600. Consider, therefore, the example of opinion polls which show that, by February,
201
202
Research Methods in Politics
support for the government is slowly rising, whilst support for the main opposition party is falling. The very latest poll (of 1600 electors) shows that 39% of the electorate say that they will vote for the government ‘if an election were held tomorrow’ and 33% for the main opposition. This is the highest level of support given for the government for two years. Should the prime minister risk an election in May, or wait another year? What is the margin of error? In this case, the standard error of the proportion, SE p is 0.01219. There would be a 95% probability that support for the government could lie between 36.6 and 41.4%. Conversely, opposition support could lie between 30.7% and 35.3%. At the 99% probability level, support for the opposition could conceivably be as great as 36.6% and therefore greater than the lowest level of support for the government at 35.5%. So, despite a 6% lead over the opposition and a national random sample of 1600, the government could lose the election. The margins of error are therefore much larger than newspapers – who commission polls as a low-cost source of ‘news’ – admit. This is important because of the potential effect of opinion polls in manipulating attitudes and discouraging people from voting where they believe that the outcome is a ‘foregone conclusion’, e.g. UK general elections in 2001 and 2005 (Heath and Taylor, 1999: 180).1 However, where the proportion of the sample sharing the same characteristic rises to nearly 100% then the margin of error falls considerably. So, if 90% of the sample of 1600 support the government’s policy on, say, ‘tougher sentences’, then, at the 95% confidence level, the margin of error falls to 1.5% ‘either way’, i.e. between 88.5 and 91.5%. The null hypothesis The null hypothesis is effectively the practical application of the test of falsifiability or fallibalism added by Popper (see Chapter 3). It is the assumption – similar to the assumption of innocence in UK and US law – that the data is untrustworthy until shown conclusively otherwise. It is generally explained as the concept of ‘no difference.’ Its general application is to the control and experimental samples. Any difference in the experimental group must first be attributed entirely to chance. It can only attributed to the effect of some stimulus (cause) when chance has been eliminated. The null hypothesis is termed H0 and the alternative hypothesis – our hunch – is termed H1 . H0 must be disproved – shown to be false – before H1 can be accepted. In the case of samples and populations where only one variable is being studied – so-called univariate statistics – the null hypothesis is that the sample has not been drawn randomly from the population and is therefore not representative. Is the sample representative? To accept a sample as representative and therefore dismiss the null hypothesis, two questions must be satisfactorily answered. First, is the sample genuinely drawn from
Using and Understanding Inferential Statistics
and statistically representative of the population? And, second, could the findings from the sample be entirely by chance? To answer the first question, you must verify that the sample is really drawn from the population. In your initial example, the population was the national membership of the Labour party. To be statistically reliable, the sample must be randomly drawn from that population. So, if the sample were gained from only one region or constituency of the UK, it would not be statistically representative. A very small sample would be unlikely to be representative. You can also check the sample against known characteristics of the population – e.g. age or sex. So, if the male/female ratio of the population is, say, 60:40 then you would expect that of the sample to be similar. You can test this mathematically. The average age of all party members is 59 (source: Labour party, London, January 2007) whereas the average age of the sample is 57 with a standard deviation of, say, 3. You can use the formula: X = x ± 1.96SEp Therefore, in your example: √ X = 59 ± 1.96(3/ 100) = 59 ± 8.88 = 50.12 or 67.88 As you know that X is 59 and therefore between the upper and lower levels predicted above, you can say that you are 95% confident that the sample could indeed have been drawn from the population. You can also investigate the significance (the likelihood that any similarity between sample and population means occurred entirely by chance) of the difference between a population mean X , and the sample mean x, by using the t-test for a population mean. Where the variance of the population is unknown, then the formula to calculate the test statistic, t is √ t = (x − X)/(sd/ n) In the case of the ages of the sample and population of the Labour party X = 59 X = 57 sd = 3 n = 100 You will calculate that t = −6.67. By referring to the t-distribution tables published in most statistical tables, you can see that, for degrees of freedom, df of 99 (i.e. n − 1) and
203
204
Research Methods in Politics
a probability level of 0.05 (i.e. 95% confidence levels), the critical value of t is ±1.65. As your calculated t statistic is greater than the critical value, then you can conclude with 95% confidence that the data from the sample has not arisen by chance. The null hypothesis can therefore be rejected and your alternative hypothesis – that the sample is statistically representative of the population – accepted.
Questions for discussion or assignments
1. Work your way through each of the worked examples shown above at your own pace to test and improve your understanding of the procedures. 2. Discuss what is meant by confidence level, confidence interval and confidence limits. Where and why should they be used? 3. Develop your own table for calculating the confidence limits of opinion polls. In column A enter sizes of sample from 10, 20, 30, …100, 200, 300, … 3,000, 4,000, 5,000, 6,000, … 10,000, 15,000, 20,000 … 100,000.You can use the Autofil routine to simplify this process. In row 1, enter the percentages of the sample sharing the same characteristic or attitude from 01, 02, 03 ,04, 05,… 99%. Write your own formula for calculating the confidence limits at 95% confidence levels, i.e. values √ of 1.96( p(p − 1)/n. Copy your formula across the spreadsheet and save.
FURTHER READING Clegg, F. (2005) Simple Statistics: A Course Book for the Social Sciences. Cambridge: Cambridge University Press. pp. 31–124. This is an excellent text for researchers who lack confidence or experience of statistical methods. The text is supported by operation schedules which demonstrate the use of the statistical techniques. A major strength is that the demonstration calculations are undertaken manually rather than computed elsewhere. Levine, D. and Stephan, D. (2005) Statistics: Even You Can Learn. London: Pearson Prentice Hall. pp. 103–23. This is a very readable account which includes step-by-step instructions for using Microsoft Excel to complete calculations and downloadable practice files. De Vaus, D. (2002) Analysing Social Science Data: 50 Key Problems in Data Analysis. London: Sage. pp. 147–234. This is an advanced text which uses SPSS for calculations. Its discussion of the principles involved in sample sizes, significance and confidence intervals is good.
Using and Understanding Inferential Statistics Kanji, G. K. (1999) 100 Statistical Tests. London: Sage. p. 27 t -test for a population mean (variance unknown). This is another advanced text. It shows clearly which tests should be carried out and includes simple worked examples and statistics table.
Note 1 Heath, A., and Taylor, B. (1999) Chapter 9: New Sources of Abstention. In Evans, G. and Norris, P. (eds.) Critical Elections. London: Sage. pp. 164–80.
205
Chapter 15
Testing for Association
Teaching and learning objectives: 1. To consider the concept of association between variables. 2. To learn how to test for association using correlation analysis and applying tests of significance to the results. 3. To learn how association between variables can be expressed as equations. 4. To learn what is meant by regression and how to produce explanatory equations between two or more variables by applying linear regression analysis and multiple linear regression analysis.
Introduction In the previous chapter (Chapter 14), you considered how individual measures of characteristics of a sample could be generalised into statements about the population from which they were drawn. However, only one characteristic was measured for each member of the sample: whether they voted or belonged to a union, income and strengths of attitude, etc. These are called univariate statistics, i.e. one number. But a greater ambition of Politics research is to identify causes and effects – the independent variables and dependent variables. Because they involve two (paired) variables, they are referred to as bivariate statistics. Where two or more independent variables cause a particular outcome or effect they are termed multivariate statistics. By convention, the independent variable (cause) is termed x, and the dependent variable (effect or outcome), y. This relationship is expressed mathematically as: y = f(x) i.e. y is a function of x. The relationships between bivariate or multivariate variables may be linear (i.e. a straight-line relationship), exponential, logarithmic or curvilinear. For example, empiric research has shown that, given perfect choice between comparable locations, then demand is a function of the ratio of their perceived utility and the perceived
Testing for Association
costs – expressed as the inverse of travel time to the power 1.8. This is known as the transport equation which can be expressed as: Di = µ{i (u/C 1.8 ) i−n (u/C 1.8 )} What this means is that the likelihood of a decision-maker choosing to travel to location i rather than any other location is a function of its perceived benefits u, and perceived costs C, by comparison to the perceived benefits and costs of travelling to other locations. This equation belongs to a family of so-called gravity models. Politics research concentrates instead on linear relations between cause and effect. Consider the data in Table 15.1 that shows paired data for relative pay and union density (% of workers in a particular industry belonging to a trade union) in the ten highest paid and ten lowest paid industries and services. The issue of pay and union density has occupied politicians and Politics researchers – particularly labour historians – for some time. The subject was of particular importance in the 1970s when the Conservatives – especially under Mrs Thatcher – believed that the unions were to blame for UK national decline
Table 15.1 Pay and union density. MLH
Industry
485 262 101
Newspaper printing Mineral oil refining Coal mining Underground workers Air transport Electricity and gas Other printing Port and inland Water General chemicals Aerospace eng. Wholesale distribution Textiles Motor repairs and distribution Industrial materials Clothing Retail distribution Woollen worsted Educational services Catering Agriculture
707 601–2 489 706 271 383 812 XIII 894 831–2 441–9 820–1 414 872 884–8 1
Pay as % of all industry and services
Union density %
157.8 143.2 133.6 145.9 130.8 122.9 120.7 119.8 117.5 117.3 86.7 86.2 85.5 85.3 84.0 83.6 82.3 80.6 77.9 72.4
94 59 97 99 85 95 94 83 59 80 15 99 60 15 42 15 47 78 8 23
Source: Department of Employment, 1981: Table 54 where MLH is Minimum List Heading
207
Research Methods in Politics
because of their power to inflate wages under threat of industrial action (strikes) and their influence on Labour governments. On the other hand, the trade unions argued that they had been responsible for securing real improvements to living conditions for workers and their families through ‘free collective bargaining’ and responsible partnership with Labour governments. (Few argued that increases in pay caused union membership to rise.) You can chart the data in Excel as an X–Y scatter graph: Pay as % of all other industries and services
208
180 160 140 120 100 80 60 40 20 0 0
50 100 Union density %
150
Figure 15.1 X–Y graph of Table 15.1, Union density and pay, 1981
You will see that, as union density rises, pay also appears to rise. The graph would appear to support the belief – shared by Left and Right – that higher union density can be associated (related to or connected with) with higher pay. However, note that there are some sectors where high union density is not associated with high pay – especially in textiles and education services. But generally the relationship could be represented as a straight line and, therefore, described as linear. This linear relationship might be expressed as an equation: Pay = po + (g) Union density Where po is (basic) pay levels where unionisation is nil (zero) and g is the gradient (steepness, slope) of the straight line.
Measuring association Apparent association can be measured by calculating the coefficient of correlation, r. This coefficient was first devised by Sir Francis Galton (1822–1911), a gentleman mathematician, brother-in-law of Charles Darwin. He was also the father of eugenics, a discipline founded on the scientific belief that the physical and mental abilities of
Testing for Association
society should be improved by selective breeding. Galton found that, when he compared the heights and weights, etc, of people, shorter people produced taller children and vice versa and underweight people produced heavier children etc. He described this tendency as regression – literally going back. Furthermore, the tendency of pairs of characteristics to regress could be measured by the coefficient of correlation and expressed as a linear regression equation. The statistical formulae used in regression analysis are complex. They essentially perform ANOVA: the analysis of variances between variables. Simply put, this involves calculating the means and seeing whether the variances of the paired variables show commonality. There are various statistical formulae to calculate the coefficient of correlation, r. Pearson’s r is the most widely used. Correlation is expressed on the scale between (+1.00) and (−1.00) where +1.00 indicates perfect positive correlation, 0.00 means no correlation and −1.00 indicates perfect, negative correlation. Their respective graphs are shown in Figures 15.2a and 15.2b. Figure 15.2c shows a distribution with a low or nil correlation. 12
12
10
10
8
8 Series1
6
6
4
4
2
2
0
Series1
0
0
A
5
10
0
15
5
10
15
B
9 8 7 6 5 4 3 2 1 0
Series1
0
5
10
15
C
Figure 15.2 a: Perfect positive correlation, b: Perfect negative correlation, c: little or no correlation
A correlation coefficient r of greater than ±0.3 is statistically important. However, in the social world, a correlation of greater than ± 0.7 may indicate collinearity, i.e. both variables may be the product of the same, unseen variable. Positive correlations are recorded where the variables both increase (or decrease) together, e.g. income and expenditure. Negative correlations are recorded when one variable increases and the other decreases, e.g. car-ownership and the likelihood of owners supporting the Labour Party. The general rule-of-thumb is that where: r = 0.1: the association is termed of small importance r = 0.3: the association is termed of medium importance r = 0.5: the association is termed of large importance
209
210
Research Methods in Politics
This interpretation is derived from the coefficient of determination, R 2 . This is the square of the coefficient of correlation, r. It is the proportion of the variance in one variable that can be explained or attributed to the other. So, where: r = 0.1, R 2 = 0.01, this means that 1% of the association can be attributed r = 0.3, R 2 = 0.09, this means that 9% of the association can be attributed r = 0.5, R 2 = 0.25, this means that 25% of the association can be attributed
Correlation is a measure of association. But evidence of association does not necessarily mean causation: theoretical exposition and explanation are required.
Calculating regression statistics The equations for calculating coefficients of correlation is complicated. Fortunately, regression statistics can readily be performed using Excel. Calculation of regression statistics should only be carried out after the data has been plotted as a chart to show whether or not there is prima facie (at first sight) signs of apparent association. The data must be entered in a spreadsheet in which the paired data are recorded side-by-side in columns (see Table 15.1). The next steps involve clicking on Tools and then Data Analysis. When the Data Analysis dialogue box opens, click on Regression and OK. The Regression dialogue box will appear. By way of example, use the data from Table 15.1 (Union density and pay). In the Input Y Range, enter the addresses of the first and last cells of the columns of pay, i.e. C8:C27. In the Input X Range, enter the addresses of the first and last cells in the column of union density, ie, D8:D27. Click on New Worksheet Ply and OK. The following results will appear: The most important statistics have been highlighted. They are: Multiple R 0.688452 Intercept 71.48414 X Variable 1 0.564809
What do they mean? They mean that the coefficient of correlation (termed ‘Multiple R’ in Excel) is +0.69, that the point where the straight line crosses the Pay axis is £71.50 and the gradient of the straight line is 0.56 (which means that, for every change in union density of 1%, pay will increase by 0.56%). The linear equation between pay as an outcome of union density is therefore: Pay = 71.5 + 0.56 Union Density
Testing for Association Table 15.2 Summary output of regression analysis of data in Table 15.1 Summary Output Regression Statistics Multiple R R square Adjusted R square Standard error Observations
0.688452 0.473966 0.444742 20.01117 20
ANOVA Regression Residual Total
Intercept X Variable 1
df 1 18 19
SS 6494.574 7208.046 13702.62
MS 6494.574 400.447
F 16.21831
Significance F 0.00079
Coefficients
Standard Error 9.822873 0.140249
t-value
P-value
Lower 95%
Upper 95%
7.277314 4.027196
9.18E-07 0.00079
50.84703 0.270157
92.12124 0.859461
71.48414 0.564809
Other results are also useful. As previously explained, R Square (written R 2 ) is the proportion of Y (outcome) that can be attributed to X (cause). In this case, it is 0.47. So 47% of pay can be attributed to union density. So there must be other drivers. What might they be? The Adjusted R Square is a more sophisticated and reliable estimate than R Square. In this case, Adjusted R Square is 0.45. However, before you can accept relatively high coefficient of correlation of +0.69 as evidence of association, you must be able to refute the null hypothesis. If your starting hypothesis, H1 was that union density affects pay, then the alternative hypothesis, H0 must be that there is no causal relation and that the association between data has arisen entirely by chance. In other words, you must be able to show that the data and results are statistically significant There are various statistical tests of significance for paired data. They include the Pearson product-moment correlation, Z-test and t-test. Perhaps the simplest way, is to use the confidence interval estimate of the slope. This is calculated by multiplying the t-statistic by the standard error of the slope and then adding and subtracting this product to the gradient. This sounds complicated. But, fortunately, Excel has already done this. The results are at the foot of the summary output of Table 15.2: This shows that, at 95% confidence levels, the upper and lower levels of the gradient are 0.270 and 0.859. Because these figures are greater than 0, then you can be satisfied that a significant relationship exists between the union density and pay data. The null hypothesis can therefore be refuted. Alternatively, you can refer in statistical tables to the Critical Values of Pearson’s r. These will tell you critical
211
212
Research Methods in Politics Table 15.3 Extract from summary output in Table 15.2
Intercept X Variable 1
Coefficients
Standard Error
t-value
P-value
Lower 95%
Upper 95%
71.48414 0.564809
9.822873 0.140249
7.277314 4.027196
9.18E-07 0.00079
50.84703 0.270157
92.12124 0.859461
values of r for a given sample size and levels of significance. You will note that, for a level of significance of 0.05 (95%) for a one-tailed test for a sample size of 20, the critical value of r is 0.378. Because your calculated r (of 0.69) is greater than the critical value, you can conclude that the association between union density and pay is statistically significant at 95% confidence limits. Another simple alternative is to use the P-value automatically calculated by Excel and included in Table 15.3. The P-value is the observed level of significance. Basically, you can reject the null hypothesis Ho if the P-value is less than the level of significance you have chosen (typically 0.05 in Politics research) and vice versa: ‘if the P-value is low, then Ho can go’. In the example, the P-values are well below 0.05 and the null hypothesis can be rejected. You will note that statistics tables distinguish between one-tailed and two-tailed tests. The default position in politics research is the one-tailed hypothesis: that, as one variable increases (say, union density), the other (pay) will also increase. However, there may be circumstances where, although you hypothesise that, as one variable changes, the other will also change but you can’t predict whether it will go up or down. In that case, your hypothesis will be two-tailed. What this means is that, for the same confidence limits, the critical value of r will have to be much higher if significance is to be attributed. Regression analysis also enables predictions to be made of variables for which data is not available. You will recall that the linear regression example for pay/union density was: Pay % = 71.5 + 0.56 Union density So you can calculate that, for a union density of, say, 50%, pay will be: = 71.5 + (0.56)(50) = 98.5% The predicted value of a term is denoted by a ‘hat’ sign. So, a predicted value of y is denoted by yˆ and called ‘y hat’. In the example above, the predicted value of pay is called ‘pay hat’. However, if we substitute data in the formula from our original paired data, there is a difference between predicted pay (pay hat) and recorded pay.
Testing for Association
For example, in Table 15.1, the pay and union density for distribution are 85.5 and 60%. But if we substitute a union density of 60 in the formula: Predicted pay (pay hat) = 71.5 + (0.56)(60) = 71.5 + 33.6 = 105.1 This difference between predicted value (105.1) and actual data (85.5) is called the residual error. It is represented by the Greek letter (pronounced ‘epsilon’). The general equation for linear regression is therefore: y = a + bx + where: y is the outcome (effect) a is the y-intercept (value of y when x is 0) b is the gradient x is the cause is the residual error.
The use of regression analysis to calculate y can only be used between the range of values for x for which data is available.
Multiple linear regression analysis The example of pay/union density showed that, despite a high coefficient of correlation of 0.69, less than half (0.47) of the change of pay could be attributed to changes of union density. Inevitably, in a complex social and political world in which everything seems related to everything else, other variables will be involved. So most regression analysis undertaken by Politics researchers involves multiple linear regression analysis whose general equation is: y = A + B1 x1 + B2 x2 + B3 x3 + . . . + Bn xn where x1 , x2 , x3 , … xn are the various independent variables. Returning to the example of pay/ union density, you can anticipate that other, subordinate independent variables are involved. What might they be? Your theoretical and literature review is likely to have identified some of these other variables. They may include the physical concentration of workers, the proportion of male workers (whose pay and conditions have been historically much greater than women), and the extent to which the industries have to compete globally (thereby creating downward pressures on pay), and whether they are protected by tariffs or enjoy other advantages which give them a near monopoly in the home market.
213
214
Research Methods in Politics
Calculating multiple linear regression equations using Excel Assume that you have found other, comparable, additional data from government sources for the proportion of the workforce employed in plants of 500 or more, the male proportion of the workforce, and UK market share. These are combined with your initial data in Table 15.4. Table 15.4 Pay/union density, workplace size, males and UK market share, 1981 Pay
Newspaper printing Mineral oil refining Coal mining Underground workers Air transport Electricity and gas Other printing Port and inland Water General chemicals Aerospace eng. Wholesale distribution Textiles Motor repairs and distribution Industrial materials Clothing Retail distribution Woollen worsted Educational services Catering Agriculture
157.8 143.2 133.6 145.9 130.8 122.9 120.7 119.8 117.5 117.3 86.7 86.2 85.5 85.3 84.0 83.6 82.3 80.6 77.9 72.4
Union density
% of Workers in Plants of 500+
94 59 97 99 85 95 94 83 59 80 15 99 60 15 42 15 47 78 8 23
40 65 70 10 35 22 24 65 78 85 34 71 18 69 47 26 73 12 11 0
% Men
95 92 99 60 55 85 73 97 96 88 82 22 77 88 28 62 26 45 13 79
% UK Market Share 95 90 85 100 45 100 60 100 37 38 91 36 87 33 47 82 31 91 77 65
Your first step is, once again, to produce an X–Y graph of the data: You will note in the chart that there is prime facie evidence of association between pay and the proportion of male workers and proportion of the domestic market. However, there is little evidence of a strong relationship between pay and the proportion of workers employed in plants of more than five hundred workers. You can complete the multiple linear regression analysis in entirely the same way as the linear regression analysis. The summary output is shown in Table 15.5.
Testing for Association
Pay, union density, worplace size, men and UK market share
Pay/other variables 200
Pay
150
Union density
100
% of workers in plants of 500+
50
% Men
0 0
5
10
15
20
% UK market share
25
Industrial and service sectors Figure 15.3 X–Y chart of variables in Table 15.4
Table 15.5 Multiple regression output summary from Table 15.4 Regression Statistics Multiple R R Square Adjusted R square Standard error Observations ANOVA df Regression 4 Residual 15 Total 19 Coefficients Intercept X Variable 1 X Variable 2 X Variable 3 X Variable 4
35.2295 0.463829 0.144154 0.315062 0.214856
0.813484 0.661756 0.571558 17.57806 20 SS 9067.796 4634.824 13702.62
MS 2266.949 308.9883
F 7.336683
Significance F 0.001757
Standard Error 18.32924 0.130775 0.206786 0.174371 0.221124
t Stat
P-value
Lower 95%
Upper 95%
1.922038 3.546763 0.697117 1.806843 0.97165
0.073805 0.002929 0.496393 0.090885 0.346633
−3.83837 0.185088 −0.2966 −0.0566 −0.25646
74.29738 0.74257 0.584909 0.686726 0.686171
By default, Excel has automatically designated the variables of union density, workers in plants of 500 staff or more, etc., as X variables 1, 2, 3, and 4. You will note that the overall coefficient of correlation for the entire range of variables has risen to 0.81 (high, positive correlation) and that 66% of changes in relative pay can be attributed to the four, potential causal factors. The multiple regression equation is therefore: y = 35.2 + 0.46x1 + 0.14x2 + 0.31x3 + 0.21x4 + ε
215
216
Research Methods in Politics
That is: Relative Pay % = 35.2 + 0.46 [Union Density %] + 0.14 [%Workers in plants of 500+] + 0.31 [% male workers] + 0.18 [% UK market share] + residual error However, when you examine the values for the Lower and Upper 95% confidence limits, you will note that these extend across zero (e.g. Variable 2 from −0.32 to + 0.49). The null hypothesis cannot therefore be rejected. Similarly, the P-values for X2, X3 and X4 are higher than 0.05 which also means that the null hypothesis cannot be rejected. The regression summary does not include the coefficients of correlation between pay and each of the four individual variables. To calculate these, click once again on Tools and Data Analysis. In the Data Analysis dialogue box, scroll upwards to correlation and click. In the Correlation dialogue box, insert the first and last cells of the five columns of data (B3:F22) in the Input Range box and click OK. The output appears in Table 15.6. Table 15.6 Correlation coefficients of all variables
Column 1 Column 2 Column 3 Column 4 Column 5
Column 1
Column 2
Column 3
Column 4
Column 5
1 0.688452 0.199587 0.537148 0.32949
1 0.158195 0.199638 0.168877
1 0.225699 −0.52341
1 0.314504
1
Where Column 1 is pay, Column 2 is union density, Column 3 is workers in larger plants, Column 4 is male workforce and Column 5 is UK market share. The coefficients of correlation between pay and union density is (as previously calculated) 0.69, workers in large plants 0.20, male workforce 0.54 and UK market share 0.32. Note also how Excel calculates the coefficients of correlation between all the variables, e.g. the coefficient of correlation between UK market share and workers in large plants is −0.49. This additional data indicates that there may be some previously overlooked relationship between apparently independent variables. The display in Table 15.6 is called a correlation matrix – also termed the R-matrix. It is the building block from which factor and cluster analysis are developed. These are the subject of the next chapter (16).
Pitfalls of linear regression analysis Excel and other software enable linear regression analysis to be completed quickly. Paradoxically, the greater simplicity increases the likelihood that errors will be made
Testing for Association
in the process and interpretation of the summary output. The greatest errors arise from the pitfalls of: 1. applying the formulae before first considering the likelihood of relationship – if any – in theoretical terms and without creating X–Y graphs of variables 2. failing to identify collinear variables 3. 4. 5. 6.
failing to interpret the summary output correctly misapplying or misinterpreting the tests of significance consequently, wrongly rejecting or accepting the null hypothesis predicting values of yˆ outside the range of known x data.
Questions for discussion or assignments
1. ‘Correlation does not necessarily mean causation’. Discuss. Explain how you would investigate a high correlation for causation. 2. The table shows paired data for the total number of UK workers registered as unemployed and membership of the British Communist party, 1929–39. Table 15.7 Paired data:1929–39 Year
UK Unemployed (000s)
CP Membership
1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939
1,216 1,917 2,630 2,745 2,521 2,159 2,036 1,755 1,484 1,791 1,514
3,200 2,555 6,279 5,600 5,700 5,800 7,700 11,500 12,250 15,570 17,756
Is there any evidence of association between unemployment and party membership between 1929 and 1939? Using Excel, draw an X –Y graph of unemployment on the x-axis and CP membership on the y-axis. Calculate the coefficient of correlation. Calculate the linear regression equation.
217
218
Research Methods in Politics Calculate the contribution made by unemployment to CP membership. Test the statistical significance of the calculation. Can the null hypothesis be dismissed? The data shows that, after 1934, CP membership increased whilst unemployment fell. What other causes of increasing CP membership can you suggest and why? 3. Read carefully the extract from the publication by Rallings, C., and Thrasher, M., (1997) Local Government Elections in Britain, London, Routledge, pp. 46–63. Examine the relevant 2001 census data for the wards of a UK city of your choice from web-site www.neighbourhood.statistics.gov.uk. On the basis of the information given in the selected text and the census data available, set out an hypothesis to the research question: which three factors are most likely to have affected electoral turnout in local elections in your chosen city? Find and save the most recent headline election data for ward turnout for all of the wards in your selected city from its web-site www.[selected city].gov.uk Select three independent/collinear variables from the census information for testing your hypothesis. Transpose the data for ward names, turnout and your three selected census variables into a single spreadsheet. Produce X –Y charts of the data. Using Microsoft Excel spreadsheet software, create a spreadsheet consisting of all wards, the turnout data and the three variables you have selected from the census data. Calculate the coefficients of correlation between turnout and selected census characteristics. What inferences can you draw from the association of turnout and selected census data? Are the data significant? What limitations do you attach to these inferences? Using the appropriate formula within Excel, calculate the multiple linear regression equation between ward turnout (Y ) and the independent variables (X1 . . .Xn ). Your submission should be no less than 2,000 words in report form. It must critically review the text by Rallings and Thrasher and clearly justify your choice of potential independent variables. You must then explicitly describe, justify and explain the analytic techniques you have adopted, the ‘results’, the linear regression equation calculated and the limitations attached to the output.
FURTHER READING Clegg, F. (2005) Simple Statistics. Cambridge: Cambridge University Press. pp. 124–40. Clegg provides a readable and authoritative introduction to correlation and provides worked examples in pp. 167–87.
Testing for Association Levine, D. and Stephan, D. (2005) Chapter 10: Regression Analysis. In Statistics: Even You Can Learn. London: Pearson Prentice Hall. pp. 182–208. This is a very readable account which includes step-by-step instructions for using Microsoft Excel to complete calculations and downloadable practice files. Kanji, G. K. (1999) 100 Statistical Tests. London: Sage. This is another advanced text. It shows clearly which tests of significance should be carried out with simple worked examples and statistics tables. Lewis-Beck, M. S. (1995) Data Analysis: An Introduction. London: Sage. This relatively short book (77 pages) provides a good introduction for students with experience of higher mathematics. It also provides additional, valuable guidance on tests for association for nominal data (using Goodman and Kruskals’ Lambda) and ordinal data (Kendall’s tau) which will be especially helpful to researchers intending to use comparative method (pp. 22–8). Pennings, P., Keman, H. and Kelinnijenhuis, J. (2006) Chapter 6: Multivariate Analysis and Causal Inference. In Doing Research in Political Science. London: Sage. pp. 132–82. This is a very advanced text which demonstrates the application of regression analysis to Politics case studies.
219
Chapter 16
Applying Factor Analysis and other Advanced Techniques
Teaching and learning objectives: 1. To understand the meaning and application of factor analysis and cluster analysis. 2. To learn how to carry out factor analysis. 3. To introduce other advanced functions and techniques including the binomial and Poisson distributions and time series analysis.
Introduction You may have noticed that UK research methods textbooks rarely go beyond linear regression analysis because of space and complexity. This is unfortunate for two reasons. First, the data and software now available to researchers make more advanced statistics much easier to apply. And, second, the more advanced techniques are, paradoxically, more useful they are than the basic descriptive or inferential statistics with which you are more likely to be familiar. The underlying purpose of this chapter is, therefore, to widen your appreciation of the new possibilities in quantitative data analysis readily available to you to achieve more practical and adventurous research.
Factors and factor analysis Chapter 15 ended with an example of a correlation matrix which showed how the independent variables shared some small degree of association. However, no causal connection was inferred from the analysis. This demonstrates the old adage (saying) that, in the social world, everything seems related to everything else in complex ways and patterns. Factor analysis enables you to simplify that complexity.
Applying Factor Analysis and other Advanced Techniques
Consider, for example, the physical and social character of many of the inner city areas of western cities located between the city centre (US ‘central business district’, CBD) and the outer rings of ‘better suburban housing’ (first identified by Park, 1925 and Wirth, 1928 of the ‘Chicago School’).1 Physically, the inner city remains characterised by decaying, predominantly older, high-density, terraced housing. Most is privately rented. The larger houses have been sub-divided into ‘houses in multiple occupation’ (HIMOs). The schools, parks and shopping areas are dilapidated. The roads are congested with traffic moving between the suburbs and the city centre. Most of the mills, factories and other industrial buildings have closed. Air quality is poor. Socially, the area has become the entry point for successive ‘waves of immigrants’. Other temporary migrants include students and other single people or single parents seeking lowest-cost housing. Turnover is high. There is also a large number of single, older pensioners living on low-incomes. Many suffer chronic illnesses. There is a relatively high level of drug and alcohol abuse, and street crime. Economically, job opportunities and wages are very low. Most of the residents rely on some form of income support from the state. In other words, these physical, social and economic characteristics come together – cluster – in the inner cities of the western world. The geographical clusters are termed areas of multiple deprivation. Governments and local authorities have developed statistical indices of deprivation. They combine census data on: population age, household size, employment, health, and housing conditions, with other data on: incomes, benefits and health treatments. The UK government uses the indices as part of the funding formulae (developed by multiple linear regression) to distribute central government grants in an objective (but contested) framework to local authorities, police and health trusts. Local authorities also use the indices to identify the localities in greatest need and to lobby government for additional resources. Both use indices to measure changes over time and the success or otherwise of intervention. Why these characteristics co-locate is disputed. That dispute is fundamental to political divisions in society. The Right explains the inner city as the manifestation of conditions which are created where individuals choose a lifestyle of welfare dependency: they get what they deserve. The policy solution therefore lies in welfare cuts to force them to work and become self-reliant. (My grandfather, a self-made businessman, alderman and church warden subscribed to this view despite both his grandmothers having died in workhouses.) On the Left, inner city residents are seen as the victims of a capitalist economy in which those unable to work or find work and therefore with the least economic power are consigned to the worst areas by market forces. They are prescribed meagre handouts and a large police presence to prevent them from challenging the system. The solutions advocated include redistribution of income, greater income support and empowerment of local institutions. The Centre takes a slightly ambivalent view in which the inner city is seen as a cost to the national economy, in which the residents are underused resources. The solutions therefore
221
222
Research Methods in Politics
rely on education, training, capacity building and a combination of incentives and sanctions to encourage participation. So, what’s all this to do with factor analysis? Well, the underlying principle of factor analysis is that, where characteristics (like poor health, housing and incomes) occur together, they may be the observable symptoms of underlying common factors also called latent variables. Where the symptoms can be measured quantitatively, then the number and strength of factors can be statistically identified. These factors point to underlying causes, for example, the role of institutions and those whose interests are served by them. But, whilst factor analysis can identify the factors, the labels – social exclusion or underclass – must be assigned by the researcher. Factor analysis serves three main purposes. First, the analysis enables the underlying structure of a complex set of variables to be identified. Second, it can reduce and compress the raw data to more manageable evidence. And, third, it enables researchers to develop more effective and efficient follow-up questionnaires or other surveys which directly target the key underlying factors.
Factor analysis Factor analysis involves four stages: 1. First, an R-matrix is produced. This enables the variables with the highest coefficients of correlation to be identified. It also enables the clusters of variables to be detected by visual analysis. Consider for example, the (simplified) R-matrix in Table 16.1. Note that the R-matrix is called a ‘square matrix’ because there are as many rows as there are columns: Table 16.1a R-matrix
Variable 1 Variable 2 Variable 3 Variable 4 Variable 5
Variable 1
Variable 2
Variable 3
Variable 4
Variable 5
1 0.5 0.4 0.1 0.1
1 0.3 0.1 0.1
1 0.1 0.2
1 0.7
1
2. Second, the underlying factors are extracted from the R-matrix. The simplest method is called principal factor extraction. When you shade or colour the cells containing coefficients of correlation with a value of 0.3 or more, then two clusters become apparent: You will note from Table 16.1a that variables 1, 2 and 3, and variables 4 and 5 show high degrees of association. The two separate clusters they form are termed the two
Applying Factor Analysis and other Advanced Techniques Table 16.1b R-matrix : higher correlations shaded
Variable 1 Variable 2 Variable 3 Variable 4 Variable 5
Variable 1
Variable 2
1 0.5 0.4 0.1 0.1
1 0.3 0.1 0.1
Variable 3
Variable 4
Variable 5
1 0.1 0.2
1 0.7
1
principal factors. These factors can be named A and B. In this stage, the eigenvalues are calculated. The eigenvalue is the measure of the variance in the original variables that can be attributed to a particular factor. 3. Thirdly, the principal factors are rotated using the varimax procedure (which maintains the independence of the factors) in which the factor axes remain at right angles to one another. What this means is that the variables are plotted along the axes of the factors. This is shown diagrammatically below: 0.8 0.7
Factor B
0.6 0.5 Factor B
0.4 0.3 0.2 0.1 0 0
0.1
0.2
0.3
0.4
0.5
0.6
Factor A Figure 16.1 Factor chart
4. Lastly, the factor scores are calculated. These are estimates of the people’s individual standing with regard to the factors identified.
This procedure sounds complicated. But advanced computer software enables the process to be completed in a small number of commands. However, whilst the R-matrix can be calculated using Microsoft Excel, the subsequent steps are currently beyond its capacity. You have two alternative courses of action. First, you can use add-ons to Excel such as WinStat taking advantage of the 30-day free trial period. These add-ons can be bought for around £100. Or second, you can use one of the advanced statistical software packages such as SPSS, STATA or Minitab. You are likely to find that your university already supports one or more of these
223
224
Research Methods in Politics
on its network. Alternatively, you may find that you can buy your own copy from your computer services under special annual licences for the cost of the CD-ROM, i.e. £5. But you will only be able to renew the licence as long as you are a student or employee of the university. After that you will have to pay to purchase the software (£1200 for SPSS) and pay the annual licence renewal (around) £200. But you can download a copy for a 30-day trial period.
SPSS SPSS will be used for factor analysis. SPSS was originally the acronym for Statistical Package for the Social Sciences. More recently, the acronym has changed to Statistical Product and Service Solutions as the company has extended its range of products and customers beyond its original academic base. The web-site is www.spss.com. Remember that learning to use a different type of software is like learning a new language: it is useful and develops your intellect. The software can appear initially perplexing but it is very easy and has become more intuitive to use. SPSS uses a full Windows environment. Like Excel, all the data is held on a single spreadsheet called the Data Editor. This has two alternative screens: a Variable View and a Data View. The Variable View is the spreadsheet where you enter the variables as separate rows and nominate the name, type, width, decimals label, values etc. For example, these might be: name (age of respondent), type (numeric), width (4), decimals (2), label (age), etc. The Data View is the spreadsheet where you enter your data. In this spreadsheet, the variables occupy the columns whilst each case is entered as a new row.
How to carry out factor analysis using SPSS Consider the example where you want to find out what the public thinks should be the most important characteristics and qualities of candidates for the post of prime minister. Following a pilot study, you reduce the characteristics to twenty-one attributes: (their) age (youthfulness); sex (male or female); appearance; religion; origins (class); peccadilloes (known or alleged personal misbehaviour); family (partner and children); wealth; identity (notably English, Scots, etc); speech (speaking and debating abilities); modernity (not old-fashioned); honesty; sincerity; integrity; guile (cunning); principle; loyalty; courage; patriotism; and, Euroscepticism (anti-EU). You enter these variables in the opening Variable View shown in Table 16.2. Note how an additional variable, ‘cases’ has been added to the Variable View. The ‘cases’ are the individual members of the public whom you have asked to rate the importance they attach to each of these short-listed characteristics on a scale of 0 to 10 (where 0 is least importance and 10 is highest importance). The members of the
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Age Sex Appearance Religion Origins Peccadillos Family Wealth Identity Speech Modernity Honesty Sincerity Integrity Guile Intellect Principles Loyalty Courage Patriotism Eurosceptic Cases
Name
Numeric Numeric Numeric Numeric Numeric Numeric Numeric Numeric Numeric Numeric Numeric Numeric Numeric Numeric Numeric Numeric Numeric Numeric Numeric Numeric Numeric Numeric
Type
Table 16.2 Variable view
3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 8
Width 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Decimals Age Gender Looks Religion Background Peccadillos Private/family life Wealth Regional identity Speaking skills Modernity Truthfulness Sincerity Integrity Guile Intellect Principles Loyalty Courage Patriotism Euroscepticism Case number
Label None None None None None None None None None None None None None None None None None None None None None {1, case number}
Values None None None None None None None None None None None None None None None None None None None None None None
Missing 4 3 4 4 3 3 4 4 3 4 4 4 4 4 4 4 4 4 4 4 4 8
Columns Right Right Right Right Right Right Right Right Right Right Right Right Right Right Right Right Right Right Right Right Right Right
Align Scale Scale Scale Scale Scale Scale Scale Scale Scale Scale Scale Scale Scale Scale Scale Scale Scale Scale Scale Scale Scale Scale
Measure
226
Research Methods in Politics
general public will have been randomly selected. The sample size will be 630, i.e. calculated at 30 people per variable. You will enter the data collected in the Data View. The Data View for the first ten cases is shown in Table 16.3. Starting the factor analysis procedure is relatively simple. On the Data View, click in turn on: Analyze Data Reduction Factor Analysis
On the Factor Analysis dialogue box, highlight all the variables listed in the left-hand pane except ‘cases’. Then, click in turn on: The higher ‘radio button’ this will transfer the list of variables to the pane marked ‘Variables’ Descriptives
In the Factor Analysis: Descriptives dialogue box, click in turn on: Univariate descriptives Initial solution Coefficients (Correlation Matrix) Reproduced Continue
You will return to the Factor Analysis dialogue box. Now click in turn on Extraction. In the Factor Analysis: Extraction dialogue box, click in turn on: Principal Components (Method) Correlation matrix Unrotated factor solution Scree plot Eigenvalues over 1 Continue
You will return once again to the Factor Analysis dialogue box. Click on Rotation. In theFactor Analysis: Rotation dialogue box, click in turn on: Varimax Rotated solution Continue
1 2 3 4 5 6 7 8 9 10
7 4 9 1 2 0 3 0 7 2
6 8 1 1 2 9 9 0 9 9
8 8 8 4 8 9 8 2 8 8
0 4 2 1 0 1 0 0 5 7
5 8 1 1 0 7 0 0 6 5
3 8 2 1 0 1 0 0 7 8
5 8 4 3 1 3 1 0 8 7
2 8 2 1 0 5 0 0 5 8
4 9 5 2 5 5 9 0 8 9
7 8 8 6 9 7 8 9 9 8
6 2 9 4 7 2 8 9 2 2
6 7 7 9 7 6 4 5 6 5
7 8 7 9 7 7 4 5 6 5
7 6 7 9 6 8 4 5 6 5
7 8 8 5 8 9 8 8 8 8
8 5 6 7 8 4 7 9 5 7
9 7 7 8 6 6 5 8 7 7
7 8 7 7 7 7 8 8 7 7
7 8 8 6 6 8 9 8 8 9
8 9 7 7 4 8 9 5 9 9
5 9 3 2 0 9 9 5 9 9
1 2 3 4 5 6 7 8 9 10
Age Sex Appear- Religion Origins Peccadi- Family Wealth Identity Speech Modernity Honesty Sincerity Integrity Guile Intellect Principles Loyalty Cour- Patriot- Euroscepti- Cases ances llos age ism cism
Table 16.3 SPSS Data View (cases 1–10 only of 630 cases)
Research Methods in Politics
You will finally return to the Factor Analysis dialogue box where you will click on OK. The output will appear immediately on a new screen. This will consist of nine tables and one chart. The most important for your purposes are the Total Variance Explained, the Scree Plot and the Rotated Component Matrix Part of the Total Variance Explained is shown below in Table 16.4. You will note that Component 1 (i.e. Factor 1) has an eigenvalue of 8.7 and explains 41.6% of the variance. Component 2 explains 24.6% of the variance. Five components (factors) have eigenvalues greater than 1.0 and explain 88.9% of the variances overall. The table is illustrated in the Scree Plot which shows the components having the highest eigenvalues. Table 16.4 SPSS output: total variance explained Total Variance Explained Component Initial Eigenvalues Total 1 2 3 4 5 6 7 8 9
% of Variance
8.7 5.2 1.9 1.7 1.2 0.8 0.8 0.5 0.2
41.6 24.6 9.0 8.0 5.7 3.9 3.7 2.4 1.2
Cumulative % 41.6 66.2 75.2 83.1 88.9 92.7 96.4 98.8 100.0
8 Eigenvalue
228
6 4 2 0 1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21
Component number Figure 16.2 SPSS output: scree plot
Applying Factor Analysis and other Advanced Techniques Table 16.5 SPSS factor analysis: rotated component matrix Component 1 Wealth Peccadillos Private/family life Religion Modernity Background (class) Patriotism Euroscepticism Gender Regional identity Truthfulness Sincerity Integrity Courage Loyalty Principles Looks Guile Intellect Speaking skills Age
.961 .948 .917 .883 −.867 .862 .664 .641 .620 .560 .136 .350 −.104 .143 .324 .149 −.570
2
3
4
5
.168 −.114 .115 .235 .335 .606 .395 .354 −.922 −.911 −.883 .819 .664 −.291 .444 .211 .442
.125
−.195 .228 .200 .294 .522 .539 −.205 −.126 .235 −.151 −.835 .784 .641 −.619 .108
−.205 −.194 .196 .566 .301 .313
.245 .376 .151 .237 .266 −.105 .312
.180 .387
.106
−.320 .168 .445
−.408 −.247 −.846 .948
Notes: Extraction method: principal component analysis. Rotation method: varimax with Kaiser normalization. A rotation converged in seven iterations.
The rotated component matrix is shown in Table 16.5. This matrix identifies the relationship between the original 21 variables and the five principal components into which the data can be reduced. Components 1 and 2 are clearly the most important. Component 1 shows very high values for wealth, peccadilloes, private/family life, religion, background (class), patriotism, euroscepticism and gender. You must now interpret this result to identify what is the family of characteristics incorporated in Component 1 interpretation as ‘traditional values’. Indeed, these are the characteristics associated with the Conservative party. The strongest characteristics of Component 2 are courage, loyalty and guile. (Sincerity and truthfulness are negative aspects here.) One interpretation of these characteristics might be ‘street fighting qualities’. The analysis shows that two factors dominate the characteristics that the British public wish to see in candidates for prime minister. These can be labelled ‘traditional values’ and ‘street fighting qualities’. Age and intellect are not important.
229
230
Research Methods in Politics
Margaret Thatcher in the period 1975–90 appears to fit these preferred characteristics well and may well explain her popular appeal. The additional advantage of reducing the variables to two key factors is that a previous battery of potential questions in questionnaires can be reduced to just two thereby reducing the number of questions, costs and time required and allowing additional alternative questions to be added.
Time series analysis Consider the data in Table 16.6 of civilian casualties in Iraq from March 2003 and October 2006. Table 16.6 Iraqi civilian casualties 2003–6 2003 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
2778 2859 507 541 593 735 525 461 433 504
2004
2005
2006
542 561 917
946 1111 644 833 1024 1060 1336 1941 1117 953 1053 828
1186 1236 1423 1200 1658 1897 3590 3009 3345 3709
541 747 682 724 805 827 1323 802
Source: Iraqbodycount.org
You will be aware that coalition spokespersons (especially for US forces) continued to claim for some time that, despite occasional ‘blips’, the level of violence was actually declining. Using Excel, you can chart the table: In Figure 16.3, you can see that the general movement for monthly casualties is to increase over time. You can identify the underlying trend by using the Moving Average function in Excel (click in turn Tools, Data Analysis, Moving Average). Essentially, this calculates the average of the months March–May, then April–June, May–July and so on. The calculated moving averages for the first year are shown in bold in Table 16.7 The moving averages can be charted to show the trend (forecast): You will note that there is a variance between the actual data (Y) for casualties and the trend.
4000 Civilian casualties
3500 3000 2500
Month no.
2000 Casualties
1500 1000 500 0 0
10
20
30
40
50
Time (months) Figure 16.3 Chart of Iraqi civilian casualties 2003–6
Table 16.7 Calculation of moving averages Year
Month
2003
Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Month No.
Casualties
#N/A
3 4 5 6 7 8 9 10 11 12
2778 2859 507 541 593 735 525 461 433 504
#N/A 2048 1302 547 623 618 574 473 466 493
Moving average 4000 3500 3000 Value
2500 Actual Forecast
2000 1500 1000 500 0 1
6
11
16
21
26
Data point Figure 16.4 Trend line
31
36
41
232
Research Methods in Politics Table 16.8 In-migration to UK from other EU States (000s) Year/Quarter 2003 2004 2005 2006
Q1
Q2
Q3
Q4
78 84 92 100
62 64 70 81
56 61 63 72
71 82 85 96
This variance has four potential components: Trend Component, T Cyclical Component, C Seasonal Component, S Residual Component, R
Where: Y = T + C + S + R The Trend Component is the underlying movement calculated by the method of moving averages. The Cyclical Component is the variation that is generated by cyclical movement of trade or business cycles. In the case of Iraqi war casualties, there may be no cyclical component. The Seasonal Component is the variation attributable to the months or seasons of the year. You may recall that there was reference during the early years of the occupation to a ‘fighting season’. When you examine the charts in Figures 16.3 and 16.4, you will see that there is some evidence of seasonal variations with casualties reducing in December–February. You can carry out sophisticated time series analysis using SPSS. However, the data output is complex and difficult to interpret. Excel is unable to go beyond calculating the moving average. The simplest method is to adopt the manual Method of Quarterly Deviations using your own formulae created in Excel. Method of quarterly deviations Consider the example of in-migration to the UK from other (‘pre-enlargement’) EU states: The trend and the deviation from the trend may be calculated easily by calculating in turn: 1. 2. 3. 4.
the four-quarter moving total the eight-quarter moving total dividing the eight-quarter moving total by 8 to calculate the trend finally, subtracting the trend from the actual data to calculate the deviation between the trend and the actual data.
Applying Factor Analysis and other Advanced Techniques Table 16.9 Calculation of trend and deviation Year
Quarter
In-migrants (Y)
2003
1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4
78 62 56 71 84 64 61 82 92 70 63 85 100 81 72 96
2004
2005
2006
Fourquarter moving total
Eightquarter moving total
Trend (T)
Deviation from trend (Y−T)
267 273 275 280 291 299 305 307 310 318 329 338 349
540 548 555 571 590 604 612 617 628 647 667 687
67.5 68.5 69.375 71.375 73.75 75.5 76.5 77.125 78.5 80.875 83.375 85.875
−11.5 2.5 14.625 −7.375 −12.75 6.5 15.5 −7.125 −15.5 4.125 16.625 −4.875
The completed calculation is shown in Table 16.9 The next step is to calculate the seasonal variation. This is calculated by finding the average of the deviations for each quarter. The deviations for the third quarter of each year are: −11.5, −12.75 and −15.50 which total −39.75 which, divided by three, give an average of −13.25. When you add the average deviations for the four quarters, you will find that these add up to +0.25. As the sum should be zero, then this error figure is divided by four and applied as a correction factor to each seasonal deviation. The residual component or irregular movement (R) can then be calculated by subtracting the seasonal deviation (S) and trend (T) from the actual data (Y) as shown in Table 16.10. You will note that the highest residual variations – irregular movements − are +2.188 in the final quarter of 2004 and −2.187 in the third quarter of 2005. These are relatively small. Your task as a Politics researcher is to explain these irregular movements. One explanation might be the policy announcement of tightening controls made in 2004, which prompted more in-migration in the final quarter before the controls came into effect in the third quarter of 2005. You will appreciate that changes in public policy often create a j-lag effect where variables continue to get ‘worse’ until the new policy begins to ‘improve’ the situation.
233
234
Research Methods in Politics Table 16.10 Calculation of residual variation (shown bold) Year
Quarter
In-migrants (Y) 000s
2003
1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4
78 62 56 71 84 64 61 82 92 70 63 85 100 81 72 96
2004
2005
2006
Trend (T)
67.5 68.5 69.375 71.375 73.75 75.5 76.5 77.125 78.5 80.875 83.375 85.875
Seasonal Variation (S)
−13.313 4.312 15.52 −6.521 −13.313 4.312 15.52 −6.521 −13.313 4.312 15.52 −6.521
Residual Variation (R)
1.813 −1.812 −0.895 −0.854 0.563 2.188 −0.02 −0.604 −2.187 −0.187 1.105 1.646
Time series analysis is a very underused but useful research method in Politics. Try to use it. It will pay dividends by identifying those irregular movements which may be the outcome of policy change. Time series analysis is therefore a very powerful tool for monitoring public policy and assessing the effectiveness of policy change.
The binomial distribution In Politics research, population characteristics are assumed to follow the normal distribution (see Chapter 14). However, there will be circumstances where the data are non-numeric and involve only two potential answers, for example, the replies YES or NO to a question, or heads or tails in respect of coins being ‘tossed’. Yes/no or heads/tails are termed dichotomous nominal data. In these circumstances, the outcomes are termed success p, or failure q. The probability of success p, can be expressed mathematically. For example, the probability of a tossed coin landing ‘heads’ is 0.5. The probability of a thrown die ‘giving’ a six is 0.167 (i.e. one divided by six). Each separate toss of the coin or throw of the die is termed an event. Tests in which dichotomous outcomes are measured and where the outcomes are independent of each other are termed Bernoulli trials (after their originator, Jacques Bernoulli, 1654–1705). In Bernoulli trials, the null hypothesis is that the probability of success in each event is the theoretical probability, e.g. for each toss of the coin, 0.5.
Applying Factor Analysis and other Advanced Techniques
Take the example of two food parcels, A and B, being prepared by an aid agency for use overseas to sustain IDPs 2 in a specific war-torn state. Both parcels contain different food of equal nutritional value. The parcels have transparent packaging so that the people can choose A or B. A two-parcel strategy has been adopted to optimise cost, local production capacity and variety. The theoretical probability of a person choosing parcel type A is therefore 0.5. An initial trial of a random sample of 100 of the target population shows that 61 preferred parcel type A. The research problem is therefore: does this higher preference represent the choice likely to be made by the population as a whole or has it occurred entirely by chance? The problem can be solved using SPSS. In the Variable View, enter ‘parcel’ in the first row and ‘frequency’ in the second row. In the Data View, enter ‘1’ (for A) and ‘2’ (for B) in the column marked ‘parcel’ and ‘61’ and ‘39’ in the column marked ‘frequency’. Now click in turn on Data View and Weight Cases. In the Weight Cases dialogue box, click on Weight Cases and, using the radio button, transfer ‘frequency’ to the Frequency Variable. Then click on OK. You will return to the Data View. Click in turn on Analyze, Nonparametric Tests and Binomial. In the Binomial Test dialogue box, use the radio button to transfer‘parcel’ to the Test Variable List. Note theTest Proportion box. This is where the theoretical probability is inserted. The default proportion is 0.5 so no new entry is required. Finally, click on OK. The Output is: Table 16.11 Binomial test Parcel
Parcel 1A Parcel 1B
Group 1 Group 2 Total
Category
N
1 2
61 39 100
Observed Prop. .61 .39 1.00
Test Prop.
.50
Asymp. Sig. (2-tailed) .035(a)
Note: Based on Z Approximation.
The key calculation is the Asymp. Sig. (2-tailed) of 0.35. This is less then the test proportion (theoretical probability) of 0.5. This means that the null hypothesis can be rejected. So the preference of parcel A cannot be attributed to chance. The implications are that the aid agency should either change the ratio of production from 50:50 to nearer 61:39 or improve the appeal of parcel B and re-test using a new sample. A special type of Binomial distribution occurs where the population is very large and the incidence of events is very small. This special type is termed the Poisson distribution (named after Siméon Denis Poisson, 1781–1840 who developed the distribution to forecast the likelihood of any of his fellow cavalrymen being kicked
235
236
Research Methods in Politics Table 16.12 Poisson distribution: data Description
Data events per 100,000 children
Illness events in city C,x Illness mean worldwide
5 3
by a horse). It is widely used to forecast the likelihood of a rare event occurring at any time or, alternatively, several rare events happening at the same time. It is therefore widely used by fire and other emergency services to plan staffing levels to ensure adequate cover. A special application is the analysis of illness and disease. For example, assume that the world wide incidence of a rare, fatal disease is 3 in every 100,000 children a year. Data for city C shows that the rate of the disease is 5 per 100,000. Does this indicate a special cluster or could the higher figure have arisen entirely by chance? In this case, Excel provides a simpler solution than SPSS. Enter the data into a spreadsheet: To calculate the probability of rate of illness in city C exceeding worldwide rate, highlight a cell and enter = POISSON. Excel will automatically complete the formula for you as: = POISSON (x, mean, cumulative) In this example, x = 5 and the mean = 3. The Help? Button will tell you that: ‘cumulative is a logical function that determines the form of distribution returned’. You can enter either TRUE or FALSE. When you enter TRUE, Excel will calculate the ‘cumulative Poisson probability that the number of events occurring will be between zero and x’. If you enter FALSE, then Excel will calculate the ‘Poisson mass probability function that the number of events will be x exactly’. First, complete the formula as: = POISSON (5, 3, TRUE) Press the Enter key. The answer will appear immediately in the highlighted cell: 0.916. Repeat the process, inserting FALSE. The answer is 0.1008. This answer tells you that there is 10.1% probability (0.1008 − 0.916 ×100) of the number of illnesses being 5 where the mean is 3. So the illnesses in city C could have arisen entirely by chance and are therefore not exceptional. However, if the incidence was ten children a year, then you will calculate that the probability is 0.0008. In other words, the incidence could not have arisen by chance. So emergency intervention is required to identify and tackle a local cause or source.
Applying Factor Analysis and other Advanced Techniques
Questions for discussion in class or seminars or for assignments
1. Work through each of the examples given in this chapter. Seek help when you encounter difficulties. 2. Collect new data from the class population of their assessment of candidates for UK prime ministers using the criteria listed in the chapter. Add four additional criteria of your own choice. Carry out a factor analysis and comment on the results. 3. Obtain unadjusted data for registered unemployment in the UK since May 1979. Apply time series analysis. Identify the size of irregular movements and offer cogent, potential explanations. 4. Two Departments of Politics, having equal and good RAE and TQA ratings in ‘new universities’ A and B both have 30 places available for similar Politics degrees. A attracts 56 applicants whilst B attracts 73. Could the variation have arisen entirely by chance or does it indicate that B’s department or university are significantly more attractive than A? If so, what should A do? 5. The Conservative party has a working majority of 11 over all other parties combined in the House of Commons following the general election. On average, five MPs die each year causing by-elections to be held. The new prime minister believes that harsh economic policies are essential for the first three years. He assumes that half the Tory-held seats will be lost in by-elections. Calculate the probability that the party can survive in office for three years. (Remember that each time a government loses one seat, their majority reduces by two.)
FURTHER READING A notable feature of SPSS is the number of very good textbooks that are available which combine excellent explanations of the statistical concepts and instructions how to carry them out using SPSS software. Two titles can be recommended: Kinnear, P. R. and Gray, C.D. (2004) SPSS for Windows Made Simple. Hove: Psychology Press Ltd. This offers a very good introductory and intermediate text. Earlier editions are also available. Field, A. (2005) Discovering Statistics Using SPSS (and Sex, Drugs and Rock ‘n’ roll). 2nd edn. London: Sage. This more extensive and expensive book is very readable and provides a comprehensive range of material for new and more experienced researchers.
237
238
Research Methods in Politics
Notes 1 Chicago University’s School of Sociology was the first established at a US university. As the ‘Chicago School’ it developed a reputation for ‘urban ecology’ which identified the concept of the ‘zone of transition’ and, later ‘inner city’ that provided the initial theoretical underpinning for US and UK urban programmes and urban regeneration. See Park, R.E. and Burgess, E.W. (1925) The City and Wirth, L. (1928) The Ghetto. 2 IDPs: acronym used by UN to distinguish ‘internally displaced persons’ from other refugees.
Part IV B: Qualitative Analysis
Chapter 17
Analysing Qualitative Information: Classifying, Coding and Interpreting Information
Teaching and learning objectives: 1. To learn how to develop and apply your own codes. 2. To learn how to interpret coded information using tables and diagrams. 3. To understand how to use the techniques of ‘memoing’, case analysis meetings and interim case summaries. 4. To learn how to construct your own typology.
Introduction In Chapter 12, you were introduced to the hierarchy of analysis which begins by assembling all the raw data, validating it and reducing it to ordered information. At the end of this essential first stage, you will have uncluttered the chaos of your desk and reduced the paperwork etc. to the ‘nuggets of gold’ on which you can begin your analysis. There are three principal methods for analysing qualitative information. Most qualitative information takes the form of talk-and-text - spoken or written words. The methods are: • • •
coding content analysis discourse analysis.
Content analysis is probably the oldest form of analysis. It is essentially a quantitative process in which you count the frequency of key words individually and, in more advanced methods, their concurrence. Its use has been recently revived thanks in part to developments in communications science, semiotics, the Internet and new computer software. Content analysis is considered in Chapter 18.
242
Research Methods in Politics
Discourse analysis is considered in Chapter 19. It is founded on the belief that words are never neutral or merely descriptive, but part of vocabularies which are written by others and laden with meanings. The three methods are not necessarily mutually exclusive.
Coding This has become the most widely used method of qualitative analysis. Coding is essentially the process of replacing or substituting groups of words, phrases or sentences by letters or numbers (or a combination). Unlike a cipher, a code is not designed to hide the original meaning of the word or phrase. Indeed, an essential property of an effective code is that it should be readily understandable to users and readers. A word or phrase can be given several different codes. Codes can be theoretical, descriptive or inferential. Theoretical codes are derived from the independent and dependent variables identified in your theoretical framework. A descriptive code merely describes what it seeks to represent. An inferential code is used when you draw an inference usually from repeated patterns or clusters of juxtapositions of different codes. One special type of inferential code is a factor code. A factor is a latent, underlying variable like ‘power’. Chapter 16 showed you how to identify underlying factors from quantitative data. There will be occasions when you identify important variables which fall outside your initial range. In this case, you can develop new codes as you continue. These are called in vivo codes (‘code-as-you-go’). Codes are usually designed generically and given alphabetical references to help identify and distinguish them intuitively. For example, where ‘education’ is the theoretical variable, then a family of codes could be, say: Education Pre-school Early Years Nursery Infants Junior Secondary Sixth-Form Teacher Further Education Higher Education University Uni-Tutor Uni-Friend
E Ep Epe Epn Ei Ej Es E6 Et Ef Eh Ehu Ehut Ehuf
Analysing Qualitative Information
The process of designing and applying theoretical, descriptive or in vivo codes to text is termed first-level coding. They are applied by underlining or making bold all the key passages and recording the codes in the right-hand margin. Alternatively, you can use Nud*st, ATLAS.ti or other (‘CAQDAS’) software to code the information electronically. In this way, passages can be multi-coded. Authorities suggest that you will be able to remember and use as many as 60 generic codes without difficulty (Miles and Huberman, 1994: 58).1 Tagging and coding You must have already reduced your raw data to organised information before you begin tagging and coding. This will have involved transcribing your interviews into a three column format and having omitted all extraneous clutter and ‘noise’. However, you don’t need to have collected all your data before you begin the process of reducing available data and coding. Indeed, there are great advantages in designing your codes and applying them at an early stage of your fieldwork to test their appropriateness and the value of your initial data and information. The initial part of the process – identifying key, codable words or phrases before they are coded – is termed tagging. You are likely to be already familiar with the most basic tagging and coding of text. This involves highlighting or underlining words, phrases or sentences with coloured pens. Each different type of text is marked with a different colour. So the text becomes colour-coded. However, you will encounter two or more problems. First, you are likely to run out of colours. And, second, many words or phrases may be coloured two or more times so the coding becomes unreadable. Yet, words or phrases which attract many codes are particularly important because they demonstrate the links between different characteristics and concepts. The case study below has been developed to demonstrate how to design and apply simple codes and to analyse the coded information. Case study The research question is: why are UK leading political elites first drawn into politics and particular parties? You have chosen a comparative approach which relies, in the first instance on primary, published sources. Your starting, theoretical hypothesis is that choices of party and political careers are caused by a combination of environmental factors. You believe that they will include: family, education, religion, class, occupation and circumstances. You have chosen a modified form of grounded research in which you start with a small sample of two cases. You will analyse these before seeking further cases until you achieve theoretical saturation.
243
244
Research Methods in Politics
You have been fortunate to obtain transcripts of interviews by the same interviewer of two UK party leaders and prime ministers in the late 1960s early 1970s. They are Harold Wilson (Labour) and Ted Heath (Conservative). They are both described as ‘self-made’ men. Wilson was the first Labour prime minister to be born into the Labour party. You could argue that his career trajectory was determined by the set of institutions into which he was born. Heath was the first Tory prime minister since Disraeli to be born outside the circle of rich, Anglican, landed, aristocrats or wealthy businessmen. His acceptance by the Tories can be attributed to his Oxford education, wartime service and the need for the party to rid itself of its archaic, grouse-moor image of previous premiers, Macmillan and Lord Douglas-Home.
Harold Wilson, 1963 (1916–95), UK Prime Minister, 1964–70, 197–76 (Transcript of interview first published in The Observer from Harris, K. (1967) Conversations. Hodder and Stoughton. pp. 266–86.)
Harris: Some men go into politics almost as a matter of course and some form of personal Damascus. Why are you in politics?
Wilson: I suppose the short answer is because politics are in me, as far as I can remember. Farther than that: they were in my family for generations before me, as they were in the families of dozens of members of the Labour party. The first time that I can remember thinking systematically about politics was when I was seven. I was in hospital with appendicitis. My parents came in to see me the night after my operation and I told them not to stay too long or they’d be late to vote – for Philip Snowden. Then when I was 10 I went to West Australia where my uncle was an Australian MP and later President of the Upper House. That was my first visit to a politician.
Harris: Why was your family so politically minded?
Wilson: They were non-conformist by religion and radical by temperament. The Lib-Lab tradition. The day after the 1906 election results came out in Manchester, my grandfather – he was a Sunday school superintendent – chose the hymn: ‘Sound the loud tumbrel
Analysing Qualitative Information
o’er Egypt’s dark sea! Jehovah hath triumphed, his people are free’. It’s the old story of the pursuit of religious freedom and indignation with social injustice combining to conflict with the established social order. My other grandfather, too, was a deeply religious man who believed that politics represented the nation’s application of religious principles. In my childhood, it was chapel and the scout movement, that kind of pattern. My wife’s the daughter of a Congregationalist minister – I met her at a sports club. So I was impregnated with nonconformity. It was the soil out of which the Labour party grew. Incidentally, there are plenty of second-generation nonconformist radicals in the Labour party, men and women whose approach to politics stems from the religious values their parents planted in them. There are a lot of first generations, too. I don’t think that you can understand the Labour party if you don’t bear that in mind. Or the trade union movement. Or the Co-op[erative Movement].
Harris: Does the religious side of the inheritance mean anything to you today? Are you a religious man?
Wilson: I have religious beliefs, yes, and they have much affected my political views. But I’m no theologian…
Harris: You talk of the nonconformist radical influence on the development of your political views. How much has Marx influenced you?
Wilson: Not at all. I’ve studied the subject as history: you can’t understand the Russians without it. But, quite honestly, I’ve never read Das Kapital. I only got as far as page 2 – that’s where the footnote is nearly a page long. I felt that two sentences of main text and a page of footnote were too much.
Harris: Who has influenced you?
Wilson: Well, as I have said, my parents and the Scout movement … but there was another indirect influence. The two men who influenced me most directly were schoolmasters. One of them is still alive … a lifelong socialist. Not by argument, but by example. He was
245
246
Research Methods in Politics
a great teacher – unselfish and unstinting. When I was fourteen I caught typhoid, and missed two terms at school.When I got back to school, I was miles behind – miles. It looked hopeless. I reckoned I could catch up in things like history. But not in mathematics. The second day of term he called me up to his desk. He said: ‘if you put in an extra hour in the afternoon after school closes, you should be able to make it all up this term. If you’re willing, I am’. I did.
Harris: Was the other teacher a socialist?
Wilson: Yes, a much younger man. He died very young …
Harris: You seem to have a high regard for teachers.
Wilson: Coming from my kind of background, the teachers were the most important adults in your life. And my mother and my sister were teachers, and two uncles. Politics is education.
Harris: I’ve taken you off the point. I was asking who influenced you?
Wilson: At Oxford, my tutors: for example, my tutor in politics, R B McCallum, who is Master of Pembroke now. A Liberal politically. He taught me so much about the mechanisms of politics, such as parliamentary standing orders, the works of the public accounts committee and so on. When I became a don at University College, there was G D H Cole, a socialist. Both had a lot of influence on my ideas, and on the training of my mind.
Harris: Who had the most personal influence on you at Oxford?
Wilson: Beveridge. I was his assistant – his research assistant – for a couple of years before the [1939–45] war … Then there was Attlee. A great prime minister. The story of Attlee has yet to be told, though I think that the public has been getting a fuller notion of his real contribution to history in the last four or five years.
Analysing Qualitative Information
Harris: Is it true that as leader of the party you are modelling yourself on him?
Wilson: I would like to think that I was. I learnt a great deal from him. I think that I’m driving the party at a faster rate than Clem [Attlee] did – at the moment. His method was to sit and listen and then say: ‘We all seem to want it this way’. I talk more, far too much in fact, put more cases, am more positive. But the circumstances are different …
Harris: Did anybody else influence you personally? How?
Wilson: Nye [Aneurin Bevan]. It’s very difficult to abstract from Nye’s influence on me. It was the whole man, you see. He taught me the power of the public platform. I wish I had a tenth of his power on the public platform. And he corrected my interest in detail. Nye always took the broad view. In many ways he was lazy, and avoided detail, if he could, anyway, but he had the gift of instinctively seeing the horizon. He never failed to see the wood for the trees. I regard him as the best-educated man I have ever met, particularly in philosophy. He could see everyday political issues in the deep human perspective and he could communicate them simply and instantly to anybody.
Edward Heath, 1966 (1916–2005), UK Prime Minister, 1970–74. (Transcript of interview first published in The Observer from Harris, K. (1967) Conversations. Hodder and Stoughton. pp. 255–65.)
Harris: What made you a Conservative in the first place? Are you a Conservative because your father was a Conservative?
Heath: No, in more ways than one. The other day my father said that he had been a Liberal. No, I certainly didn’t become a Conservative because my family were Conservative. I’m not
247
248
Research Methods in Politics
sure what they were. We hardly ever talked about politics in the family because as a family we weren’t particularly interested in politics.
Harris: What were your parents like?
Heath: My father was a very likeable and good man, and a good craftsman – a carpenter. I was born in Broadstairs [Kent] but during the First World War he went to work at Vickers – woodwork for aircraft – and he came back to Broadstairs when I was seven or so, and the family with him. I still have a sense of everything beginning when I came back to Broadstairs at the age of seven. My mother was a fine character with a strong personality, a high sense of morality and public responsibility, but none of which she projected on anybody, including the family. She had the spiritual sense of her Christian faith, not that she was all ethereal about it – she kept her feet on the ground. I think that she was rather a remarkable woman, because she had high standards and strong views and yet she was a wonderfully forbearing mother towards her children. For instance, when I was thirteen I went to Europe on my own, or with one or two other boys, and I only discovered later how much she worried about me without letting me ever know.
Harris: Even if your parents weren’t politically minded, do you think that the home background had any effect on you becoming a Conservative?
Heath: Oh yes. I am what I am politically because of a combination of home background and the times I was living in as a boy and a certain amount of thinking about both. Living in Broadstairs [Kent], we didn’t see the mass unemployment they suffered in S Wales, Scotland, N Ireland and parts of England. We knew about it, and felt about it, but it didn’t stamp itself upon our minds as it did on some of the people who lived in the middle of it. But conditions were tough for nearly everyone nearly everywhere, and I saw my father working hard, showing enterprise, taking risks, and gradually developing his capacity in spite of very difficult circumstances. I had the feeling then – and I have it more clearly now – that father could not have achieved what he did – I’m thinking about personality, and capacity, not income or status – if he had been left free to do so. My mother’s influence – it follows from what I’ve said already – had
Analysing Qualitative Information
to do with freedom too. She left us boys free – to go one’s own way, and because she did that, I had the sense of moving forward, growing up, without the feeling that I was doing it against my parents. On the contrary, I was doing it with them, because of them, and that one’s parents were behind one – which makes such a difference. As I got to 17, 18 years old, the country as a whole was moving out of the period in which it was preoccupied with the suffering of the unemployed – things were looking up – and moving into the threat of dictators. This was the moral dimension – of my youthful associations with the attractions of freedom. By the time I went up to Oxford, if there was one political idea in the broadest sense of the term which governed me it was a deep feeling about and for the idea of freedom. I was very conscious of this. I joined all the three parties on my first term at Oxford, to see what they were up to, but by then I had accepted that I was a Conservative as part of accepting, so to speak, the facts of life.
Harris: Who influenced you most at Oxford?
Heath: Just after I got to Balliol – I was an organ scholar there – they were putting a new organ into the chapel, and, partly because of this, and partly because I later became President of the Junior Common Room, I saw a great deal of the Master of College, A D Lindsay.
(Harris: A socialist).
Heath; yes, and in a way because of it he strengthened my own Conservatism. He was completely non-dogmatic and a non-doctrinaire thinker and doer. Any way, what influenced me most about him was that he was a great believer in the democratic expression as central to the whole system of political democracy, which in term, he thought the only political system which enabled men to apply the freedom which is the prerequisite for doing the only thing in life that really matters – the chance for each man to live a full life – not a prescribed full life, but his own full life …
Harris: Oxford, then, was a very formative experience for you?
Heath: Yes, formative, inspirational. But, as I say, it shaped and gave a clearer meaning to what … But it also started processes for me. It wasn’t merely that I went up to Oxford but
249
250
Research Methods in Politics
that I went to Balliol. Balliol is a college of high academic standards and strong democratic traditions where old Etonian sons of dukes or millionaires are expected – and expect – to meet on equal terms, well, the son of a carpenter. Balliol opened all the doors to me …
Harris: You were quite a prominent Christian at Oxford, weren’t you? How much of that remains with you? Does Christianity mean something fundamental to you?
Heath: Good lord, yes. It’s been the same all along. I was brought up in a Christian home, as they say in the obituaries, and at Oxford – well, I just went on as I do today … I believe in God, very much the kind of God, I would say, that ministers of the Church of England would believe in. I believe in Christ and in the divinity of Christ. I believe in an after-life …
Harris: How far does your religion come into your politics?
I don’t think that there should be too much talk about religion and politics. And I think that Temple [war-time Archbishop of Canterbury, 1942–44] was right: the Christian should not aim at programmes of action. It’s the declaration of values and principles and inward personal attitudes that come from faith …
Harris: Apart from your religion, has any system of ideas, or any particular person’s ideals, had much influence on your views?
Heath: Yes, a great many. I don’t get much time to read now outside of what I have to read, but I’m a reader by temperament and when I’m free I read a lot. But it isn’t the people I’ve read most of that come to mind. It’s more what I’ve read about them. Disraeli, for instance. His one-nation approach, his refusal to be distracted from the search for the national interest by the existence of class and sectional interests. I found that profoundly sympathetic. And I admired him because he didn’t come from the top men of politics of its day, and he was opposed by them in his early political life and had to struggle to preserve his confidence in himself, let alone succeed in getting the confidence in others.
Analysing Qualitative Information
Locke also comes to mind too. Locke’s ideas I found gave me a base on which to think about my own – his emphasis on liberty enshrined in political institutions based upon experience and judgement, not on doctrinaire theories and utopias.‘No man’s knowledge can go beyond his experience.’ And his view of the state. It exists for men, not men for the state. Indeed, it’s only a convenience, the state, to deal with the problems you’d have if each man tried to be a judge in his own case. And I admire the cast of his mind, Christian, calm, temperate, good-humoured.
Codes You have chosen and designed the following generic, theoretical codes: Variable
variable
variable
variable
Education Sunday school prep school secondary school sixth-Form university teachers and tutors Family Father Mother Sibling Grandparents Uncles and Aunts Wife Children Others Religion Non-conformism Methodism Church of England Ministers Lay preachers Politics Labour Conservative Leading Members MP
E Ess Ep Es Es6 Eu Et F Ff Fm Fs Fg Fua Fw Fc Fo R Rn Rnm Rcoe Rp Rlp P Pl Pc Pm Pmp
251
252
Research Methods in Politics
variable
variable
variable
Life Stage Child Student Young adult Middle-age Old age Events/Experiences Depression World War Overseas visits Voluntary Bodies Trade Union Co-op Scouts Sports clubs
L Lc Ls Lya Lm Lo EX EXd EXww EXo V Vtu Vcoop Vsc Vsp
The codes can now be applied to the text. This is shown in Table 17.1. Note how landscape format has been used to transcribe the text. This effectively condenses the text and provides greater continuity from page to page in both its printed form and on-screen display. Note how line numbers have been added to your transcript to enable you to cross-reference quotations later to their original source. You will quickly find out that this first-time (first-pass) coding is timeconsuming – especially if you code to screen. But it will save you time later. When you apply the codes, you will learn whether they capture all the independent variables that you identify in the text. You may decide that additional, in vivo codes would be useful. Two variables for additional coding might be ‘significant others’ and ‘music’. Examine your coded sheet. You will see that substantial parts of the transcript have been coded. This shows that the material is an information-rich source. Note also how many passages have been coded many times. For example, the part of the sentence lines 012 and 013 attracts three codes (Fg, Ess, Rlxp). These codes show the concurrence of family (grandfather), education (Sunday school) and religion (superintendent). This appears to confirm your original hypothesis of family, religion and education as causal variables of political consciousness. However, what is missing from this and other parts of the Wilson interview is any mention of contemporary events, despite his childhood in the late 1920s in a northern UK mill town. He makes no reference to the Depression, the rise of Fascism and the appeal of Communism. His politics appear to reflect an entirely historical rather than experiential basis. Compare this with Heath’s account. Heath describes the influence of experiencing his own family’s hardship, reading of unemployment in the depressed areas, the growth of Fascism and joining all the political parties at university to experience their appeal. Perhaps this comparison may explain the greater sense of
Harris: Why was your family so politically minded?
010
Wilson: They were non-conformist by religion and radical by temperament. The Lib-Lab tradition. The day after the 1906 election results came out in Manchester, my grandfather – he was a Sunday school superintendent chose the hymn; ‘Sound the loud tumbrel o’er Egypt’s dark sea! Jehovah hath triumphed, his people are free’. It’s the old story of the pursuit of religious freedom and indignation with social injustice combining to conflict with the established social order. My other grandfather, too, was a deeply religious man who believed that politics represented the nation’s application of religious principles. // In my childhood, it was chapel and the scout movement, that kind of pattern. My wife’s the daughter of a Congregationalist minister – I met her at a sports club. So I was impregnated with nonconformity. It was the soil out of which the Labour party grew. Incidentally, there are plenty of second-generation nonconformist radicals in the Labour party, men and women whose approach to politics stems from their religious values their parents planted in them. There are a lot of first generations, too. I don’t think that you can understand the Labour party if you don’t bear that in mind. Or the trade union movement. Or the Co-op.
Wilson: I suppose the short answer is because politics are in me, as far as I can remember. Farther than that: they were in my family; for generations before me, as they were in the families of dozens of members of the Labour party. The first time that I can remember thinking systematically about politics was when I was seven. I was in hospital with appendicitis. My parents came in to see me the night after my operation and I told them not to stay too long or they’d be late to vote – for Philip Snowden. Then when I was ten I went to West Australia where my uncle was an Australian MP and later President of the Upper House. That was my first visit to a politician.
Harris: Some men go into politics almost as a matter of course and some form of personal Damascus.1 Why are you in politics?
003 004 005 006 007 008 009
001 002
Line Coding Number
011 012 013 014 Social and 015 political 016 engineering? 017 018 019 Is Wilson 020 being entirely 021 truthful or is 022 this post-facto 023 justification to appeal to readers?
His political career seems ‘written for him’. Didn’t he ever rebel against the orthodoxy or look at alternatives? No mention of Damascene events
Remarks
Table 17.1 Coding of Wilson interview (paras 1 and 2 only)
P , Vtu, Vcoop
Fg, R Lc, Rn, Vsc Fw , Rlp, Vsp Rn, Pl Pl , F , Lc
Rn Fg, Ess, Rlp
P F P , Lc Ef , Fm Pm, Lc EXo, Fa, Pmp
254
Research Methods in Politics
political conviction and obduracy displayed by Heath when he was prime minister in 1970–74. However, you must beware that these observations are being made from only two, relatively short interviews and with the benefit of hindsight. The next stage is to abstract the coded passages into a table whose columns are formed of the primary codes used. In your research, the outcome is membership of a particular political party. So place this dependent variable – politics - in the first (left-hand) column. The process of abstracting highlighted passages is relatively easily done by copying and pasting the highlighted passages and their codes. The line numbers are added in brackets to enable cross-referencing to their source. Where passages are multiply-coded, then they should be copied in each of the relevant columns and the relevant sub-sections highlighted. This is shown in Table 17.2. The next task is to look for patterns and clusters. In this context, a pattern is a repeated set of relationships. One of the simplest methods of identifying these is to compress the analysis table and to colour-code or grey-scale the contents. This is shown in Table 17.3. This tabulation enables patterns and clusters to be seen to emerge. For example, religion and family can be seen to be more strongly associated with politics. However, that represents only the first two paragraphs of the interview. Suppose that the tabulated coding of the full interview produced Table 17.4. You can see how education, events and significant others become associated with politics. If you wish, you can calculate the number of passages.
Second-level pattern coding Second-level pattern coding builds on the original coding by developing metacodes which group various interviews, etc. into sets, themes or constructs (analogous to cluster or factor analysis in quantitative analysis). This technique can be used to develop maps and flow charts illustrating the linkages interpreted between variables. For example, in the Wilson case, you could begin to sketch out the variables, overlaps and links. In this way, you can explore and demonstrate overlaps using a combination of text boxes, circles and arrows. The preceding text boxes demonstrate the sequence of influencing variables and how they relate to one another. What emerges from this analysis of the Wilson interview is the enormous overlaps between family, religion, voluntary organisation and early education. This suggests that they are visible manifestations of an underlying latent variable or factor. What might that be? One strong contender would be ‘class’ – northern, industrial working class (although Wilson described his parents as lower, middle-class). And what (from a Marxist perspective) is the determinant of class: the mode of production. You have therefore revived the explanatory power of class which was central to post-war UK political science.
Family
Religion
Education
I was ten when I went to Australia where my uncle was an MP (008) Non-conformist by Non-conformist by religion (011) religion My grandfather was a My grandfather was My grandfather was Sunday school a Sunday school a Sunday school superintendent (012) superintendent superintendent Other grandfather Other grandfather was a Other grandfather was a deeply deeply religious man was a deeply religious man who believed that religious man who believed that politics represented who believed that politics the nation’s politics represented the application of religious represented the nation’s principles (015) nation’s application of application of religious religious principles principles
My uncle was an MP
Politics… were in Politics … were in my my family; for family; for generations generations (004) Remember thinking about politics when I was seven
Politics
Table 17.2 Wilson interview: coded passages tabulated Events
Remember thinking about politics when I was seven (005) I was ten when I went to went to Australia Australia
Life Stages
(Continued)
Vol. Organisations
So I was impregnated with nonconformity. It was the soil out of which the Labour Party grew. Incidentally, there are plenty of secondgeneration nonconformist radicals in the Labour party, men and women whose approach to politics stems from their religious values their parents planted in them.
Politics
Table 17.2 Cont’d Religion
Secondgeneration nonconformist radicals in the Labour party, men and women whose approach to politics stems from their religious values their parents planted in them. (019)
In my childhood, it was chapel and the scout movement, that kind of pattern My wife’s the daughter of My wife’s the a Congregationalist daughter of a minister – I met her at Congregationala sports club (017) ist minister – I met her at a sports club So I was impregnated with nonconformity. It was the soil out of which the Labour party grew (018)
Family
Education In my childhood, it was chapel and the scout movement, that kind of pattern
Life Stages
Events In my childhood, it was chapel and the scout movement, that kind of pattern My wife’s the daughter of a Congregationalist minister – I met her at a sports club
Vol. Organisations
Analysing Qualitative Information Table 17.3 Wilson interview: coded passages grey-scaled Politics
Family
Religion
Education
Life Stages
Events
Vol. Orgs.
The next task is to apply this potential explanation to the Heath interview. Heath attributes the appeal of Conservative politics to a ‘combination of home background and the times’. He extends the influences to ideas of freedom. But perhaps the most telling passage is his description of his father: ‘. . . working hard, showing enterprise, taking risks, and gradually developing his capacity in spite of very difficult circumstances’. His father was a self-employed carpenter working in a predominantly rural county. In short, the Heaths were petty-bourgeoisie and church – rather than chapel – goers. However, the evidence is not wholly conclusive. But both appear to have conformed to the political norms of their social class. Another step would be to prepare a comparative table drawing together material from the texts. End stage The last stage in your analysis is to construct a typology – an explanatory, conceptual framework which represents graphically all the cases within two, unrelated variables or, preferably, factors. It therefore provides a very powerful display of your data and analysis and provides a basis from which you can develop or extend theoretical explanations. In terms of explaining the causal factors of political identity, you might construct a typology using the factors of class-of-birth and conformity where wider experience was likely to lead to individuals rejecting the party of the class into which they had been born. Footnote However, there are no easy fixes in coding and its analysis. You will have to explore and prepare different ways of ‘differentiating and integrating’ your information until
257
258
Research Methods in Politics
Table 17.4 Wilson interview: tabulation of complete interview Politics Family Religion Education Life Stages Events Vol. Orgs. Sig. Others Music
Analysing Qualitative Information Table 17.5 Calculated number of passages Politics Family Religion Education Life Stages Events Vol. Orgs. Sig. Others Music 29
16
15
21
3
8
2
14
0
Family Religion
Education
Voluntary organisations
Significant others
Figure 17.1 Graphical representation of relationship between variables
some signs of clarity appear. Be prepared to invest considerable time in the process: as much as (if not more than) you spent collecting and transcribing the data. Other methods of analysis These include memoing, case analysis meetings, interim case studies and pre-structured cases (Miles and Huberman, 1994: 73–85).1 Memoing Writing a memo generally means writing an informal note to yourself or colleagues of new information. In the context of research, memoing has a more specific meaning: the theorising write-up of ideas about codes and their relationships as they strike the analyst while coding … it can be a sentence, a paragraph or a few pages … it exhausts the analyst’s momentary ideation based on data with perhaps a little conceptual elaboration. (Glaser, 1978: 83)2
259
260
Research Methods in Politics Table 17.6 Coding: Wilson and Heath compared Code
Wilson
Rn Nonconformism
Allied to social justice; soil for Labour party
Heath
Rcoe Church of England
Mother’s spiritual sense of Chr faith *[his] belief in God that ministers of CoE share
Rnm Chapel Rp Ministers Er School (Religious music)
Essential part of childhood Father-in-law (Implied) Chapel hymns reflected political events
F f Father F m Mother F w Wife F o Other relatives
(No specific mention) Teacher
E t School teachers
Two: most important adults in your life R B McCallum G D H Cole Beveridge Attlee Bevan (Nye)
E t Tutors, etc.
So Significant others
Organ scholar
A good man, craftsman, self-made Fine character, strong personality (never married)
Uncle, grandfathers
A D Lindsey
Disraeli ‘one-nation’ Locke on liberty and the state
In other words, when you have a Eureka moment, insight or brainwave during your coding and analysing of text, write it down and continue with your activity. Copy the memo to your colleagues or supervisor for their views. Return to your memos when you have finished your coding. Many of your bright ideas will prove false dawns. But you will not have forgotten them and have evidence of all the leads that you explored and discounted.
Table 17.7 An analytical typology
Class
Conforming
Non-conforming
Conforming working class (Wilson) Conforming middle class (Heath) Conforming upper class (Macmillan)
Nonconforming working-class (D. Davis) Nonconforming middle class (Attlee) Nonconforming upper class (Mosley)
Analysing Qualitative Information
Case analysis meetings Case study meetings are similar to supervision meetings or peer-reviews. They are designed to seek formalised discussions on your analysis so far. They are particularly valuable to review progress against timetables, or to talk-through problems, new opportunities or unexpected findings. Interim case study Miles and Huberman strongly recommend researchers to write a 25-page interim case study report to: ‘prevent the nightmare of bad or opaque data, systematic error, or blindingly obvious or trivial conclusions’. You should complete this when you are one-third the way through the research (comparable to the ‘up-grading’ review procedure through which postgraduates must pass before the initial MPhil registration can be raised to DPhil/PhD). This interim report should demonstrate the type of data and information that you have collected and the effectiveness of your analytical methods. If you are unable to demonstrate a functional relationship between the data and its analysis, then clearly you must consider collecting different types of data or adopting other methods of analysis – or both. Questions for discussion or assignments
1. Consider and discuss the transcripts of the interviews with Harold Wilson and Edward Heath. On the basis of your theoretical understanding gained from other courses, what causal influences on their political choices would you suggest? Copy them from www.sage.co.uk/pierce. Develop your own generic codes and apply these. What additional, in vivo codes would you suggest? Tabulate the codes and extracts for Wilson and Heath. Tabulate a comparison of the transcripts. Develop a typology of your own. What inferences would you suggest? 2. Visit www.bbc.co.uk On the home page, enter ‘Paxman Interviews’. Select an interview with a UK political elite. If in doubt, use interview with Tony Blair on 27 April 2005. Click on ‘printable version’. Download, transcribe and code. Note that you can also watch video recordings of many BBC interviews.
FURTHER READING Gubrium, J. F. and Holstein, J. A. (1997) The New Language of Qualitative Method. Oxford: Oxford University Press. p. 244. This concise textbook explores the ‘new language’ of analysis within the discipline of sociology and the perspectives of naturalism, emotionalism, ethnomethodology and postmodernism.
261
262
Research Methods in Politics Miles, M. B. and Huberman, A. M. (1994) Qualitative data Analysis. London: Sage. p. 338. This US textbook provides detailed guidance on coding of text and its analysis. The authors pay particular attention to the value of matrices, tables and charts in data analysis. The book draws its examples from US public policy practice, especially education reform. Ritchie, J. and Lewis, J. (eds.) (2003) Qualitative Research practice: A guide for social science students and researchers. London: Sage. p. 336. This is a well-written and readable textbook which is written for a readership of all social science disciplines. It draws on examples from UK practice. However, only two chapters describe the analysis of qualitative data. The analytic hierarchy shown on p. 212 is invaluable to understand the process of analysis. Silverman, D. (2001) Interpreting Qualitative Data: Methods for analysing talk, text and interaction. London: Sage. p. 325. This is an excellent monograph by a leading UK academic. He promoted three models for interpreting interview data: positivism, emotionalism and constructionism. Methods used include content and discourse analysis. Exceptionally, the textbook includes a chapter (Chapter 7) on the analysis of visual images.
Notes 1 Miles, M. B., and Huberman, A. M. (1994) Qualitative Data Analysis. London: Sage. 2 Glaser, B. G. (1978) Theoretical Sensitivity: Advances in the Methodology of Grounded Theory. Mill Valley: Sociology Press.
Chapter 18
Using Content Analysis
Teaching and learning objectives:
1. 2. 3. 4.
To consider alternative definitions of ‘content analysis’. To compare qualitative and quantitative content analysis. To consider the sub-types of content analysis. To learn how to carry out content analysis using latest available software.
Introduction Content analysis is also called textual analysis and, somewhat pejoratively, text mining. In this context, content is words, texts, transcripts of speeches or conversations, pictures, ideas, themes or messages. Content analysis is concerned with the bits and pieces – words – of communication rather than the generality. It is a long-standing tradition and was widely used by church authorities to search out heresies and to prosecute heretics. Max Weber encouraged its use, in Weimar Germany, to monitor and compare press reports – in terms of coverage and bias – of his Social Democratic Party. During the Cold War, content analysis was widely used by Kremlinologists and Sinologists to monitor the stream of radio traffic, official reports and statements from USSR and Chinese sources. Its use has become even more widespread as a result of new software designed to automatically scan all telephone and emails etc. to detect key words as part of the ‘war on terrorism’. In a sense, content analysis has become part of the apparatus of the vigilant – or oppressive – state and a source of increasing paranoia among political activists. It is a ‘key concept’ of the new popular discipline of media studies (O’Sullivan et al, 1997: 62).1 In this context, text is considered to have an independence from its sender or intended receiver. It can be any message written – visual, spoken or sung – as a medium for communication including books, newspapers, advertisements, speeches, official documents, films of video, musical lyrics, photographs, clothing (especially T-shirts), graffiti, works of art and national anthems.
264
Research Methods in Politics
The advantages to researchers of using content analysis include: • ‘there’s plenty of it’: widespread availability of texts – especially on the Internet • greater ease of scanning or downloading text for analysis • relatively low cost • simple, covert research method which minimises the research effect: (Hawthorne, etc.) • distance from subjects • quantifiability • consequently, claims to objectivity.
On the other hand, the criticisms that can be made of content analysis include: • • • • • •
researcher bias in the selection of texts conceptual assumptions are highly contestable application of Boolean and other mathematical software is bad science problem of identifying the population of texts and, therefore, of obtaining a random, representative sample of texts heavy promotion by commercial interests benefits are exaggerated by users.
I regard content analysis as a potentially useful but supplementary tool of analysis.
Qualitative and quantitative content analysis Content analysis may be qualitative or quantitative. Both can use the same body of textual and other data. Qualitative content analysis is highly interpretive. It essentially involves the reading of texts, etc. to determine the extent of bias in terms of supportive, critical or (more or less) neutral accounts of organisations, institutions, concepts or figures. In York, it was used by students over many years to analyse the bias of British print media towards the Loyalists and Nationalists in Northern Ireland. The approach here, as elsewhere, is to select a sample of texts over a long period of time for independent reading and analysis by a number of researchers whose own biases are recorded at the beginning of the exercise. The assessments are then compared. Alternatively, qualitative content analysis can be used to compare different perspectives on the same topic by different speakers. In 2004, as an exercise at the Essex Summer School, I compared the interpretations of ‘democracy’ of world leaders in their speeches to the UN following 9/11. My analysis is shown below. You will note the differences in the meanings of democracy adopted and promoted by the US President and European leaders. Note also how Putin adopts an instrumental view of democracy.
Democracy
opposite of communism
+
choose raise families vote assemble worship
fear tyranny censorship taxes
Liberty
responsibility to defend greater vigilance, security military action sacrifice intervention BURLESCONI response pre-emption freedom and democracy can't be neutral defence of human rights peace and development freedom from totalitarianism route from poverty
+
PUTIN market economies decent standard of living choice European humanism anti-totalitarianism myth of partnership (NATO) caliphate
SCHROEDER institutions to guarantee human rights popular participation shared trade sustainable development disarmament and non-proliferation full state sovereignty social and material security minority rights
CHIRAC liberty responsibility freedom and dignity multilateralism state sovereignty rule of law under UN equal dignity of all cultures respect for diversity dialogue
Figure 18.1 Interpretation of world leaders of ‘democracy’ in speeches to the UN following 9/11
God
from freedom to
BUSH
BLAIR cooperation human rights peace and prosperity peace and development peace and prosperity hope rule of law respect for others liberty rather than freedom stability partnership uniting disparate groups tolerance justice hard work offering asylum to refugees merit worth
266
Research Methods in Politics
This rough and ready approach has been replaced by the development of discourse and narrative analysis as the principal method for the qualitative analysis of text.
Quantitative analysis Most content analysis is now entirely quantitative. It counts the incidence and frequency of words. Its theoretical underpinning is provided by Zipf’s Law – an experimental law developed by George Zipf (1902–50), Harvard professor of linguistics. Zipf’s Law is, technically, a power-law distribution. The law states that the frequency of occurrence of some event (P), as a function of the rank (i) when the rank is determined by the above frequency of occurrence, is a power-law function Pi ∼ 1/i a with the exponent a close to unity. What this means is that, in any body of text, the frequency of any word is inversely proportional to its rank in the frequency table (of most commonly-used words). In English texts, the word ‘the’ is the highest ranking word. So, in any text, it is likely to be the most frequently used word and, indeed, is used twice as often as the next, highest-ranking word. What Zipf means in plain English is that, where uncommon words (or phrases) are used often in a text, then they express and reflect the greatest concerns of the communicator. These are termed key words. For example, a Zipfian analysis of G. W. Bush’s presidential inaugural address of 20 January 2005 shows that, in a speech of only 2,083 words, the key words used were: Key word freedom liberty hope history tyranny God
Frequency 27 15 8 7 5 4
However, the word ‘Iraq’ was not mentioned at all. This example readily demonstrates the great weakness of content analysis as a tool of Politics research: that the key words used by political elites (and their speech-writers) do not necessarily express what may be their greatest concerns. Hence the criticism made of Margaret Thatcher by a cabinet colleague that: ‘The trouble with Margaret is that, when she speaks without thinking, she says what she thinks’.2 Indeed, the key words may be used deliberately to conceal or divert attention from real policy concerns. Political speeches have become carefully-planned, ‘communications events’ whose contents will have been analysed to ensure that the ‘right words’ are used the ‘right’ number of times and ‘resonate’ (appeal) to the target audiences. However, paradoxically, content analysis by others can enable omissions to be identified.
Using Content Analysis
You can carry out your own simple content analysis by copying texts of speeches from the Internet and then using the Edit/Replace function in Microsoft Word. First, identify your key words. Then ask Word to replace all your first key word by, say, xyz. Word will tell you how many occurrences have been replaced. Quantitative content analysis can be either structural or substantive. Whichever you adopt, you must firstly clearly identify the population of texts and sample frame and justify fully the (random or non-random) sampling method you adopt.
Structural content analysis Structural content analysis is primarily concerned with how the text is presented and reported rather than the frequency of key words used. It seeks to measure: • • • • • •
space (or time) devoted to the text volume: headline type and font size position or prominence of the text use of accompanying illustrations or photographs indications of direction or bias (for example, by stereotyping) persistence over time (for example, Daily Mail ’s campaign over ‘asylum seekers’ and Daily Express’ pre-occupation with Princess Diana)
Structural content analysis is widely used as a comparative technique by media watchers. It is also employed by political parties to make complaints of bias against public broadcasters and thus to secure better, corrective coverage. But the public broadcasters also use structural content analysis in the design of their programmes to rebut complaints. One very useful, simple and instructive exercise that you can perform is to track and compare the front page editions of all the national newspapers over one week. Note how the choice of front-page stories varies between the ‘quality’ (Times, Guardian, Independent, Telegraph) and popular, ‘redtop’ newspapers. Then compare them with the headlines on the BBC, ITN and Sky Internet news sites. For example, the headlines for 13 January 2007 are: BOX 18.1 National Newspapers, circulation (000s) and headlines, 13.01.2006: TV News headlines, 13.01.2007 The Times The Guardian Daily Telegraph
635 365 899
NHS faces treatment rationing Revealed: the 11 government ministers fighting NHS cuts Brown’s manifesto for Britishnesss (Continued)
267
268
Research Methods in Politics
The Independent Daily Express Daily Mail The Mirror The Sun The Star TV news BBC News ITN news Sky News
238 Shoot the messenger: Blair blames press for anti-war mood 773 Big new tax on house owners 2,311 Now a school bans crucifix 1,540 Big brother shot my dad dead 3,026 Shoot him, mother’s anger at beast who raped girl, 3 750 Big Bro on skids (audience for post-watershed news) 5,000 Reid moves to quell offender row 3,000 Boys killed by Tube train 2,000 Men killed in Tube tragedy
Source: Audit Bureau of Circulation www.abc.org.uk, BBC, ITN and Sky News web-sites
Substantive content analysis Substantive content analysis analyses selected texts by counting the frequency and distribution of key words. More advanced, software-based techniques use Boolean search parameters which enable you to refine your research by counting the frequency and distribution of combinations of key words. A very good example of traditional, substantive content analysis is provided by Budge’s use of the method to analyse changes of UK party policy and ideology in the period 1945–97 (Budge, 1999: 1–21).3 He uses the party manifestos published by the major parties as election programmes which he argues are: not widely read by the British public.Their importance is that they are read by the political and media elite and reported intensively in newspapers, TV and radio. Thus their textual emphases set the tone and themes of campaign discussion (1999: 2).
He explains that the task of analysing the data is simplified by the coding of all Western European manifestos since 1945 by the Manifesto Research Group of the European Consortium for Political Research (ECPR). The Group uses a base of 46, major, policy areas which are coded generically. They include: 101 Foreign special relationships: positive; 102 Foreign special relationships: negative; 103 Decolonisation; … 706 Non-economic demographic groups. Each sentence is coded and counted under one policy area. The percentage of sentences devoted to each policy area is then calculated. In this way, the coverage given to any policy area can be plotted over time by extending the analysis over all of the parties’ manifestos in chronological order. Budge extended the research by analysing the movement of manifestos across the political spectrum to establish how far (and often) the parties have changed their
Using Content Analysis
ideological positions and appeals to the electorate. He adopted the left-right coding scale developed by Klingermann et al (1994).4 This is shown in Table 18.1. Table 18.1 Left-right coding scale Left-Wing Emphases (Sum of %s for)
Right-Wing Emphases (Sum of %s for)
Decolonisation Anti-military Peace Internationalism Democracy Regulate capitalism Economic planning Pro-protectionism Controlled economy Nationalisation Social services expansion Educational expansion Pro-labour
Pro-military Freedom, human rights Constitutionalism Effective authority Free enterprise Economic incentives Anti-protectionism Economic orthodoxy Social services limitation National way of life Traditional morality Law and order Social harmony
Source: Klingermann, et al 1994: 40
Left wing (−) / Right wing (+) scores for each party
Budge applied this scale by adding all the sentences in Klingermann’s ‘left list’ and subtracting them from all the sentences in the ‘right list’ to provide a scale between +100% (manifesto wholly ‘right wing’) to –100% (all ‘left wing’). The scores are charted in Figure 18.2.
40 30 20 10 Conservative
0 −10 0
20
40
60
−20
Liberal Labour
−30 −40 −50 −60 General Election years (where 1945 is 0)
Figure 18.2 British parties ideological movement on a left-right scale, 1945–97 Source: Budge, 1999: 5, Figure 11.1
269
270
Research Methods in Politics
Budge interpreted the chart to argue that: Labour moves sharply rightwards from 1992 and for the first time in post-war history shows a preponderance of right wing positions over left ones positions (+5%) … Labour moved rightwards and ‘leapfrogged’ over the [Liberal–Democrats] … in relative terms, Labour became the most centrist party. (Budge, 1999: 6)
You may criticise Budge’s method for its reliance on assumptions that you regard as very problematic (e.g. the categorisation of ‘human rights’ as a right-wing phenomenon). However, you must accept that his analysis does appear to chart accurately the ideological emergence and drift of New Labour to the (more electable) centre-ground. However, this very good example of effective content analysis also demonstrates that the method is not a quick fix. In this research project, it has involved reading and coding 42 election manifestos. So structural content analysis can be very laborious. And users can also be criticised for substituting hard labour for the less-laborious but more intellectually demanding methods such as discourse and narrative analysis (see Chapter 19). Content analysis software New software removes the hard labour of traditional content analysis but compounds criticisms of its use as a principal research method. There are now a large number of English language software programs for quantitative content analysis. They include: AutoMap; CatPac; General Inquirer (Harvard); Hamlet II; Leximancer; TACT; Textpack (Cologne); Texstat (Berlin); VBPro; Wordsmith; WinMax. You can find reviews of and information on many of these programs at: http://lboro.ac.uk/research/methods/ research/software/stats.html Each of these types of content analysis software perform a wide range of functions.5 The core functions include: • • • • • • •
calculating word frequencies excluding stopwords (‘the’, ‘and’, ‘in’ etc.) automatically adopting lemmitisation to combine words with the same stem, e.g. go, going, gone using synonyms to categorise as one word all others having the same meaning, e.g. gone, quit, departed, etc. recording concordance by showing each word in its context (termed KWIC: key words in context) using cluster analysis to group together words used in similar contexts using co-word citation to identify the concurrence of key words. This is used by the US government’s Echelon Project to scan emails for terrorist activity. So, emailers using a
Using Content Analysis combination of, say, ‘bomb’ and ‘Islam’ are likely to automatically attract the attention of the security services.
The software is designed to analyse relatively short texts of up to 10,000 words. However, no coding is required. All you have to do is to enter the text you wish to analyse and the key words. Your selection of text and appropriate key words is therefore critical to the effectiveness and value of the analysis. Many other programs designed primarily for coding and analysing texts can perform simple content analysis. They include ATLAS.ti, Nud*st and SPSS for text. Hamlet II Hamlet II can be recommended for a number of reasons. First, it was developed uniquely by a political scientist, Dr Alan Brier of Southampton in conjunction with the foremost continental researchers, Ekkehard Mochmann and Bruno Hopp of the Central Archive for Empirical Social Research in Cologne. Second, the software provides supplementary graphical displays of completed analysis in the form of dendograms and three-dimensional (Minissa) displays (which can be rotated). These dendograms and displays can illustrate what might otherwise be uninteresting tables and bring welcome, additonal interest to your research report. Third, you can download a free, 30-day trial copy of Hamlet II from www.apb.cwc.net. A free tutorial guide is also available. Hamlet II offers a number of analytical procedures including: Joint Frequency Analysis simple cluster analysis multi-dimensional scaling PINDIS (Procustean Individual Difference Scaling) KWIC (key words in context) Wordlist Compare (to compare two or more texts) Profile (which displays the distribution of words and sentence lengths)
To provide an example, Hamlet II has been used to complete a content analysis of the full text of President Bush’s State of the Union Address 2005. This was downloaded from the White House’s Internet site at www.whitehouse.gov/news/release/ 20050202 as a plain text file. A list of key words was selected from a quick skim of the speech. They were (in alphabetical order) better world, budget, democracy, economy, free*, freedom, God, history, hope, justice, liberty, sacrifice, tax, terror* and tyranny. (The wild card * is used to enable all words beginning with the chosen stem, e.g. free*, to be counted together.) No synonyms were used. However, the word ‘Applause’
271
272
Research Methods in Politics
(included in the transcript as ‘(Applause)’) was added to find out which key words generated the greatest support from Members of Congress. A demonstration copy of Hamlet II was installed and the instructions followed. The analysis was saved and copied and is shown in Boxes 18.2a, b, and c:
BOX 18.2a
HAMLET II analysis: word search
HAMLET II analysis: word search HAMLET - Computer-assisted Text Analysis - 17/01/2007 17:32:57 ================================================================= The text is read from the file: State of the Union Address 2005.txt Counting collocations within a span of 50 words WARNING : No collocates for "freedom" WARNING: some characters were not recognised when reading this file! WORD-SEARCHING IS INSENSITIVE TO CASE. There are 17 main entries in the search list. No synonyms / related items are recognised: CATEGORY/WORD COUNTS VOC.LST. Applause better world budget democracy economy free* God History Hope Justice Liberty Sacrifice Society Tax terror* tyranny
FREQUENCY % VOC.LST. % TEXT CONTEXT UNITS 67 6 3 8 11 27 1 5 3 4 7 1 5 6 27 2
36.61 3.28 1.64 4.37 6.01 14.75 0.55 2.73 1.64 2.19 3.83 0.55 2.73 3.28 14.75 1.09
1.30 0.12 0.06 0.15 0.21 0.52 0.02 0.10 0.06 0.08 0.14 0.02 0.10 0.12 0.52 0.04
67 6 3 8 11 27 1 5 3 4 7 1 5 6 27 2
Using Content Analysis
You will note that the most frequently used key words were free* (27), terror* (27), democracy (8), liberty (7), ‘better world’ (6), tax (6) and history (5). ‘Applause’ was recorded in the transcript on 67 occasions
BOX 18.2b 5162 words were read from the text file. 183 of these were in the search list, and collocations within up to 50 words were counted. JOINT FREQUENCIES ...................................... COLLOCATIONS within up to 50 words: i 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 +----------------------------------------------------Applause 1 | better world 2 | 4 budget 3 | 7 0 democracy 4 | 9 0 0 economy 5 | 20 3 2 0 free* 6 | 25 0 0 4 2 God 8 | 1 0 0 0 0 1 History 9 | 6 0 0 1 2 6 Hope 10 | 4 1 0 1 0 3 | 0 justice 11 | 7 0 0 1 2 1 | 0 0 liberty 12 | 6 0 0 3 1 9 | 1 1 0 sacrifice 13 | 1 0 0 0 0 0 | 0 0 0 0 society 14 | 4 0 0 0 0 1 | 0 0 0 0 0 tax 15 | 11 1 2 0 6 0 | 1 0 0 0 0 0 terror* 16 | 26 0 0 6 0 23 | 0 6 0 3 0 0 tyranny 17 | 2 0 0 2 0 3 | 0 1 0 0 0 0 4
273
274
Research Methods in Politics
BOX 18.2c
HAMLET II analysis: Joint Frequency
The joint frequency table clearly shows that ‘Applause’ was most strongly associated with terror* (26 times), free* (25 times) and economy (20 times). Other frequent associations are; free* with terror* (23 times); liberty with free* (9 times); history and free* (6 times); and, hope and free* (3 times). The probability of a specific pair of words being present in any pair of words is measured by the Jaccard coefficient. This is shown calculated in Box 18.3c: HAMLET II analysis: Jaccard coefficients STANDARDISED JOINT INDEX VALUES Jaccard coefficient - ignores joint non-occurrence i 1
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 +------------------------------------------------------Applause better world budget democracy economy free* God History Hope justice liberty sacrifice society tax terror* tyranny
1| 2| 3| 4| 5| 6| 8| 9| 10| | 11| | 12| | 13| | 14| | 15| | 16| | 17| |
0.06 0.11 0.14 0.34 0.36 0.01 0.09 0.06 0 0.11 0 0 0.09 0.01 0.06 0.18 0.10 0.38 0 0.03 0
0 0 0.21 0 0 0 0.13
0 0.17 0 0 0 0
0 0.13 0 0.08 0.10
0.06 0 0.14 0
0.04 0.23 0.11
0 0 0
0.20 0
0
0
0.09
0.15
0.03
0
0
0 0.09 0 0 0 0 0.09 0 0 0.25 0 0.25
0 0.11 0 0 0 0 0.29 0 0 0 0 0
0.25 0 0 0 0 0 0 0 0.21 0.10 0.25 0
0.06
0.36
0
0
0 0 0
0
0
0
0.03 0 0 0 0.74 0 0.12 0
0 0 0
0
0 0 0 0
0
0.55 0 0 0 0 0
0
0 0.16
Using Content Analysis
Figure 18.3 Dendogram
The dendogram and minissa display are shown in Figures 18.3 and 18.4. They demonstrate the functional relationship between the key words used and their relative proximity. The example demonstrates the main strengths and weaknesses of using content analysis software. The main strengths are: • • • • •
the ready availability of texts the scope for comparative analysis easy-to-use (with practice) analytical software good tabulations of results eye-catching illustrations.
The main weaknesses are: •
concurrence of key words does not necessarily indicate causal links
275
276
Research Methods in Politics
Figure 18.4 Minissa 3-D display
•
•
the meanings of key words change over time and can be deliberately changed by political parties who exploit ‘feel good’ or ‘feel bad’ words to make their policies more attractive (e.g. ‘community’, ‘homeland’) overall, the analysis is not really conclusive enough to provide sufficiently reliable evidence to confirm your research hypothesis.
Content analysis can therefore arguably be recommended only as a supplementary research method.
Questions for discussion or assignments
1. Where and when should content analysis best be used? 2. Compare and contrast two or more of the available software packages.
Using Content Analysis 3. Working in teams, complete a structural content analysis of last week’s national TV and print news media accounts of national and international news from Monday to Sunday. What inferences and conclusions can you draw? 4. Complete a content analysis of all President George W. Bush’s State of the Union Addresses. Compare the analyses. What are your main conclusions? What are their limitations? 5. Select two quality newspapers. Using CD-ROMs or Internet sources, evaluate their (emergent) bias for or against the Republican and Loyalist movements in N. Ireland between the Good Friday Agreement (1998) and the St Andrew’s Agreement (2006). What conclusions can you draw? What reasons can you suggest for any changes of support? 6. A comparison of the key words used by President Bush in his Inaugural Address (2,083 words) and State of the Union Address in 2005 (5,162 words) shows a variation in their frequency: Key word Freedom Liberty Hope History Tyranny God
Inaugural Address 27 15 8 7 5 4
State of Union Address 27 7 3 5 2 1
Are these differences significant? What alternative explanations can you give?
FURTHER READING Most textbooks of research methods in Politics mention content analysis but few give more than a few pages’ coverage. Krippendorff, K. (1980) Content Analysis: An Introduction to its Methodology. London: Sage. Despite its age, this remains an essential text for students contemplating the use of content analysis. Its early publication (1980) means that it pre-dates PCs and the availability of software. Hence, Krippendorff provides a very good, detailed account of the assumptions and processes adopted, and shows how these can be applied manually. Harrison, L. (2001) Political Research: An Introduction. London: Routledge. pp. 113–20, provides an authoritative account of the scope and application of content analysis. She provides a case study of the portrayal by UK news media of political leadership qualities.
277
278
Research Methods in Politics Burnham, P., Gilland, K., Grant, W. and Layton-Henry, Z. (2004) Research Methods in Politics. Basingstoke: Palgrave Macmillan. pp. 236–42, provides a well-referenced introduction to content analysis including two case studies. Budge, I. (1999) Chapter 1: Party Policy and Ideology: reversing the 1950’s? In Evans, G. and Norris, P. (eds.) Critical Elections: British Parties and Voters in Long-Term Perspective. London: Sage. pp. 1–21, provides an excellent example of the application of content analysis. The book also shows how content analysis can be used to scope and foreground other methods of analysis. Klingermann, H-D., Hofferbert, R. and Budge, I. (1994) Parties, Policies and Democracy. Boulder: Westview Press.
Notes 1 O’Sullivan, T., Hartley, J., Saunders, D., Montgomery, M. and Fiske, J. (1997) Key Concepts in Communication and Cultural Studies. London: Routledge. 2 Matthew Parris, The Times, 13 January 2007. 3 Budge, I. (1999) Chapter 1: Party policy and ideology: reversing the 1950s? In Evans, G. and Norris, P. (eds.) Critical Elections: British Parties and Voters in Long-Term Perspective. London: Sage. 4 Klingermann, H-D., Hofferbert, R. and Budge, I. (1994) Parties, Policies and Democracy. Boulder: Westview Press. 5 Brier, A.P. and Hopp, B. HAMLET a Multidimensional scaling approach to text-oriented policy analysis. In Journal of Diplomatic Language 2(1) (2005) (online at http://www.jdlonline.org// IIbrier.html), illustrates the comparison of a number of sources.
Chapter 19
Understanding and Adopting Discourse and Narrative Analysis
‘… the world is structured by discourse’ (Foucault) ‘We cannot get iffy over other people’s power-games with language, and then pretend we are not players in the game too’ (Curt, 1994:19)1 Teaching and learning objectives:
1. 2. 3. 4. 5. 6.
To understand some of the origins of the discipline of discourse. To consider and compare various schools of discourse analysis. To consider in greater detail what is meant by critical discourse analysis. To learn how critical discourse analysis can be applied. To discuss what is meant by narrative. To learn how to apply narrative analysis.
Introduction Discourse analysis is characterised, at one hand, by growing enthusiasm for its use by ambitious students and researchers and, at the other, by a variety of interpretations and advice from leading authorities. The paradox is that discourse analysis has many interpretations: there is a discourse on discourse. Some regard it as a method. Others profoundly disagree: they conceptualise discourse analysis as a discipline. It is, therefore, another ‘essentially contested concept’ (Gallie, 1956).2 Indeed, there are times when, faced with the wide range of differing authorities, you may be forgiven to likening discourse analysis to Churchill’s description of Russia as: a riddle, wrapped in a mystery, inside an enigma.3
280
Research Methods in Politics
In everyday speech, discourse is generally used to describe a discussion, conversation, talk or text. Discourse is essentially a ‘mode of communication’. However, within the social sciences, a variety of explanations are given. The most readily understandable and succinct are: •
language as social practice determined by social structures (Fairclough, 1989)4
•
systems of meaning, including all types of social and political practice, as well as institutions and organisations (Howarth, 1995)5
•
a representation of what we want the world to be like, rather than a representation of how the world is, the correctness of which [can] be tested (Carver, 2002; 5)6 a specific ensemble of ideas, concepts and categorisations that are produced, reproduced, and transformed in a particular set of practices and through which meaning is given to physical and social realities (Hajer, 1995: 44).7
•
In other words, ‘language is political’. Similarly, discourse analysis has many explanations including: • • •
a description for studies focusing only on linguistic units above the level of the sentence (Stubbs, 1983)8 analysing the way systems of meaning or ‘discourses’ shape the way that people understand their roles in society and influence their political activity (Howarth, 1995)9 an examination of the relationships between units of talk, writing, or other representational forms, and of the significance of these relationships for our subjective experience (Squire, 1995)10
•
discourse analysis does not look for truth – but rather at who claims to have truth (Carver, 2002: 53).11
Essentially, discourse involves language (rather than individual words). But the modes of discourse also include other sounds (a referee’s whistle), non-verbal communication (gestures, holding a child’s hand) and signs (for example, traffic lights). The primary uses of language are: • • •
ideational (the expression of ideas) referential propositional.
Additionally, language is used as a means of: • • •
emotional expression social interaction an instrument of thought
Understanding and Adopting Discourse and Narrative Analysis • •
expression of identity co-ordination and control of others.
Examples of the use of language for co-ordination and control include paradeground commands, children’s nursery rhymes and the litany of religious rituals. But the meaning of the language will be determined by the context and societal norms. Consider the example of a brick. A brick on a building site is a component of construction. A brick as part of a wall is a means of enclosure. A brick to an artist is a highly textured and variegated surface within a regular and repeated shape. But if you carry a brick through a crowded shopping street, then – especially if you are wearing a ‘hoodie’ – it may well be regarded as a potential weapon and you, the bearer, as a would-be criminal.12
Origins of discourse theory and discourse analysis Discourse theory is an ever-evolving discipline. Its roots lie in linguistics, semiotics, philosophy, psychology, and social theory. Politics departments – especially at the University of Essex – have played an increasing role in its more recent development.
Linguistics Linguistics is ‘the science of language’ (Crystal, 1995: 425).13 Its original focus was grammar and the rules of language. In the Enlightenment period, the rationalist scholars investigated what appeared to be universal features of language as evidence of universal rationality. Their nineteenth-century successors studied the evolution of language and grammar. In the twentieth-century, the focus moved to psychology and social theory. A paradigmatic shift was provided by the Course in General Linguistics (1916) by Ferdinand de Saussure (1857–1913). He distinguished between language (facility of speech), langue (language system) and parole (speech act). Langue is essentially a system of signs which has two parts: the signifier (what signifies) and the signified (what is signified or meant). He also distinguished between diachrony (study of the historical development of language) and synchrony (study of the ‘lived’ language of the present). The synchronic approach of concentrating study on the structural forms of language – rather than diachronic study – became termed structuralism. Its name derived from the belief that that individual agency (personal autonomy) is superficial: behaviour is determined by underlying structures of society. Individuals are not the architect of their universes but their products. This approach was extended by the French scholars: Claude Levi-Strauss (1908–),who was particularly concerned with the role of myths in society); cultural seminologist, Roland Barthes (1915–80), and psychoanalyst; Jacques Lacan (1901–81) and others (Jary, 1995: 660–2).14
281
282
Research Methods in Politics
Anthropologist, Franz Boas (1858–1939) argued that language is essential to the acquisition and transfer of knowledge. Later, post-structuralism was developed by Jacques Derrida (1930–2004) and Jean Baudrillard (1929–2007) to focus on the nature of surface knowledge. Special attention was given to the rhetorical devices of which language was constructed. Their principal method of analysis was therefore, deconstruction. A paradigmatic shift was achieved by the MIT Professor of Linguistics and subsequent political activist, Noam Chomsky (1928–). He distinguished between competence (the individual’s knowledge of the rules of grammar) and performance (their use of the language). Linguistics should concentrate more on linguistic competence as part of the individual’s psychological capacity: How [is] it that human beings, whose contacts with the world are brief and personal and limited, are nevertheless able to know as much as they do? (Chomsky, 1986: xxvi)15
Semiotics Semiotics is the: study of the properties of signs and signalling systems especially as found in all forms of human communication. (Crystal, 1995: 430)16
More recently, the signifiers have been extended to include consumer durables, gadgets, gifts, music and architecture. For example, archaeological research has shown that the great fortified gatehouses built by the Romans had little military value: their role was to express to the native population the overarching power of Rome.
Philosophy The philosophers principally associated with the development of discourse theory are the Austrian-born philosopher, Ludwig Wittgenstein (1889–1951) and the French philosopher, Michel Foucault (1926–84). Wittgenstein argued that language pictured the world. Furthermore, there could never be a ‘private language’. Language was bounded by language games where each game had its own rules and was grounded in its own ‘form of life’. Foucault was, in part, a poststructuralist and iconoclast (a person who challenges accepted beliefs). He re-examined topics and prevailing beliefs ‘through the other end of the telescope’. He argued that there was no absolute truth or
Understanding and Adopting Discourse and Narrative Analysis
objective knowledge. The maxim that knowledge is power was a myth. In practice, power is knowledge. Elites use language to promote their interpretations of knowledge through fabricated truth regimes. Discourse was a system of representation in the form of epistemes and discursive formations. In particular, language is a vehicle for the production of new knowledge and systems of classification of society. Knowledge did not liberate individuals. Instead, it has become part of the apparatus of control by the state. He was particularly concerned (and personally affected) by the emergent discourse of scientific and medical knowledge by which individuals became labelled as mad or homosexual, and panoptically complicit in their own exclusion and confinement. Madness and sexuality were entirely social constructs. In this way, individuals were objectified and structured by discourse. His approach has been widely adopted by Foucauldian critical discourse analysts. Other significant contributions were made by Canadian-born sociologist Erving Goffman (1922–82), US sociologist, Harold Garfinkel (1917–) and Canadian media guru, Marshall McLuhan (1911–80). Goffman studied face-to-face interaction through the lens of social interactionism where the meanings of language were never static or fixed but negotiated by the participants. Garfinkel was the founder of ethnomethology which studied the everyday methods used by members of different societies (ethnomethods) in their conduct and interaction. McLuhan contributed the maxim that: ‘the medium is the message’ (1962).17 So ‘television is more significant than the content of its programmes’ (O’Sullivan, 1997: 176–7).18
The schools of discourse theory and discourse analysis The leading UK authority on discourse in Politics, David Howarth, identifies five different theoretical schools of discourse and, therefore, discourse analysis: positivists; realists; Marxists; critical discourse analysis; and post-structuralists (Howarth, 2000: 2–5).19 Positivists (and empiricists) see discourses as ‘frames’ made by groups to interpret their worlds in the same way and for specific purposes. These frames legitimate (and, therefore, justify or excuse) collective action. This is: essentially a political definition, focusing on attempts by groups to impose their own assumptions and values on others in order to promote their own interests. (Burnham, 2004: 243)20
Members of this school see the task of discourse analysis as, therefore, to identify these frames, their effectiveness and the consequences of their use. Realists argue that objects have an existence independent of society’s conception or perception. The objects have their own properties and causal capacity. So discourses
283
284
Research Methods in Politics
are objects in their own right which contribute to events and social development. The task of discourse analysis is to identify the discourses, show how they contribute to processes of development and to expose the ‘underlying material resources which make discourses possible’ (Parker, 1992: 1).21 Marxists see discourses as ‘ideological systems of meaning’ which legitimate the unequal distribution of resources and power. The discourses are economic. They are promoted by capitalists via their political parties, state education system and news media. The task of discourse analysis is: first, to expose the false consciousness sustained by discourse; second, to identify the mechanisms and means adopted: and third, to liberate those people whom it imprisons. Critical discourse analysis is essentially similar to the Marxist conception of discourse. However, it privileges social over economic influences. This is described in greater detail later in the chapter where it is adopted to analyse a case study. Post-structuralists (and Post-Marxists) regard: social structures as inherently ambiguous, incomplete and contingent systems of meaning … discourses constitute symbolic systems and social orders … the task of discourse analysis is to examine their historical and political construction and functioning. (Howarth, 2000: 4–5)
The proponents of this view include Derrida, Foucault, and, more recently, Howarth’s colleagues, Laclau and Mouffé.22 Laclau and Mouffé set out ten propositions. These include the argument that a political discourse can never dominate a discursive field to the exclusion of other discourse: it is defined by a subordinate one. Their propositions are very sophisticated and probably too advanced for first-time discourse analysts.
Confused by discourse theory? This brief review demonstrates one essential element of discourse analysis: don’t think about using discourse analysis unless you are willing to confront a very challenging literature. If your research project is merely a means to an end outside academia, then choose another method of analysis. Go for an easy option. But if you regard your research as a unique opportunity to develop your intellect by tackling new concepts ranging beyond the conventional boundaries of political science, then consider using discourse as both the theoretical framework and method of analysis. On the practical side, an understanding of discourse can help you to compete better in the growing communications and media sectors of the economy.
You may well feel intimidated by Laclau and Mouffé’s post-structuralist approach. In that case, you are likely to find that critical discourse analysis provides a
Understanding and Adopting Discourse and Narrative Analysis
better starting point for your journey into discourse analysis. It offers three great advantages to the political scientist. First, critical discourse analysis sees its primary role as ‘emancipatory’: to expose the way in which language and meaning are used by the powerful to deceive and oppress the dominated. (Howarth, 2000: 4).
Second, critical discourse theory draws on wide-ranging authorities with whom you are likely to be already familiar: Gramsci, Althusser, Foucault, Giddens, Habermas and Bakhtin (1895–1975). And third, a leading school is led by Norman Fairclough, formerly Professor of Language in Social Life at the University of Lancaster, whose very readable, authoritative texts draw examples from Politics – especially the discourses of Thatcherism and New Labour.
Fairclough’s critical discourse analysis In the introduction to his major text, Language and Power (2000), Fairclough explains that his language study is critical (CLS) because it seeks to expose otherwise hidden connections between language, power and ideology (2000: 4).23 He differentiates this approach from others by concentrating on how the audience interprets the text (rather than the intentions of the ‘text-producer’) drawing on their members’ resources (MR, i.e. accumulated experience). He also includes within the scope of discourse analysis the turn-taking of everyday conversations (which others tend to treat as a specialist sub-field). Critical language study adopts a concept of discourse which Fairclough defines succinctly as: ‘language as social practice determined by social structure’ (2000: 14). So language is integral (rather than external) to society and a socially-conditioned process. The relevant conditions are the social conditions of production and the social conditions of interpretation. These social conditions have three levels of organisation: the immediate social situation; the social institution; and the level of society. Fairclough develops three arguments about the relationships between language, discourse and social practice. First, the discourse used is socially determined by orders of discourse conventions set out in social institutions. Second, orders of discourse are ideologically shaped by power relations of class and power in capitalist society. Third, (following Giddens’ structuration theory) discourse shapes social structures and vice versa. So this dialectic of structures and practices achieves both social continuity and change. Fairclough’s critical language study develops Gramsci’s concept of hegemonic power. Discourse is used by the state to win consent and acquiescence. However,
285
286
Research Methods in Politics
this non-coercive power can be in or behind discourse. Power in discourse is used by dominant groups to control or constrain the contributions of others in terms of what is said (contents), the social relations of the discourse (e.g. teacher and pupil) and the subject positions they occupy. In particular, news media limit the contents of bulletins and representation of minorities. This constraint is an example of the hidden power of discourse. But the power behind discourse is more subtle. One example is the promotion of an elite dialect as the national language. Standard English became the essential qualification for advancement in UK professions (early phonographs record Gladstone’s Worcestershire accent). Whilst ‘Estuarial’ (or ‘Mockney’) English may be the dialect of popular television, it is not used in news bulletins – especially of serious news. Standard English remains authoritative. It remains authoritative by mutating (as notably chronicled by Raymond Williams, 1976).24 Access to discourse is subtly restricted. Free speech is a myth. Professions monopolise technical language and develop their own narrow jargon. Similarly, turn-taking is dictated by social class especially in formal situations like court proceedings. Fairclough is particularly concerned with the relationships between discourse, common sense and ideology. He argues that ideology prevails when it becomes elevated to and accepted as common sense. As such, it is natural and therefore not only acceptable but desirable. Many older people believe that the unequal distribution of wealth and income is common sense and ‘for the best’ (Panglossian). The meaning of words is also common sense. For example, the word ideology is equated with totalitarianism, fascism, communism and Marxism. So political leaders will argue that their views are, basically, common sense, whilst their opponents are entirely ideological. Similarly, each will preface their arguments by stating that ‘the truth of the matter is …’. Another device is to use scare quotes. For example, the popular press undermined the scientific discourse of GM foods by calling them Frankenstein foods. Fairclough’s exposition of the concept and components of language as discourse is relatively straightforward. Where his particular discourse becomes more difficult is when it becomes operationalised as a complex system of discourse analysis. He argues that you can’t just cite a speech by President Bush and say that this is a good example of hegemonic discourse: formal analysis is necessary. The conceptual framework for Fairclough’s critical discourse analysis is the relationship between text, interactions and contexts, illustrated in Figure 19.1. Fairclough develops a three-stage system of critical discourse analysis: 1. description: identifying the formal properties of the text 2. interpretation: identifying the relationship between the text and interaction 3. explanation: identifying the relationship between the interaction and the social context.
Description answers ten questions in three sections.
Understanding and Adopting Discourse and Narrative Analysis Social conditions of production Social conditions of interpretation Context
Process of production Process of interpretation Interaction Text
Figure 19.1 Discourse as text, interaction and context Source: (after Fairclough, 2000: 21, Figure 2.1)
BOX 19.1 92–3)
Description: ten questions (Fairclough, 2000:
Vocabulary 1. What experiential values do the words have? 1. What classification schemes are being used? 2. Are there words which are ideologically contested? 3. Is there rewording or over-wording? 4. What ideologically significant meaning relations are there between words? 2. What relational values do words have? 1. Are there euphemistic expressions? 2. Are there markedly formal or informal words? 3. What expressive values do the words have? 4. What metaphors are used? Grammar 5. What experiential features do grammatical features have? 1. What types of process and participant predominate? 2. Is agency unclear? 3. Are processes what they seem? 4. Are nominalisations used? 5. Are sentences active or passive? 6. Are sentences positive or negative? Continued
287
288
Research Methods in Politics
6. What relational value do grammatical features have? 1. What modes (declarative, grammatical question, imperative) are used? 2. Are there important features of relational modality? 3. Are the pronouns we or you used, and, if so, how? 7. What expressive values do grammatical features have? 1. Are there important features of expressive modality ? 8. How are simple sentences linked together? 1. What logical connectors are used? 2. Are complex sentences characterised by co-ordination or subordination? 3. What means are made for referring inside and outside the text? Textual structures 9. What interactional conventions are used? 1. Are there ways in which one participant controls the turn of others? 10. What larger-scale structures does the text have?
These terms need explanation. Experiential value refers to traces of how the text producers’ own experiences of the world – social or otherwise – are represented. By way of example, Fairclough cites the difference between the use of the terms ‘solitary confinement’ and ‘seclusion’ by different groups of psychiatrists. The two terms reflect two opposing ideologies. The replacement of ‘solitary confinement’ by ‘seclusion’ is an example of rewording. Relational value refers to traces of the social relationship in the discourse. In other words, how the words used create social relations between the participants (for example, addressing each other by title, family name, first name or nickname). Expressive value refers to the text producer’s view of reality (for example, ‘globalisation’ and ‘internationalisation’). Euphemisms provide traces of expressive values, for example, ‘defence forces’. In terms of grammatical description, Fairclough analyses the grammar of sentences into subject (S), verb (V), object (O), complement (C) and adjunct (A). Their combination and order express three types of process: actions (SVO, e.g. Thatcher attacked the welfare state), events (SV, e.g. Thatcher was speaking), or attributions (SVC, e.g. the welfare state was attacked by Thatcher). Attributions identify causes (or allocate credit or blame to agents). In contrast, where agency is unclear, then causes are concealed – perhaps for legal reasons. Nominalisation is used to convert a process into a noun. In this way, meaning and attribution become (conveniently) ambiguous. A verb requires a subject and may cite an object. For example, using the word ‘development’ rather than the verb ‘to develop’ enables
Understanding and Adopting Discourse and Narrative Analysis
the speaker to be much less specific about who is going to develop what, how and when. ‘Community development’ is a wonderful example of nominalisation in practice. Fairclough distinguishes between modes of sentence, modality and pronouns. Modes can be declarative ( SV), grammatical question (no O) or imperative (no S). Each treats the subject in a different way. Using grammatical questions or imperatives underlines the power of the subject. Modality refers to the authority of the speaker. Relational modality is the social authority of the speaker, whereas expressive modality is concerned with the speaker’s authority of truth or reality, and can be recognised by their use of ‘is’, ‘may’ or ‘can’. Similarly, using ‘we’ or ‘you’ can evidence power relations. Aspects of Fairclough’s textual analysis are sophisticated and, at first sight, complicated, He can perhaps be criticised for appearing to omit other key contributions to the discourse of particular importance to the Politics researcher. These are primarily contextual. They include the when, where, to whom (the audience) and how (the medium), the role of the speaker or writer (the ‘text-producers’) and the position of the discourse in preceding or contemporary events. For example, you may consider that contextual factors like a formal speech by the British prime minister to the UN in the wake of 9/11 may be more significant in the first instance than whether individual sentences have objects or not.
Interpretation Fairclough distinguishes between interpretation and explanation. Interpretation is the process of mediation between text and social structures in which the audience draws on the common-sense assumptions embedded in the resources (MR). Explanation concerns: ‘the relationship of discourses to processes of struggle and power relations’ (2000: 117). Interpretation boils down to answering three questions of the text: first, what interpretation are the speaker and audience giving to the situation and ‘intertextual context’? Second, what discourse type is being used? And, third, are participants likely to interpret the context and discourse type in different ways? (And does this affect the course of the interaction?)
Explanation Fairclough argues that, whilst the processes of production and interpretation reproduce members’ resources (MR), explanation seeks to locate the discourse as social practice, to show how it is affected by social structures and to identify how these structures are reproduced or modified. Explanation therefore has two analytical
289
290
Research Methods in Politics
dimensions: first, to reveal the role of discourses as elements of social struggles in which structures are contested, sustained or re-shaped; and, second, to identify how and which power relations determine discourses. Explanation can be achieved by answering three questions of the text: first, what power relations at situational, institutional and societal levels shaped the discourse? Second, what aspects of the members’ resources used are ideological? And third, does the discourse sustain or change previous power relations? (2000: 138) Fairclough concludes by arguing that the researcher must draw on their own MR to explain how the participants use theirs. The analysis must be an ‘inside job’. So, the analyst must be reflexive and seek to develop their own MR by developing their own theoretical understandings and experience of society and politics. The critical discourse analyst can therefore be likened to an ‘expert forensic witness’. Fairclough demonstrates the application of critical discourse analysis by a case study of a radio interview of Margaret Thatcher in 1985 (2000: 143–5). You will note that his analysis is eight times the length of the transcript! A small extract demonstrates the level of analysis:
‘Question 1: What relational values do textual features have? Are there inconsistencies in relational values which could indicate a new articulation of discourse type?
1. We. Mrs Thatcher uses the pronoun we mainly in lines (11–9) and (79–81), both inclusively and exclusively … The inclusive use (e.g., ‘ now we do enjoy a standard of living which was undreamed of then’) is rationally significant in that it represents her, her audience and every one else in the same boat. It assimilates the leader ‘to the people’ … (Fairclough, 2000: 148)
He also subjects New Labour to discourse analysis in a separate text (Fairclough, 2000a).25 His essential argument is that the re-naming of Labour as New Labour signals ideological change and manipulates language to control public perception. Great emphasis is given to ‘getting the language right’ by, for example, substituting the ‘privatisation’ of Thatcherism to ‘public-private partnership’. (This is effectively the opposite of Thatcherism in which radical social and economic policy was offset by appeals to the traditional Conservative party discourse of the virtues of patriotism, strong pound, strong defence, law and order, and the ‘3Rs’.)26 New Labour was wholly committed to the neo-liberal global economy. It portrayed globalisation as a ‘natural’ and ‘inevitable’ process in the face of which government was forced to respond by ‘allowing’ greater ‘labour flexibility’, etc.
Understanding and Adopting Discourse and Narrative Analysis
Fairclough identifies a discourse of New Labour characterised by lists – of effectively motherhood objectives – in which the connections and means of implementation go undisclosed. So, by speaking of, say, ‘not only economic dynamism but social justice’, the contradictions, antagonisms and unpleasant consequences of the duality are omitted. But he argues that New Labour portrays its portfolio of platitudes with an overriding consensus formed from the ‘old’ clash of (outdated) socialism and capitalism. The New Labour keywords are: we, Britain, welfare, partnership, new, schools, people, crime, reform, deliver, promote, business, deal, tough and young (2000a: 17). He also pinpoints how the moral discourse becomes authoritarian when expressed by statements like: ‘if you can work, you should work’ and ‘if you can save, you have a duty to do so’ (2000a: 42). The rhetoric and presentation of Tony Blair are also honed by the discourse. ‘Tony’ is an everyday man who uses glottal stops, says ‘yeah’ rather than ‘yes’ and speaks of ‘Cherie and the kids’.27 Yet he combines ‘moral righteousness with toughness’ (2000a: 148). Fairclough ends the case study by re-asserting that: … (a) that politics and government are social practices in which language is salient – this is a durable feature of these social practices in comparison with others [he cites agriculture as a case where language is less salient], (b) language is becoming more salient within these practices. (Fairclough, 2000a: 156).
Case Study: President George W Bush’s State of the Union Address, 2007 The annual State of the Union Address is a constitutional requirement of the US President. It is made to a joint meeting of Congress. It effectively combines the UK Parliament’s Queen’s Speech and the Chancellor of the Exchequer’s Budget statement. Read the introduction, passages on Iraq and Afghanistan and the conclusion from www.whitehouse.gov/news/releases/2005/02/20050202-11.html. Note that you may also be able to watch the video of the speech at the same web-site. The task is to apply discourse analysis to the section of President Bush’s speech dealing with Iraq and the Middle East. However, before beginning the detailed analysis, you should ponder its context. This is highly significant. You will note that: • •
The State of the Union Address is a formal speech whilst the President occupies centre-stage, he is overlooked by Vice-President Cheney and Democrat Nancy Pelosi, Speaker of the House of Representatives elected on Continued
291
292
Research Methods in Politics
4 January as the first woman speaker and the most senior elected politician in the country. She is ‘two heartbeats away from the Presidency’ •
The President is effectively reading a prepared speech written by speech-writers and honed to perfection by rehearsals
•
the speech is being made to the Members of Congress face-to-face and, via the news
• • •
• •
• • • •
media, to the citizens of the US the speech will also be considered carefully by the US allies, supporters, critics and enemies by tradition, the speech is made unchallenged by opposition; it is punctuated only by applause which is triggered by pauses in the President’s delivery President Bush is ‘beleaguered’: the Iraq project has become the dominant political issue in the US and, in consequence, the President’s Republican Party has lost control of the Senate and House of Representatives The President’s approval ratings are at an all-time low the ‘separation of powers’ set out in the US constitution means that, if bills are not to be vetoed by the President or Congress, then a new ‘bi-partisan’ approach will have to be negotiated by the two institutions if progress is to be made on new policy on social security, Medicare, taxation, home security and defence big business, the Pentagon, the ‘neocons’, and potential candidates for the presidential election will be watching for signals of support for their own constituencies The President has just rejected the recommendations of the bi-partisan Iraq Study Group to involve Syria and Iran in the resolution of Iraq a predecessor, President Reagan was called the ‘Great Communicator’; George W. Bush is no great communicator US commentators tell us that, whatever the public’s view of any individual President, they retain great respect for the Presidency.
A comparative content analysis of key words used in all President Bush’s State of the Union Addresses shows the changing emphasis given to key issues over the years: The table is illustrated in Figure 19.2
Analysing the case study The analysis will adopt Fairclough’s three-stage system of description, interpretation and explanation.
Understanding and Adopting Discourse and Narrative Analysis
Table 19.1 State of the Union Addresses 2001–7 Topic
2001
2002
2003
2004
2005
2006
2007
Iraq Afghanistan Economy Iran Oil
− − 6 − −
2 13 7 2 1
21 3 13 3 −
24 5 17 1 −
27 3 14 3 −
16 2 23 6 2
34 4 8 5 9
Number of mentions of keywords
Source: New York Times: 21/01/07
40 35 30 25 20 15 10 5 0
Iraq Afghanistan Economy Iran Oil 0
2
4 6 Years (2000 onwards)
8
Figure 19.2 President Bush’s State of The Union Addresses 2001–07: frequency of key words Source: Table 19.1 from New York Times: 21/01/07
Explanation You will recall that the questions to be asked of the text are: what are the experiential, relational and expressive values and metaphors used? Experiential values refer to the traces of ‘text-producer’s’ experience of the natural and social world included in the text. Aspects include the classification schemes used, whether words are ideologically contested and over-wording used. In this case, the classificatory schema adopted by the President is primarily to reduce the world to the US, ‘Americans’ and US interests. Ideologically contested words are minimised in favour of the language of consensus and learning-through-experience. This is also achieved by portraying the enemy in overworded, graphic terms: … possessed by hatred and commanded by a harsh and narrow ideology. They preach with threats, instruct with bullets and bombs, and promise paradise for the murder of the innocent. Continued
293
294
Research Methods in Politics
Note also the use of alliteration (the rhetorical style which uses the repetition of opening consonants to achieve emphasis): ‘hatred … harsh, bullets and bombs …, promise paradise…’. Notice how ‘our American values’ are contrasted with their ‘extremism, ideology, and totalitarianism’. By contrast, ‘our government’ is using ‘lawful and proper use of intelligence, etc.,’ in a list in which ‘military action’ is mentioned last, implying a ‘last resort’. What they (the terrorists) ‘fear most is human freedom’. There is a superabundance of purple prose: free societies where men and women make their own choices, answer to their own consciences and live by their hopes rather than resentments. Relational values refer to traces of social relationships enacted via the discourse. In this case, the social relationship sought with his immediate audience of Congressmen is that of friendship in adversity: e.g. ‘Like many before us, we can work through our differences, and achieve big things for the American people’ and: ‘our success in the war’. You will note that, in the video recording, emphasis is given by the President to we. In terms of the wider audience of the American people, President Bush is seeking to project himself as their resolute protector. But the social imagery induced is more like the ‘strong father’ than ‘benevolent mother’ or ‘kindly grandfather’ (which President Reagan was able to evoke). But this President is no wise old man sitting out on the porch: he is a military scout and fire-fighter. Are there any euphemistic expressions? Well, the whole speech seems to avoid saying either ‘my disastrous invasion of Iraq, my failure to secure peace there, the worsening situation, and 3,000 US dead’. Instead, he asserts that ‘our success in this war is often measured by the things that do not happen’. Are there any markedly formal or informal words? The speech is ‘US formal’ which seeks to underline the dignity and authority of the Presidency and the seriousness and fitness-for-office of the current President. Expressive values refer to traces of the speaker’s evaluation of reality and their ideological significance. In this case, the President ‘congratulate[s] the Democrat majority’. He appears to avoid terms ideologically confined to the Republican Party and uses instead terms common to the Republicans and Democrats: ‘extending the nation’s prosperity, to spend the people’s money wisely … to guard America against evil’. The ideological struggle is between America and its enemies. There is no accommodation possible which does not compromise US freedom: ‘the security of our nation is in the balance’. What metaphors are used? There appears to be very few metaphors used. The President avoids using the popular metaphors of baseball or American football or the ‘tough talk’ (e.g. ‘I say, bring it on’ for which he apologised later). He’s ‘talking Presidential’, using measured words.
Understanding and Adopting Discourse and Narrative Analysis
What experiential value do grammatical features have? The main grammatical features are short, grammatically correct sentences. Most sentences are given in the active (rather than passive) tense in which the subject–verb–object predominates. So does the present tense. The subject is ‘we’ rather than ‘I’ or ‘you’. Agency is clear: terrorists. The sentences are mainly positive and confident. There are few nominalisations. The grammatical mode is almost wholly declarative (rather than grammatical questioning or imperative). The relational modality is President to Congress and ‘our citizens’. The expressive modality is characterised by the use of ‘can’, ‘will’ and ‘know’ rather than ‘may’ or ‘might’. The President is confident and certain. The main logical connectors employed are hyphens. There are no lists. Applause is cultivated to maintain control and to enable topics to be changed. Indeed, applause is the main interactional element between speaker and audience. The President speaks direct to his audience rather than to the ceiling and notes. There are no asides or quips. He is reading from the transparent video-screens at each side.
Interpretation There will be many interpretations of a Presidential address. The interpreters will include: Congressmen, the television audience, the newspaper readers and the ‘expert commentators’ who mediate between the text and the wider audience. Each group is diverse. They will all draw upon very different members’ resources (MR). Then there will be your own member’s resources as the Politics researcher. Knowing how the State of the Union Address will ‘play in Peoria’ will be difficult for a British or Australian researcher, and doubly-difficult for a researcher who is not wholly fluent in American-English. So, ideally you should be able to identify which interpretations are likely to be given by the main constituents to whom the President is seeking to appeal and influence. There are many. You must identify the principal constituencies and suggest how they might interpret the speech. Take, for example, the US military who, despite forces of over a million men and women and a budget of $600 billion, appear unable to stem the rising violence in Iraq. What’s in it for them? Well, their Commander-in-Chief declares that: on this day, at this hour, it is still within our power to shape the outcome of this battle. Let us find the resolve, and turn events towards victory’. Victory will be achieved by sending ‘more than 20,000 additional soldiers and Marines to Iraq. Continued
295
296
Research Methods in Politics
An additional 92,000 troops will be recruited, and civilians hired for rear echelon duties. Our country is pursuing a new strategy in Iraq, and I ask you to give it a chance to work. And I ask you to support our troops in the field, and those on their way. This last statement evokes memories of Vietnam when President Nixon argued that it was primarily the lack of support for US forces at home that undermined the war effort over there. You may conclude that US soldiers based in Baghdad would interpret the speech to mean ‘more of the same’ with the options of withdrawal or ‘taking the fight to the real enemy in Iran and Syria’ ignored. They may be disappointed or even angry that the death of so many soldiers goes unstated.
Explanation Fairclough’s critical discourse analysis concludes by identifying the power relations, ideologies and effects. He is particularly concerned to identify how the audience’s members’ resources are reproduced unconsciously through their interpretation of the text. In this case, the President achieves – or seeks to achieve – reproduction of his discourse by minimising the differences between Republicans and Democrats, by wrapping himself in the flag of consensual American values and by painting his enemies as wholly evil and hell-bent on destroying our America. His discourse is therefore of integration and differentiation. American values are being defined by the enemy. It is their ideology which is defined. He continues to link the war in Iraq with 9/11 and seek popular support for a war of righteous retribution against evil which threatens ‘the security of [our] nation’. So the war is essential to defend American values and way of life. He raises the sword of ‘human freedom’ which ‘every terrorist fears most’. The appeal is to emotions rather than intellect. But – despite the stentorian rhetoric – the speech is comparatively uninspiring and lacks the power of John Kennedy or Churchill to reinforce the determination of the people.
Postscript In his State of the Union Address, the President spoke of the budget that would ‘shortly be sent to Congress’. His budget was announced on 15 February. Despite his pledges to balance the budget and to protect Medicare provision, the budget proposed to increase military spending by nearly 12%. This would be achieved by reducing federal spending on Medicare by $66 billion over five years and cutting back $12 billion from the Medicare
Understanding and Adopting Discourse and Narrative Analysis
healthcare scheme for lowest-income citizens. However, he argued that these reductions could be achieved by attacking costs rather than by reducing the level and volume of services.
Conclusion: How useful is critical discourse analysis? The case study demonstrates the strengths and weaknesses of applying Fairclough’s approach to critical discourse analysis. The highly-structured framework of analysis enables you to offer a wealth of critical assessments. The framework provides a good starting point for the ambitious researcher keen to adopt discourse analysis. However, there are a number of weaknesses which arise in part from its roots in linguistics. Too little emphasis is given to the context in which the discourse is produced. It is better suited to the analysis of interactive discourse than the speeches and monologues which characterise politics. It emphasises the interpretations given by the audience rather than the intention of the ‘text-producer’. Identifying the interpretations given by the audience is highly speculative. This requires understanding of their members’ resources. Furthermore, the audience is likely to consist of a number of constituencies with very different resources You may therefore wish to modify the approach by including other methods, such as: • • • • •
randomly selecting texts from a sample frame identifying the contexts considering the intentions of the text-producer by asking: what is the text seeking to achieve? using content analysis to calculate the actual frequency of key words like ‘will’ and ‘may’ and sentence length critically using interviews, focus groups and questionnaire surveys to identify both the members’ resources drawn upon by the audience to interpret the text, and how they are fixed or changed by the text.
Narrative analysis Narrative analysis is a sub-type of discourse analysis concerned with the analysis of narratives. A narrative is essentially an account of a past event. It can be verbal, written, visual or aural. Often discourse is a combination of media. The narrators mediate between the subjects and the audience. They can be a politician, journalist, commentator, teacher or other authoritative narrator. They will seek to locate the
297
298
Research Methods in Politics
narrative in the audience’s members’ resources and to strengthen or modify them. Modern politics has been likened to a battle of narratives in which the victor is the most successful in providing the most popular, enduring and damning narrative of their opponents. For example, the Labour leader, Harold Wilson succeeded in devastating the Conservative government in 1964 by repeating the narrative of ‘thirteen wasted years of Tory misrule’. Various definitions and explanations have been given of narrative in the social sciences. They include (in chronological order): •
any form of communication (Barthes, 1966)28
• •
the main mode of human knowledge (Bruner, 1986)29 a socially-symbolic act in the double sense that it (a) takes on a meaning only in a social context, and (b) plays a role in the construction of that context as a site of meaning within which social actors are implicated (Mumby, 1993)30
• •
stories that take place in time (Berger, 1997);31 a story with a beginning, middle and end that reveals someone’s experiences (Manning and Cullum-Swan, 1998).32
Origins The origins of narrative analysis include linguistics, social and literary theory, history and discourse theory. The classical theory underpinning narrative analysis is the Morphology of Fairy Tales (Propp, 1927). Vladimir Propp (1985–1970) examined the main Russian fairy tales. He found that the underlying narrative structure was remarkably common. Each fairy tale could be deconstructed into 5 categories of elements and 31 narratemes. The five categories were: the dramatis personae (‘cast’); the conjunctive elements of misfortune and good fortune; motivations; forms of appearance of the cast (e.g. the flying fairy godmother); and, lastly, attributive elements (e.g. Jack’s beanstalk, the old lady’s shoe). The narratemes began with α, the initial situation when the hero and members of family are introduced, followed by when one of the members of the family goes missing. The stories all end with the last narrateme, , when the hero marries and ascends the throne. Western fairy tales also followed this pattern. Propp therefore argued that fairy tales constituted pre-knowledge – comparable to Fairclough’s earliest members’ resource – by which people became conditioned to believe that their own lives would follow a similar fairy tale course. These expectations enabled them to accept that early misfortune was inevitable and part of the essential tapestry of life that – ultimately – would end in happiness. Parents and grandparents become reconditioned and reproduced by their role as household narrators. The concept of pre-knowledge can explain how readily the British public came to see Princess Diana as the ‘fairy princess’ and Camilla Parker-Bowles as the ‘wicked witch’
Understanding and Adopting Discourse and Narrative Analysis
and ‘step-mother’ to the young princes. Indeed, the western antipathy to stepmothers and mothers-in-law can be attributed to the bad press they receive in fairy tales. Whilst Propp’s work has been hugely influential, he has been criticised for overlooking the nuances of context and mood. He can also be criticised for overlooking the competing pre-conditioning role of religion and Biblical narrative on western society. The narrative of Christ is the antithesis of the fairy tale. Through this combination of fairy tales, religious tracts and local folk stories (King Arthur, etc.), the structures of stories become both historically embedded and conservative (retrospective). The participant or observer is conditioned to believe that there is little opportunity for – or point in – agency. This historicist model can be likened to Heidigger’s ‘horizons of meaning’. History – especially national history – is grand narrative. History shapes national identity. Identity shapes and reinforces the boundaries of the state. History is shaped by dominant power: ‘it is the victors who write history’ (Carr, 1990).33 So whose history prevails is critical. Hence the continuing argument in the UK about the contents of the unified, national curriculum for history in which – as one insider told me – the central question was: ‘how much can we afford to tell the working class about their past?’ Some concentration camp survivors argue that narratives of the Holocaust were suppressed in Western Europe during the Cold War to facilitate reconciliation and military coalitions against the ‘Soviet threat’. In the same way, the historical narrative of Japan was re-written by the British government in the 1980s to air-brush out Japanese wartime atrocities and therefore to reduce opposition to Japanese inward investment.
Narrative analysis Definitions of narrative analysis include: • •
the analysis of act, scene, agent, agency and purpose (Burke, 1966)34 how respondents in interviews impose order on the flow of experience to make sense of events and actions in their lives (Kohler Reissman, 1993)35 • the analysis of how stories mark out identities, mark out differences, differences define ‘the other’, and the other helps structure the moral life of culture, group and individual (Plummer, 1995)36 • the analysis of the production of narratives: the process of association, building and ‘the and, and and’ connections between actions and events and negotiating them with readers (Czarniawska, 1998)37 • the study of lives from the narrator’s experience as a shared production with social scientists (Social Anthropology, Feminist Research, McRobbie, 1982).38
299
300
Research Methods in Politics
Narrative analysis essentially asks: • • • • •
Who is ‘writing’ the story? Who is telling the story? How? Who is the ‘target audience’? What is the story trying to achieve? What are its effects?
The approaches to narrative analysis include: • •
identifying the formal structures (properties) of stories in terms of the plot, setting, characterisation, and temporal ordering identifying the social roles of stories in terms of: • the ways they are reported • the ways they are read • how they change • their role in the political process • stories as speech acts.
Speech Act Theory is central to narrative analysis in explaining how words ‘work’. The underlying concept is that words ‘do action’. The speech acts can be distinguished and classified as (Austin, 1962):39 • • •
locutionary speech acts: constative, descriptive statements, e.g. ‘it’s hot today’ illocutionary speech acts that achieve action ‘in’ saying (directly): e.g. ‘in saying it I was warning him’ perlocutionary speech acts that ‘do something’ indirectly. They use the language of a constative statement: e.g. ‘students who do not attend classes are more likely to fail’.
Often these categories of speech act are linked. Not all speech acts are felicitous (successful): some are infelicitous. Narratives can also compete for support. For example, consider the famous case of the Congressional hearings in 1991 when President Bush’s nominee for the US Supreme Court was contested. The nominee was Judge Clarence Thomas, a 43-year old, conservative, African-American from Georgia. Critics argued that, whilst Thomas would maintain the racial composition of the Supreme Court, he would replace a black liberal judge by a conservative who would oppose reform in decisions involving Affirmative Action and abortion. The hearing considered the objection by Anita Hill (a former colleague) of sexual harassment. Both were high-achieving, black Republicans from poor backgrounds whose successful careers were assisted by early ‘affirmative action’ programmes. The hearing was held in the context of the imminent presidential elections in which Republicans
Understanding and Adopting Discourse and Narrative Analysis
were arguing that affirmative action was discriminatory. The full transcript of the hearing is available on http://chnm.gmu.edu/courses/122/hill/hillframe.htm. When you read them, you will note how both Thomas and Hill adopted ‘narrative strategies’. She sought to argue on the basis of the ‘facts’ (detailed descriptions of diarised events). He presented himself as a ‘victim’ citing his deprived background and the ‘high-tech lynching for uppity Blacks’ by the liberal news media. Furthermore, he criticised his sister, Mae as a ‘welfare queen’ and portrayed Hill as a ‘black lady’. Thomas’ narrative prevailed by appealing to embedded US narratives of the self-made man and ‘victimhood’.40 You may note how the contemporary narrative metaphor of ‘black’ - which had recently replaced the more pejorative ‘negro’ - has now been superseded by ‘African-American’ in the same period in which ‘Red Indian’ has been superseded by ‘Native American’ (cf., Canadian ‘First Nations’).
Narrative constituents Narrative analysts adopt a scheme of terms to classify the components of narrative discourse. They are: • • • • • • • • • • • •
•
story : the raw, temporarily sequenced or causal narrative of life plot : emerges from unexpected twists in the narrative that draw attention to differences from the conventional story narrative of the self : the story or stories by which self-identity is reflexively understood by the individual and others reflexive projects of the self : the process by which self-identity is constituted by the reflexive ordering of self-narratives (Giddens, 1991) discursive register : the style of narrative, especially stylistic narratives ‘borrowed’ from other narratives, e.g. ‘ball-park figure’ from US sport; and ‘take out’ from US military texts epiphany : a crucial turning-point (Damascene conversion) which stimulates a new, radical consciousness text : a finite, structured whole composed of language signs actors: passive subjects, objects agents: active subjects who affect others by their actions events: transitions from one state to another fabula: the series of logically and chronologically related events that are caused by or experienced by actors or agents kernels: narrative moments that give rise to cross-roads or branches in the direction taken by events. They are nodes or hinges in the structure which form cross-roads or branching points which force a movement into one or more possible paths satellites: relatively minor events which embellish the kernels, add detail or ‘flesh them out’
301
302
Research Methods in Politics
The narrative-of-the-self is critical. Narrative analysts argue that ‘the self is nothing but stories’. And it is also unstable. The self and its narratives are heavily contextualised. So you add, change, edit and embellish stories of yourself. In particular, you avoid recounting the tedium of boring, everyday life in favour of tales of rare, interesting moments. So the self is in a constant state of flux. To paraphrase Shakespeare:
all the world’s a stage in which you act the part dictated by the combination of the set and your interests at the time.
(This view contradicts the argument of Sartre and others that, despite peripheral flux, people have solid, stable, cores of authentic self.)
Case Study Consider the example of the famous interview of Princess Diana by Martin Bashir broadcast on BBC Panorama programme in 1995. You can watch the video-recording and inspect the transcript by inserting ‘Diana, Bashir’ in the search box on the web-site www.bbc.co.uk You can identify the narrative of the self employed by the Princess: • • • • • • • • • • • • • • •
innocent foal/Bambi eyes/ fairy tale princess brave, bright-eyed innocent (virgin) let down by advisers trapped in media spotlight frequent use of ‘I’ inference of brutal, uncaring husband, isolated, alone epithanies of engagement, marriage, Alice Springs walkabout other people are disappointing her support for other victims: drug addicts, battered people her ill-health, depression, self-disgust abused but loyal wife and mother relations with Charles are damned with ‘false praise’ intuitive knowledge of Charles’ relationship un-loved tactics adopted by Charles’ ‘friends’ to portray her as ill, unbalanced
Understanding and Adopting Discourse and Narrative Analysis
•
embarrassment, isolation and desolation • ‘Three of us in the marriage’ • ‘The [Royal] Family’ • the news media – abusive relations • denial of self-pity • • • •
ambassadorial aspirations … to do good Mother Theresa Diana as victim I will survive … the fairy tale is not yet ended
You can also identify the many discursive registers adopted: • • • • •
psychological language register confessional register packed with language of therapy marketing language register romantic love register (seduced by Hewitt)
Her victim strategy was successful in her narrative struggle with Prince Charles for the ‘heart of the nation’. Her narrative was defined and reinforced by Prince Charles’ own TV interview later (with Jonathan Dimbleby) in which he admitted his adultery.
Questions for discussion or assignments
1. Working in small groups, discuss your understanding or misunderstandings of discourse, discourse analysis, narrative and narrative analysis. 2. Critically assess Fairclough’s process of discourse analysis. 3. Select and obtain transcripts of two or more of the State of the Union Addresses by President George W. Bush. Complete a comparative discourse analysis of the transcripts. What conclusions can you draw? 4. Contrast and compare the narratives of the most recent general election manifestos of the British Conservative, Liberal-Democrat and Labour parties and US Republic and Democrat presidential elections. How do they compare to the narratives of French presidential rivals?
303
304
Research Methods in Politics FURTHER READING Howarth, D. (2000) Discourse. Buckingham: Open University Press. pp. 166. This excellent textbook is ‘essential reading’ for students contemplating the adoption of discourse analysis in their research. The author critically examines and compares the various concepts of discourse. Fairclough, N. (2000) Language and Power. 2nd edn. Harlow: Longman. pp. 224. This text is also ‘essential reading’ for students contemplating discourse analysis and wishing to learn how critical discourse analysis can be applied. Fairclough, N. (2000a) New Labour: New Language? London: Routledge. pp. 178. This is a very readable and wry demonstration of the application of critical discourse analysis. It is less technical than his previous text (Fairclough, 2000). It might have been strengthened by a comparison with the discourse of the earlier, new Labour (of the ‘white heat of technology’ and economic restructuring) successfully forged by Harold Wilson in the 1960s (which proved equally disappointing in the long term). Austin, J. L. (1962) How to Do Things with Words. Oxford: Oxford University Press. Williams, R. (1988/ 1976) Keywords. London: Fontana. Raymond Williams (1921–90) describes and analyses how the meanings of ‘keywords’ like culture, class, community and democracy have evolved through adaptation and struggle. He defines keywords as: ‘significant, binding words in certain activities and their interpretation; they are significant, indicative words in certain forms of thought. Certain uses [bind] together certain ways of seeing culture and society …’ (1988: 15).
Notes 1 Curt, Beryl C. (1994) Textuality and Tectonics; Troubling Social and Psychological Science. Buckingham: Open University Press, cited by Bergman, M. at Essex Summer School, 2003. 2 Gallie, W.B. (1956a), Essentially Contested Concepts. In Proceedings of the Aristotelian Society, 56: 67–198. 3 Churchill’s quote is widely attributed to his (post-war) assessment of China. However, he first coined it in 1939 about the Soviet Union in the wake of the Germany-Soviet Union Non-Aggression Pact. 4 Fairclough, N. (1989) Language and Power. London: Longman. 5 Howarth, D. (1995) Discourse Theory. In Marsh, D. and Stoker, G. (eds.) (1995) Theory and Methods in Political Science. Basingstoke: Macmillan. 6 Carver, T. (2002) Discourse analysis and the ‘linguistic turn’. In European Political Science, Autumn 2002, cited by Bergman, M. at Essex Summer School, 2003. 7 Hajer, M. (1995) The Politics of Environmental Discourse. Oxford: Oxford University Press, cited by Bergman, M. at Essex Summer School, 2003. 8 Stubbs, M. (1983) Discourse Analysis. Oxford: Blackwell, cited by Bergman, M. at Essex Summer School, 2003.
Understanding and Adopting Discourse and Narrative Analysis 9 Howarth, D. (1995) Discourse Theory. In Marsh, D. and Stoker, G. (eds.) (1995) Theory and Methods in Political Science. Basingstoke: Macmillan, cited by Bergman, M. at Essex Summer School, 2003. 10 Squire, C. (1995) Discourse Analytical Psychology. In Wilkinson, S. and Kitzinger, C. (eds.) (1995). Feminism and Discourse: Psychological Perspectives. London: Sage, cited by Bergman, M. at Essex Summer School, 2003. 11 Carver, T. (2002) Discourse analysis and the ‘linguistic turn’. In European Political Science, Autumn 2002, cited by Bergman, M. at Essex Summer School, 2003. 12 A ‘hoodie’ is the slang term popularly given in the UK to the hoods of jackets worn over the head by young men. It is interpreted both as a means of concealing their identity from CCTV cameras and as a self-proclaimed sign of the wearer’s separate and alienated identity. 13 Crystal, D. (1995) The Cambridge Encyclopaedia of Language. Cambridge: Cambridge University Press. 14 Jary, D. and Jary, J. (1995) Sociology. London: HarperCollins. 15 Chomsky, N. (1986) Knowledge of Language. New York: Praeger, cited by Crystal, D. (1995) The Cambridge Encyclopaedia of Language. Cambridge: Cambridge University Press. p. 409. 16 Crystal, D. (1995) The Cambridge Encyclopaedia of Language, Cambridge, Cambridge University Press. 17 McLuhan, M. (1962) The Gutenberg Galaxy. London: Routledge and Kegan Paul. 18 O’Sullivan, T., Hartley, J., Saunders, D., Montgomery, M. and Fiske, J. (1997) Key Concepts in Communications and Cultural Studies. London: Routledge. pp. 176–7. 19 Howarth, D. (2000) Discourse. Buckingham: Open University Press. pp. 2–5. 20 Burnham, P., Gilland, K., Grant, W. and Layton-Henry, Z. (2004) Research Methods in Politics. Basingstoke: Palgrave Macmillan. 21 Parker, I. (1992) Discourse Dynamics. London: Routledge, cited by Howarth, D. (2000) Discourse, Buckingham, Open University Press. 22 Laclau, E. and Mouffé, C. (1985) Hegemony and Socialist Strategy. London: Verso. Laclau, E. and Mouffé, C. (1987) Post-Marxism without Apologies, New Left Review, 166: 79–106. 23 Fairclough, N. (2000) Language and Power. 2nd edn Harlow: Longman. 24 Williams, R. (1976) Keywords. London: Fontana. 25 Fairclough, N. (2000a) New Labour: New language? London: Routledge. 26 The ‘3Rs’ is popular shorthand for ‘reading, rewriting and arithmetic’ widely regarded by older people as the core curriculum of education which had been marginalised by trendy, socialist, schools reform which had allegedly ‘left our children ill-qualified for the world of work’. 27 A ‘glottal stop’ occurs when air from the larynx suppresses the ‘t’ in words. In its extreme form, ‘bottle’ becomes ‘bo’ol’. A milder form is very popular with young middle-class people who are trying to play down their accents. Blair suppresses his consonants in interviews. 28 Barthes, R. (1966) Introduction to the Structural Analysis of the Narrative. Birmingham: University of Birmingham, cited by Bergman, M. at Essex Summer School, 2003. 29 Bruner, J.S. (1986) Actual Minds, Possible Words. Harvard, Mass.: Harvard University Press, cited by Bergman, M. at Essex Summer School, 2003. 30 Mumby, D.K. (1993) Narrative and Social Control. Newbury Park, Calif.: Sage, cited by Bergman, M. at Essex Summer School, 2003. 31 Berger, A.A. (1997) Narratives in Popular Culture, Media and Everyday Life. Thousand Oaks: Sage, cited by Bergman, M. at Essex Summer School, 2003. 32 Manning, P. K., and Cullum-Swan, B. (1998). Narrative, content, and semiotic analysis. In Denzin, N.K. and Lincoln, Y.S. (eds.), Collecting and Interpreting Qualitative Materials. pp. 246–73, Thousand Oaks, CA: Sage Publications, cited by Bergman, M. at Essex Summer School, 2003. 33 Carr, E.H. (1990/1961) What is History? London: Penguin.
305
306
Research Methods in Politics 34 Burke, K. (1966) Language as Symbolic Action. Berkeley: University of California Press, cited by Bergman, M. at Essex Summer School, 2003. 35 Kohler Reissman, C. (1993) Narrative Analysis. London: Sage, cited by Bergman, M. at Essex Summer School, 2003. 36 Plummer, K. (1995) Telling Sexual Stories: Power, Change and Social Worlds. London: Routledge, cited by Bergman, M. at Essex Summer School, 2003. 37 Czarniawska, B. (1998) A Narrative Approach to Organization Studies. London: Sage Publications, cited by Bergman, M. at Essex Summer School, 2003. 38 McRobbie, A. (1982) The Politics of Feminist Research: Between Talk, Text and Action. In Feminist Review, 12, pp. 46–58, cited by Bergman, M. at Essex Summer School, 2003. 39 Austin, J. L. (1962) How to Do Things with Words. Oxford: Oxford University Press. 40 Lubiano, W. (1992) Black Ladies, Welfare Queens and State Minstrels. In Morrison, T. (ed.) Race-ing Justice, En-Gendering Power: Essays on Anita Hill, Clarence Thomas and Social Reality. New York: Pantheon. Compare with Carver, T.(1997) Identity and Narrative in Prime-Time Politics: The Hill-Thomas Hearings. In Carver, T. and Hyvarinen, M. (eds.) Interpreting the Political: New Methodologies. London: Routledge.
Part V Communicating Research
Chapter 20
Writing-up
‘… and in the end was the beginning’ (Chapter XIII, The Last Time)
Teaching and learning objectives:
1. To consider when to begin writing up your research. 2. To identify essential contents. 3. To learn how to communicate your research so that it stands out favourably from other similar works. 4. To consider how best to maintain the interest of your readers.
Introduction Paradoxically, most research methods textbooks – and this is no exception – end by reviewing the process of writing the research report. This is termed – ungrammatically – writing-up. This implies that writing-up is the last part of the research process. This was true many years ago when academic research was divided into three separate stages of reading-up, fieldwork and writing-up. But, nowadays, writing-up is the first and last stages of the research process: Report-writing is not, or should not be, a frantic activity carried out at the end of the project. It is a process of varied stages all of which need to be recorded at the time they are completed. Your first drafts will certainly need to be revised and in some cases completely re-written but the foundations of the report should have been established at the planning stage. (Bell, 1997:152)1
310
Research Methods in Politics
Writing-up research Clegg memorably describes writing-up as the response to four questions (1998: 141):2 Why? How? What? So what?
Where ‘so what’ means: why is this research really so important? However, I believe that there are two additional, more significant questions, to be asked at the outset: To whom? That is, who or what is the readership? Where? That is, in what publication? What are their rules?
Readership The readership is all important. This will determine the language used. The reader’s prior knowledge or experience will – or should – also determine how concise or expansive the theoretical and conceptual exposition will need to be and, critically, how the conclusions are expressed. Conclusions must be relevant to the reader. Remember that your initial readers are likely to be your external examiners or the referees for journal articles. They have been selected because they are leading authorities in your research field. Readership will also affect the style of writing-up. The discipline of Politics contests objectivity. There are few facts but many interpretations. Political researchers are interpreters. So the clinical style of the natural scientist is rarely adopted. Instead, Politics researchers admit their own subjectivity, communicate their interest in the research topics and conclude with almost jaw-jutting assertions. Most researchers seek, especially in case study research, to tell a good story. However, the danger in this enthusiastic, assertive style of writingup is that the empiric experience – the facts such as they are – can become so inseparable from the interpretation as to appear highly selective or fabricated. Similarly, the great emphasis given to theoretical exposition can lead to springs of conclusions being drawn from a few swallows of evidence. A good research report should lead your readers stage-by-stage and paragraph-by-paragraph through the research process and enable them to fully share (or reject) your conclusions.
Writing-up
Publication rules The readership – or their publisher – will also have their own rules to maintain comparability and scholarship. They may seem anachronistic or petty, e.g. margin size and line spacing. But they cannot be ignored – especially with regard to word length – if your dissertation is to be approved or your research published. Remember, words count: illustrations, tables and graphs rarely do so. When you are preparing an MA or PhD dissertation, check the format required by the university regulations. These will include printing margins. In many cases, the double-spaced, single-sided pages of ink-and-parchment days have been superseded by more recent formats which include 1.5 line spacing and verso (printing on both sides). These new formats reduce the volume and weight of your research report and, arguably, make your dissertation easier to read. But note that regulations allowing double-sided submissions set specific additional requirements for gutter widths. You may be tempted to exceed the maximum word limit on the grounds that your department or university won’t count them. That’s correct. However, departments receiving a large number of dissertations of 20,000 words length know that these should be about 95 double-spaced pages. So they will recognise when the word limit is likely to have been exceeded. New regulations also require that an electronic copy is also submitted for interrogation by plagiarism software. The software will automatically calculate the word length.
Structure An important issue is the degree of structure (chapters, sub-headings, paragraphs) that you include in your writing-up. Following a rigid structure can help to order and organise material. But a structure of paragraph headings and sub-headings – especially where generic paragraph numbers and insets are used – can compartmentalise your material and arrest the flow and development of argument. Similarly, the incorporation of extensive tabulated data and statistical tests can cut across what you are trying to say. You can avoid this by using graphs and diagrams to simplify complex data (which can be annexed as technical appendices). Similarly, you can give ancillary evidence or explanation without compromising the continuity of the text by using extensive footnotes. Similarly, many researchers prefer to give full citation of texts in footnotes rather than using the Harvard system. Where footnotes (or endnotes) are used, they should be annotated at the end of the sentence and after the full stop. Do not use the Latinism op. cit. in the footnote to refer to a previously cited reference: repeat it in full. (Regrettably, the software used by Sage and other publishers to scan texts for publication precludes the use of footnotes.)
311
312
Research Methods in Politics
When to write-up As already noted, writing-up has traditionally been regarded as the last stage of the research process. However, the practice of leaving it all until the end is neither encouraged nor accepted. Instead, you will be expected to ‘write-up-as-youproceed’ in parallel with the research process. This is particularly important in the initial phases of contextualisation, theoretical and literature review, and hypothesis setting. Indeed, you will begin the research process by writing your outline and detailed dissertation proposals which will provide the foundation for your first chapter of the research report. Writing is an exploratory and creative process. Ideas can emerge in this process which might have eluded mere abstract contemplation. Writing-up-as-you-go provides a substantive record of progress for circulation to, and discussion with, supervisors and colleagues and as back-up in the event of some disaster befalling you. Most sponsors demand this parallel writing-up as a basis for monitoring – and controlling – the research process and budget. Most supervisors require their supervisees to submit draft chapters before any meeting as part of the teacher-student ‘supervision contract’ so that there is a substantive document to discuss. You will have already received expert advice on essay-writing in your earlier module guides. Most of that advice applies to research reports and need not be re-stated. Whilst there is no right way to write-up research in politics, the following conventions are recommended: • •
• • •
• • • • •
write generally in the third person; e.g. ‘it was considered that …’ however, you can use the pronoun ‘I’ to emphasise your own position – especially where you disagree with accepted theory: but use this sparingly so that it retains its impact avoid writing ‘one’: use ‘you’ to share the text with the reader, e.g. ‘as you have seen’, etc. avoid gender problems of ‘his/her’ by referring to subjects as ‘they’ if possible using the present tense to describe historical events may be a good literary device to create excitement: however, it can become very confusing in academic reports. Instead, use the past tense to describe the methodology and field-work; reserve the present tense for your analysis and conclusions qualify your assertions – but not excessively always state your assumptions clearly strike an appropriate balance between the coverage and erudition given to theoretical discussion, methodology, data, interpretations and conclusions don’t write the data speaks for itself : you must interpret the data try to avoid excessive and spurious accuracy in the text, e.g. don’t say ‘48.73%’, write ‘nearly half’: where you think that precise figures are helpful, give them in footnotes
Writing-up • •
follow normal grammatical conventions avoid excessive acronyms: after a time, the reader may forget what they stand for and their significance. Avoid inventing acronyms which are the same or similar to well-established ones, e.g. don’t label independent research analysis as IRA
• •
•
seek to maintain pace and interest ensure that your report is read by another person before you submit it: we are all very poor at spotting our own errors. If English is not your first language, then seek the assistance of a proof-reader be positive and up-beat: if you don’t sound confident of your research, then your readers are even less likely to have confidence in your report.
Structure and contents A suggested structure for research reports is (after Denscombe) threefold:3 BOX 20.1
Research report contents
I Title: Short, interest-grabbing, classifiable. A two-part title can be useful, consisting of a snappy, headline question and brief technical description (which can be readily identified by a search machine), e.g.: Left-right position matters, but does social class? An investigation into the outcome of 1992 British general election Note that capital letters are now much less used in titles than previously Abstract: This is a synopsis of the research of 200–300 words. The abstract is a statement of what you have done – not of your starting intentions. Dedications: UK academics generally find dedications somewhat embarrassing especially when they adopt the emotional style and length of Hollywood Oscar-winners’ acceptance speeches. If you believe that the sacrifices made by your family and friends should be made public, then include them in the copies which you have made after your degree has been awarded. (Continued)
313
314
Research Methods in Politics
Preface A personal statement explaining why you, as the researcher, chose the topic, your starting biases and prejudices, and what the research means to you, i.e. the reflexive relations between you and your social world. Often students speak of carrying out research which enables them to shed fresh light on specific experiences and to make sense of their past. Acknowledgements This provides thanks to academic supervisors, sponsors, respondents and research gatekeepers. The real purpose of the acknowledgements is to show off the many, many people who you have consulted or interviewed. You will normally end your acknowledgements by stating: ‘However, any errors are entirely my own’. Declaration A statement to the effect that: ‘The dissertation is entirely my own, original work with the exception of those academic and other sources which are separately identified in the text, fully attributed in the footnotes and listed in the bibliography’. In many scientific research papers, a statement of conflicts of interest is normally given here – especially in medical research. Such admissions or declarations would be a useful addition to Politics research papers. Contents List of Figures List of Tables Acronyms and Abbreviations II Chapter One Introduction The context, the aims of the research, research question and hypothesis, overview and (chapter by chapter) contents. Chapter Two Theory and literature review Critical review of theory, contemporary research and literature review leading to the identification of the gap in the literature.
Writing-up
Chapter Three Method of data collection • • • • • • • • • • •
concept/indicator link what methods were used why when did the research take place where how was access to the data and subjects obtained who was involved (population, sample, case studies) how many were involved how were they selected why the procedures were chosen limitations of methods employed
Chapter Four Data: e.g. the case study Chapter Five Analysis and Discussion Chapter Six Conclusions Has the hypothesis been confirmed or not? What contributions to theory has the research made? What are the implications and recommendations for follow-on research (and grants)? Provide a retrospective, critical evaluation of your research and proposals for its improvement. This shows that you are fully aware of the limitations and know how these might be overcome. III Appendices • • • • •
organised tabulated data questionnaires transcripts technical specification statistical methods
References and Bibliography Index
315
316
Research Methods in Politics
Bell provides a very good check-list for reviewing the final draft report (1997: 162):4 • • • • • • • • • • • • • • • • • • • • • •
Is the meaning clear? Is the report well written? (tenses, grammar, spelling, punctuation) Is the referencing well done? Does the abstract really give the reader a clear idea of what is in the report? Does the title indicate the nature of the study? Are the objectives of the study stated clearly? Are the objectives fulfilled? If hypotheses were postulated, are they proved or not proved? Has a sufficient amount of relevant literature been studied? Does the literature review provide an indication of the state of knowledge in the subject? Is your topic placed in the context of the area of study as such? Are all the terms clearly defined? Are the selected methods of data collection accurately described? Are they suitable for the task? Why were they chosen? Are any limitations of the study clearly presented? Have any statistical techniques been used? If so, are they appropriate? Is the data analysed and interpreted or merely described? Are the results clearly presented? Are tables, diagrams and figures well drawn? Are conclusions based on evidence? Have any claims been made that cannot be substantiated? Is there any evidence of bias? Any emotive terms or intemperate language? Is the data likely to be reliable? Would another researcher get the same results? Are recommendations feasible? Are there any unnecessary items in the appendices? Would you give the report a good grade of you were the examiner? If not, an overhaul is necessary.
Re-writing You are likely to find that your chapters are too long. Trimming or editing is unlikely to be sufficient. You must expect to completely rewrite every chapter at least once and plan your work programme accordingly. Word distribution There are no rules for the breakdown of words to individual chapters. However, following the advisory model distribution given below should ensure that proper
Writing-up
coverage is given to the theory and literature review and to your analysis and discussion of your data. Chapter 1 Chapter 2 Chapter 3 Chapter 4 Chapter 5 Chapter 6
Introduction Theory and Literature Review Method of Data Collection The Data, e.g., Case Study Analysis and Discussion Conclusions and Implications
5% 30% 5% 30% 20% 10%
Avoid the temptation to reduce the analysis and discussion. Adequate coverage is essential if your conclusions are to be shared.
Questions for discussion or assignments
1. Select three research reports from approved journals on the same topics, e.g. election turnout. Contrast and compare these. Which do you regard as being most successful in attracting your interest? Why? What improvements would you suggest to the other two? 2. Select a dissertation previously approved by your department for BA, MA or PhD purposes appropriate to your own degree. Critically evaluate the writing-up. What improvements would you make? Give specific examples. 3. Prepare a programme for your research project on a week-by-week basis. Integrate your proposed supervision meetings and writing, and re-writing of chapters.
FURTHER READING Similar dissertations by former students. Bell, J. (1997) Doing your own Research Project; A Guide for First-Time Researchers in Education and Social Sciences. Buckingham: Open University Press. pp. 244. Judith Bell gives very good advice on the research process. Chapter 11, pp. 127–50, Interpretation and Presentation of the Evidence is especially valuable as is Chapter 12, pp. 151–64, Writing the Report. Clegg, F. (1998) Simple Statistics: A Course Book for the Social Sciences. Cambridge: Cambridge University Press. pp. 200. This relatively short textbook focuses on the application of statistical techniques. However, Chapter 16 ‘… In the last analysis’, pp. 141–52 provides a very good and amusing account of the writing-up process.
317
318
Research Methods in Politics Denscombe, M. (1998) The Good Research Guide for Small-Scale Social Research Projects. Buckingham: Open University Press. pp. 247. This authoritative textbook provides complementary advice on writing-up – Chapter 12, pp. 225–37 – and an excellent checklist on p. 238.
Notes 1 Bell, J. (1997) Doing your own Research Project; A Guide for First-Time Researchers in Education and Social Sciences. Buckingham: Open University Press. 2 Clegg, F. (1998) Simple Statistics: A Course Book for the Social Sciences. Cambridge: Cambridge University Press. 3 Denscombe, M. (1998) The Good Research Guide for Small-Scale Social Research Projects. Buckingham: Open University Press. 4 Bell, J. (1997) Doing your own Research Project; A Guide for First-Time Researchers in Education and Social Sciences. Buckingham: Open University Press.
Glossary of Terms
α data: β data: Abscissa:
new data generated in the course of the research. data abstracted from other sources. the horizontal or x-axis on a graph. (Vertical or y-axis is termed the Ordinate.)
Abstract: a synopsis of 200–300 words which follows the title page of the research report. The abstract is a summary of the research done – not of intent. An abstract should clearly identify the research topic, research question, main theoretical framework, methodology, conclusions and assess its contribution to the topic and research field. Access: means by which the subject is approached and contacted. Acknowledgements: statement in the research report which names and thanks the academic supervisors, sponsors, respondents and research gatekeepers who helped the researcher to complete their research project. Action research (method): (generally) obtaining data by direct participation by the researcher in a group of actors experiencing particular conditions in order to understand better the institutional and other barriers they face in the struggle towards transformation. Analysis: an objective-seeking, rational process which essentially involves breaking down – deconstructing – the whole into its constituent parts to identify their relationships and structure. Anecdotal evidence: evidence in story form based on unique, personal experience. It is often used by quantitative researchers as a pejorative term to describe qualitative researchers’ evidence. Antinaturalism: belief that using methods developed in the natural sciences is inappropriate to social science research, i.e. opposed to Positivism. Attitude: learned and persisting positive or negative evaluation or predisposition to people, groups, policies, etc. Attribute: qualitative variable, e.g. gender. Baseline or Benchmark: starting level of dependent variable before the independent variable is changed. Behaviouralism: specific to Politics: advocacy of research based on empirical observation of behaviour. Behaviourism: the school of psychology associated with Harvard professor B.F. Skinner (1904–90), architect of the controllable ‘Skinner box’ and ‘programmed learning’. Its key tenet is that only observable behaviour may be scientifically studied. Bias: prejudice: pre-disposition towards or against subject, movements and interpretations: a source of systematic errors in research findings. Case study: a ‘sample of one’ event, instance, state or sub-unit at one point in time.
320
Glossary of Terms Case study method: gathering and analysing data from an individual example as a means of making broader generalisations. Causality (‘cause and effect’ or causation): the relationship between two variables when one causes a change in the character of the other. Census: count of 100% of population. Central limit theorem: this states that, the larger the sample, the greater the tendency for the sampling distribution of the means to follow a normal distribution. Central tendency: concentration (cf. dispersion) of values statistically represented by mean, mode, median, variance and standard deviation. Classification data: information sought in questionnaire surveys which relate to the circumstances of the subject rather than their beliefs and attitudes. They include: sex, age group, socio-economic group, education, housing tenure and dependants. Some can be obtained by proxy (e.g. newspaper readership as an indicator of social class). Others can be guessed by careful observation but should be checked with the subject. These variables may well include important independent variables. For example, research has shown that older people are more likely to participate in political activity than others. Class interval: groupings of data selected by the researcher to enable a frequency distribution to be constructed to represent and analyse data, e.g. ages 36–45, etc. Closed questions: questions which seek – or receive – closed answers. Closed answers are generally short and confined to yes, no or don’t know or specific answers, e.g. date of birth. Coding: essentially the process of replacing or substituting groups of words, phrases or sentences by letters or numbers (or a combination). Unlike a cipher, a code is not designed to hide the original meaning of the word or phrase. Indeed, an essential property of an effective code is that it should be readily understandable to users and readers. A word or phrase can be given several different codes. Cohort: groups sharing characteristic(s), e.g. males 18–29. Comparative method: obtaining data from a population or sample of equivalent states or sub-units, sectors or groups at the same points in time. It is the principal method used in comparative politics and international relations. Concept: basic idea. Content analysis: set of techniques to systematically analyse contents of books, reports and transcripts of speeches, etc. by measuring repetitions and bias. Contingency: conditional relationship between variables. Control Group: sample not subjected to changes in independent variable. Correlation: statistically proven relation between two variables. But correlation does not necessarily indicate causation. It is measured statistically by the correlation coefficient which extends from +1 (perfect positive correlation) through 0 (no correlation) to −1 (perfect negative correlation, i.e inverse relationship). Counterfactual: what you would expect to happen but didn’t (e.g. Sherlock Holmes’ ‘dog that didn’t bark in the night’). Counterfactuals may be possible evidence of unseen, dominant power.
Glossary of Terms Covariance: phenomenon where two variables appear to change at the same time. Data: raw material which may be sorted and processed into information. Data has no meaning by itself. Note that although data is plural, writing data are is now considered pedantic. Decile:
a tenth part.
Declaration: a statement that follows the Acknowledgements page of the research report which states that ‘This dissertation is entirely my own, original work with the exception of those academic and other sources which are separately identified in the text, fully attributed in the footnotes and listed in the bibliography. Signed ... Date ...’. Deductive research: the research approach which develops specific hypotheses from general principles or theories and then seeks to confirm or infirm the hypotheses: ’moving from the general to the specific’ (cf. deductive research: moving from specific to the general). Demography: the study of population and influences. Demographic data therefore includes population size and density, age and male/female distribution, household size, income, health and life expectancy, socio-economic distribution, ethnicity and migration, etc. Dependent variable: consequent event; outcome; variable which changes as a consequence of change in the independent variable; in social research, this is generally represented by y. Descriptive statistics: quantitative, numeric information which describes or summarises primary data. Diachrony: the study of the historical development of language. Discourse: ‘a specific ensemble of ideas, concepts and categorisations that are produced, reproduced, and transformed in a particular set of practices and through which meaning is given to physical and social realities’ (Hajer, 1995: 44). Discourse analysis: ‘analysing the way systems of meaning or discourses shape the way that people understand their roles in society and influence their political activity’ (Howarth, 1995). Directional hypothesis: (one-tailed test): an experimental hypothesis which states a particular direction of outcome, e.g. that the dependent variable will increase as the independent variable changes. A non-directional hypothesis – where a causal relationship is hypothesised but what happens to the dependent variable when the independent variable changes is unclear – is called two-tailed. Ecological fallacy: false reasoning which allocates attributes to individuals from information about groups. Eigenvalue: (Eigenwerte) used in Principal Component and Factor Analysis as a measure of the variation in a group of variables attributable to one factor. Empirical: data collected by systematic observation or experience (cf. theoretical). Enumeration District (ED): the basic building block of the UK census consisting of around 200 households. They are the lowest level of census data publicly available until individual household forms are released after 100 years. Epistemology: study of knowledge. Ethnography: study of social and cultural life of social or ethnic groups often by participant observation.
321
322
Glossary of Terms Experimental method: a ‘sample of two’; a control sample and an experimental sample drawn (ideally randomly) from the population under study. Experimental sample:
the sample subjected to the independent variable.
Facts: the available information. They present a picture of events. Truth is the reality behind the facts. Sometimes the facts may obscure the truth – perhaps deliberately so. Fallacy of composition: false reasoning that accords individual’s attributes to groups. Falsification: after Popper, the doctrine that theories can never be confirmed: they can only be refuted. So verification by repeated experiment can never be conclusive. Field research: research undertaken in and of the real, social world (cf. laboratory research). Focus group: group of subjects recruited by researchers to emulate the spontaneous discussion of people who broadly share the same circumstances or identity. In particular, the focus group should draw upon the respondents’ attitudes, feelings, beliefs, experiences and reactions in a way that would not be feasible by other methods, for example, observation, one-to-one interviewing or questionnaire surveys’ (Gibbs, 1997). Gambler’s fallacy: prediction based on continuation of or non-continuation of coincidence of random events, e.g. the belief that the prior incidence of heads or tails will influence the outcome of the next spin of the coin. Generalisability: the facility to draw reliable inferences about a population from a sample. Grounded research: approach first developed by Glaser and Strauss where the findings are entirely grounded in the data collected. Guttman scale: a scale used in questionnaire surveys as a means of determining the respondent’s intensity and consistency of attitudes to a specific topic by asking them a series of related questions. Hermeneutics:
the study of interpretation.
Hypothesis: a specific theoretical proposition presented for testing by research. It is the researcher’s initial answer to their research question. Independent variable: causal event or driver; the variable causing change to the dependent variable. Inductive research: the approach which develops general hypotheses from observation of specific phenomena (cf. deductive research: moving from general to the specific). Inferences: generalisations about the research population drawn from samples or other limited information Inferential statistics: ‘a branch of applied mathematics or statistics based on a random sample. They let the researcher make precise statements of the levels of confidence they have in the results of a sample being equal to the population parameter’ (where a parameter is a characteristic of the entire population drawn from a sample’) (Neuman, 2003: 539, 541); a statistical method of drawing reliable generalisations about a population from sample data. Information: data which has been sorted and classified to inform the research; information has meaning. Integer: whole number, e.g. 1, 2, 113.
Glossary of Terms Iteration: repeated procedure. Language: the facility of speech. Law of averages: doctrine that variations and errors in numeric data will be reduced by repeated experiments. Law of large numbers: doctrine that, as the sample size increases, it will assume the general character of the population from which it has been drawn. In particular, the sample mean will approach the population mean. Likert scale: a scale used in questionnaire surveys to measure the intensity of attitudes of subjects. The subject is asked whether they strongly agree, agree, neither agree nor disagree, disagree or strongly disagree with a statement. Linguistics:
‘the science of language’ (Crystal, 1995: 425).
Longitudinal method: gathering data from the observation of a cohort – a group of people, states or organisations, etc. sharing one or more common characteristics – over an extensive period of time. The common characteristics may include age, education, place or specific condition. Mean: a measure of central tendency. There are arithmetic, geometric and harmonic means. The arithmetic mean, X is generally called the average and is the sum of terms X of a population, divided by the number of terms in the population N, i.e. X = X/N. In a sample, the (arithmetic) mean is denoted by x and the number of terms by n and is given by the formula x = x/n. Median: another measure of central tendency. It is the middle term in a ranked series, e.g. in the series 2,3,4,5,6, then 4 is the median. In the series 2,3,4,5, then the median is 3.5. Mode: another measure of central tendency. It is the most common or frequent term, e.g. in the series 2,2,3,3,3,4 then 3 is the mode. Method of agreement: term used by Mill to describe the comparative method adopted in which apparently different cases having the same outcomes are compared to identify the common – potentially causal – elements. Method of difference: term used by Mill to describe the comparative method adopted in which apparently similar cases having different outcomes are compared to identify the absent or common – potentially causal – elements. Method of concomitant variations: comparative method used when outcomes of different magnitude occur to identify the potential causal variable by comparing their magnitude. Moving average: a technique used in Time series analysis to smooth out seasonal and irregular movements to identify the underlying trend. Naturalism: this has two, almost uniquely, contradictory conceptualisations. The first – the minority view held by Giddens – is that naturalism refers to the adoption in the social sciences of models of inquiry from the natural sciences. So naturalism is effectively a new, nonpejorative name for positivism. But the dominant view is that naturalism rejects positivism in favour of a free-standing method of inquiry which is essentially humanistic and hermeneutic (after Hermes, interpretative) and concerned with social meanings (actor’s beliefs, motives, purposes, reasons etc. which lead to social actions) rather than frequency. Whilst positivism emphasises scientific, controlled, replicable experimentation, naturalism seeks to study everyday
323
324
Glossary of Terms life in naturally-occurring situations. Naturalism is experiental: researchers seek to see the world through the eyes of the subject and to understand what people feel, interpret and do. Moods, ideas, identity and beliefs are the stuff of naturalistic research. Nomination: method of recruiting a non-probability sample in which a social gatekeeper or other intermediary is asked to nominate – name – potential recruits. Non-probability samples: samples where members of the population do not have an equal chance of being selected. They are not statistically reliable. They cannot generate generalisable data. Null hypothesis: ‘the statistical proposition of no difference’ (Clegg, 1998: 61). The proposition – which must be refuted – that data favourable to our experimental hypotheses occurred by chance (rather than any causal relationship between the variables). In sampling research, the null hypothesis is that the sample has not been drawn randomly from or is representative of the population and that therefore no generalisable inferences can be made. Ockham’s razor: ‘the principle of parsimony’: the simplest theory is the best. Odds: ratio of probable outcome in sequence of events, e.g. odds of a coin falling heads is 1:2. Also expressed as a probability of 0.5 (i.e., 5 out of 10) or 50%. Operationalisation: the process of transposing abstract theoretical concepts into observable, measurable variables. Outliers: exceptional lowest or highest terms (or first or last ranked terms) which lie outside the general distribution and which can skew measures of central tendency and distort interpretation. Panel: a sample of individuals from a population who are repeatedly surveyed over time to provide longitudinal or tracking data. They do not meet. Poisson distribution: a special distribution used when the sample size is very large and the probability of incidents, e.g. illness, is very small. (cf. Binomial Distribution used where there are only two potential outcomes, e.g. heads or tails). Political elites: individuals who exercise disproportionately high influence on the outcome of events or policies in the research topic. Population: the entirety of the group – of people or objects – under study and from which the sample is drawn. Positivism: After Comte, the doctrine that the only true knowledge is scientific knowledge. Positivism emphasises scientific, controlled, replicable experimentation. Auguste Comte (1798– 1859) argued that all societies were fated to move from a theological stage of ‘fictitious knowledge’ (in which all otherwise inexplicable phenomena were attributed to ‘spiritual’ forces) via an intermediate metaphysical stage to a ‘positive’ stage. The underlying beliefs of the theological stage were medieval faith and custom in which the family was the social base. Those of the metaphysical stage were Enlightenment philosophy, the ‘scientific revolution’ and the nation state. The state of knowledge of the positivist stage was scientific when rational logic and humanity prevailed. Power of a test:
sensitivity of a test to identify relationships between variables.
Glossary of Terms Preface: a personal statement explaining why the researcher has chosen the topic, their starting biases and prejudices and what the research means to them, i.e. the reflexive relations between them and their social world. Primary: data or information is original, unedited and first-hand (cf. secondary information). Probability: the likelihood that an event will occur. For example, if an event is certain – like death albeit in the distant future – then the probability is 100% or 1.00. Alternatively, a remote possibility – like winning the UK national lottery next week – may be 0.001% or 0.00001. A one-in-four likelihood would be expressed as a probability of 25% or 0.25. A bookmaker would give odds of 4-1 on an event having a 25% probability. Proposition: an unproven, generalised, theoretical explanation. R2 : used in multivariate analysis as a measure of the extent to which the dependent variable can be attributable to two or more independent variables. Random Walk: method adopted by interviewers to access a sample of households. The researcher begins at the centre of the town or city calling at every nth house in the first street, turning left at the first intersection, right at the second and so on until the boundary of the urban area is reached. Rapport: French word used to describe a desirable state of mutual confidence, empathy, trust and liking between the researcher and their subject(s). Reflexivity: intellectual self-awareness gained through self-examination to identify deepseated biases. Regression analysis: analysis carried out to calculate the relationship between the dependent variable (y) and one (linear regression) or more (multivariate regression) independent variables (x, x 1 . . .) represented in the formula y = a + bx where b is the regression coefficient and gradient (or slope) of the best fit line. Reliability: literally, the extent to which we can rely on the source of the information and, therefore, the information. Reliable information is dependable, trustworthy, unfailing, sure, authentic, genuine, reputable. Consistency is the main measure of reliability. Note that a measure may be reliable but not accurate and vice-versa, e.g. a reliable weighing machine may give an inaccurate but consistent recording of the same weight. Research component analysis: method used to evaluate published research in which the researcher dissects the published research into 15 or more components which are assessed separately. Research effect: the tendency for the population being studied to modify its behaviour and, therefore, provide unrepresentative data. Residual error: the difference between predicted and actual data. Sample: a group of subjects chosen randomly or non-randomly from the population and from whom it is intended that generalisations can be made. Sampling error: potential variance between an attribute of the sample and the population. Sample frame: a list or schedule of the population from which the sample will be drawn. It may be a membership list, a directory of engineering firms or electoral roll.
325
326
Glossary of Terms Secondary data or information which is second-hand and may be previously edited or interpreted material. Semantic differential: a five or seven-point scale used in questionnaire surveys to assess respondents’ beliefs, etc., between pairs of words with diametrically opposite meanings, e.g. honest and dishonest. Semi-structured interview: the most widely-used type of interview – especially in elite interviews. The researcher uses a pre-designed schedule of a limited number of topic-related questions and, pre-determined, alternative supplementary questions which question aspects of the answer received. The format of the semi-structured interview is essentially one of question-and-discussion. Semiotics: ‘study of the properties of signs and signalling systems especially as found in all forms of human communication’ (Crystal, 1995: 430). Show-cards: printed lists used by researchers administering questionnaire surveys to ask sensitive questions. For example, a card might contain a list of weekly incomes marked A, B, C, etc. The subject can then be asked to give the letter corresponding to their own income range. Significance: characteristic of importance of data or calculated result. Statistical Significance: the likelihood that any similarity between sample and population means occurred entirely by chance. Snowball sampling: non-random sampling in which subjects identify others similar to themselves for further research, e.g. drug-users. Split-half strategy: an approach adopted by some researchers towards focus groups. Half the group are interviewed individually before the focus group and the others afterwards. In this way, the role of the group in developing or imposing new collective perceptions or understandings can be identified better. Spurious relation: where variables appear statistically related or correlated but bear no dependent or causal relationship to one another. Standard deviation (SD or σ ): another measure of central tendency expressing the average by which individual terms are less or more than the mean. It is calculated as the square root of the variance. Standard error of the mean (SE): a statistical measure of the variance between a sample and the population: the standard deviation of a sampling distribution of the mean. Structural content analysis: assesses how the text is presented and reported rather than the frequency of key words used. Substantive content analysis: distribution of key words.
analyses selected texts by counting the frequency and
Subject: the smallest unit of population adopted for research, e.g. individual people, households, firms. The term is most widely used by qualitative researchers to de-objectify the respondent or interviewee. Synchrony: the study of the ‘lived’ language of the present.
Glossary of Terms t-Test: Tells:
statistical test of the significance of the variance between two groups. term given by body language analysts to the inadvertent signs given by subjects.
Theoretical saturation: in grounded research, term used by Glaser and Strauss to describe the stage reached when additional data fail to reveal any further significant findings. Theory: simply stated, a statement of general principles of the underlying relationship in phenomena or events. Theory may be expressed as laws, propositions, arguments or hypotheses. Historically, theory has generally been descriptive in terms of describing and, therefore, explaining relationships. Some descriptive theories may be called laws, for example, Newton’s law of universal gravitation, where they are accepted as having been universally verified by observation. Newton’s law was subsequently superseded by Einstein’s theory of relativity. The title law is therefore rarely given to theories. Alternatively, theory may be normative where it proposes what ought to be the relationship. For example, the theory of egalitarianism is that all people should be treated equally regardless of origin or circumstances. ‘thick/thin’: case study and a quarter; term used to describe the research method used when the researcher seeks to extend the findings of their single case study by quickly testing them against another comparable case. Triangulation: term derived from navigation and surveying; a method of corroborating data by seeking accounts from three or more perspectives and media. Truth: the reality behind the facts (the available information). Sometimes the facts may obscure the truth – perhaps deliberately so. Type I Error: wrongly rejecting the null hypothesis. Type II Error: wrongly accepting the null hypothesis. Universe: population. Validity: accuracy and appropriateness of measurement; ‘the extent to which a measure, indicator or method of data collection possesses the quality of being sound or true as far as can be judged. . . . in the social sciences generally, the relationship between indicators and measures and the underlying concepts they are taken to measure is often contested’ (Jary & Jary, 1995: 714). Variance: a measure of central tendency calculated as the sum of the squares of the differences between each term and the mean divided by the number of terms. Squaring the difference makes negative differences positive. In a sample, the sum of the squared differences is divided by the number of terms minus one, i.e. n − 1. Verstehen: term adopted by Max Weber (1864–1920) to describe the necessary empathetic understanding by researcher of their subject; ‘trying to see the world through the subjects’ eyes’. Vignette: ‘Short story generated by the researcher and focusing on hypothetical characters in particular situations. Interviewees are asked what they would do in these circumstances or, alternatively, how they think that a third party might react. The latter mode of question specifically distances the interviewer from the issues being studied and, in this sense, is impersonal and less threatening’ (Arksey & Knight, 1999: 94–5).
327
328
Glossary of Terms Wave: term used in longitudinal research to describe each phase of cross-sectional survey of the research cohort or panel. For a more comprehensive and technical glossary of terms with examples and illustrations, see: Jary, D. and Jary, J. (1995) Sociology Dictionary. Glasgow: HarperCollins. Vogt W. P. (1999) Dictionary of Statistics & Methodology. London: Sage Publications.
Key Formulae and Symbols
Number of cases or terms Individual values Arithmetic mean Variance Standard deviation Variance Standard deviation Standard error of the mean, SE
Population N X X σ2 σ or SD (X − Xi )2 √ N (X − Xi )2 N n/a
Confidence limits 95% i.e. in 95 out of 100 samples, the population mean will lie between x + (1.96 × SE ) and x − (1.96×SE) Standard error of the proportion, SEp √ (or percentage) (pq/n)
Sample n x x s2 s or sd (x − xi )2 √(n − 1) 2 (x − xi ) (n − 1) sd √ n ±(1.96 × SE )
(where q = 1 − p)
Bibliography
Amis, K. (1997) The King’s English – A Guide to Modern Usage, cited by Mount, H. (2006) Amo, Amas, Amat and All That: How to Become a Latin Lover. London: Short Books. p. 40. Arksey, H. and Knight, G. (1999) Interviewing for Social Scientists. London: Sage. Austin, J. L. (1962) How to Do Things with Words. Oxford: Oxford University Press. Bachrach, P. and Baratz, M. (1962) Two faces of power. In American Political Science Review, 56: 947–52. Barthes, R. (1966) Introduction to the Structural Analysis of the Narrative. Birmingham: University of Birmingham. Bell, J. (1997) Doing your own Research Project; A Guide for First-Time Researchers in Education and Social Science. Buckingham: Open University Press. Beresford, P. and Hoban, M. (2005) Effective Participation in Anti-poverty and Regeneration Work and Research. York: Joseph Rowntree Foundation. Berg, B. (2001) Qualitative Research for the Social Sciences. London: Allyn & Bacon. Berger, A.A. (1997) Narratives in Popular Culture, Media and Everyday Life. Thousand Oaks: Sage. Brier, A.P. and Hopp, B. (2005) HAMLET a multidimensional scaling approach to text-oriented policy analysis. In Journal of Diplomatic Language, 2 (1). Bruner, J.S. (1986) Actual Minds, Possible Words. Harvard, Mass: Harvard University Press. Bryman, A. (1988/96) Quantity and Quality in Social Research. London: Routledge. Budge, I. (1997) Chapter 1: Party policy and ideology: reversing the 1950s? In Evans, G., and Norris, P. (1999) Critical Elections. London: Sage Burke, K. (1966) Language as Symbolic Action. Berkeley: University of California Press. Burnham, P., Gilland, K., Grant, W. and Layton-Henry, Z. (2004) Research Methods in Politics. Basingstoke: Palgrave Macmillan. Carr, E.H. (1990/1961) What is History? London: Penguin. Carver, T. (2002) Discourse analysis and the ‘linguistic turn’: In European Political Science, Autumn 2002. Chomsky, N. (1986) Knowledge of Language. New York: Praeger. Clegg, F. (1998) Simple Statistics: A Course Book for the Social Sciences. Cambridge: Cambridge University Press. Cole, G.D.H. (1938/66) The Common People. London: Methuen. Cole, J. (1996) As it Seemed to Me: Political Memoirs. London. Weidenfeld & Nicolson. p. 64. Crossman, R.H. (1976) The Diaries of a Cabinet Minister 1 Minister of Housing 1964–66. London: Hamish Hamilton and Jonathan Cape. p. 590. Crouch, C. (2004) Post-Democracy. Cambridge: Polity. Crystal, D. (1995) The Cambridge Encyclopaedia of Language. Cambridge: Cambridge: University Press. Curt, Beryl C. (1994) Textuality and Tectonics; Troubling Social and Psychological Science. Buckingham: Open University Press.
Bibliography Czarniawska, B. (1998) A Narrative Approach to Organization Studies. London: Sage Publications. Dahl, R. (1969) Who Governs? Democracy and Power in an American City. New Haven: Yale University Press. A new edition was published in 2005, ISBN 03001 03921. Denscombe, M. (1998) The Good Research Guide for Small-Scale Social Research Projects. Buckingham: Open University Press. De Vaus, D.A. (2001) Surveys in Social Research. London: Routledge. De Vaus, D. (2002) Analysing Social Science Data: 50 Key Problems in Data Analysis. London: Sage. Downs, A. (1957) An Economic Theory of Democracy. New York: Harper & Brothers. Engels, F. (1845/1993) The Condition of the Working Class in England. Oxford: Oxford University Press. Evans, G. and Norris, P. (1999) Critical Elections. London: Sage. Fairclough, N. (2nd edn 2000) Language and Power. Harlow: Longman. Fairclough, N. (2000a) New Labour: New Language? London: Routledge. Field, A. (2005) Discovering Statistics Using SPSS (and sex, drugs and rock ‘n’ roll) 2nd edn. London: Sage. Filstead, W.J. (1979) Qualitative methods: a needed perspective in evaluation research. In Cooke, T.D. and Reichardt, C.S. (eds) Qualitative and Quantitative Methods in Evaluation Research. Beverly Hills: Sage. Foddy, W. (1993) Constructing Questions for Interviews and Questionnaires. Cambridge: Cambridge University Press. Foot, M. (1973) Aneurin Bevan: A Biography, 2. 1945–60, London: Davis-Poynter. Franke, R.H. and Kaul, J.D. (1978) The Hawthorne experiments: first statistical interpretation, American Sociological Review, 43: 623–43. Friedrich, C.J. (1937) Constitutional Government and Politics. New York: Harper & Brothers. Galbraith, J.K. (1992) The Culture of Contentment. London: Penguin. Gallie, W.B. (1956a), Essentially contested concepts. In Proceedings of the Aristotelian Society, 56: 167–98. Gibbs, A. (1997) Focus groups. In Social Research, 19. Guildford: University of Surrey. Gilbert, N. (2003) Researching Social Life. London: Sage. Glaser, B.G. and Strauss, A.L. (1967/1995) The Discovery of Grounded Theory: Strategies for Qualitative Research. New York: Aldine de Gruyter. Glaser, B.G. (1978) Theoretical Sensitivity: Advances in the Methodology of Grounded Theory. Mill Valley: Sociology Press. Greenfield, S., Singh, S., Tallack, P. et al. (2001) The Science Book. London: Cassell & Co. p. 302. Grigg, J. (1978) Lloyd-George: The People’s Champion. London. Eyre Methuen. Gubrium, J. F. and Holstein, J. A. (1997) The New Language of Qualitative Method. Oxford: Oxford University Press. Hajer, M. (1995) The Politics of Environmental Discourse. Oxford: Oxford University Press. Harris, K. (1967) Conversations. London: Hodder and Stoughton. Harrison, L. (2001) Political Research: An Introduction. London: Routledge. Hart, C. (1998) Doing a Literature Review. London: Sage. Harvey, L. (1990) Critical Social Research. London: Unwin Hyman. pp. 19–32. Heath, A. and Taylor, (1999) New sources of abstention. In Evans, G. and Norris, P. (eds) Critical Elections: British Parties and Voters in Long-Term Perspective. London: Sage.
331
332
Bibliography Held, D. and Leftwich, A. (1984) Chapter 8: A discipline of politics? In Leftwich, A. (ed.) What is Politics? Its Activity and Study. Oxford: Blackwell. Heywood, A. (2000) Key Concepts in Politics. Basingstoke: Palgrave Macmillan. Hoinville, G., Jowell, R. et al. (1977) Survey Research Practice. London: Heinemann. p. 61 cited in De Vaus, D.A. (2001) Surveys in Social Research. London: Routledge. p. 73 Honderich, T. (ed.) (1995) The Oxford Companion to Philosophy. Oxford: Clarendon Press. pp. 405–6. Hopkin, J. (2002) Chapter 12: Comparative methods. In Marsh, D. and Stoker, J. (eds) Theory and Methods in Political Science. Basingstoke: Palgrave Macmillan. Howarth, D. (2000) Discourse. Buckingham: Open University Press. Hume, D. (1739) A Treatise on Human Nature. 1: 467–8. Huxley, A. (1932/1994) Brave New World. London: HarperCollins. Jary, D. and Jary, J. (1995) Sociology. Glasgow: HarperCollins. Jenkins, R. (2001) Churchill: A Biography. London: Macmillan. Kanji, G. K. (1999) 100 Statistical Tests. London: Sage. Kinnear, P. R. and Gray, C.D. (2004) SPSS for Windows Made Simple. Hove: Psychology Press Ltd. Klingermann, H-D., Hofferbert, R. and Budge, I. (1994) Parties, Policies and Democracy. Boulder: Westview Press. Kohler Reissman, C. (1993) Narrative Analysis. London: Sage. Krippendorff, K. (1980) Content Analysis: An Introduction to its Methodology. London: Sage. Lacey, A. (1995) In Honderich, T., The Oxford Companion to Philosophy. Oxford: Oxford University Press. pp. 145, 705–6. Laclau, E. and Mouffé, C. (1985) Hegemony and Socialist Strategy. London: Verso. Laclau, E. and Mouffé, C. (1987) Post-Marxism without apologies, New Left Review, 166: 79–106. Landman, T. (2003) Issues and Methods in Comparative Politics: An Introduction. London: Routledge. Lasswell, H. (1958) Who gets What, When, How. New York: Meridian. Lee, R. L. (1995) Dangerous Fieldwork. London: Sage. Leftwich, A. (ed.) (1984) What is Politics? Its Activity and Study. Oxford: Blackwell. Leftwich, A. (2000) States of Development: On the Primacy of Politics in Development. Cambridge: Polity. Leftwich, A. (ed.) (2004) What is Politics? Cambridge: Polity Press. Levine, D. and Stephan, D. (2005) Statistics: Even you can Learn. London: Pearson Prentice Hall. Lewis-Beck, M. S. (1995) Data Analysis: An Introduction. London: Sage. Lubiano, W. (1992) Black ladies, welfare queens and state minstrels. In Morrison, T. (ed.) Race-ing Justice, En-Gendering Power: Essays on Anita Hill, Clarence Thomas and Social Reality. New York: Pantheon. Lukes, S. (1974) Power: A Radical View. London: Macmillan. Mace, C.A. (1962) The Psychology of Study. Harmondsworth: Penguin. Manning, P. K. and Cullum-Swan, B. (1998). Narrative, content, and semiotic analysis. In Denzin, N.K. and Lincoln, Y.S. (eds), Collecting and Interpreting Qualitative Materials, pp. 246–73. Thousand Oaks, CA: Sage. Marsh, D. and Stoker, G. (eds) (1995) Theory and Methods in Political Science. Basingstoke: Macmillan.
Bibliography Marsh, D. and Furlong, D. (2002) Chapter 1: A skin is not a sweater: ontology and epistemology in Political Science. In Marsh, D. and Stoker, G. (eds) Theory and Methods in Political Science. Basingstoke: Palgrave Macmillan. Marsh, D. and Stoker, G. (eds) (2002) Theory and Methods in Political Science. Basingstoke: Palgrave. pp. 1–16. Maslow, A.H. (1970) Motivation and Personality. New York: Harper and Row. Mayer, L.C. (1989) Redefining Comparative Politics: Promise versus Performance. London: Sage. Mayo, E. (1949) Hawthorne and the Western Electric Company. London: Routledge. McEvoy, J. (2006) Elite interviewing in a divided society: lessons from Northern Ireland. In Politics 26 (3): 184–91. McLuhan, M. (1962) The Gutenberg Galaxy. London: Routledge and Kegan Paul. Miles, M.B. and Huberman, A.M. (1994) Qualitative Analysis: An Expanded Sourcebook. London: Sage. McRobbie, A. (1982) The Politics of feminist research: between talk, text and action. In Feminist Review, 12, pp. 46–58. Mill, J.S. (1843) A System of Logic. London: Longman. Moore, B. (1966) Social Origins of Dictatorship and Democracy. Boston: Beacon Press. Morris, D. (1977) Manwatching; A Field Guide to Human Behaviour. London: Jonathan Cape. Morris, D. (1994) Bodytalk: A World Guide to Gestures. London: Jonathan Cape. Morris, D., Collett, P., Marsh, P., and O’Shaughnessey, M. (1979) Gestures: Their Origins and Distribution. London: Jonathan Cape. Mumby, D.K. (1993) Narrative and Social Control. Newbury Park, Calif.: Sage. Neuman, W.L. (2003) Social Research Methods: Qualitative and Quantitative Approaches. London: Allyn and Bacon. Nicolson, N. (1953) King George V. London: Routledge. Chapter XXVII. Norris, P. and Lovenduski, J. (1995) Political Recruitment: Gender, Race and Class in the British Parliament. Cambridge: Cambridge University Press. Oakley, A. (2000) Experiments in Knowing: Gender and Methods in the Social Sciences. Cambridge: Polity Press. pp. 37–8. O’Sullivan, T., Hartley, J., Saunders, D., Montgomery, M. and Fiske, J. (1997) Key Concepts in Communications and Cultural Studies. London: Routledge. Parker, I. (1992) Discourse Dynamics. London: Routledge. Pennings, P., Keman, H. and Kleinnijenhuis, J. (2006) Doing Research in Political Science: An Introduction to Comparative Methods and Statistics. London: Sage. Plummer, K. (1995) Telling Sexual Stories: Power, Change and Social Worlds. London: Routledge. Przeworski, A., Alvarez, M., Cheibub, J.A. and Limongi, F. (1996) What makes democracies endure? In Journal of Democracy, 7 (1): 35–55. Race, K.E., Hotch, D.F. and Parker, T. (1994) Rehabilitation programme evaluation: use of focus groups to empower clients. In Evaluation Review, 18 (6): 730–40. Reed, M. and Marsh, D. (2002) Chapter 11: Combining quantitative and qualitative method. In Marsh, D. and Stoker, G. (eds) Theory and Methods in Political Science. Basingstoke: Macmillan, pp. 231–48. Richards, D. (1996) Elite interviewing: approaches and pitfalls. In Politics, 16 (3): 199–204. Ritchie, J. and Lewis, J. (2003) Qualitative Research Practice: A Guide for Social Science Students and Researchers. London: Sage. Robertson, D. (1993) Politics. London: Penguin.
333
334
Bibliography Robson, C. (1993) Real World Research. Oxford: Blackwell. Rose, G. (1982) Deciphering Sociological Research. London: Macmillan. Russell, B. (1971) A Liberal Decalogue. In The Autobiography of Bertrand Russell, 3. 1944–67. London: George Allen & Unwin. Sanders, D. (2002) Chapter 2: Behaviouralism. In Marsh, D. and Stoker, G. (eds) Theory and Methods in Political Science. Basingstoke: Palgrave Macmillan. p. 52. Schattschneider, E.E. (1961) The Semisovereign People: A Realists’s View of Democracy in America. New York: Holt, Rinehart. In The Process and Effects of Mass Communication. Urbana, Ill.: The University of Illinois Press. Silverman, D. (2001) Interpreting Qualitative Data: Methods of Analysing Talk, Text and Interaction. London: Sage. Smith, M. (1993) Personal interviewing: a mini-training workshop. In Breakwell, G.M., Foot, H. and Gilmour, R. (eds) Doing Social Psychology: Laboratory Exercises. Leicester: BPS. Smith, M.J. (2003) Social Science in Question. London: Sage. Spencer, L., Ritchie, J. and O’Connor, W. (2003) Chapter 8: Analysis: practices, principles and processes. In Ritchie, J. and Lewis, J. (eds) Qualitative Research Practice: A Guide for Social Science Students and Researchers. London: Sage. pp. 199–218. Squire, C. (1995) Discourse analytical psychology. In Wilkinson, S. and Kitzinger, C. (eds) (1995). Feminism and Discourse: Psychological Perspectives. London: Sage. Stringer, E. (1996) Action Research: A Handbook for Practitioners. London: Sage. Stubbs, M. (1983) Discourse Analysis. Oxford: Blackwell. Webb, S(ydney) and B(eatrice) (1975/1932) Methods of Social Study. LSE: Cambridge University Press. Williams, R. (1976) Keywords. London: Fontana. Woodruff, W. (2003) Beyond Nab End. London: Abacus. Yates, S.J. (2004) Doing Social Research. London: Sage.
Index
3Rs, 305 n.26 Absorbed actions, 163 Accessibility, 52 ACORN, 148 Action research method, 52, 62–63 All-postal voting, 54–55 Amis, Kingsley, 80 ANOVA, 209 Archives, 104–105 Aristotle, 175 Arithmetic mean, 185 Arksey, H. and Knight, G., 133, 137 Assessment letter, sample, 77 (box) Association, 31 measuring, 208–210 multiple linear regression analysis, 213–217 regression statistics, calculating, 210–213 testing for, 206–208 Association of Colleges on data, 14 Auden, W.H., 140 Austin, J. L., 300, 304 Bachrach, P. and Baratz, M., 5, 26, 36 n.9 Barthes, Roland, 281, 298 Bashir, Martin, interview of Princess Diana: case study: narrative analysis, 302–303 Baton signals, 164–165 Baudrillard, Jean, 282 BBC News, 161 Behaviouralism, 25–27 Behaviourism, 25 Bell. J., 309, 316, 317 Benn, Tony, 83–84 Beresford, P. and Hoban, M., 63, 66 n.16 Berg, B.L., 129 (box) Berger, A.A., 298
Bergman, M., 129 (box) Bernoulli, Jacques, 234 Bernoulli’s trials, 234–235 Bevan, Aneurin, 84 Bias, 17–18 Biblical narrative, 299 Bibliographies, 107 (box) Binomial distribution, 234–236 Biography, 84 Blair, Tony, 291 Boas, Franz, 282 Body language, 161–162 claims made for, 162–163 Morris on, 163–168 recording and analysing, 168–171 Bogardus Social Distance Scale, 149–150 Brier, A.P. and Hopp, B., 278 n.4 British Election Study (BES), 44, 49, 61 British Household Panel Survey (BHPS), 61 Bruner, J.S., 298 Bryman, A., 41, 42, 49 Budge, I., 268–270, 278 Burke, K., 299 Burnham, P., Killand, K., Grant, W., and LaytonHenry, Z., 4, 8, 21–22, 137, 149–150, 196, 278, 283 Bush, George W.: State of the Union Address (2005), 271–276 State of the Union Address (2007): case study: Fairclough’s critical discourse analysis, 291–297 CACI, 148 CAQDAS (Computer-Assisted Qualitative Data Analysis System) software, 41–42
Cardinal numbers, 193–194 Carr, E.H., 299 Carver, T., 280 case studies: case study meetings, 260 Evaluating information 86–88 Fairclough’s critical discourse analysis, 290–291, 291–297 interim case study, 260 narrative analysis, 302–303 qualitative information analysis case study, 243–259 single case study, 53–54 Categorical numbers, 193 Causality, 29–32 Census returns, 103 Central limit theorem, 199 Chicago School, 221, 238 n.1 Chomsky, N., 282 Churchill, Winston S., 279, 304 n.3 Clegg, F., 196, 204, 218, 310, 317 Coding, 153–154, 242–243 Coefficient of correlation, 208 Cohort, 60–61 Cole, G.D.H., 88 Cole, John, 81–82, 83 Collett, Peter, 172 n.2 Comparative method, 51, 55–60 Comte, Auguste, 23 Constructed memory, 60 Content analysis, 263–264 Hamlet II, 271–276, 278 n.4 qualitative, 264–266 software for, 270–271 structural, 267–268 substantive, 268–270 Counterfactual, 27, 36 n.12 Covariation, 31 Critical social theory, 62 Crossman, Richard, 81–82, 83 Crouch, C., 44, 48 Crystal, D., 281, 282
336
Index
Curt, Beryl C., 279 Czarniawska, B., 299 Dahl, Robert, 25–27, 35 Data: accuracy of, 85 dichotomous nominal data, 234 reliability of, 83 validation, 178–179 Data analysing, 175–176 assembly of data, 178 hierarchy of, 177–178 process, 176–177 reduction, 179 validation, 178–179 Data Protection Act (1988), 13 Data Protection Co-ordinator, 14 Data protection legislation, 13–15 Databases, 102 De Vaus, D.A., 21, 35, 98, 159, 204 Deductive research, 33 Denscombe, M., 313, 318 Derrida, Jacques, 282, 284 Descriptive statistics, 183–187 calculation, 190–193 cardinal numbers, 193–194 categorical numbers, 193 grouped frequency distribution, 188–190 and n, 188 and N, 188 standard derivation, 188 variance, 187–188 Diachrony, 281 Dialects, 286 Dichotomous nominal data, 234 Discourse analysis, 242, 279–281 critical discourse analysis (CDA), 284 constituents, 301 linguistics, 281–282 Marxist school, 284 philosophy, 282–283 positivist school, 283 post structuralists, 284 realist school, 283–284 semiotics, 282 See also Fairclough’s critical discourse analysis
Discovered actions, 163 Disraeli, Benjamin, 183 Domhoff, D.W., 27 Downs, A., 74 Durkheim, Emile, 23, 24–25 EEC Directive 96/96, 13 EFTA (European Free Trade Area), 21 n.4 Electoral Commission, 54 Elite dialect, 286 Elite interviews: approaching elites, 121–123 defining elite, 119 introductory letter, 122 (box) post interview, 127 preparation and procedure, 123–125 reasons for interviews, 119–120 rules, 120–121 structure, 125–127 Empiricism, 24–25 Engels, F., 55–56 Enlightenment, 23 Enumeration district (ED), 103 Epistemology, 22 ESRC, 12, 21n.2 Ethics. See harm; Nuremberg Code Eugenics, 25, 208–209 Eurobarometer, 61 European Convention of Human Rights & Fundamental Freedoms, 13, 21 n.3 Evaluating information case study, 86–88 Excel, 190–193, 200, 223 calculating multiple linear regression equations, 214–216 calculating regression statistics, 210–212 Poisson distribution, 236 time series analysis, 230–232 Experimental effect, 17 Experimental method, 51, 54–55 Eye-pupils, 166 Factor analysis, 220–224 using SPSS, 224–230
Facts, 85 Fairclough, Norman, 280, 304 Fairclough’s critical discourse analysis: case study: George W. Bush’s State of the Union Address (2007), 291–297 case study: interview of Margaret Thatcher, 290 case study: New Labour, 290–291 description, 286–289 explanation, 289 interpretation, 289 Fairy tales, 298 Falsification/falsibility, 24, 43 Feminism, 28–29 Field, A., 237 Fields of experience, 128 Filstead, W.J., 41 Focus groups, 134–136 Foddy, W., 132, 138 Foot, Michael, 84 Footnotes, 106 (box), 311 Foucault, Michel, 81, 279, 282–283, 284 Freedom of Information Act (2000), 103–104 Friedrich, C.J., 119 Galbraith, J.K., 50 n.8 Gallie, W.B., 279 Galton, Francis, 208–209 Garfunkel, Harold, 283 Gaze behaviour, 165 George V, 89 Gestures, 164 Gibbs, A., 135 Giddens, Anthony, 27, 285, 301 Gilbert, N., 65, 159 Gini coefficient, 166 Glaser, Barney, 33–34, 259 Glaser, B.G. and Strauss, A.L., 35 Glottal stop, 305 n.27 Goffman, Erving, 283, 50 n.6 Gramsci, Antonio, 285 Greenfield, S. et al, 4 Grigg, J., 84 Grounded research, 33–34
Index Grouped frequency distribution, 188–190 Gubrium, J.F. and Holstein, J.A., 35, 261 Guttman scale, 149–151, 151 (table) Hajer, M., 280 Halo effect, 17 Hamlet II, 271–276, 278 n.4 Harm, types of, 10 Harris, K., 244–251 Harrison, L., 49, 98, 138, 159, 277 Hart, C., 110 Harvard system, 106 (box) Harvey, L., 65 Hawthorne effect, 15–16 Healey, Dennis, 84 Heath, A., and Taylor, B., 44 Heath, Ted and Harold Wilson: qualitative information analysis case study, 243–259 Heidegger, Martin, 299 Held, D. and Leftwich, A., 8 Heywood, A., 29 Hill, Anita, 300–301 Hobbes, Thomas, 23 Honderich, T., 32 Hopkin, J., 64 Hopp, Bruno, 271 Howarth, D., 280, 283–285, 304 Hume, David, 23, 30–31, 32 Hypothesis, 22, 23 Ideology, 286 Inductive research, 32 Inferential statistics, 197–198 central limit theorem, 199 national opinion polls, 201–202 normal distribution, 198–199 null hypothesis, 202–204 standard error of the proportion, 201 standard error of the sample mean, 199–201 Intermediaries, 92 Interquartile range, 186 Interviews, 85–86 with elites. See elite interviews focus groups, 134–136
with non-elites, 127–129, 129 (box) pariah topics, 130–132 projective technique, 132–133 questions, 117–118 recording, 129–130, 138 n.1 vignettes, 133–134 Jary, D. and Jary, J., 74, 83, 281 Jenkins, Roy, 82 John Henry effect, 16 Journals of political science, 70 Kanji, G.K., 205, 219 Kinnear, P.R. and Gray, C.D., 237 Kohler Reissman, C., 299 Krippendorff, K., 277 Lacan, Jacques, 281 Laclau, E. and Mouffe, C., 284 Landman, T., 59–60, 64 Language, 281 Langue, 281 Large-n samples, 58, 59 Last Time, The, 309 Law of large numbers, 53, 65 n.1 Laws, 22 Lee, R.L., 65 Leftwich, A., 8, 64 Levine, D. and Stephan, D., 204, 219 Levi-Strauss, Claude, 281 Lewis-Beck, M.S., 219 Likert, Rensis. See Likert scale Likert scale, 149, 153–154, 187 Literature reviews: examples of reviews, 107–108, 110–116 (box) official records, 103–105 plagiarism, 108–109 purpose of, 100–102 reading literature, 105–106 referencing, 106–107 searching for literature, 102–103 writing the review, 106 Longitudinal method, 51, 60–62 Lovenduski, J., 29 Lukes, Stephen, 5, 27
337
Mace, C.A., 4 Mach, Ernst, 23 Manifesto Research Group, 268 Manning, P.K. and Cullum-Swan, B., 298 Marlow, A., 177 Marr, Andrew, 84–85 Marsh, D. and Furlong, D., 35 Marsh, D. and Stoker, G., 5, 8 Mayer, L.C., 64 Mayo, Elton, 15–16, 124 McEvoy, J., 138 McLuhan, Marshall, 283 McRobbie, A., 299 Measures of central tendency, 185 Median, 185 Meek, James, 175 Members’ resource, 285 Memoing, 259–260 Method of agreement, 57 Method of concomitant variations, 57 Method of difference, 57 Miles, M.B. and Huberman, A.M., 177, 243, 259, 261, 262 Mill, John Stuart, 23, 55, 56, 57 Mixed actions, 164 Mixed methods, 47–49, 48, 49 Mochmann, Ekkehard, 271 Modernity, 185 Moore, B., 59 Morris, Desmond, 163–168 Most different systems design (MDSD), 58 Most similar systems design (MSSD), 58 Multiple linear regression equations, 213–217 Mumby, D.K., 298 N and n, 188 Narrative analysis, 297–298 case study: Martin Bashir’s interview of Princess Diana, 302–303 definitions, 299–301 origins, 298–299 Narrative-of-the-self, 302 National opinion polls, 201–202
338
Index
National Readership Survey, 146–147 Naturalism, 27–28 Natural science, 23, 36 Neuman, W.L., 29, 36, 98, 138, 177, 178, 197, 37 n.17 New Labour: case study: Fairclough’s critical discourse analysis, 290–291 Nicholson, Harold, 86–89 Nietzsche, Friedrich, 47 Nomination, 91 Non-probability samples, 91–93 Non-verbal leakage, 165 Normal distribution, 198–199 Norris, P., 29 Null hypothesis, 202–204 Nuremberg Code (1947), 10–12 Oakley, A., 28 Official records, 103–105 Ontology, 22 Orwell, George, 13 Osgood, Charles E., 148 O’Sullivan, T., 283 Outlier effect, 185 Panel, 61 Parker, I., 284 Park, R.E., 221 Peacock effect, 17 Pennings, P., Keman, H. and Kleinnijenhuis, J., 64, 65, 196, 219 Personal life histories, 60 Pilot project, 54–55 Placebo effect, 16 Plagiarism, 108–109 Plummer, K., 299 Poisson distribution, 235–236 Poisson, Siméon Denis, 235 Politics as an academic subject, 4–6 Popper, Karl, 23, 24, 43, 202 Population, 91, 185 Positivism, 23–24 See also Feminism; Naturalism Post-modernism, 29 Postural echo, 165 Powell, Enoch, 120
Prejudices proforma, 19–20 Pre-knowledge, 298–299 Probability, 198 Probability samples, 93–95 Property ownership, 109 (table) Propositions, 22–23 Propp, V., 298–299 Przeworski, A., Alvarez, M., Cheibub, J.A. and Limongi, F., 64, 66 n.8 Pygmalion effect, 17 QAA, 5 Qualitative analysis compared with quantitative analysis, 178 (table) Qualitative information, analysing, 235 case study: Harold Wilson and Ted Heath, 243–259 case study meetings, 260 coding, 242–243 content analysis, 241 discourse analysis, 242 interim case study, 260 memoing, 259–260 second-level pattern coding, 254–257 tagging and coding, 243 typology, 257 Qualitative methods: characteristics of, 43 (table) criticisms and weaknesses, 46 defence and counter-claims on behalf of, 46–47 strengths, 45–46 See also Mixed methods Quantitative analysis compared with qualitative analysis, 178 (table) Quantitative methods: characteristics of Table, 43 defence and counter-claims on behalf of, 44–45 strengths, 42–44 weaknesses and criticisms, 44 See also Mixed methods Quarterly deviations, method of, 232–234
Questionnaire surveys: access, 141–143 administering, 143 coding, 153–154 definition, 140–141 demonstration questionnaire, 154–158 design, 144–145 Guttman scale, 149–151, 151 (table) Likert scale, 149, 153–154, 187 motivating respondents, 143–144 n-th address technique, 142 personal safety, 152–153 postcodes, 148 proxy-indicators, 145–147 random walk technique, 142 ranking, 151–152 scale rating, 150–151 semantic differential, 148 show cards, 147 Questions. See Interviews; Questionnaire surveys Quota samples, 95–96 Race, K.E., 136 RAE (Research Assessment Exercise), 78 n.3 Range, 185–186 RDN (Research Discovery Network), 103 Reed, M. and March, D., 50 Referencing, 106–107 Regression analysis, 209 Regression analysis, calculating, 210–213 Reinharrz, 9 Reliability of data, 83 Research: action research method, 52, 62–63 choice of, 52–53 comparative method, 51, 55–60 deductive research, 33 definition, 3–4 experimental method, 51, 54–55 grounded research, 33–34 inductive research, 32 longitudinal method, 51, 60–62
Index RAE (Research Assessment Exercise), 78 n.3 research component analysis (RCA), 70–73 Rose’s ABCDE validity analysis deciphering model, 73–76 single case study, 53–54 research component analysis (RCA), 70–73 Research effects, 15–17 Researchers and subjects, 9–10 Richards, D., 138 Ritchie, J. and Lewis, J., 177, 262 Robson, C., 65 Rose, G. See Rose’s ABCDE validity analysis deciphering model Rose’s ABCDE validity analysis deciphering model, 73–76 Rotated component matrix, 229 Rumsfeld, Donald, 101 Russell, Bertrand, 79 Saint-Simon, Comte de, 23 Sampling: errors, 94, 199 large-n samples, 58, 59 non-probability samples, 91–93 probability samples, 93–95 quota samples, 95–96 small-n samples, 58, 59–60 snowball samples, 92 volunteer samples, 92–93 Sanders, D., 27 Saussure, Ferdinand de, 281 Scepticism, 4 Schattschneider, E.E., 32 Schramm, W., 128 Second-level pattern coding, 254–257 Serendipity, 3–4 Series, 185 Shakespeare, William, 302 Silverman, David, 176, 262 Single case study, 53–54 Skinner, B. F., 25 Small-n samples, 58, 59–60 Smith, M., 129 (box)
Smith, M.J., 36 Snowball samples, 92 Social facts, 24 Social gatekeeper, 91–92 Social Sciences Citation Index, 102 Socio-economic classification scheme, 146 (box) Sources 80–83 Spearman, Charles, 25 Spencer, L., Ritchie, J. and O’Connor, W., 180 SPSS, 224 binomial test, 235 data view, 226–227 factor analysis using, 224–230 rotated component matrix, 229 text books on, 237 total variance explained, 228 variable view, 224–226 Squire, C., 280 Standard error of the proportion, 202 Standard error of the sample mean, 199–201 Statistics, 206 official, 84 regression statistics and association, 210–213 See also descriptive statistics; inferential statistics Strauss, Anselm, 33–34 Stringer, E., 65 Structuralism, 281 Stubbs, M., 280 Subjects and researchers, 9–10 Suicide, 24–25 Synchrony, 281 Tagging and coding, 243 Taylor, W.F., 21 n.7 Terms, 185 Thatcher, Margaret, 266 case study: Fairclough’s critical discourse analysis, 290 Theoretical saturation, 34 Theory, 22 Thick/thin approach, 54 Thomas, Clarence, 300–301
339
Thomas, Mae, 301 Thompson, Walter, 82 Time series analysis, 230–232 Trained actions, 163–164 Treaty of Rome (Article 108), 13 Triangulation, 89–90, 120, 138 n.4 Truth, 85 Twain, Mark, 183 United Nations Declaration of Human Rights, 13 Validity, 74–76, 83 Variable-orientated analysis, 58 Variance, 187–188 verstehen, 45 Vienna Circle, 23, 24 Voice-transcription software, 130 Volunteer samples, 92–93 Webb, B. and Webb, S., 18, 21 n.9 Webb, Beatrice, 18 Weber, Max, 29, 32, 263 Williams, Raymond, 286, 304 Wilson, Harold, 298 and Ted Heath: qualitative information analysis case study case, 243–259 WinStat, 223 Wirth, L., 221 Wittgenstein, Ludwig, 23, 282 Woodruff, William, 105 Writing-up, 309–310 final draft, review of, 316 literature reviews, 106 publication rules, 311 readership, 310 re-writing, 316 structure, 311 structure and contents, 313–316 when to write up, 312–313 word distribution, 316–317 Yates, S.J., 160 Zipf, George, 266 Zipf’s Law, 266–267
