Mixed Methods Research: What it means and why it is useful for organisational and social change

1

Mixed Methods Research What it means and why it is useful for organisational and social change Facilitator: Bronwyn Hegarty |Doctor of Education|MSc of  Education|MSc (Dist)|BSc (Hons)|DipTeaching

(Tertiary). Prepared for Otago Polytechnic Mixed Methods Research and Enterprise Seminar.

Evolution of Mixed of  Mixed Methods Research (MMR) Using Mixed Methods Research to investigate complex organisational and social issues is becoming more popular, and the field has steadily gained credibility in the research community. MMR is now an accepted research methodology and although it has been around for 25 years, much debate has surrounded this approach. When exploring how MMR has evolved, you may read about the “paradigm wars” and the third methodological movement” (Cameron, 2011, p. 99). No matter whether you are a quantitative or a qualitative researcher, mixed methods requires a shift in worldview (or paradigm) about how to conduct research. It has also been used as an evaluation method and became popular for evaluating eLearning innovations (Reeves & Hedberg, 2004). Supporters of this of  this research methodology see value in a design that combines not only quantitative and qualitative methods but also the mixing of  paradigms or worldviews (e.g., postpositivist, interpretive, constructivist etc.). The direction of a of  a Mixed Methods Research project may be driven by one type of theoretical of  theoretical foundation or by a mix of theories. of  theories. For example, social science theories (e.g., leadership, behavioural change, marketing, learning and teaching etc.) or emancipatory theories (associated with marginalized groups) or a mix of these of  these (Cresswell, 2015). However, MMR as a methodology is the main consideration here rather than the paradigms or theories informing the research. Three basic designs for MMR will be explained. In this workshop, we will examine what MMR is and what it is not. Discuss the theoretical and practical implications of undertaking of  undertaking Mixed Methods Research, and explore some examples where it has been applied to complex problems in the workplace.

Definition of MMR of  MMR John Creswell’s (2015) definition is used here and it is worth investigating others. An approach to research in the social, behavioural, and health sciences in which the investigator gathers both quantitative (close‐ended) and qualitative (open‐ended) data, integrates the two, and then draws interpretations based on the combined strengths of both of  both sets of data of  data to understand research problems (Creswell, 2015, p. 2). The characteristics of MMR, of  MMR, when compared to quantitative and qualitative research methods, fit somewhere in between depending on how the research is designed (see Figure 1). Dr Bronwyn Hegarty, 23 September 2015 Research & Enterprise Research Training Te Kura Matatini ki Otākou (Otago Polytechnic)

2 Quantitative

MMR

Qualitative

Deductive logic

Inductive

Objective inference

Subjective inference

Value neutral

Value involved

Politically neutral

Transformative

Figure 1: Positioning Mixed Methods Research (MMR).

Advantages and disadvantages exist for each type of research. of  research. For example, quantitative research generally deals with larger numbers of participants of  participants and findings may be used to make predictions and examine cause and effect. In contrast, qualitative research, although considering the perspectives of  participants in more depth, involves fewer people and findings cannot be generalized to other populations. Mixed Methods Research can potentially integrate the good and bad of both of  both quantitative and qualitative approaches, therefore, particular care must be taken to ensure the design is rigorous and the validity and reliability of the of  the data is established (Cresswell, 2015).

Choosing a Research Method When choosing and designing a research methodology, generally the worldview (philosophy or paradigm), the approach (deductive or inductive) and the strategies (case study, ethnography etc.) are considered (Figure 2). However, Creswell (2015) recommends that the easiest way to enter into Mixed Methods Research is to put most emphasis on the methods that will be used to collect, analyse and interpret data, and how the findings will be integrated.

Figure 2: Research layers

Roslyn Cameron (2011) has developed the Five Ps Framework (Figure Framework  (Figure 3) to assist researchers in understanding the challenges of Mixed of  Mixed Methods Research and the historical debate surrounding it. This framework is useful in helping researchers when choosing a MMR approach and the best way forward. For example, does the worldview (beliefs, assumptions and values) of the of  the researcher drive the design of  Dr Bronwyn Hegarty, 23 September 2015 Research & Enterprise Research Training Te Kura Matatini ki Otākou (Otago Polytechnic)

3 the research or is the ‘best fit’ practical approach used to investigate the research problem? More information about the framework can be found in Cameron’s (2011) paper.

Figure 3: Five Ps Framework for Mixed Methods research (based on Roslyn Cameron, 2011).

Research Designs in MMR Several research designs are recommended by Creswell (2015), and three will be discussed in this workshop – workshop  – Convergent Parallel  Convergent  Parallel Design Design,, Explanatory Sequential  Explanatory  Sequential Design Design and Exploratory Sequential  Exploratory  Sequential  Design (see Figure 4). Each design has a different phase for collecting, analysing and interpreting data, and the process for integrating the quantitative and qualitative findings varies. For example, in a Convergent Parallel  Convergent Parallel Design Design,, both types of data of  data are collected and analysed separately and synchronously, and then the results are merged and interpreted to explain the overall outcomes. The research problem, and the intent of the of  the research as well as the desired outcomes drive the choice of design of  design (Creswell, 2015). For example, is the research problem well understood or is further exploration needed? Do the researchers feel confident in using a specific method? In which case, the well‐known method may provide a starting point. Does qualitative data need to be used to explain quantitative data?

Advantages Mixed Methods Research is useful for investigating complex problems as the combination of  quantitative and qualitative data brings different perspectives (Azorín & Cameron, 2010). The approach helps to ensure triangulation of data of  data (more than one source aids verification) and also bracketing (different perspectives or levels) (Reeves & Hedberg, 2004). The process can be enriching for both participants and researchers, and can enhance the integrity and significance of the of  the research (Azorín & Cameron, 2010).

Barriers Often researchers are more skilled in either quantitative or qualitative methods so a lack of expertise of  expertise can prevent them engaging in MMR. This type of research of  research works better with a team as it tends to take more time and work especially when researchers need to develop their skills and knowledge. The Dr Bronwyn Hegarty, 23 September 2015 Research & Enterprise Research Training Te Kura Matatini ki Otākou (Otago Polytechnic)

4 approach can also be more financially taxing. Getting published can be an issue due to the size and complexity of the of  the research being reported, and also the preferences of some of  some journals  journals for either qualitative or quantitative research (Azorín & Cameron, 2010).

Figure 4: Three Mixed Methods Research Designs taken from Cresswell (2013, p. 40).

Examples of Mixed of  Mixed Methods Research projects Examples 1 and 2 from: Cronholm, S. & Hjalmarsson, A. (2011). Experiences from Sequential Use of 

Mixed Methods. The Electronic Journal  Electronic  Journal of  of Business Business Research Methods, 9(2), 87‐95). Retrieved from http://www.ejbrm.com

Example 1: Explanatory Sequential design of  web based text designs when secondary Aim: the researcher investigated the readability and legibility of web school students used PC devices with a small screen (7‐9 inches). The research: a quantitative study was carried out first as an experiment measuring the type of texts of  texts

that were read and also a Likert scale questionnaire was used. The effect of variables of  variables such as the length of a of  a row of text of  text and font size on readability and legibility were examined. Statistical analysis of the of  the questionnaire responses was used to look for relationships between participants’ reading speed and the design of the of  the text.

Dr Bronwyn Hegarty, 23 September 2015 Research & Enterprise Research Training Te Kura Matatini ki Otākou (Otago Polytechnic)

5 The qualitative component of the of  the research occurred in the second phase and consisted of interviews of  interviews using semi‐structured questions. Respondents to the questionnaire were chosen and asked their preferences regarding the text they were reading and the devices used. Characteristics of the of  the researcher: high pre‐knowledge about quantitative methods but uncomfortable

and skeptical with qualitative methods. The researcher’s high existing knowledge about the phenomenon to be studied was the reason she decided to begin with an experiment based on a well‐ developed hypothesis. She had little experience performing interviews and the outcomes of the of  the questionnaire provided a base for developing the semi‐structured interview questions. of  the qualitative phase since the interviews were Outcome: the design did not enable the full potential of the only used to confirm the outcomes of the of  the quantitative phase as opposed to exploring different conundrums.

Example 2: Exploratory Sequential design of  users when searching for information using a web‐based e‐ Aim: in this research, the behaviour of users service was investigated. The researcher wanted to find out if the if  the structure and content of resources of  resources provided for higher education was suitable and usable. of  the study began with qualitative text analysis with the The research: Phase one of the aim of generating of  generating categories that could be used for the quantitative part of the of  the study. The categories obtained from the qualitative phase were used to generate quantitative questions. Phase two followed with a questionnaire administered to first year and final year students in a secondary school. Characteristics of the of  the researcher: the researcher was knowledgeable and

confident with qualitative methods but less so with quantitative methods. Outcome: using quantitative and qualitative methods helped to highlight different perspectives. The

quantitative data confirmed the importance of the of  the categories obtained qualitatively and identified different features associated with the e‐service. Qualitative analysis enabled similarities and differences amongst the groups to emerge. However, generalization of the of  the results was not considered possible.

Example 3: Digital Information Literacy research project – project – Intervention Design. Hegarty, B., Penman, M., Kelly, O., Jeffrey, L., Coburn, D. and McDonald, J. (2010). Digital Information Literacy: Supported Development of Capability of  Capability in Tertiary Environments. Final report. Ministry of  Education, Wellington. http://www.educationcounts.govt.nz/publications/tertiary_education/80624 Aim: to investigate different strategies and tools that could be used to develop the digital information

literacy capability of staff  of  staff and and students from four tertiary institutions in New Zealand (Otago Polytechnic, University of Otago, of  Otago, Manukau Institute of Technology, of  Technology, and Massey University).

Dr Bronwyn Hegarty, 23 September 2015 Research & Enterprise Research Training Te Kura Matatini ki Otākou (Otago Polytechnic)

6 of  experiential workshops over a 10 week period were facilitated by the The research: a series of experiential researchers to support participants (staff and (staff  and students) to work on solutions for digital information issues in their work, study and personal lives. A Mixed Methods Research methodology was used and both qualitative and quantitative data was collected and analyzed within an Action Research framework. Data was collected using pre and post‐surveys (quantitative), observations and discussions in face‐to‐face workshops, participants’ reflections on their blogs, action learning cycles and journals, and  journals, email correspondence and focus group interviews (qualitative). A process of descriptive of  descriptive statistics, and content and thematic analysis and interpretation was used to integrate all forms of evidence of  evidence when developing case studies to present the findings. By integrating both forms of data, of  data, rich case studies were constructed to understand and articulate how participants developed their digital information capability over the research period. of  six researchers included people familiar with both Characteristics of the of  the researchers: the team of six quantitative and qualitative methods. Some members had previously used mixed methods in evaluation projects but not in research contexts. Outcome: the results were presented as an overall case study constructed from both individual case

studies and four institutional case studies. Therefore, different methodologies were used to collect, analyze and interpret the data and a mix of research of  research strategies (Action Research and Case Study) were incorporated.

Dr Bronwyn Hegarty, 23 September 2015 Research & Enterprise Research Training Te Kura Matatini ki Otākou (Otago Polytechnic)

7

Scenario – Scenario  – Pedestrian behaviour at Gardens intersection. Issue: The Dunedin City Council has received complaints about the number of pedestrians of  pedestrians who do not use the designated crossing to walk across North Road at the Gardens intersection. Pedestrians have been observed crossing in traffic even when they only have a short distance to walk to the traffic lights. Motorists have had near misses with pedestrians. The situation is potentially dangerous because even if  traffic is stopped it can start up again quickly when the lights change, and it tends to flows constantly in several directions. Cars often come round the corner quickly from two directions, and south bound traffic has to move into one of three of  three lanes on North Road once it reaches the intersection.

As researchers you are interested in how to solve this situation. You need to find out what determines whether pedestrians cross North Road at the traffic lights or cross through traffic.

Initial observations show several conditions concerning pedestrians. 1. Traffic may be stopped at the lights. 2. Traffic may be moving. 3. Pedestrians may cross at the crossing as instructed by the traffic lights. Traffic is stopped or waits for pedestrians. 4. Pedestrians cross through traffic – traffic  – some traffic is stopped but can start moving, or is moving from another direction. Characteristics of people of  people crossing in traffic: pedestrians may stop in the middle, some run quickly if 

traffic is moving, agility and age is variable, some cross slowly, and physical impairment or use of a of  a walking stick is not a deterrent .

Dr Bronwyn Hegarty, 23 September 2015 Research & Enterprise Research Training Te Kura Matatini ki Otākou (Otago Polytechnic)

8

Develop a Mixed Methods Research methodology to investigate the issue. Organise into groups (three or four) with a mix of quantitative of  quantitative and qualitative researchers. 1. What aspects do you want to investigate? 2. How will you design the research? 

Develop a research inquiry question.



How will you collect and analyse your data? Use both Quantitative and Qualitative methods.

3. Which MMR design does your project fit? 

Convergent Parallel Design, Explanatory Sequential Design or Exploratory Sequential Design?

MMR Description – Description – Example of Convergent of  Convergent Parallel Design – Design  – Gardens intersection In this study, surveys and observations (quantitative) will be used to ascertain how pedestrians cross North Road at the Gardens intersection and test the theory that attitudes and motivation may positively or negatively influence the behaviour of pedestrians. of  pedestrians. Data from observations and interviews (qualitative) will be used to describe the behaviour of pedestrians of  pedestrians and investigate factors that influence how they cross the road.

Conclusion MMR is most common in social, behavioural, and health sciences and has gained standing in the research community as the third methodological movement. Whether the approach is influenced by paradigms, theories or methodology true MMR must ensure that quantitative and qualitative methods, data collection, analysis and interpretation are integrated. Otherwise, the research is regarded as multi‐ methods. Advantages and barriers exist. For example, MMR is excellent for investigating complex problems due to the opportunity for many different forms of data of  data and perspectives. Conversely, MMR projects are best with teams due to the time and work involved and the difficulties in recruiting researchers with the desired mix of quantitative of  quantitative and qualitative proficiency. Researchers are advised to explore the challenges described in Cameron’s (2010) Five Ps Framework – Framework  – Paradigms, Pragmatism, Praxis, Proficiency and Publishing – Publishing  – before embarking on this type of research. of  research. Three research designs have been considered here (Convergent Parallel Design, Explanatory Sequential Design and Exploratory Sequential Design), and some examples outlined.

Facilitator: Bronwyn Hegarty |Doctor of Education|MSc of  Education|MSc (Dist)|BSc (Hons)|DipTeaching (Tertiary). Bronwyn has led and been involved in several collaborative and nationally funded research and evaluation projects in education where mixed methods designs were implemented. Bronwyn currently works with Capable NZ as a Principal Lecturer in Tertiary Teacher education. She is also a facilitator and academic mentor in the Masters of Professional of  Professional Practice. email: [email protected]

Dr Bronwyn Hegarty, 23 September 2015 Research & Enterprise Research Training Te Kura Matatini ki Otākou (Otago Polytechnic)

9

References Azorín, J., M. & Cameron, R. (2010). The Application of Mixed of  Mixed Methods in Organisational Research: A Literature Review. The Electronic Journal  Electronic  Journal of  of Business Business Research Methods, 8 (2), 95‐105). Retrieved from http://www.ejbrm.com Cameron, R. (2011). Mixed Methods Research: The Five Ps Framework. The Electronic Journal  Electronic  Journal of  of  Business Research Methods, 9(2), 96‐108. Retrieved from http://www.ejbrm.com Creswell, John W. (2013). Steps in Conducting a Scholarly Mixed Methods Study. DBER Speaker Series. Speaker  Series. Paper 48. Retrieved from http://digitalcommons.unl.edu/dberspeakers/48 Cronholm, S. & Hjalmarsson, A. (2011). Experiences from Sequential Use of Mixed of  Mixed Methods. The Electronic Journal  Electronic  Journal of  of Business Business Research Methods, 9(2), 87‐95). Retrieved from http://www.ejbrm.com Reeves. T. & Hedberg, J. (2004). Interactive Learning Systems Evaluation. Englewood Cliffs, New Jersey: Educational Technology Publications.

Dr Bronwyn Hegarty, 23 September 2015 Research & Enterprise Research Training Te Kura Matatini ki Otākou (Otago Polytechnic)