CHAPTER 10

MIXED METHOD
PROCEDURES
REPORTER: KENNETH ERIGA SAVANDAL
 What is Mixed Methods Research?
Mixed methods involves the collection and “mixing” or integration of both quantitative and
qualitative data in a study. It is not enough to only analyze your qualitative and quantitative
data. Further analysis consists of integrating the two databases for additional insight into
research problems and questions.

It involves the collection of both qualitative (open-ended) and quantitative (close-ended) data in
response to research questions or hypothesis.

It includes the rigorous methods (i.e., data collection, data analysis, and interpretation) of both quantitative and
qualitative data.

The two forms of data are integrated in the design analysis through merging the data, explaining the data,
building from one database to another, or embedding the data within a lager framework.

These procedures are incorporated into a distinct mixed methods design that indicates the procedures to be used
in a study.

These procedures are often informed by a philosophy (or worldview) and a theory.
 Background of Mixed Methods Research
There are many different terms used for this approach, such as integrating, synthesis, quantitative
and qualitative methods, multimethod, mixed research, or mixed methodology but that recent writings,
such as the SAGE Handbook of Mixed Methods in the Social & Behavioral Sciences and SAGE’s Journal
of Mixed Methods Research, tend to use the term mixed methods (Bryman, 2006; Creswell, 2015;
Tashakkori & Teddlie, 2010)

This can be seen as a methodology originating around the late 1980s and early 1990s in its current
form based on work from individuals in diverse fields such as evaluation, education, management,
sociology, and health sciences. It has gone through several periods of development and growth, and it
continues to evolve, especially in procedures. Several texts outline these developmental phases (e.g.,
Creswell & Plano Clark, 2011, 2018; Teddlie & Tashakkori, 2009). This section could also include a brief
discussion about the importance or rise of mixed methods today through indicators such as federal
funding initiatives, dissertations, and the discipline-specific discussions about mixed methods found in
journals across the social and health sciences (see Creswell, 2010, 2011, 2015).
 Why should Researchers use Mixed Methods Research?

At a general level, mixed methods is chosen because of its strength of

drawing on both qualitative and quantitative research and minimizing the
limitations of both approaches.
At a practical level, mixed methods provides a sophisticated, complex
approach to research that appeals to those on the forefront of new research
procedures. It also can be an ideal approach if the researcher has access to both
quantitative and qualitative data.
At a procedural level, it is a useful strategy to have a more complete
understanding of research problems and questions, such as the following:
 Why should Researchers use Mixed Methods Research?

1. Comparing different perspectives drawn from quantitative and qualitative data

2. Explaining quantitative results with a qualitative follow-up data collection and
analysis
3. Developing better contextualized measurement instruments by first collecting and
analyzing qualitative data and then administrating the instruments to a sample
4. Augmenting experiments or trials by incorporating the perspectives of individuals
5. Developing cases (i.e., organizations, units, or programs) or documenting diverse cases
for comparisons
6. Developing a more complete understanding of changes needed for a marginalized group
through the combination of qualitative and quantitative data
7. Evaluating both the processes and the outcomes of a program, an experimental
intervention, or a policy decision
 Challenges to the design

These include the need for extensive data collection, the time-intensive
nature of analyzing both qualitative and quantitative data, and the requirement
for the researcher to be familiar with both quantitative and qualitative forms of
research. The complexity of the design also calls for clear, visual models to
understand the details and the flow of research activities in this design.
 THREE CORE MIXED METHODS DESIGN

Explanatory Exploratory
Convergent Mixed Sequential Sequential
Methods Design
Design Design
 Convergent Mixed Methods Design
Description of the Design

The Convergent Mixed Methods Design is probably the most familiar of the core and
complex mixed methods approaches.

In this single-phase approach, a researcher collects both quantitative and qualitative data,
analyzes them separately, and then compares the results to see if the findings confirm or
disconfirm each other.

The key assumption of this approach is that both qualitative and quantitative data provide
different types of information – often detailed views of participants qualitatively and scores
on instruments quantitatively – and together they yield results that should be the same.
 Convergent Mixed Methods Design
Data Collection

Ideally, the key idea with this design is to collect both forms of data using the same or
parallel variables, constructs, or concepts. In other words, if the concept of self-esteem is
being measured during quantitative data collection, the same concept is asked during the
qualitative data collection process, such as in an open-ended interview.

Some researchers will use this design to associate certain themes with statistical data using
different forms of data for the quantitative and qualitative data collection.

Typically, mixed methods researchers would include the sample of qualitative participants in
the larger quantitative sample, because ultimately researchers make a comparison between
the two databases and the more they are similar, the better comparison.
 Convergent Mixed Methods Design
Data Analysis and Integration

Data analysis in a convergent design consists of three phases:

First, analyse the qualitative database by coding the data and

collapsing the codes into broad themes. Second, analyze the quantitative
database in terms of statistical results. Third comes the mixed methods
data analysis. This is the analysis that consists of integrating the two
databases.
 Convergent Mixed Methods Design
Data Analysis and Integration

This integration consists of merging the results from both the qualitative and
the quantitative findings. One challenge in this design is how to actually merge the
two databases since bringing together a numeric quantitative database with a text
qualitative database is not intuitive. There are several ways to merge the two
databases:

The first approach is called a side-by-side comparison.

These comparisons can be seen in the discussion sections of mixed methods
studies. The researcher will first report the quantitative statistical results and then
discuss the qualitative findings (e.g., themes) that either confirm or disconfirm the
statistical results. Alternatively, the researcher might start with the qualitative
findings and then compare them to the quantitative results.
 Convergent Mixed Methods Design
Data Analysis and Integration

The second approach is called data transformation.

Researchers can also merge the two databases by changing or

transforming qualitative codes or themes into quantitative variables and
then combining the two quantitative databases—a procedure in mixed
methods research called data transformation. The researcher takes the
qualitative themes or codes and counts them (and possibly groups them) to
form quantitative measures.
 Convergent Mixed Methods Design
Data Analysis and Integration

The final procedure involves merging the two forms of data in a table
or a graph. This is called a joint display of data, and it can take many
different forms. It might be a table that arrays the themes on the horizontal
axis and a categorical variable (e.g., different types of providers such as
nurses, physician assistants, and doctors) on the vertical axis. It might be a
table with key questions or concepts on the vertical axis and then two
columns on the horizontal axis indicating qualitative responses and
quantitative responses to the key questions or concepts (Li, Marquart, &
Zercher, 2000).
 Convergent Mixed Methods Design
Interpretation

 The interpretation in the convergent approach is typically written into a

discussion section of the study. Whereas the results section reports on
the findings from the analysis of both the quantitative and qualitative
databases, the discussion section includes a discussion comparing the
results from the two databases and notes whether there is convergence
or divergence between the two sources of information.
 Typically the comparison does not yield a clean convergent or
divergent situation, and differences exist on a few concepts, themes, or
scales. When divergence occurs, steps for follow-up need to be taken.
The researcher can state divergence as a limitation in the study without
further follow-up. This approach represents a weak solution.
 Convergent Mixed Methods Design
Interpretation

Alternatively, mixed methods researchers can return to the analyses

and further explore the databases, collect additional information to resolve
the differences, or discuss the results from one of the databases as possibly
limited (e.g., the constructs were not valid quantitatively or the qualitative
themes did not match the open-ended questions). Whatever approach the
researcher takes, the key point in a convergent design is to further discuss
and probe results when divergent findings exist.
 Convergent Mixed Methods Design
Validity

 Validity using the convergent approach should be based on establishing both

quantitative validity (e.g., construct) and qualitative validity (e.g., triangulation)
for each database.

 There are certainly some potential threats to validity in using the convergent
approach, and several of these have already been mentioned. Unequal sample
sizes may provide less of a picture on the qualitative side than the larger N on
the quantitative side.

 Generally we find the use of unequal sample sizes in a convergent design study,
with the researcher acknowledging the different perspectives on size taken by
quantitative and qualitative researchers. The use of different concepts or
variables on both sides, quantitative and qualitative, may yield incomparable
and difficult-to-merge findings.
 Convergent Mixed Methods Design
Validity

 The recommended approach is to use the same concepts for both the
quantitative and qualitative arms of the research study, but we
acknowledge that some researchers use the convergent design to
associate different qualitative and quantitative concepts.

 A lack of follow-up on conclusions when the scores and themes

diverge also represents an invalid strategy of inquiry. In this discussion
we have recommended several ways to probe divergence in more detail
and would recommend the use of one or more of these strategies in a
convergent design project.
 Explanatory Sequential Mixed Methods Design
Description of the Design

The explanatory sequential mixed methods approach is a

design in mixed methods that appeals to individuals with a
strong quantitative background or from fields relatively new
to qualitative approaches. It involves a two-phase data
collection project in which the researcher collects quantitative
data in the first phase, analyzes the results, and then uses the
results to plan (or build on to) the second, qualitative phase.
 Explanatory Sequential Mixed Methods Design
Description of the Design

The quantitative results typically inform the types of

participants to be purposefully selected for the qualitative
phase and the types of questions that will be asked of the
participants. The overall intent of this design is to have the
qualitative data help explain in more detail the initial
quantitative results, thus it is important to tie together or to
connect the quantitative results to the qualitative data
collection.
 Explanatory Sequential Mixed Methods Design
Description of the Design

A typical procedure might involve collecting

survey data in the first phase, analyzing the data, and
then following up with qualitative interviews to help
explain confusing, contradictory, or unusual survey
responses.
 Explanatory Sequential Mixed Methods Design
Data Collection

The data collection proceeds in two distinct phases

with rigorous quantitative sampling in the first phase and with
purposeful sampling in the second, qualitative phase. One
challenge in this strategy is to plan adequately what
quantitative results to follow up on and what participants to
gather qualitative data from in the second phase.
 Explanatory Sequential Mixed Methods Design
Data Collection

The key idea is that the qualitative data collection

builds directly on the quantitative results. The quantitative
results that then are built on may be extreme or outlier cases,
significant predictors, significant results relating variables,
insignificant results, or even demographics.
 Explanatory Sequential Mixed Methods Design
Data Collection

For example, when using demographics, the researcher

could find in the initial quantitative phase that individuals in
different socioeconomic levels respond differently to the
dependent variables. Thus, the follow-up qualitatively may
group respondents to the quantitative phase into different
categories and conduct qualitative data collection with
individuals representing each of the categories.
 Explanatory Sequential Mixed Methods Design
Data Collection

Another challenge is whether the qualitative sample

should be individuals that are in the initial quantitative
sample. The answer to this question should be that they are the
same individuals, because the intent of the design is to follow
up the quantitative results and explore the results in more
depth. The idea of explaining the mechanism—how the
variables interact—in more depth through the qualitative
follow-up is a key strength of this design.
 Explanatory Sequential Mixed Methods Design
Data Analysis and Integration

The quantitative and the qualitative databases are analyzed

separately in this approach. Then the researcher combines the
two databases by the form of integration called connecting the
quantitative results to the qualitative data collection. This is
the point of integration in an explanatory sequential design.
Thus, the quantitative results are then used to plan the
qualitative follow-up.
 Explanatory Sequential Mixed Methods Design
Data Analysis and Integration

 One important area is that the quantitative results cannot only inform
the sampling procedure but it can also point toward the types of
qualitative questions to ask participants in the second phase.

 These questions, like all good qualitative research questions, are

general and open-ended. Because analysis proceeds independently for
each phase, this design is useful for student research and perhaps easier
to accomplish (than the convergent design) because one database
explains the other and the data collection can be spaced out over time.
 Explanatory Sequential Mixed Methods Design
Interpretation

The mixed methods researcher interprets the follow up

results in a discussion section of the study. This interpretation
follows the form of first reporting the quantitative, first-phase
results and then the qualitative, second phase results.
However, this design then employs a third form of
interpretation: how the qualitative findings help to explain the
quantitative results.
 Explanatory Sequential Mixed Methods Design
Interpretation

A common misstep at this point by beginning researchers

is to merge the two databases. While this approach may be
helpful, the intent of the design is to have the qualitative data
help to provide more depth, more insight into the quantitative
results.
 Explanatory Sequential Mixed Methods Design
Interpretation

Accordingly, in the interpretation section, after the

researcher presents the general quantitative and then
qualitative results, a discussion should follow that specifies
how the qualitative results help to expand or explain the
quantitative results. Because the qualitative database questions
narrows the scope of the quantitative questions, a direct
comparison of the overall results of the two databases is not
recommended.
 Explanatory Sequential Mixed Methods Design
Validity

 As with all mixed methods studies, the researcher needs to

establish the validity of the scores from the quantitative
measures and to discuss the validity of the qualitative
findings. In the explanatory sequential mixed methods
approach, additional validity concerns arise.

 The accuracy of the overall findings may be compromised

because the researcher does not consider and weigh all of
the options for following up on the quantitative results.
 Explanatory Sequential Mixed Methods Design
Validity

We recommend that researchers consider all options for

identifying results to follow up on before settling on one
approach. Attention may focus only on personal demographics
and overlook important explanations that need further
understanding. The researcher may also contribute to
invalidated results by drawing on different samples for each
phase of the study.
 Explanatory Sequential Mixed Methods Design
Validity

If explaining the quantitative results in more depth, then it

makes sense to select the qualitative sample from individuals
who participated in the quantitative sample. This maximizes
the importance of one phase explaining the other. These are a
few of the challenges that need to be built into the planning
process for a good explanatory sequential mixed methods
study.
 Exploratory Sequential Mixed Methods Design
Description of the Design

 If we reverse the explanatory sequential approach and start with a

qualitative phase first followed by a quantitative phase, we have an
exploratory sequential approach.

 A three-phase exploratory sequential mixed methods is a design in

which the researcher first begins by exploring with qualitative data and
analysis, then builds a feature to be tested (e.g., a new survey
instrument, experimental procedures, a website, or new variables) and
tests this feature in a quantitative third phase.
 Exploratory Sequential Mixed Methods Design
Description of the Design

 The intent of this design is to explore with a sample first so that a later
quantitative phase can be tailored to meet the needs of the individuals
being studied. Sometimes this quantitative feature will include
developing a contextually sensitive measurement instrument and then
testing it with a sample.

 Other times it may involve developing new variables not available in

the literature or attuned to a specific population being studied, or
designing a website or an Internet application shaped to the needs of the
individuals being studied.
 Exploratory Sequential Mixed Methods Design
Description of the Design

 In this design, the researcher would first collect focus group data,
analyze the results, develop an instrument (or other quantitative feature
such as a website for testing), and then administer it to a sample of a
population. In this case, there may not be adequate instruments to
measure the concepts with the sample the investigator wishes to study.

 In effect, the researcher employs a three-phase procedure with the first

phase as exploratory, the second as instrument (or quantitative feature)
development, and the third as administering and testing the instrument
feature to a sample of population.
 Exploratory Sequential Mixed Methods Design
Data Collection

In this strategy, the data collection would occur at two

points in the design: the initial qualitative data collection and
the test of the quantitative feature in the third phase of the
project.
 Exploratory Sequential Mixed Methods Design
Data Collection

Several options exist, and we will use the approach of

developing a culturally sensitive instrument as an illustration.
The qualitative data analysis can be used to develop an
instrument with good psychometric properties. The qualitative
data analysis will yield quotes, codes, and themes .
 Exploratory Sequential Mixed Methods Design
Data Analysis and Integration

In this strategy the researcher analyzes the two databases

separately and uses the findings from the initial exploratory
database to build into a feature that can be analyzed
quantitatively. So integration in this design involves using the
qualitative findings (or results) to inform the design of a
quantitative phase of the research such as the development of
a measurement instrument or new variables.
 Exploratory Sequential Mixed Methods Design
Interpretation

 Researchers interpret the mixed methods results in a discussion section

of a study. The order of interpretation is to first report the qualitative
findings, the development or design of the feature to be tested (e.g., the
development of an instrument, the development of new quantitative
measures), and then the quantitative test in the final phase of the study.

 It does not make sense to compare the two databases, because they are
typically drawn from different samples (as noted above in the data
collection discussion) and the intent of the strategy is to determine if the
qualitative themes in the first phase can be generalized to a larger
sample.
 Exploratory Sequential Mixed Methods Design
Validity

 Researchers using this strategy need to check for the validity of the
qualitative data as well as the validity of the quantitative scores. Special
validity concerns arise, however, in using this design that need to be
anticipated by the proposal or mixed methods report developer.
 Finally, as previously mentioned, the sample in the qualitative phase
should not be included in the quantitative phase as this will introduce
undue duplication of responses. It is best to have the sample of
qualitative participants provide information for scale, instrument, or
variable (or website) design, but the same individuals should not
complete the follow-up instruments.
Three Core Mixed Methods Design