See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/380371546

Experimental Research

Chapter · May 2024

CITATIONS READS
0 1,117

1 author:

Rizgar Qasim Mahmood

University of Wollongong, Wollongong campus
20 PUBLICATIONS 26 CITATIONS

SEE PROFILE

All content following this page was uploaded by Rizgar Qasim Mahmood on 26 July 2024.

The user has requested enhancement of the downloaded file.

 Suggested Citation:

206 Truong Tran Minh, N. & Mahmood, R. Q. (2024). Experimental Research. In H. Bui (Ed.), Applied Linguistics and
Language Education Research Methods: Fundamentals and Innovations (pp. 206-227). IGI Global.
https://doi.org/10.4018/979-8-3693-2603-9.ch013

Chapter 13
Experimental Research
Nhat Truong Tran Minh
Industrial University of Ho Chi Minh City, Vietnam

Rizgar Qasim Mahmood

https://orcid.org/0000-0002-5987-8884
University of Wollongong, Australia

ABSTRACT
Nowadays, researching and innovating teaching processes is vital for teachers at higher education to
improve their teaching quality, especially in applied linguistics and language education. experimental
research is a useful approach for researchers, educators, teachers and managers at universities. The
chapter may aim to achieve the following objectives: (1) Present a comprehensive overview of experi-
mental research methodologies; (2) Provide guidelines for designing experiments, including defining
research questions, selecting appropriate variables, and establishing control conditions; (3) Discuss
various data collection methods, including observation, surveys, and laboratory experiments; (4) Detail
the importance of random assignment and statistical analysis techniques to ensure robust results; (5)
Highlight ethical considerations and potential limitations in experimental research; (6) Showcase ex-
amples of successful experimental studies in different disciplines to provide practical insights.

INTRODUCTION

Imagine you are teaching second language (L2) pronunciation to a new class of learners. As you observe
their performance, you notice their pronunciation skills are inadequate, with some struggling to articu-
late basic sounds of the target language (TL). This prompts you to consider the use of Oral Corrective
Feedback (OCF) techniques (Li, 2018; Lyster, Saito, & Sato, 2013; Saito, 2021).
During each lesson, when a learner makes an error, you provide them with the correct form of their
errors. This feedback can be explicit, directly informing them of their mistake and providing the correct
form, or indirect, such as repeating the word, phrase, or sentence with the correct pronunciation (Bry-
fonski & Ma, 2020; Mahmood, 2023).

DOI: 10.4018/979-8-3693-2603-9.ch013

Copyright © 2024, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Experimental Research

After some time, you question whether the provision of OCF effectively improves learners’ pronun-
ciation abilities. How can you find answers to this question? These inquiries aren’t limited to the realm
of L2 teaching; people ask similar questions in their daily lives and seek answers (Gonzalez-Marquez,
Mittelberg, Coulson, 2003). Therefore, it can be concluded that academic research is a process of in-
vestigation that revolves around posing questions and seeking corresponding answers (Nunan, 1992).
In that sense, academic research is defined as “an intellectual act to discover new facts or knowledge
by attempting to go beyond existing knowledge” (Phakiti, 2015, p. 4). Therefore, the following sections
will explore how to approach such questions using the scientific methodology, focusing on experimental
research. Additionally, practical considerations will be addressed to serve as a preliminary guide for
conducting experimental research.

EXPERIMENTAL RESEARCH

Experimental Research in Applied Linguistics and Language Education

As a reader, when you encounter the term “experiment” for the first time, you might immediately envision
a scenario where a team of scientists meticulously observes subjects, perhaps plants, within a controlled
setting or working with chemical substances, striving to formulate cures for serious diseases. In language
pedagogy, experimental research methods resemble those utilised in the aforementioned example. How-
ever, they focus on language learners and seek to comprehend various aspects of their learning process.
For instance, in language learning, researchers may investigate whether specific teaching strategies (e.g.,
the provision of OCF or activities can improve learners’ learning outcomes. They might also examine
whether certain linguistic conditions pose challenges for language acquisition among learners.
Unlike the botanical example mentioned earlier, experimental research in language learning typically
does not take place in a highly controlled scientific laboratory. Instead, participants, namely language
learners, are not confined and can interact with the external environment. It is impractical for language
learning researchers to attempt complete control over these interactions, and doing so could raise ethi-
cal concerns (Phakiti, 2015). Experimental research entails examining the impact of one variable on
another. Essentially, a researcher begins with a hypothesis about why a certain phenomenon occurs and
then alters at least one variable while controlling others to observe its effect on another variable. For
instance, suppose we aim to determine if directing a learner’s attention to a specific aspect of language
enhances their learning of that aspect. One approach is to select two groups of learners matched in their
pre-existing knowledge of the language aspect. Subsequently, the experimental group undergoes a treat-
ment session where their focus is directed towards the particular language feature being investigated,
whereas the control group receives exposure to the same aspect of language without the intentional
direction of attention. Following the treatment, a post-test is administered to assess improvement from
the pre-test. In essence, experimental research involves manipulating at least one variable (independent
variable) while keeping other relevant variables constant and observing its effect on another variable
(dependent variable), such as a test score (Gass, 2015). In addition to this example mentioned above,
several other examples illustrate how researchers conduct experimental research among targeted groups
of participants. For instance, Mahmood (2023) explored the impacts of explicit and implicit pronuncia-
tion instruction on the development of pronunciation skills among Kurdish EFL learners. Similarly,
Saito (2021) examined the effects of different types of OCF, such as “recasts” and prompts, on learners’

207
 
Experimental Research

pronunciation. Based on these two examples, it is evident that experimental studies compare research
outcomes, such as learners’ performance, based on the conditions to which learners are exposed.
Experimental research serves as a valuable methodology for studies aiming to elucidate causal relation-
ships. This method helps researchers meticulously control extraneous factors by maintaining consistency
across groups while intentionally varying the degree of the variable of interest among different learner
groups. In so doing, the researcher(s) attempts to explore causal-like relationships. Experimental research
is typically situated within a quantitative research framework which is frequently utilized in language
learning research. As this method is rooted in the active theory of causation, it attempts to pinpoint
variables which can be manipulated by the researchers. These manipulations help researchers to induce
changes (Cook & Shadish, 1994; Johnson & Christensen, 2019; Phakiti, 2015). Experimental research
is a primary method for testing hypotheses and establishing cause-effect relationships. It offers the most
robust form of reasoning regarding the connections between variables. In experimental research, the
researcher(s) intentionally alters at least one independent variable, while maintaining control over other
relevant variables, and then observes the effects on one or more dependent variables (Mills & Gay, 2018).
In experimental research, it is essential to differentiate between experimental and quasi-experimental
designs. In experimental designs, a primary focus is typically on the dependent variable(s), which represents
what the researcher is investigating for potential change. Concurrently, one or more independent variables
are systematically manipulated by the researcher to assess their impact on the dependent variable(s).
Researchers have the flexibility to include or exclude specific independent variables and adjust their
intensity as needed. To evaluate the effects of independent variables, researchers often conduct pre-tests
and post-tests, with the treatment administered between these testing sessions. The experimental design
also encompasses random assignment of participants to groups and the inclusion of a control group.
However, many research studies in Second Language Acquisition (SLA) are quasi-experimental rather
than truly experimental. Unlike experimental designs, quasi-experimental designs typically lack random
group assignment and instead rely on convenience sampling. Moreover, quasi-experimental designs may
not require a control group and might involve intact classes for different groups. Another key distinc-
tion between experimental and quasi-experimental designs is the presence of a control group. While
quasi-experimental studies may feature multiple treatment groups, without a control group, it becomes
challenging to ascertain whether these groups would have performed significantly better than a group
that did not receive any treatment (Cook & Wong, 2008; Kirk, 2009, 2012; Loewen & Plonsky, 2015;
Phakiti, 2015; Rogers & Révész, 2023).

Advancing Scientific Knowledge through Experimental Research

Experimental research plays a crucial role in advancing scientific knowledge in various fields, includ-
ing applied linguistics and L2 education. Through experimental studies, researchers can systematically
investigate causal relationships between variables and test hypotheses. This approach allows for rigorous
testing of theories and provides empirical evidence to support or refute existing beliefs or assumptions
(Ledyard, 1995). Moreover, experimental research can help researchers establish cause-and-effect rela-
tionships in the research process. Through carefully designed experiments, researchers can manipulate
and control variables to determine their effects on a particular outcome (Rogers & Révész, 2023). This
allows them to identify the factors that contribute to the success of language learning or teaching, pro-
viding valuable information for practitioners and policymakers in the field. Furthermore, experimental
research allows for the replication of studies, enhancing the credibility and generalizability of findings

208

Experimental Research

(Higgins, Murphy, & Hogg, 2021). Replication is essential in science as it helps to validate or challenge
previous results, ensuring that the knowledge generated is reliable and robust. Experimental research
also promotes transparency and openness in scientific inquiry since researchers are required to provide
detailed descriptions of their methodologies and procedures, allowing transparency in the research pro-
cess (Escueta, Nickow, Oreopoulos, 2020).
In applied linguistics and language education, experimental research is particularly valuable for in-
forming instructional practices in teaching and policy decisions in management. By examining the effects
of specific instructional techniques, interventions, or technologies, researchers can provide evidence-
based recommendations for improving language learning and teaching (Fang, Yeh, Luo, 2021). This
knowledge can then be used to develop more effective teaching materials, design targeted interventions
for language learners, and inform educational policies and practices. In general, through experimental
research, researchers can provide valuable information and evidence-based recommendations to improve
language learning and teaching practices.

KEY ELEMENTS IN EXPERIMENTAL RESEARCH

Experimental research demands a robust methodology to explore whether and to what extent specific
factors, such as types of instructions, input, or interactions, can enhance or impede language learning
(Phakiti, 2015). Before conducting a study using experimental research, a researcher must consider vari-
ous specifics to ensure methodological rigour and ethical conduct. These include formulating research
questions and hypotheses, identifying variables, determining participant assignment methods, selecting
appropriate statistical analyses, and addressing validity and reliability concerns. Within validity consid-
erations, researchers must assess content validity, face validity, construct validity, and criterion-related
validity, while also addressing internal and external validity concerns. Additionally, it is vital for the
researcher(s) to carefully consider the techniques and instruments used in data collection and analyses.
Moreover, ethical considerations, such as participant rights and confidentiality, must be thoroughly ad-
dressed throughout the research process (Creswell & Creswell, 2022; Gass, 2015; Kaiser, 2014; Kirk,
2009; McDonough, 2019; Phakiti, 2015).

Research Questions

Understanding the nature of research questions in experimental research is crucial. These questions
should be articulated and grounded in previous literature (Gass, 2015; Phakiti, 2015). For instance,
previous research has investigated the effects of OCF and its types on the development of L2 learners’
pronunciation and sound production. These studies have explored how different forms of Corrective
Feedback (CF) affect the written or spoken aspects of an L2 within specific contexts. Subsequently, an
experimental study could be conducted to compare the outcomes of learners’ accurate sound production
development based on different types of feedback provided under uniform conditions or across varied
contexts. By employing an experimental research design, researchers may address the research question:
“To what extent does explicit OCF enhance L2 learners’ ability to accurately produce L2 target sounds?”
Ideally, researchers aim to maintain consistent environmental conditions across all comparison groups,
except the variable under investigation. For instance, group 1 might receive explicit OCF, group 2 might
receive implicit OCF, and group 3 might undergo traditional pronunciation instruction without any OCF

209
 
Experimental Research

provision. Subsequently, researchers can statistically compare learners’ observed learning outcomes, such
as sound production scores obtained from pre-tests, post-tests, or even delayed post-tests, both within
and across groups (Mahmood, 2023).
The following examples are experimental research questions:

Do recasts have differential effects on the acquisition of regular versus irregular English past tense
forms? Do prompts have differential effects on the acquisition of regular versus irregular English
past tense forms? (Yang & Lyster, 2010, p. 242).
Do different Form-Focused Instructions (FonFs and FonF) have different effects on L2 Chinese learners’
oral production? Does L2 proficiency modulate the learning outcomes in different pedagogical
approaches? (Chen & Li, 2022, p. 4).

Research Hypotheses

Definition

The research hypothesis serves as a cornerstone for conducting a research study. Its absence would hinder
our understanding of key research components, limiting the scope of conclusions drawn. A hypothesis
serves as a tentative idea or proposition proposed to ascertain its validity. In research, a hypothesis is a
statement that poses a question and predicts potential outcomes (Toledo, Flikkema, & Toledo-Pereyra,
2011). The protocol’s findings should substantiate a well-defined research hypothesis (Bartoszynski
& Niewiadomska-Bugaj, 2007; Toledo-Pereyra, 2011). Mills and Gay (2018, p. 85) defined a research
hypothesis as “A researcher’s prediction of research findings, a statement of the researcher’s expecta-
tions about the relations among the variables in the research problem.” They, further, stated that the
researcher’s objective is not to confirm a hypothesis but to gather data that either supports or refutes it.
A hypothesis assumes a crucial role within the scientific method as it establishes a foundation for testing
established theories. For instance, a hypothesis could examine the efficacy of OCF on the development
of pronunciation skills among L2 learners.

Types of Hypotheses

Hypotheses can be categorized based on their derivation (inductive versus deductive hypotheses) or
their statement (directional versus null hypotheses). An inductive hypothesis is a generalization based
on specific observations. Researchers observe certain patterns or associations among variables in mul-
tiple situations and use these tentative observations to formulate an inductive hypothesis. For instance, a
researcher noticed that in some eighth-grade classrooms, learners who take essay tests appear to exhibit
less test anxiety than those who take multiple-choice tests. This observation could serve as the foundation
for an inductive hypothesis. On the other hand, a deductive hypothesis stems from theory and provides
evidence that supports, expands, or challenges the theory. A research hypothesis outlines an anticipated
relationship or difference between variables. Research hypotheses may take the form of nondirectional or
directional. A nondirectional hypothesis states that a relationship or difference between variables exists.
Conversely, a directional hypothesis specifies the expected direction of the relationship or difference
(Creswell & Creswell, 2022; Gass, 2015; Mills & Gay, 2018; Sheskin, 2011).

210

Experimental Research

Statistically, researchers employ both null and alternative hypotheses. The null hypothesis posits no
significant difference and is denoted as Ho. Conversely, the alternative hypothesis indicates a signifi-
cant difference and is represented by HA or H1(Sheskin, 2011; Toledo et al., 2011). Take the effects of
OCF on L2 learners’ accurate sound production, the null hypothesis states that There is no statistically
significant difference between variables, that is, if the researcher found that the L2 learners’ ability to
produce L2 sound accurately did not improve after the provision of OCF, then, the null hypothesis will
be accepted. However, if the results show statistically significant differences between the variables,
then, the alternative hypothesis will be accepted. As Sheskin (2011, p. 58) puts “The null hypothesis is
a statement of no effect or no difference… The alternative hypothesis, on the other hand, represents a
statistical statement indicating the presence of an effect or a difference.”

Variables

The term “variable” originates from the word “vary.” Age serves as a prime example of a variable due
to the wide age range observed among individuals. Phakiti (2015) gives other examples of variables
such as gender, first language (L1), length of learning, intelligence, proficiency in the English language,
motivation, anxiety, and feedback.
Experimental research designs can be classified based on the number of independent and dependent
variables utilized in the study (Loewen & Plonsky, 2015). The independent variable, manipulated by
the researcher, contrasts with the dependent variable, which is measured. For instance, in a hypothetical
investigation concerning accent judgments, the independent variable might be whether the participant
is a native or non-native speaker, while a potential dependent variable could be the scores obtained on
the accent judgment test. In this scenario, with only one independent variable, the experiment is termed
single-factor, and as this independent variable encompasses two groups, it is regarded as having two
levels. Lastly, since there is just one dependent variable (scores), the design would be categorized as
a univariate design (Abbuhl, Gass, & Mackey, 2014). Furthermore, Gass (2015) also stated that in ex-
perimental research, two primary variables are of utmost concern: independent variables and dependent
variables. “An independent variable is a variable that exists freely and is hypothesized to affect other
variables” (Phakiti, 2015, p. 27). Dependent variables, on the other hand, are influenced by the indepen-
dent variable, reflecting the variables upon which the independent variable exerts its effect. For instance,
in the example of OCF and sound production improvement, OCF is the independent variable and sound
production improvement is the dependent variable. Thus, “Dependent variables are those variables that
the independent variable is affecting” (Gass, 2015, p. 91).

Participants and Recruitment

Research in the field of language always involves human participants. For example, when a researcher
conducts research with L2 learners, the participants are those who speak, write, read, or listen to the
language under study. Researchers may interact with these individuals in various ways, including in-
person interactions, communication over phones and other media, or analysis of texts produced by
these individuals for different purposes. Depending on the specific context and nature of the research,
individuals involved may be referred to by different terms such as subject, informant, participant, col-
laborator, consultant, and possibly others (Abbuhl et al., 2014; Creswell & Creswell, 2022; Loewen &
Plonsky, 2015).

211
 
Experimental Research

Once the researcher has decided to conduct the research experimentally, the next step involves re-
cruiting participants. Typically, specific conditions are established for participant selection (Creswell
& Creswell, 2022), such as being over 18 years old, speaking a particular language as their L1, being
in English as a Foreign Language (EFL) or English as a Second Language (ESL) context, or having a
certain proficiency level. Researchers frequently inquire, “What is an adequate number of participants?”
Qualitative researchers typically respond with, “It varies.” There are no rigid guidelines dictating the
“ideal” number of participants. Qualitative inquiries can be conducted with just one participant or with
larger groups comprising 60 to 70 individuals, encompassing diverse contexts (Mills & Gay, 2018). In
addition to that, Mills and Gay (2018) propose two primary criteria to indicate the adequacy of participant
selection. Firstly, researchers assess the extent to which the chosen participants mirror the diversity found
within the broader population. Secondly, researchers consider the redundancy of information obtained
from participants. Finally, after the recruitment process, the participants are divided may be divided into
several groups: control and experimental group(s).
A control group is a group of participants who do not receive the intervention or manipulation of the
independent variable (Nieuwburg, Ploeger, & Kret, 2021). The control group serves as a baseline against
which the effects of the independent variable are compared. In the case of the feedback study, the control
group would be the group of students who do not receive any feedback on their writing.
An experimental group is a group of participants who receive the intervention or manipulation of the
independent variable (Mattila, Luo, Xue, 2021). They are exposed to the condition being studied. In the
feedback study, the experimental group would be the group of students who receive corrective feedback
or no feedback on their writing. The purpose of the experimental group(s) is to observe and compare
the effects of the independent variable on the dependent variable. By comparing the results of the ex-
perimental group(s) with those of the control group, researchers can determine whether the independent
variable had a significant impact on the dependent variable (Li, Cheng, Liu, 2013).
A key characteristic of experimental research is the utilization of random assignment. Random as-
signment involves randomly allocating participants to different study conditions involving a manipulated
variable. When individuals are assigned to groups in this manner, the research design is referred to as a
true experiment (Creswell & Creswell, 2022; Phakiti, 2015). The fundamental structure of a true experi-
ment comprises the experimental subject, the independent variable, and the measured dependent variable.
True experiments serve not only to establish the relationship “If X then Y” but also to demonstrate the
converse: “If no X then no Y.” (Pickard, 2013, p. 121). Designs where researchers have limited control
or are unable to randomly assign participants to levels of a manipulated variable of interest are termed
quasi-experiments(Creswell & Creswell, 2022). Regarding random assignment in experimental research,
Gass (2015, p. 91) states:

A hallmark of experimental research is the random assignment of participants to one group or another.
Random assignment of individuals means that each individual has an equal chance of being assigned
to any of the conditions of the study (experimental or control). That is, the process of assignment is
random. Randomization is intended to eliminate the possibility that extraneous variables will creep into
the research design.

For example, In Goo’s study (2012), which assessed the efficacy of recasts versus metalinguistic
feedback on English that-trace filter acquisition, six intact classes were randomly allocated to one of
three conditions: “recasts,” “metalinguistic feedback,” and “control” (p. 454).

212

Experimental Research

Figure 1. Major Experimental Designs<Fig_Large></Fig_Large>

RESEARCH DESIGN

When researchers select an experimental design, they must address several inquiries to enhance both the
internal and external validity of the study. These considerations involve evaluating the nature of variables
under examination, the number of independent variables under scrutiny, the decision to conduct pretests,
determining the frequency of treatment sessions, and determining the size and characteristics of the sample
to be chosen (Mackey & Gass, 2022). A research design encompasses a structured framework detailing
the procedures, phases, and methodologies employed in various experimental research endeavours. It
typically comprises four primary types: pre-experimental, single-case, randomised experimental, and
quasi-experimental designs (Phakiti, 2015). Figure 1 illustrates the major types of experimental research
design.

Pre-experimental Designs

Pre-experimental designs are considered to be the least rigorous type of experimental design. They lack
comparison or control groups and do not allow for strong causal inferences. The most common types of
pre-experimental designs are (Phakiti, 2015):
One-group post-test-only design: In the one-group post-test-only design, the absence of a pre-treatment
measure of the dependent variable, such as grammatical and vocabulary knowledge, makes it challenging
to attribute high post-test scores solely to the treatment. This is because there is no baseline to compare the
post-test scores to, nor are there scores from a comparison group to assess the treatment’s effectiveness.
The one-group pretest-post-test design: This design represents an advancement from the one-group
post-test-only design. In this design, a single group of participants undergoes both pre-treatment and
post-treatment assessments. This setup resembles action research, where a teacher-researcher endeavours
to enhance student learning by introducing activities thought to address a specific issue.
Post-test-only with non-equivalent groups design: This design seeks to overcome the shortcomings of
previous two designs by introducing comparison groups. This design involves two groups of participants:
one group receives the experimental treatment, while the other does not. Unlike the previous designs,
there is no assumption of equivalence between the groups. Following the treatment period, both groups
undergo a post-test, and their scores are compared to assess the treatment’s impact.

True Experimental Designs

True experimental designs are considered more rigorous than pre-experimental designs, as they involve
random assignment of participants to different groups. This allows greater control over confounding
variables and enables stronger causal inferences (Gass, 2015; Inna, Ihor, Olena, 2021; Phakiti, 2015).
For example, in a study investigating the effectiveness of a new language teaching method, one group of

213
 
Experimental Research

participants may receive the new method, while the control group receives the traditional method. The
differences in post-test scores between the two groups can then be attributed to the effectiveness of the
new method. The most commonly used true experimental designs are (Leppink, 2019). In a true experi-
ment, researchers randomly assign participants to different treatment groups. According to Creswell and
Creswell (2022) there are several types of true experimental design as categorised below.
A single-subject design, also known as N of 1, involves observing the behaviour of one individual or
a small group over a period of time. In experiments, comparisons are essential. Between-subject designs
involve each participant experiencing only one condition of a manipulated variable. One-way designs
involve manipulating one independent variable, while factorial designs manipulate multiple independent
variables to explore their effects. In contrast, within-subject designs involve participants experiencing
multiple conditions of a manipulated variable. For example, a repeated measures design uses the same
measurement before and after an intervention, with “time” being a within-subjects variable. Many studies
combine between-subjects and within-subjects variables, termed mixed designs. For instance, in a study
on working conditions and burnout symptoms among nurses, participants could be randomly assigned
to work part-time or full-time (between-subjects variable) while their burnout symptoms are measured
before and after the work change (within-subjects variable), resulting in a mixed design.

Quasi-experimental Designs

Quasi-experimental designs are used when random assignment is not possible or practical and it is hard
to control unexpected variables, but researchers still want to compare the effects of different treatments
or interventions (Reichardt, 2009; Rogers & Révész, 2023). Quasi-experimental designs are often used
in educational settings where it is not practical or ethical to randomly assign participants to groups.
Creswell and Creswell (2022) have classified quasi-experimental designs into as follows:
Non-equivalent (Pretest and Post-test) Between-Group Design: In this widely used quasi-experimental
approach, both experimental Group A and control Group B are selected without random assignment.
Both groups undergo pretest and post-test assessments, with only the experimental group receiving the
treatment.
Single-Group Interrupted Time-Series Design: This design involves recording measures for a single
group both before and after a treatment is administered.
Between-Group Interrupted Time-Series Design: A modification of the Single-Group Interrupted
Time-Series design, this approach observes two groups of participants, not randomly assigned, over
time. One group receives the treatment, while the other serves as a control. For instance, if there are two
groups, A and B, the treatment is administered solely to one group (Group A). For example, researchers
could measure students’ language proficiency at regular intervals before and after the implementation
of a new teaching strategy (Rogers & Révész, 2023).

Reliability and Validity in Experimental Research

Reliability and validity can be the multifacted concepts (Hung, 2017). Reliability pertains to the internal
consistency of instrument scores, ensuring that item responses remain consistent across constructs. It
also involves the stability of scores over time, as evidenced by the correlation between the tests, and the
consistency in test administration and scoring procedures. In essence, reliability reflects the consistency
with which an instrument measures a particular construct. Therefore, it assesses whether subsequent

214

Experimental Research

measurements would yield similar results. Reliability is a fundamental prerequisite for establishing
instrument validity (Creswell & Creswell, 2022; Gass, 2015; Loewen & Plonsky, 2015). For example,
when a researcher investigates L2 learners’ reading performance, a reliable test would be using reading
tests, or when the researcher examines the effectiveness of OCF on L2 sound production, a reliable test
would be a test that measures learners’ sound production such as word pronunciation or analysing their
produced spoken utterances.
The validity, on the other hand, concerns the precision and accuracy of the measurements and ob-
servations conducted during data collection, as well as the integrity of the conclusions drawn from the
collected data. In essence, it examines whether the research findings are resulting from reliable data and
analysis methods. Validity is a fundamental concept in all research fields, though it is often emphasized
more in quantitative research. It encompasses both the substantive and methodological robustness of a
study (Gass, 2015; Loewen & Plonsky, 2015; Phakiti, 2015).
However, generating a single definition of validity that encompasses all aspects of research processes
is impractical. Research methodologists outline various types of validity that quantitative researchers
need to consider. These include internal and external validity, construct validity, and statistical validity
(Phakiti, 2015).
First, internal validity, the primary form of research validity, focuses on establishing the logical con-
nection between the independent and dependent variables being investigated. It evaluates the extent to
which other potential variables may affect the research outcomes. On the other hand, external validity
pertains to the extent to which findings from an experiment can be applied to other learners with similar
characteristics and different settings. This aspect is crucial for securing research funding, as studies that
are limited to specific learner groups, settings, or timeframes may have limited generalizability. Further-
more, the researcher(s) must critically examine the definitions of constructs during the literature review.
Researchers must assess differing perspectives on the same construct and propose clear definitions and
operationalizations for their research. It gauges the extent to which a construct is accurately defined,
measured, and inferred (Cash, Stanković, & Štorga, 2016; Gass, 2015; Phakiti, 2015).
Another type of validity is content validity, a component of construct validity that assesses how well a
measurement represents the construct of interest. It evaluates whether the behaviours or abilities sampled
are a good match for what they are supposed to measure. For example, in assessing learners’ ability to
find the main ideas in a given academic passage, the selection of texts, questions, and tasks determines
the content validity of the assessment (Creswell & Creswell, 2022; Loewen & Plonsky, 2015).
Then, there is criterion-related validity, closely linked to construct validity, which examines the rela-
tionship between an instrument and other measures of the same or similar constructs. Finally, statistical
validity comes into play. It ensures the reliability of inferences drawn from experimental studies through
sound statistical analyses. It is essential for comparing the performance of groups subjected to different
treatments and arriving at valid conclusions(Cash et al., 2016; Creswell & Creswell, 2022).

Identifying Common Threats and Mitigation

Strategies in Experimental Research

There are several threats to internal validity in experimental research that researchers must be aware of
and mitigate. Creswell and Creswell (2022) recognised several common threats to internal validity as
summarized in Table 1.

215
 
Experimental Research

Table 1. Types of Threats to Internal Validity<Tbl_Large></Tbl_Large>

In Response, Actions the Researcher Can

Type of Threat to Internal Validity Description of Threat
Take
External events occurring during the
Both experimental and control groups can
History experiment can influence outcomes apart
experience the same external events.
from the treatment.
Select participants who mature or change
Participants may mature or change during
Maturation at a similar rate (e.g., same age) during the
the experiment, impacting the results.
experiment.
Participants with extreme scores tend to Select participants with non-extreme scores
Regression to the mean
move towards the average over time. for the experiment.
Randomly select participants to distribute
Participants with certain characteristics
Selection characteristics equally among experimental
may predispose them to specific outcomes.
groups.
Recruit a large sample to account for
Participants dropping out during the
Mortality (study attrition) dropouts or compare dropouts with
experiment may affect outcomes.
continuers in terms of outcomes.
Communication between control and
Keep the groups as separate as possible
Diffusion of treatment experimental groups may influence
during the experiment.
outcomes.
Provide benefits to both groups, such as
offering the control group the treatment
Unequal treatment may lead to resentment
Compensatory/resentful demoralization after the experiment ends or providing
in the control group.
a different type of treatment during the
experiment.
Take steps to ensure equality between
Control group participants may feel
groups, such as managing expectations or
Compensatory rivalry devalued compared to the experimental
explaining the value of the control group
group.
clearly.
Increase the time interval between
Participants may become familiar with the
administrations of the outcome or use
Testing outcome measure, affecting responses on
different items on later tests than those used
later testing.
in earlier tests.
Changes in the instrument between pretest Use the same instrument for both pretest
Instrumentation
and post-test may affect outcome scores. and post-test measures.

ETHICAL CONSIDERATIONS

When conducting experimental research involving human participants, ethical considerations are essential
principles. The researcher’s foremost responsibility is to honour the rights, needs, values, and prefer-
ences of the participants. Ethical research conduct entails adherence to essential principles, including
participant consent (i.e., informed consent), anonymity, and confidentiality (Phakiti, 2015). Research-
ers must adhere to ethical protocols to ensure the protection of their research participants. Participants
should provide voluntary consent to participate in the study (Pickard, 2013), fully informed of its nature
and procedures (Kaiser, 2014). They should not be coerced into participation. Additionally, researchers
must carefully evaluate and mitigate any potential risks, such as physical or psychological harm, that
participants may encounter during the study. Furthermore, honesty in reporting is paramount, particularly
concerning participant exclusion. If participants are excluded from the study, it must be justified and

216

Experimental Research

carried out judiciously while maintaining transparency and integrity in the research process (Abbuhl et
al., 2014; Gass, 2015; Phakiti, 2015).
Ethical concerns extend beyond research participants to encompass the professional integrity of
researchers. The public must view research professionals as trustworthy, and researchers must conduct
themselves in a manner that upholds their credibility. Researchers bear responsibility for the societal
implications of their work. Thus, ethical considerations in experimental research underscore the inter-
connectedness between research and society. A groundbreaking experimental study, regardless of its
design, will not garner acceptance from society if conducted without ethical adherence (Abbuhl et al.,
2014; Phakiti, 2015).
When engaging in experimental research, researchers must uphold specific ethical principles and
guidelines to safeguard the rights and welfare of participants. Some of these principles and guidelines
are outlined by (Bryman, Bell, Reck, 2021; Drager, 2018; Kirk, 2009, 2012; McDonough, 2019; Polio
& Lee, 2019; Rogers & Révész, 2023).

Informed Consent

Central to ethical research practice is the requirement to obtain voluntary informed consent from par-
ticipants. This entails providing participants with a clear understanding of the study’s purpose, potential
risks and benefits, and their rights as participants. Participants need to have the autonomy to make an
informed decision about their participation, and researchers have to ensure that consent is obtained in a
transparent and comprehensible manner.

Privacy and Confidentiality

It is important to safeguard the privacy of participants and maintain the confidentiality of any personal or
sensitive information obtained during the study. To uphold confidentiality, it is crucial to adopt measures
to anonymize data and protect participants’ identities. By ensuring the confidentiality of participants,
researchers uphold the trust and integrity of the research process.

Minimization of Harm

It is important to prioritize the well-being of participants and take precautions to prevent harm during the
experimental research. This involves anticipating and addressing potential risks to participants’ physical
and psychological well-being, establishing protocols for addressing any adverse effects that may arise,
and providing participants with support mechanisms as needed to mitigate harm.

DATA COLLECTION

In experimental research, data are collected over various periods. For instance, before commencing the
experiment, researchers gather data through pre-tests. Following this, interventions are applied, and data
may be collected midway through or at the end of the intervention, typically in the form of post-tests.
The pre-test-post-test control-group design is commonly employed in language learning research. In
numerous L2 studies, participants typically undergo a pretest to establish the comparability of partici-

217
 
Experimental Research

pant groups before treatment, followed by a post-test to gauge treatment effects. While a pretest/post-
test design enables researchers to assess the immediate impact of treatment, the fundamental inquiry
in an L2 learning study lies in determining the extent to which treatment genuinely facilitates learning.
To evaluate longer-term effects, researchers often incorporate delayed post-tests alongside immediate
post-tests, which occur shortly after treatment (Mackey & Gass, 2022). Rogers and Révész (2023, p.
135) stated that “The pre-test–post-test control group design is probably the most common experimental
research design.”
In experimental research, participants are initially assigned randomly to one of three conditions, in-
cluding a control condition. Initially, they undergo pretesting on the dependent variable. Subsequently,
the experimental groups receive the designated treatments, while the control group may undergo typical
classroom practices. Following the treatment period, both groups undergo testing on the dependent vari-
able (Phakiti, 2015). For example, Saito and Lyster (2012) investigated the effectiveness of form-focused
instruction and CF on Japanese L2 learners in the production of English /r/. They randomly assigned
65 learners to three groups: the FFI-only group, the FFI + CF group, and the control group. Each group
underwent 4 hours of meaning-based instruction on argumentative skills, delivered by two ESL teach-
ers. Additionally, the FFI-only and FFI + CF groups received instruction on the English /r/ sound. After
data collection, they analysed the data acoustically. The results revealed that (a) The F3 values of the
FFI + CF group significantly decreased following the intervention, not only in controlled speech but
also in spontaneous speech, irrespective of the following vowel contexts. (b) There was no statistically
significant change in the F3 values for the FFI-only group and the control group.
There are other types of design in experimental research. For example, time-series design exempli-
fies a longitudinal approach wherein researchers gather language samples at regular intervals over a
defined duration (Kirk, 2009; Mackey & Gass, 2022; Rogers & Révész, 2023). Through multiple data
collection points, time-series designs offer insights into the temporal dynamics of language development,
encompassing immediate, gradual, delayed, incubated, or residual changes (Mackey & Gass, 2022; Mel-
low, 2012; Rogers & Révész, 2023), as well as the durability of treatment effects. Furthermore, there
are repeated-measures designs, also termed within-participants designs, that involve a single group of
participants engaging in various treatment conditions and/or being assessed at multiple time points
(Abbuhl & Mackey, 2017). Finally, some researchers may use another type of design: factorial designs.
This design includes more than one independent variable. This design enables researchers to explore
the effects of multiple independent variables on the dependent variable. These designs enable research-
ers to evaluate the individual impacts of each independent variable and their combined effects on the
dependent variable (Mackey & Gass, 2022; Rogers & Révész, 2023).

Data Organization and Coding

Once the researcher(s) collected the quantitative data to address the research objectives and inquiries,
it is important to ensure the completeness of all participants’ data. For instance, some participants may
have left certain questionnaire items or test items unanswered. This results in incomplete data, commonly
referred to as missing data. In this situation, the researcher(s) must determine the appropriate approach
for dealing with such missing data. Then, the data will be entered into a computer software program
(e.g., SPSS, Microsoft Excel, or Jamovi). Once the data are coded and assigned numerical values for
each participant, they can be entered into a statistical software application for analysis (Phakiti, 2015).
Finally, the next step is to select an appropriate statistical test for the analysis purposes.

218

Experimental Research

Data Analysis and Interpretation

This section delves into the procedures undertaken when dealing with quantitative data. Initially, it outlines
the various phases encompassed in statistical analysis, emphasizing the significance of understanding this
process in experimental research before going into fundamental statistical principles. After elucidating
the stages of statistical analysis, the section introduces descriptive statistics as a means to portray the
attributes of quantitative data and delves into essential notions in inferential statistics, such as hypothesis
testing, probability, significance values, and sample sizes. Additionally, it offers a synopsis of prevalent
statistical tests applied in language learning and experimental investigations.
After the researcher(s) collected the data, the initial step in interpreting results involves conducting
statistical analyses. However, a comprehensive discussion regarding various types of statistics and de-
signs is not within the scope of this chapter. In experimental research, quantitative social science data
undergoes specialized statistical analysis, often facilitated by special software such as the Statistical
Package for the Social Sciences (SPSS) and Jamovi (Cash et al., 2016).
In experimental research, there are two main categories of statistical analyses: descriptive statistics
and inferential statistics, each serving distinct purposes. Descriptive statistics focus on summarizing data
collected for a specific variable, typically by calculating measures of central tendency (such as mode,
median, and mean) and variability (such as range and standard deviation) (Johnson & Christensen, 2019;
Phakiti, 2015). These statistics provide insights into the characteristics of the data. In contrast, infer-
ential statistics allow researchers to explore relationships between variables, test hypotheses, and draw
generalizations beyond the immediate research context. They enable the examination of differences or
associations among two or more variables (Field, 2017).

Descriptive Statistics

Descriptive statistics provide a summary of the data collected in the experiment (Ross & Morrison,
2013). This includes measures such as means, standard deviations, and frequency distributions, which
help researchers understand the central tendency and variability of the data. For example, in a study
comparing the effectiveness of two teaching methods, descriptive statistics can provide information on
the average performance of students in each condition and the variability in their scores. Descriptive
statistics serve as metrics for quantitative data, encompassing indicators of central tendency, variability,
and relative position. Central tendency can be defined through various measures, one of which is the
mean, calculated as the sum of scores divided by the number of scores.
In applied linguistics research, including both experimental and qualitative studies, the mean is
extensively utilized as a descriptive statistic, representing the average of the data or scores. Another
measure of central tendency is the median, which is the value that divides a dataset into two halves,
with half the scores falling below and half above it. To determine the median, the data must be arranged
in ascending order.
Additionally, central tendency can be assessed through the mode, which identifies the value that ap-
pears most frequently in the dataset. When considering the distribution of data, the normal distribution
is characterized by being unimodal (having one mode), symmetric around the mean, and bell-shaped.
However, achieving a perfectly normal distribution is rare in research. An assumption often made in
inferential statistics is that the data follows a normal distribution, particularly in larger sample sizes (e.g.,
30 or more), where the mean, median, and mode converge or closely approximate each other.

219
 
Experimental Research

Furthermore, measures of dispersion, also known as measures of variability, describe the spread of
data. Dispersion can be categorized as homogeneous, indicating little variation in numerical values, or
heterogeneous, suggesting variability among values. Common measures of dispersion include the range,
which is the difference between the highest and lowest scores in the dataset. A smaller range indicates
a more homogeneous dataset.
The variance and standard deviation are more precise measures of dispersion. The variance quanti-
fies the average squared deviation from the mean, while the standard deviation, obtained by taking the
square root of the variance, represents how much individual values deviate from the mean on average. A
smaller standard deviation indicates a more homogeneous dataset. Both the variance and standard devia-
tion are integral to various inferential statistics, such as ANOVA and ANCOVA. However, it is important
to note that extreme values, or outliers, can influence measures like the mean and standard deviation.

Inferential Statistics

This section concentrates on fundamental concepts within inferential statistics, including hypothesis
testing, probability, significance values, and sample sizes. It aims to offer a comprehensive review of
the typical statistical tests employed in language learning and experimental research. Inferential statis-
tics are employed to enhance comprehension of the association between two or more variables, such as
linear or causal-like relationships.
To measure a potential relationship between two variables, we follow a three-step process. Initially, we
assume that no relationship exists between the variables as our starting point. Second, we assess whether
this assumption holds true by gathering data or evidence. If the data refute the idea of no relationship,
we can infer that a relationship does exist. Third, we determine the nature of this relationship, whether
it is positive or negative, such as a positive or negative linear connection or a causal-like relationship
(Phakiti, 2015).
Hypothesis testing is a statistical method utilized to evaluate how well quantitative data support a
hypothesis, known as the null hypothesis, which researchers typically consider false. The null hypothesis
(Ho) posits that there is no relationship between the variables or no distinction among learner groups.
This method directly scrutinizes the null hypothesis, and if the data fails to support it, researchers accept
an alternative hypothesis (HA or H1), which asserts the presence of a relationship between the variables
or differences among learner groups (Field, 2017; Gray & Kinnear, 2011). Two types of alternative hy-
potheses exist, first, a non-directional alternative hypothesis, which posits a relationship or difference
between the variables without specifying the direction, and second, a directional alternative hypothesis,
which specifies the direction of the relationship or difference, such as asserting a positive relationship
between the variables or stating that Group A statistically outperforms Group B (Phakiti, 2015).
To reject the null hypothesis, researchers establish a probability threshold, known as the p-value.
This value is specifically designated for testing the null hypothesis rather than the alternative hypoth-
esis. Essentially, the null hypothesis can be rejected when the probability of obtaining the observed
result under the assumption of a true null hypothesis is extremely low. In language learning research,
for instance, it is common to set the p-value to be less than 0.05 (p < 0.05), although some researchers
may opt for a threshold of less than or equal to 0.05 (p ≤ 0.05). It is crucial to understand that if the
established probability value is less than 0.05 and the test statistic yields a p-value of exactly 0.05, the
null hypothesis should still be rejected. This threshold of 0.05 indicates a 5 percent probability that the
null hypothesis under scrutiny is accurate, allowing for a 5 percent margin of error in rejecting it. In

220

Experimental Research

experimental research, a p-value of 0.05 is commonly recommended, although researchers often opt for
stricter thresholds, such as 0.01. For instance, a p-value of less than 0.01 (p < 0.01) indicates a prob-
ability of less than 1 percent that the null hypothesis is correct, while a p-value of 0.001 signifies a one
in a thousand chance that the null hypothesis holds true (Gray & Kinnear, 2011; Johnson & Christensen,
2019; Loewen & Plonsky, 2015; Robinson, 2012).
Researchers new to statistics often grapple with distinguishing between the set probability value and
statistical significance. To clarify this disparity, it is essential to understand that the significance value,
also known as the alpha value, remains constant (e.g., it must be less than or equal to 0.05 or 0.01),
whereas the probability value is determined by the test statistics and is data dependent. For instance, if
we establish a p-value threshold of less than 0.05 to denote significance, and the data yields a p-value
of 0.06, this data-driven p-value isn’t statistically significant at the 0.05 level because 0.06 exceeds 0.05.
Conversely, if we set a probability value of 0.10, a p-value of 0.06 would be considered statistically sig-
nificant at the 0.10 level since 0.06 falls below 0.10. It is crucial to recognize that the significance level
simply indicates a high probability of correctly rejecting the null hypothesis. Moreover, it is important
to note that in statistics, the term “significance” does not equate to importance in everyday language.
Instead, it signifies that researchers have strong grounds to reject the null hypothesis (Ary, Jacobs, &
Sorensen, 2006; Field, 2017; Gray & Kinnear, 2011; Phakiti, 2015).
Correlation is a statistical method utilized to explore associations between two variables without
implying a cause-and-effect relationship. Through correlation analysis, researchers investigate whether
one variable consistently changes in tandem with another, rather than one variable directly influencing
the other (Field, 2017; Gray & Kinnear, 2011).
Regression analysis is used to examine the relationship between a dependent variable and one or
more independent variables (Rogers & Révész, 2023). It helps to determine the strength and direction
of the relationship and can be used to predict future values of the dependent variable based on the in-
dependent variable.
T-test in SPSS: The T-test in SPSS is a statistical analysis tool that is used to determine whether there
are significant differences between the means of two groups. Compare the means of two groups on a
single variable to determine if there is a statistically significant difference (Neumark, 2018).
Analysis of variance (ANOVA): ANOVA is a statistical technique used to compare means between
two or more groups. It helps determine if there are significant differences among the group means and
identifies which groups differ from each other (Field, 2017; Gray & Kinnear, 2011).

Importance of Statistical Significance and Effect Size

Statistical significance and effect size are two important aspects of data analysis in experimental research.
Statistical significance refers to the probability that the observed results are due to the independent
variable and not to the chance (Li et al., 2013). It helps determine whether the results are meaningful
and not just a result of random variation. Researchers typically set a significance level (e.g., p < 0.05)
to determine statistical significance. If the p-value is lower than the significance level, the results are
considered statistically significant.
For example, imagine a study investigating the effectiveness of a new language teaching method
compared to a traditional method. The researchers collect data on the language proficiency of the stu-
dents before and after treatment. By analysing the data, they found that the students who received the
new teaching method improved significantly in their language skills compared to those who received the

221
 
Experimental Research

traditional method. However, to determine the statistical significance of this finding, researchers would
need to compute a p-value using a statistical test, such as a t-test or an analysis of variance (ANOVA).
The p-value represents the likelihood of obtaining the observed results if there is no real difference
between the two teaching methods (Inna et al., 2021). If the p-value is below a predetermined threshold
(usually 0.05 or 0.01), the researchers can conclude that the difference in language skills is statistically
significant. This means that the observed results are unlikely to be due to chance and provide evidence
to support the claim that the new teaching method is more effective.
The effect size, in addition, measures the magnitude or strength of the relationship between variables
(Escueta et al., 2020; Phakiti, 2015). It helps researchers understand the practical significance of the
results and the size of the effect. While statistical significance focuses on the presence or absence of
an effect, effect size focuses on the magnitude of the effect. Effect size is particularly important when
interpreting statistically significant results. Even if a finding is statistically significant, it may have little
practical relevance if the effect size is small. Conversely, a statistically nonsignificant finding may still
have practical significance if the effect size is large.
Both statistical significance and effect size are important in the interpretation of experimental find-
ings. Statistical significance helps determine whether the observed effects are statistically reliable, while
effect size helps determine the practical significance of the effects. It is possible to have statistically
significant results with small effect sizes, which may have little practical significance. Therefore, it is
important to consider both statistical significance and effect size when interpreting experimental findings.

Guidelines for Interpreting and Reporting the Findings

When interpreting and reporting experimental findings, researchers should follow some guidelines to
ensure clear and accurate communication of the results. These guidelines include (Bänziger, Mortillaro,
& Scherer, 2012):
First, Considering the research questions and hypotheses. The interpretation of the experimental
findings should be guided by the research questions and hypotheses. Researchers should examine the
data about these research questions and hypotheses and determine if the results support or refute them.
Second, use appropriate statistical techniques. Depending on the nature of the data and research
questions, different statistical techniques may be used for data analysis (Bänziger et al., 2012). For ex-
ample, if the data are continuous and normally distributed, parametric tests such as t-tests or analysis of
variance (ANOVA) may be used.
Third, look for patterns and trends. Researchers should examine the data to find patterns and trends
that can help explain the results. This may involve visualizing the data using graphs or charts and de-
scriptively analysing the data.
Fourth, consider effect sizes. In addition to statistical significance, researchers should also consider
effect sizes. The effect sizes provide information on the magnitude of the observed effects and can help
determine the practical significance of the findings.
Fifth, provide descriptive statistics. When reporting the findings, it is important to include descrip-
tive statistics that summarize the data. This can include measures such as means, standard deviations,
medians, and interquartile ranges. Descriptive statistics provide a snapshot of the data and help the reader
understand the central tendency and variability of the results.

222

Experimental Research

Sixth, interpret and discuss the findings concerning previous research. Researchers should interpret
and discuss the findings with previous research in the field. This involves comparing the results with
existing literature and discussing similarities or differences.
Seventh, consider limitations and potential confounders. Researchers should also discuss the limita-
tions of the study, including potential confounders that may have impacted the results. This can help
readers understand the scope and generalizability of the findings.
Finally, conclude and make recommendations. Finally, based on the interpretation of the findings,
researchers should conclude and make recommendations for future research or practice. This can involve
discussing the implications of the findings and suggesting areas where further investigation is needed.
By following these guidelines, researchers can ensure that their experimental findings are accurately
interpreted and effectively communicated to the scientific community.

CONCLUSION

In conclusion, this chapter has outlined the steps necessary to conduct experimental research, specifi-
cally in the field of applied linguistics. It begins with a general introduction to experimental research.
Then, it delves into the principles and key concepts essential for understanding and conducting any
experimental research. This chapter serves as a valuable resource for those who write and conduct
experimental research by providing comprehensive information and guidelines. Moreover, it offers an
additional reading list for individuals eager to contemplate deeper into specific aspects of experimental
research. With its thorough coverage and supplementary resources, this chapter equips researchers with
the knowledge and tools needed to undertake meaningful and rigorous experimental studies in applied
linguistics and other fields as well.

REFERENCES

Abbuhl, R., Gass, S., & Mackey, A. (2014). Experimental research design. In R. J. Podesva & D. Sharma
(Eds.), Research Methods in Linguistics (pp. 116–134). Cambridge University Press., doi:10.1017/
CBO9781139013734.008
Abbuhl, R., & Mackey, A. (2017). Second language acquisition research methods. In K. A. King, Y.-J.
Lai, & S. May (Eds.), Research methods in language and education (pp. 183–193). Springer International
Publishing., doi:10.1007/978-3-319-02249-9_13
Ary, D., Jacobs, L. C., & Sorensen, C. (2006). Introduction to research in education (8th ed.). Thomson
Wadsworth.
Association, A. P. (2019). Publication manual (OFFICIAL) 7th edition of the American psychological
association. APA. https://apastyle.apa.org/products/publication-manual-7th-edition-spiral
Bänziger, T., Mortillaro, M., & Scherer, K. R. (2012). Introducing the Geneva Multimodal expression
corpus for experimental research on emotion perception. Emotion (Washington, D.C.), 12(5), 1161–1179.
doi:10.1037/a0025827 PMID:22081890

223
 
Experimental Research

Bartoszynski, R., & Niewiadomska-Bugaj, M. (2007). Probability and statistical inference. John Wiley
& Sons.
Bryfonski, L., & Ma, X. (2020). Effects of implicit versus explicit corrective feedback on Mandarin tone
acquisition in a SCMC learning environment. Studies in Second Language Acquisition, 42(1), 61–88.
doi:10.1017/S0272263119000317
Bryman, A., Bell, E., Reck, J., & Fields, J. (2021). Social research methods. Oxford University Press.
Cash, P., Stanković, T., & Štorga, M. (2016). An introduction to experimental design research. In P. Cash,
T. Stanković, & M. Štorga (Eds.), Experimental design research: Approaches, perspectives, applications
(pp. 3–12). Springer Cham., doi:10.1007/978-3-319-33781-4_1
Chen, M., & Li, W. (2022). The influence of form-focused instruction on the L2 Chinese oral production
of Korean native speakers. Frontiers in Psychology, 13, 790424. Advance online publication. doi:10.3389/
fpsyg.2022.790424 PMID:35478761
Cook, T. D., & Shadish, W. R. (1994). Social experiments: Some developments over the past fifteen
years. Annual Review of Psychology, 45(1), 545–580. doi:10.1146/annurev.ps.45.020194.002553
Cook, T. D., & Wong, V. C. (2008). Better quasi-experimental practice. In P. Alasuutari, L. Bickman, &
J. Brannen (Eds.), The SAGE handbook of social research methods (pp. 134–165). SAGE Publications
Ltd., doi:10.4135/9781446212165.n10
Creswell, J. W., & Creswell, J. D. (2022). Research design: Qualitative, quantitative, and mixed methods
approaches. SAGE Publications, Inc.
Dechenaux, E., Kovenock, D., & Sheremeta, R. M. (2015). A survey of experimental research on
contests, all-pay auctions and tournaments. Experimental Economics, 18(4), 609–669. doi:10.1007/
s10683-014-9421-0
Drager, K. (2018). Experimental research methods in sociolinguistics. Bloomsbury Academic.
Escueta, M., Nickow, A. J., Oreopoulos, P., & Quan, V. (2020). Upgrading education with technology:
Insights from experimental research. Journal of Economic Literature, 58(4), 897–996. doi:10.1257/
jel.20191507
Fang, W.-C., Yeh, H.-C., Luo, B.-R., & Chen, N.-S. (2021). Effects of mobile-supported task-based
language teaching on EFL students’ linguistic achievement and conversational interaction. ReCALL,
33(1), 71–87. doi:10.1017/S0958344020000208
Field, A. (2017). Discovering statistics using IBM SPSS statistics. SAGE Publication.
Gass, S. (2015). Experimental research. In B. Paltridge & A. Phakiti (Eds.), Research methods in applied
linguistics _ A practical resource (pp. 88–99). Bloomsbury Academic.
Gonzalez-Marquez, M., Mittelberg, I., Coulson, S., & Spivey, M. J. (2003). An introduction to experi-
mental methods for language researchers. In M. Gonzalez-Marquez, I. Mittelberg, S. Coulson, & M.
J. Spivey (Eds.), Methods in cognitive linguistics (pp. 53–86). John Benjamins Publishing Company.

224

Experimental Research

Goo, J. (2012). Corrective feedback and working memory capacity in interaction-driven L2 learning.
Studies in Second Language Acquisition, 34(3), 445–474. doi:10.1017/S0272263112000149
Gray, C. D., & Kinnear, P. R. (2011). IBM SPSS statistics 19 made simple. Psychology Press.
Higgins, R., Murphy, F., & Hogg, P. (2021). The impact of teaching experimental research on-line:
Research-informed teaching and COVID-19. Radiography, 27(2), 539–545. doi:10.1016/j.radi.2020.11.014
PMID:33262051
Inna, H., Ihor, K., Olena, S., Oleksiy, T., & Natalia, T. (2021). Experimental research of oscillation param-
eters of vibrating-rotor crusher. Przegląd Elektrotechniczny, 3(3), 97–100. doi:10.15199/48.2021.03.19
Johnson, R. B., & Christensen, L. B. (2019). Educational research: Quantitative, qualitative, and mixed
approaches. SAGE Publications, Inc.
Kaiser, E. (2014). Experimental paradigms in psycholinguistics. In R. J. Podesva & D. Sharma
(Eds.), Research methods in linguistics (pp. 135–168). Cambridge University Press. doi:10.1017/
CBO9781139013734.009
Kirk, R. E. (2009). Experimental design. In R. E. Millsap & A. Maydeu-Olivares (Eds.), The SAGE handbook
of quantitative methods in psychology (pp. 23–45). SAGE Publications Ltd., doi:10.4135/9780857020994.
n2
Kirk, R. E. (2012). Experimental design-procedures for the behavioral Sciences. Sage Publication, Inc.
Ledyard, J. O. (1995). Public goods: A survey of experimental research. In J. H. Kagel & A. E. Roth
(Eds.), The handbook of experimental economics (pp. 94–111). Princeton University Press.
Leppink, J. (2019). Statistical methods for experimental research in education and psychology. Springer
Cham. doi:10.1007/978-3-030-21241-4
Li, S. (2018). Corrective feedback in L2 speech production. In The TESOL encyclopedia of English
language teaching (pp. 1-9). doi:10.1002/9781118784235.eelt0247
Li, X., Cheng, G., Liu, S., Xiao, Q., Ma, M., Jin, R., Che, T., Liu, Q., Wang, W., Qi, Y., Wen, J., Li, H.,
Zhu, G., Guo, J., Ran, Y., Wang, S., Zhu, Z., Zhou, J., Hu, X., & Xu, Z. (2013). Heihe watershed allied
telemetry experimental research (HiWATER): Scientific objectives and experimental design. Bulletin of
the American Meteorological Society, 94(8), 1145–1160. doi:10.1175/BAMS-D-12-00154.1
Loewen, S., & Plonsky, L. (2015). An A–Z of applied linguistics research methods. Bloomsbury Publishing.
Lyster, R., Saito, K., & Sato, M. (2013). Oral corrective feedback in second language classrooms. Lan-
guage Teaching, 46(1), 1–40. doi:10.1017/S0261444812000365
Mackey, A., & Gass, S. M. (2022). Second language research: Methodology and design (3rd ed.).
Routledge.
Mahmood, R. Q. (2023). Enhancing EFL speaking and pronunciation skills: Using explicit formal in-
struction in a Kurdish university. Issues in Educational Research, 33(4), 1421–1440. http://www.iier.
org.au/iier33/mahmood-abs.html

225
 
Experimental Research

Mattila, A. S., Luo, A., Xue, X., & Ye, T. (2021). How to avoid common mistakes in experimental re-
search? International Journal of Contemporary Hospitality Management, 33(1), 367–374. doi:10.1108/
IJCHM-07-2020-0696
McDonough, K. (2019). Experimental research methods. In S. Loewen & M. Sato (Eds.), The Routledge
handbook of instructed second language acquisition (pp. 562–576). Routledge.
Mellow, J. D. (2012). Time series. In C. A. Chapelle (Ed.), The encyclopedia of applied linguistics (pp.
1–5). Wiley-Blackwell., doi:10.1002/9781405198431.wbeal1216
Mills, G. E., & Gay, L. R. (2018). Educational research: Competencies for analysis and applications.
Pearson.
Neumark, D. (2018). Experimental research on labor market discrimination. Journal of Economic Lit-
erature, 56(3), 799–866. doi:10.1257/jel.20161309
Nieuwburg, E. G. I., Ploeger, A., & Kret, M. E. (2021). Emotion recognition in nonhuman primates:
How experimental research can contribute to a better understanding of underlying mechanisms. Neuro-
science and Biobehavioral Reviews, 123, 24–47. doi:10.1016/j.neubiorev.2020.11.029 PMID:33453306
Nunan, D. (1992). Research methods in language learning. Cambridge University Press.
Phakiti, A. (2015). Experimental Research Methods in Language Learning. Bloomsbury Academic.
Pickard, A. J. (2013). Research methods in information. Facet., doi:10.29085/9781783300235
Polio, C., & Lee, J. (2019). Experimental studies in L2 classrooms. In J. W. Schwieter & A.
Benati (Eds.), The Cambridge handbook of language learning (pp. 137–165)., https://doi.org/DOI,
doi:10.1017/9781108333603.007
Reichardt, C. S. (2009). Quasi-experimental design. In R. E. Millsap & A. Maydeu-Olivares (Eds.),
The SAGE handbook of quantitative methods in psychology (pp. 46–71). SAGE Publications Ltd.,
doi:10.4135/9780857020994.n3
Robinson, M. A. (2012). How design engineers spend their time: Job content and task satisfaction. Design
Studies, 33(4), 391–425. doi:10.1016/j.destud.2012.03.002
Rogers, J., & Révész, A. (2023). Experimental and quasi-experimental designs. In J. McKinley & H. Rose
(Eds.), The Routledge handbook of research methods in applied linguistics (pp. 133–143). Routledge.
Ross, S. M., & Morrison, G. R. (2013). Experimental Research Methods. In D. Jonassen & M. Driscoll
(Eds.), Handbook of research on educational communications and technology (pp. 1007–1029). Rout-
ledge., doi:10.4324/9781410609519
Saito, K. (2021). Effects of corrective feedback on second language pronunciation development. In H.
Nassaji & E. Kartchava (Eds.), The Cambridge handbook of corrective feedback in second language
learning and teaching (pp. 407–428). Cambridge University Press. doi:10.1017/9781108589789.020
Saito, K., & Lyster, R. (2012). Effects of form-focused instruction and corrective feedback on L2 pro-
nunciation development of /ɹ/ by Japanese learners of English. Language Learning, 62(2), 595–633.
doi:10.1111/j.1467-9922.2011.00639.x

226
 
Experimental Research

Sheskin, D. J. (2011). Handbook of parametric and nonparametric statistical procedures. CRC Press/
Taylor & Francis Group.
Toledo, A. H., Flikkema, R., & Toledo-Pereyra, L. H. (2011). Developing the research hypothesis.
Journal of Investigative Surgery, 24(5), 191–194. doi:10.3109/08941939.2011.609449 PMID:21867386
Toledo-Pereyra, L. H. (2011). Developing the research idea. Journal of Investigative Surgery, 24(3),
101–102. doi:10.3109/08941939.2011.572797 PMID:21524174
Yang, Y., & Lyster, R. (2010). Effects of form-focused practice and feedback on Chinese EFL learners’
acquisition of regular and irregular past tense forms. Studies in Second Language Acquisition, 32(2),
235–263. doi:10.1017/S0272263109990519

ADDITIONAL READING

Allen, M. (2017). The SAGE encyclopedia of communication research methods. SAGE Publications,
Inc. doi:10.4135/9781483381411
Bryman, A., Bell, E., Reck, J., & Fields, J. (2021). Social research methods. Oxford University Press.
Howitt, D., & Cramer, D. (2020). Research methods in psychology. Pearson.
Rose, H., McKinley, J., & Baffoe-Djan, J. B. (2019). Data collection research methods in applied lin-
guistics. Bloomsbury Academic. doi:10.5040/9781350025875
Wicaksono, R., Hall, C. J., & Smith, P. H. (2011). Mapping applied linguistics: Transforming data for
competitive advantage. Routledge.

KEY TERMS AND DEFINITIONS

Informed Consent: It is one of the main principles of ethical consideration, and through informed
consent, the participants must be informed and provided with adequate information about a research
study before voluntarily agreeing to participate.
Participants Confidentiality: When the data were collected from the participants, the researcher(s)
must ensure the privacy and anonymity of research participants by keeping their identity and personal
information.
Participants: When the participants are human beings, they are those whom the data will be collected
from, then, the data will be analysed and interpreted to answer the research questions.
Research Design: When researchers plan to use some certain research strategy to address the research
questions and the hypotheses is called research design.
Research Instrument: When the researcher(s) use some methods or tools to collect data in a research
study, such as surveys, questionnaires, interviews, tests, observation protocols, or physiological measures.
Variables: It is the object(s) of the study that can be measured to test hypotheses or answer research
questions.

227

View publication stats