Assignment Week 5 Sila Celebi
Assignment Week 5 Sila Celebi
Assignment Week 5 Sila Celebi
Loewen, S., Crowther, D., Isbell, D.R., Kim, K.M., Maloney, J., Miller, Z.F. & Rawal, H. (2019).
Mobile-assisted language learning: A Duolingo case study. ReCALL 31(3): 293–311.
https://doi.org/10.1017/S0958344019000065
This study was focused on the effectiveness of one specific language learning program, namely
Duolingo, for L2 learning.
INTRODUCTION
The authors initiated the study by explaining the increasing appliances of mobile technologies,
including smartphones and tablets, have impacted how individuals engage in the process of studying
and acquiring proficiency in a second language. It was emphasised that it is crucial to assess the
efficacy of this technology in facilitating second language (L2) learning.
The definition of MALL is briefly described as learners' utilisation of mobile technology to engage in
language study. Then the key components given in the study as below; (a) flexibility in time and
location of study; (b) continuity of study on different devices, such as mobile phones, tablets, and
laptop/desktop computers; (c) easy accessibility of information; and (d) adaptability to personal study
habits. Additionally, web services and apps that include built-in language learning materials and
activities are seen as the components of MALL.
Effectiveness of MALL
This study stated that the efficacy of MALL in second language development has been somewhat
limited, primarily because there is a lack of objective and quantifiable measures of learning outcomes
in studies related to MALL. However, it was stated that if suitable measures of learning are
implemented, Mobile-Assisted Language Learning (MALL) has demonstrated its capability to offer
learning benefits, particularly in the areas of reading, listening, and speaking. The study highlighted
the ongoing uncertainty regarding the effectiveness of commercial online language learning programs.
Despite research funding for programs such as Rosetta Stone (Vesselinov, 2009), Duolingo
(Vesselinov & Grego, 2012), and Babbel (Vesselinov & Grego, 2016) suggesting positive language
learning outcomes, in terms of the degree of effectiveness i.e being as effective as or even more
effective than traditional face-to-face foreign language courses there are contrasting opinions. The
limitedness of those kinds of study was supported by a previous study by Van Deusen-Scholl (2015)
who argued that despite strong claims of learner success, there is a scarcity of research available for
commercial products, and the validity of the reported outcomes is questionable.
Sıla ÇELEBİ Technologies in L2TL
Duolingo
The study under consideration investigates the efficacy of Duolingo in second language (L2) learning.
Duolingo, a free language education platform, is globally accessible via mobile devices, and its
features include interaction with the program and other learners. Despite claims on its website about
effectiveness, contrasting opinions emerge. Duolingo asserts its effectiveness by referencing
Vesselinov and Grego's (2012) research, which indicates that L1 English speakers with prior
knowledge of Spanish covered the content of a first college semester in Spanish with an average of 34
hours of Duolingo usage. On the other hand, Rachels and Rockinson-Szapkiw's (2018) quasi-
experimental study with elementary students showed no significant difference in performance between
face-to-face instruction and Duolingo. Falk and Götz (2016) found positive feedback on automated
features but limited use of interactions among learners. In response to Vesselinov and Grego's (2012)
findings, Krashen (2014) criticised Duolingo' in terms of its little reliance on the acquisition of
language competence or implicit L2 knowledge, just focusing on promoting conscious learning and
explicit L2 knowledge.
Due to the scarcity of empirical research on the efficacy of Duolingo and other Mobile-Assisted
Language Learning (MALL) apps for second language (L2) development, the current study aimed to
address the following research questions:
METHOD
In this mixed-methods study, the L2 development and experiences of nine individuals were
investigated.
Participants
Nine participants from Michigan State University took part in the study, comprising three females and
six males. Among them, eight were graduate students (two master's and six doctoral), and one was a
professor. The participants had diverse first language (L1) backgrounds, including English (six),
Chinese (one), Korean (one), and Nepali (one). All had prior experience with second language (L2)
learning, with some being bilingual (e.g., Korean/English, Chinese/English) or multilingual (e.g.,
Nepali/Hindi/English), and others having varying proficiency in additional languages. Notably, these
nine learners served in dual roles as both participants and researchers. Importantly, the research
explicitly notes that as it was conducted as a mandatory requirement for a class, the motivation for
Sıla ÇELEBİ Technologies in L2TL
studying Turkish with Duolingo was likely distinct from that of ordinary Duolingo users. Each
participant was conscious of the study's purpose, which was to evaluate the effectiveness of Duolingo
as a language learning application.
Materials
Duolingo;
Duolingo organizes learning objectives into skills, such as Basics, Phrases, and Adjectives 1,
each containing multiple lessons focusing on specific topics like question formation or food.
The platform follows a mostly linear progression in presenting these skills. However, its
technical features lack contextual meaning and communicative tasks, prioritizing lexical and
grammatical aspects. Duolingo employs both explicit methods like grammar translation
activities and implicit techniques like input enhancement to teach language. It adopts a
gamified approach to motivate learners, allowing them to set daily goals for earning
experience points (XP) by completing lessons. Meeting daily XP goals earns streaks,
indicating consecutive study days. Skills and streaks translate into Duolingo's digital currency,
called lingots, which users can use to access timed practice sessions and customize their
avatar, Duo, a cartoon owl mascot.
Participant journals;
Participants in the study kept journals to document their weekly reflections on using Duolingo.
They focused on (a) the time they spent on the app, (b) whether they used the mobile or PC
platform, (c) the levels they completed and their use of review, (d) any successes or
difficulties they encountered, and (e) connections between their Duolingo experiences and
ISLA (Instructed Second Language Acquisition) theory. These journals provided a
comprehensive record of participants' engagement, progress, and reflections on the language
learning process with Duolingo.
learners. The test was administered and graded by a Turkish university course instructor. With
a high internal reliability of α = .86, the exam comprised 10 sections encompassing various
language skills, including listening, reading, lexicogrammar, speaking, and writing.
Procedure;
The study comprised four phases as below;
1. Turkish Study and Journaling: Participants initiated Turkish language learning on
Duolingo in January 2016, committing to at least one hour of study per week for 12 weeks
as part of the course project. They were restricted to using only Duolingo resources, with
external assistance from resources like Turkish language textbooks or websites being
prohibited.
2. Language Assessment: After 12 weeks of Turkish study, all participants, regardless of
their total study time, underwent evaluation through both the Duolingo Progress Test and
the Turkish 151 Test in April 2016.
3. Additional Turkish Study: Participants who had not completed 34 hours of study and
chose to continue (n = 5) engaged in further Turkish study to meet the 34-hour target
4. Language Assessment (Second Round): In July 2016, the same five participants retook
the Duolingo Progress Test and the Turkish 151 Test.
The phases reveal that participants had varying degrees of completion in the Turkish
study, aligning with the naturalistic and self-directed learning nature of the study.
overall performance. Participants in the study achieved lower scores in the speaking (33%) and
listening (37%) sections, both involving real-time language processing. In contrast, higher scores were
observed in the reading (57%), writing (55%), and lexicogrammar (50%) sections, which emphasized
the application of explicit knowledge. The researcher interpreted these results in the context of
Duolingo's pedagogy, which relies heavily on grammar-translation and audiolingual-type activities,
common in Mobile-Assisted Language Learning (MALL). Despite the relatively short study time, the
Sıla ÇELEBİ Technologies in L2TL
gains made by participants were likely in explicit knowledge, even though Duolingo does not
extensively provide explicit metalinguistic rules. (that is competent with Krashen’s point on
acquisition part and creative usage of language)
An additional crucial finding was the moderate correlation observed between the amount of study time
and test scores (r = .58). However, the researcher acknowledged that this relationship could be
influenced by the small sample size and ceiling effects in the study time data. Importantly, the
researcher referenced Nielson's (2011) assertion that high rates of drop-off in user engagement when
using online technology may indicate that learners might not persist long enough to make substantial
gains in their second language (L2) knowledge, particularly in the absence of obligations or
encouragement from peers and teachers commonly present in traditional classroom environments. This
perspective highlights the potential impact of learner persistence and support structures on the
effectiveness of online language learning platforms.
LIMITATIONS
The study's small sample size limits its generalizability, and participants' prior successes in second
language (L2) learning and familiarity with Second Language Acquisition (SLA) theory might have
positively influenced their study's effectiveness, suggesting that typical Duolingo users may achieve
even less. The imposition of Turkish as the sole target language seemed to diminish motivation for
some learners; therefore, allowing learners to choose the target language might improve learning
gains.
My reflection;
Overall, I have some comments on this study. First of all, I agree with the fact that those types of apps
may be useful in the beginning of learning L2 because they lead you and give you a context that you
won’t be lost. It is easy practical and time-consuming. It gives you freedom and flexible working
hours. But the more you progress you have the less those apps become useful. After some degree of
improvement, you need to be exposed to the more authentic language of authentic users. The need for
interaction arises. The Krashen’s point was correct for these reasons. Also, it was both odd and
interesting to see a research whose participants were also the researchers. However, this led me to
some doubts about the credibility of the study. As they stated, they were more knowledgeable about
learning theories and the process of L2 learning. Thus, the results of other studies can be changed or
be completely different from theirs. I can’t decide if this type of research is a reliable way to do
research. I’m eager to hear everyone’s opinions about that part.
Sıla ÇELEBİ Technologies in L2TL
Chen, X. L., Zou, D., Xie, H. R., & Su, F. (2021). Twenty-five years of computer-assisted language
learning: A topic modelling analysis. Language Learning & Technology, 25(3), 151–185.
http://hdl.handle.net/10125/73454
This study is a review of the concept of CALL (computer-assisted language learning) in language
learning settings.
INTRODUCTION
The study began by defining Computer-Assisted Language Learning (CALL) as the exploration and
examination of computer applications in language education. Over time, due to the growth of diverse
information and communication technologies (ICTs) and the widespread use of digital devices both
inside and outside language classrooms, CALL was redefined as the development and utilization of
technology applications in language teaching and learning. The shift in definition reflects the evolving
landscape of technology in language education. This study uses an expansive definition of CALL,
encompassing any digital technology utilized in both formal and informal learning settings, whether
inside or outside language classrooms.
LITERATURE REVIEW
The researcher highlighted early studies that shared a similar approach. These reviews generally fall
into two main categories: those providing an overview of CALL development and technologies used
across the entire field, and those concentrating on specific types of technology, such as mobile-assisted
language learning (MALL), digital game-based language learning (DGBLL), and multimedia. Many
of those studies (Bax, 2003, Liu et al. 2002, Golonka et al. 2014, Chun 2016, Zhang and Zou 2020)
indicated that the effects of these technologies on language education were generally positive. They
were found to enhance educational practices and interactions, facilitate the delivery of instructional
content, and contribute to the restructuring of teaching methods.
Additionally, there are some reviews focused on MALL such as Sung et al. (2015) Hwang and Fu
(2019) and DGBLL related to digital games such as Hung et al. (2018) and Acquah and Katz (2020).
In addition to those studies, some reviews delved into specific technologies. Gamper and Knapp, for
instance, discussed advanced artificial intelligence (AI)-)-supported technologies, presenting potential
enhancements for CALL systems. Mohsen and Balakumar (2011) conducted a review of multimedia
glosses in CALL, while Parmaxi and Zaphiris (2016) explored the role of computer-mediated
communication (CMC) in CALL. Barrot (2018) reported on the effectiveness of Facebook in this
context, and Reinhardt (2019) synthesized 87 focal pieces on social media within the realm of CALL.
The researcher raised two main critiques of those reviews. Firstly, they pointed out a lack of attention
to tracing the developmental trends of CALL issues, which limits their utility in guiding future
research. Secondly, the majority of these reviews employed time-consuming systematic analysis and
Sıla ÇELEBİ Technologies in L2TL
meta-analysis methodologies, often focusing on a relatively small sample of articles. As a solution, the
utilization of bibliometrics was stated.
The researcher explained the term "Bibliometrics," which involves analyzing scientific output by
treating characteristics of literature as research objects. as discussed by Chen et al. (2020b).
Bibliometric analysis is utilized to compare contributions from different countries, institutions, and
publication sources. It offers methods to assess the impact and evolution of topics over time within a
specific field. Another approach for extensive literature review is topic modelling, capable of
uncovering hidden thematic structures in a corpus of text documents. This method identifies typical
topics and measures the extent to which each document relates to those topics (Chen et al., 2020c).
Social scientists have developed Structural Topic Modeling (STM) 1 to categorize terms based on the
probabilities of their co-occurrence across observations in a dataset.
The research questions (RQs) for our review of CALL articles from 1995 to 2019 are as follows.
RQ1: What was the annual frequency of CALL articles and citations?
RQ2: Who were the representative journals, countries/regions, and institutions for CALL
research?
RQ3: What were the most frequently investigated topics in CALL, and how did research
interests evolve?
RQ4: How did the identified research topics correlate?
RQ5: How were the identified research topics distributed across representative
countries/regions and institutions ranked by the Hirsch index (H-index)?
The researchers highlighted a departure from previous review methodologies, emphasizing the use of
more general terms for a comprehensive data scope. The evaluation of each article was based on four
inclusion criteria: (a) original research, (b) publication between 1995 and 2019, (c) relevance to the
application of computer-related tools in language learning, and (d) being written in English. This
process identified 2571 articles indexed by the Social Sciences Citation Index (SSCI) or Science
Citation Index (SCI). Initially, two articles lacking abstracts were excluded.
1
Structural Topic Modeling (STM) is a method used to find key themes or topics in a large set of documents.
Unlike simpler models, STM considers extra details like when the documents were created or who wrote them.
It helps researchers see how topics change over time and how other factors might influence what's being
talked about. In simpler terms, STM is a tool for understanding and organizing big sets of text, making it easier
to see patterns and trends in the information.
Sıla ÇELEBİ Technologies in L2TL
Following the initial screening process, an extra 1271 articles were excluded based on specific
predetermined exclusion criteria. These criteria included:
1. For teacher education (N=52): Articles focused on teacher education were excluded.
5. Not for language learning (N=219): Articles not related to language learning were excluded.
6. Reflection on previous research on CALL (N=5): Articles that were reflections on previous
research on Computer-Assisted Language Learning (CALL) were excluded.
7. Review paper (N=104): Review papers were excluded from the analysis.
These criteria were applied to refine the selection and ensure the inclusion of articles relevant to the
study's objectives.
Analysis
RQ1 was answered by counting the articles and citations published in each given year. Polynomial
regression2 analysis was employed to model and analyze the data to understand the trends in annual
article and citation counts.
For RQ2, the researchers utilized bibliometric indicators based on the publication source, author
address information, and citations of each article. Svensson's (2010) method for counting articles and
citations was applied to measure the productivity and influence of journals, countries/regions, and
institutions.
To address RQ3, the researchers applied Structural Topic Modeling (STM) and the M-K trend 3 test
(Mann, 1945). The STM methodology was carried out in three steps. Initially, terms were extracted
from titles and abstracts and underwent preprocessing, which involved the removal of numbers,
2
Polynomial regression is a way to model relationships between variables using curved lines instead of straight
lines. It's like fitting a curve to your data points, allowing for more flexibility in capturing non-linear patterns.
This helps when the relationship between variables isn't a straight line and requires a more complex curve to
better represent it.
3
The Mann-Kendall trend test is a method used to check for trends in data that is arranged in chronological
order, like time series data. It's handy when the data doesn't follow a specific pattern. The test helps determine if
there's a consistent upward or downward trend over time. It's a way to see if things are generally increasing or
decreasing without assuming anything specific about how the data should behave.
Sıla ÇELEBİ Technologies in L2TL
punctuation, and stop words. Following this, a term frequency-inverse document frequency model was
utilized for term selection. Subsequently, candidate models were executed with topic numbers
spanning from five to 30. The researchers determined that the 15-topic model demonstrated optimal
semantic consistency within topics and exclusivity between topics.
RQ4 was addressed using hierarchical clustering analysis, to investigate the correlation between
topics. This involved the examination of a document-level cosine similarity matrix.
To respond to RQ5, the researchers visualized the distribution of topics for prominent countries,
regions, and institutions, organized by their Hindex rankings4.
RESULTS
The annual trend curves for both article and citation counts exhibited similar growth patterns,
particularly between 2003 and 2017, despite fluctuations in their respective numbers.
In the analysis of 1295 articles from 254 SCI/SSCI-indexed journals, a noticeable trend emerged.
From 1995 to 2009, most journals had low article and citation counts, but a significant upswing
occurred from 2010 onwards. Language Learning & Technology (LLT) consistently held the top
position. At the same time, journals like ReCALL, Educational Technology & Society, System,
Australasian Journal of Educational Technology, and Interactive Learning Environments saw a notable
increase in rankings during 2010–2019 compared to the earlier period. The relationship between
article and citation counts revealed that, in many cases, a journal's citation count strongly influenced
its article count. Journals tended to have minor differences in article counts but substantial variations
in citation counts, indicating differences in academic impact. This implies that a higher article count
doesn't necessarily translate to a higher citation count. Regarding the geographical distribution,
scholars from the USA had the highest article and citation counts, followed by Taiwan and the UK.
Notably, most countries and regions experienced a substantial increase in both article and citation
counts in the most recent decade.
As an answer for RQ3 summarized list of the main findings from using STM:
1. Increasing Attention:
5. Topic Proportions:
MALL (11.99%) ranked first, followed by PBL, Blended Learning, CMC SCMC, and
DMC.
Virtual World and Virtual Reality (VW and VR) – an emerging topic in recent years.
Additionally, the analysis identified two main phases in research trends, spanning from 1995 to 2002
and from 2003 to 2019. During the first phase, topics such as Project-Based Learning (PBL),
Multimedia-Enhanced Learning (MEL) audiovisual resources, captions/subtitles, glosses/annotations,
vocabulary learning, and Digital Multimodal Composing (DMC) dominated but received less attention
after 2003. The second phase saw a shift in focus, with other topics consistently gaining more
attention. Notably, Mobile-Assisted Language Learning (MALL) remained the most researched topic
during the final decade from 2003 to 2019.
The hierarchical clustering analysis conducted for RQ4 revealed distinct topic clusters. Below there
are some examples given in the study:
2. Cluster 2: MEL Captions/Subtitles, Seamless Learning, and Digital Books show a correlation.
4. Cluster 4: Wiki-Based Learning, CMC and SCMC, and Feedback and Assessment are
correlated within the hierarchical clustering.
Sıla ÇELEBİ Technologies in L2TL
The findings from RQ5 highlight the research strengths of top-ranked countries/regions and
institutions based on the H-index. Specifically:
MEL Audiovisual Resources garnered attention from Canadian and Dutch researchers.
The USA and Taiwan's National Sun Yat-Sen University demonstrated notable interest in
Blended Learning.
Spain focused on Project-Based Learning (PBL), Taiwan on MALL, and Australia on Digital
Multimodal Composing (DMC).
DISCUSSION
The study employed a nonparametric trend test to identify topics that exhibited either increasing or
decreasing interest in Computer-Assisted Language Learning (CALL). This approach, distinct from
previous reviews that primarily focused on chronologically listing technology applications in language
learning, provided a more nuanced understanding. The findings offered valuable insights for shaping
future directions in CALL research.
In examining the evolution of technologies in language education across five distinct periods (1995–
1999, 2000–2004, 2005–2009, 2010–2014, and 2015–2019), the research aims to discern prevalent
topics in Computer-Assisted Language Learning (CALL) during each timeframe and analyze their
developmental trajectories over the past 25 years.
The subsequent period of 2005–2009 witnessed the rise of mobile technologies and Automatic
Speech Recognition (ASR), in addition to multimedia and CMC. This was reflected in studies
Sıla ÇELEBİ Technologies in L2TL
From 2010 to 2014, Digital Multimodal Composing (DMC), wikis, and mobile technologies
took centre stage in CALL, as indicated by the prevalence of studies on CMC and SCMC,
MALL, DMC, and wiki-based learning.
In the most recent period of 2015–2019, Virtual World (VW) and Virtual Reality (VR), along
with digital games, gained significance alongside DMC, mobile technologies, CMC, and
wikis. The prevalent topics included DMC, MALL, CMC and SCMC, wiki-based learning,
VW and VR, and Game-Based Learning (GBL).
The review points out that two current technologies, Artificial Intelligence (AI) and Learning
Analytics (LA), haven't been studied enough. Even though AI has advanced, especially in personalized
learning, and LA is contributing to precision education, there's a shortage of published studies in these
areas. This indicates the necessity for more research to explore the applications and impacts of AI and
learning analytics in education.
MY REFLECTION
First of all, this study was very comprehensive, thus I had difficulties in terms of summarizing. But it
was also useful in terms of filling the gap about technology-integrated ELT approaches. Even if it was
stated that the focus was on the CALL, the study gives plentiful information about other types of
technology-integrated approaches and techniques with related terminology. When it comes to
methodology, I had a chance to see new techniques for a literature review (as I added as a footnote).
However, I have a few questions about these techniques.