Computers & Education 144 (2020) 103708

Contents lists available at ScienceDirect

Computers & Education

journal homepage: http://www.elsevier.com/locate/compedu

Effectiveness of flip teaching on engineering students’

performance in the physics lab
�mez-Tejedor a, *, Ana Vidaurre a, Isabel Tort-Ausina b, Jos�e Molina-Mateo a,
Jos�e A. Go
María-Antonia Serrano a, Jos�e M. Meseguer-Duen ~ as a, Rosa M. Martínez Sala b,
a a
Susana Quiles , Jaime Riera
a
Departamento de Física Aplicada, ETS de Ingeniería del Dise~
no, Universitat Polit�ecnica de Val�encia, Camino de Vera, s/n, 46022, Valencia, Spain
b
Departamento de Física Aplicada, ETS de Ingeniería de Edificaci�on, Universitat Polit�ecnica de Val�encia, Camino de Vera, s/n, 46022, Valencia,
Spain

A R T I C L E I N F O A B S T R A C T

Keywords: The progressive introduction of the flip teaching (FT) instructional model into higher education
Adult learning has accelerated in recent years. The FT methodology seems to be especially suitable for laboratory
Distributed learning environments practice sessions: before the lab session the students are given documents and videos that explain
Improving classroom teaching
the theoretical contents and the experimental procedure. When this material is studied in
Multimedia/hypermedia systems
Teaching/learning strategies
advance, the practice session can be devoted to the discussion, clarification and practical appli­
cation of the acquired knowledge.
This paper describes the effect of the FT methodology on the students’ academic performance
when it was applied to the laboratory practice in two subjects, Physics and Electricity, of a
technical degree. The laboratory and final grades of these subjects were compared in four
consecutive years. The characteristics of all four years were quite similar, except that the tradi­
tional teaching method (TM) was used in two, while FT was applied in the other two. The sta­
tistical analysis shows that the academic results of the students were better in both subjects under
FT than those obtained using TM, and that the difference was statistically significant.

1. Introduction

1.1. Flipped classroom model

Flip teaching (FT) has become increasingly popular in higher education since it was first described as an “inverted classroom” in
2000 (Lage, Platt, & Treglia, 2000). This method changes the conditions of where and when the students perform their learning tasks.
While in the traditional methodology (TM) new concepts are explained by the teacher in class and the student later reflects on the
content at home, in the FT model the students prepare the class content alone at home (watching videos or reading texts), while in class
they work together and discuss matters as a group (Garrison & Vaughan, 2013; Hung, 2015; Hung & Chou, 2015). Implementing the FT
model involves a change in the instructional design and affects strategies, tools and teaching methods to engage learners in

* Corresponding author.
E-mail addresses: jogomez@fis.upv.es (J.A. G�omez-Tejedor), vidaurre@fis.upv.es (A. Vidaurre), isatort@fis.upv.es (I. Tort-Ausina), jmmateo@fis.
upv.es (J. Molina-Mateo), mserranj@fis.upv.es (M.-A. Serrano), jmmesegu@fis.upv.es (J.M. Meseguer-Due~ nas), rmsala@fis.upv.es (R.M. Martínez
Sala), suquica@fis.upv.es (S. Quiles), jriera@fis.upv.es (J. Riera).

https://doi.org/10.1016/j.compedu.2019.103708
Received 23 April 2019; Received in revised form 19 September 2019; Accepted 23 September 2019
Available online 26 September 2019
0360-1315/© 2019 Elsevier Ltd. All rights reserved.
J.A. G�
omez-Tejedor et al. Computers & Education 144 (2020) 103708

self-directed learning outside the classroom before face-to-face meetings with teachers in the classroom (Teo, Tan, Yan, Teo, & Yeo,
2014). FT is based on active learning models (Bonwell & Eison, 1991; Meyers & Jones, 1993) and is a very general term that means
students actually think about what they are doing. It covers a wide range of learning activities, learning strategies and instructional
models. In the FT model, the responsibility for learning is transferred from the teacher to the student through participation in
interactive activities (Pierce & Fox, 2012). In the FT model the students are engaged in more complex activities while they can obtain
teachers feedback in a timely manner (Thai, De Wever, & Valcke, 2017). FT enables teachers to encourage critical thinking in their
students, building the capacity for lifelong learning and preparing future graduates for their work-place contexts (Akçayır & Akçayır,
2018; O’Flaherty & Phillips, 2015).
Many different types of flipped models have been developed, each having a different perspective. The flipped classroom thus has
many different names and approaches. Some authors consider that terms like “blended learning”, “reverse instruction”, “inverted
classroom”, and “24/7 classroom” are interchangeable (Bergmann & Sams, 2012), while others think that FT is a subset of blended
learning (Staker & Horn, 2012). In an attempt to identify the mains aspects of the FT schema, the Flipped Learning Network and
Pearson’s School Achievement Services define the four components that support student’s engagement in flipped learning: Flexible
Environment, Learning Culture, Intentional Content and Professional Educators (F-L-I-P™) (Hamdan, McKnight, McKnight, & Arf­
strom, 2013). It has recently been pointed out that these four components are not enough for higher education for a number of reasons,
including the fact that the F-L-I-P™ schema does not account for computer learning platforms (Chen, Wang, Kinshuk, & Chen, 2014).

1.2. Information and Communications Technology as a basis for flip teaching

Information and Communications Technology (ICT) provides us with the basic tools to create “blended learning” environments,
which support flipped classroom model development (Ardid, Go �mez-Tejedor, Meseguer-Duen ~ as, Riera, & Vidaurre, 2015). Computer
learning platforms offer a wide range of tools and play an important role in the flip classroom design. Universities have adopted
different platforms specifically designed for educational purposes, e.g. that by Moodle and Sakai, which are among the most popular.
Rienties et al. (2012) found no correlation between the use of information and communication technologies (ICT) and the teaching
approach used in the course design or cognitive content. Zacharia and Olympiou (2011), when comparing the effectiveness of virtual
labs versus manipulative experimentation, found that both conditions were equally effective in promoting students’ understanding of
concepts. In other words ICT has spread to all academic levels all over the world. FT takes advantage of the fact that computer learning
platforms can organise the course content into video, tasks and assesment (Ardid et al., 2015). Video recording is one of the most
commonly used tools in FT (Pierce & Fox, 2012). The videos proved to be more effective than textbook readings (Jensen, Holt,
Sowards, Heath Ogden, & West, 2018) and were preferred by the students (Sahin, Cavlazoglu, & Zeytuncu, 2015). Moreover, Nouri
(2016) found that this pre-class material was more positively perceived by the low achievers’ group. Others technologies, like
augmented reality-based flipped learning or a remote laboratory system, have demonstrated to improve student’s performance as well
as student’s learning motivation (Chang & Hwang, 2018; Tho & Yeung, 2016).
FT is being implemented in a variety of fields, including: physics (Şengel, 2016), maths (Sun, Xie, & Anderman, 2018), chemistry
(Baepler, Walker, & Driessen, 2014), chemistry labs (Teo et al., 2014), engineering, social studies and humanities (Kim, Kim, Khera, &
Getman, 2014), and economics (Roach, 2014). Most of the papers evaluating student outcomes report improved academic perfor­
mance over the traditional methodology (TM) (Missildine, Fountain, Summers, & Gosselin, 2013; Pierce & Fox, 2012). Its application
to secondary school students was found to induce statistically significant growth in information literacy, competency and critical
thinking skills (Kong, 2014, 2015). Although some authors conclude that the flipped classroom does not bring higher learning gains or
better attitudes than other methods (Jensen, Kummer, & Godoy, 2015), learning gains are most likely the result of the active-learning
style.
In particular, and especially as related to the present paper, FT has been implemented in different physics courses. For instance,
Deslauriers, Schelew, and Wieman (2011) found evidence that flipping the classroom can produce significant learning gains by
comparing two sections of a large physics class. Şengel (2016) compared the effectiveness of the flipped-classroom approach coupled
with problem-based learning and cooperative learning, compared to that of a traditional classroom. The results of this study showed
that effectiveness was significantly better in the flipped classroom. In reference (Aşıksoy & Ozdamlı,
€ 2016) it was found that an
experimental group of students following a flipped classroom model performed better than the students in a group following a
traditional lecture format. It can thus be seen that there is general agreement in the literature on the improvement in learning per­
formance in physics courses by applying the FT methodology. Moreover, Vo, Zhu, and Diep (2017) found a higher mean effect size in
STEM disciplines as compared to non-STEM disciplines. However, all these studies are centred on theory and problem teaching ac­
tivities, and there is a lack of studies on the implementation of this methodology in the physics laboratory, which was one of this
paper’s aims. In this document, we examine the effectiveness of the use of ICT, delivering recorded videos explaining the experiment,
that students should prepare before class. This allows more time in the laboratory session to be devoted to the analysis and discussion
of the obtained results.

1.3. The research goals and questions

In this paper we compare the lab practice results of students in two academic years who followed a TM with those of another group
who followed an FT model in the subjects of Physics and Electricity for the Industrial and Automated Electronics Degree at the UPV in
the period 2013–2017. The two first academic years (2013–2014 and 2014–2015) followed a TM method, while in the other two
(2015–2016 and 2016–2017) the FT method was used. In order to obtain more detailed information on the effect of using the FT

2
J.A. G�
omez-Tejedor et al. Computers & Education 144 (2020) 103708

method, we matched the students of both methods by defining the analogous student. The students were divided into two groups
according to their academic results and the normalized gain of analogous students was calculated.
Our hypothesis is that, as we eliminated the teacher’s explanation at the beginning of the lab session, if the students can spend more
time thinking about what they are doing, asking questions and discussing things with their classmates, they will obtain more detailed
knowledge and produce better results. This will also affect the student’s motivation and improve their academic results.
In brief, the research questions of this study are:

� Are there any significant differences in learning performance between the FT ICT-based and TM methodology? How does this affect
to the different ways of measuring learning performance?
� Do these results affect all students equally, or it depends on their academic level?

2. Material and methods

2.1. Flip teaching model

This paper describes the methodological changes introduced by FT in the laboratory and analyses their effects on the academic
performance of the students in the laboratory. For this, the grades obtained by the students in the years when the FT method was used
are compared to the grades obtained in previous years using a TM model. Design plays an important role in both TM and FT and
sequencing and progression should be given careful consideration (Jovanovic, Mirriahi, Ga�sevi�c, Dawson, & Pardo, 2019). In general,
it is advisable that the lab-work be directly related (in content and timing) to the rest of the learning activities planned for the subject.
This was a key criterion when planning the lab-sessions in TM and later in FT. Neither lab-sessions not teachers were changed when we
introduced the FT model.
The lab sessions were organized in the 4 years as follows: before the session the necessary learning material was made available to
students through the Universitat Polit�ecnica de Val�encia’s online teaching platform, PoliformaT (“PoliformaT,” 2003), adapted from
Sakai (“Sakai Learning Management System,” 2018). In this platform students and teachers can share information about their courses
and use management tools such as contents resources, assignments, tests & quizzes, etc. In our case, the learning materials consisted of
documents explaining the purpose and the procedure of the experience in both cases, and also a video in the FT methodology. The
videos were screencasts with the explanation of the work that students must do in the lab session, and the equipment they had to
handle. An example of the video introducing the free fall practice can be seen at reference (Molina-Mateo, 2017).
Lab sessions, no matter the methodology used, were based on teamwork. Under continuous teacher supervision, the students
worked in groups of six. At the end of each 2-h session the groups uploaded a spreadsheet to the Assignments Section of the PoliformaT
platform, containing the data acquired and the analytical model used. Each group then had ten days to upload a report. Fig. 1 shows a
scheme with the main differences between the TM (left) and FT (right) methodologies, based on the teacher explaining the theoretical

Fig. 1. Outline of the differences between TM (left) and FT (right) methodologies. The boxes in the centre represent points common to both
methodologies.

3
J.A. G�
omez-Tejedor et al. Computers & Education 144 (2020) 103708

contents and lab procedure at the beginning of the TM session, while in the FT ones the students had studied the material in advance
and the lab session started directly with the experimental work.
There is some discussion in the literature about the convenience of gradually introducing the FT model (Mehring & Leis, 2018;
Tomas, Evans, Doyle, & Skamp, 2019). In our case we opted to do this first in the laboratory sessions. We realised that long expla­
nations before the experimental work were a waste of time. FT was introduced into the lab sessions from the first day of class and the
teacher’s explanations were reduced to answering any doubts about out-of-class learning (reading texts and watching videos). The
learning design was based on the “first principles of instruction” (Merrill, 2002) following the scheme of Lo, Lie, and Hew (2018): the
videos, as part of the out-of-class learning component, contributes to the activation phase. During the lab sessions the students start
measuring under the teacher’s supervision and are encouraged to ask questions to be sure they have understood the task (demon­
stration phase). Later, still in class, they organise, discuss and defend their results (integration phase).

2.2. Participants

The sample was formed by 1233 students enrolled from 2013 to 2017 who completed the subjects of Physics and Electricity. The
previous grade that gave access to the university has fluctuated between 6 and 7 (out of 10) during all these years. This means that the
students’ initial level was practically the same in all the years of the study for the TM and FT participants.
TM was applied in the first two academic years and FT in the other two. The sample characteristics are summarized in Table 1, also
the method used each year and course, the number of enrolled students and the number of students that completed the course.
The lab report was used to grade the group’s lab work (total ten points), which was part (20%) of the final course grade. All the
teachers used the same criteria for the years and subjects involved in the study. These criteria (see the rubric at Appendix A, Sup­
plementary data) were also available to the students on PoliformaT and could be used as a guide to prepare their reports. Moreover,
each teacher was lecturing and assessing the same courses in the first years, with TM, and in the latest ones, with FT, that makes the
evaluation consistent between the four years analysed.
To analyse the improvement in the students’ knowledge from one method to the other, the student sample was segmented ac­
cording to the final grades obtained.

2.3. Data analysis

After anonymization, grades were obtained from the courses in which the authors taught these subjects during the academic years
studied. The final grades of the different courses were obtained from the published grades records and the lab grades of each course
were collected from the documentation available to the lecturers who taught these subjects.
The laboratory grades were analysed on SPSS software on Windows Version 16 (IBM, Somers, NY, USA) (raw data can be found at
the research data section). In the first step, a student t-test was applied to the two consecutive years (with the same method) of each
course.
Secondly, the possible differences in the average grade of the laboratory between the four groups (FT and TM in both years) were
then analysed by a parametric one-way ANOVA test. Its applicability had previously been verified by a Levene test, which showed the
homogeneity of the variances of the laboratory grades. The sample was segmented into two groups (high and low performance levels)
according to the median of the total grade for the course (final grade total of 10 points). A two-way ANOVA test then analysed the
interaction between the effect of the method and the students’ level.

2.4. Measure of improvement

2.4.1. Cohen’s d and average normalized gain

In order to assess the improvement obtained by the FT methodology, students’ learning can be measured by Cohen’s d that is
commonly used in the broader field of educational research. Cohen’s d is defined as the difference between the average post-test minus
pre-test divided by the pooled standard deviation (sp) of the pre- and post-tests (Cohen, 1988):

Table 1
Number of total enrolled students and number of students that completed the course, academic year and method.
Course Academic year Method Number of students enrolled Number of students following the course

Physics 2013–2014 TM 193 159

2014–2015 TM 172 146
2015–2016 FT 176 151
2016–2017 FT 166 148
Electricity 2013–2014 TM 221 163
2014–2015 TM 194 152
2015–2016 FT 191 159
2016–2017 FT 185 155

TOTAL 1498 1233

4
J.A. G�
omez-Tejedor et al. Computers & Education 144 (2020) 103708

FT TM
d¼ (1)
sp

where TM and FT stand for the mean grades (out of 10) for the TM and FT lab grades, respectively.
The most popular metric used in physical education research is the normalized gain (g) developed by Hake (1998) and defined as
the ratio of the actual average gain to the maximum possible average gain:

FT TM
g¼ (2)
10 TM
The normalized gain g is accepted as a good index to measure the knowledge acquired by the students. Many studies have used this
method of measuring students’ learning in science and engineering fields (Bao, 2006; Coletta & Phillips, 2005; Marx & Cummings,
2007; Nissen, Talbot, Nasim Thompson, & Van Dusen, 2018; Willoughby & Metz, 2009). Nissen, Talbot, Nasim Thompson, and Van
Dusen (2018) compared the gain and effect size metric and concluded that g is biased in favour of high pre-test populations and puts
groups with lower results at a disadvantage when instructors use g to measure instructional efficacy.
Another commonly used method is to calculate the normalized gain for each student, gi, and then average the normalized gains to
calculate g for the group (Bao, 2006; Von Korff et al., 2016). Using individual gains provides more detailed information than the
average value. For instance, it makes possible to find any significant differences between two groups or define the confidence intervals.
The group can also be divided into subgroups of students in order to analyse the dependence on different parameters, such as grade
level, gender, or others (Kohlmyer et al., 2009; Şengel, 2016). Bao (2006) claims that comparing g and the average of gi, g, can indicate
any changes in a group of students, and also that when g is lower than g, students with low pre-test scores tend to have larger score
improvements than those with high pre-test scores, while the opposite happens when g is higher than g.

2.4.2. The analogous students normalized gain

In our study, as the same students did not follow the two methodologies, we could not calculate the normalized gain for each
student. To overcome this difficulty and benefit from a more robust statistical development, we introduced “analogous students”
concept between the FT and TM groups (the one that occupies the same order by overall grade) following a two-step process (Fig. 2):

1. First, we divide the groups into two levels, low and high, according to the total course grade.
2. To identify the analogous students, the students were ranked according to their lab grades in each of the two subgroups (high or low
grades) for each course. Two students were considered analogous when they were in the same position in the ranking, as can be
seen in Fig. 2.

The introduction of the concept of “analogous students” allowed us to associate confidence intervals with the results of the gain and
perform a more robust statistical study.
In cases where the number of students was not the same in the subgroups of the two methodologies, some students were excluded
randomly from the subgroup with most members. Table 2 shows the size of the student sample finally used in this analysis.
Then, for each course, the individual normalized gain of “analogous” students for the FT methodology was calculated according to
Bao (2006):
8 FT TM
> i i
>
< 10 TM ðFTi > TMi Þ
(3)
i
gi ¼
>
: FTi TMi ðFTi < TMi Þ
>
TMi

Fig. 2. Procedure to define the “analogous students” according to the average laboratory grade.

5
J.A. G�
omez-Tejedor et al. Computers & Education 144 (2020) 103708

Table 2
Size of students sample used to calculate analogous students normalized gain.
Low grade High grade Total

PHYS 150 149 299

ELEC 157 157 314
Total 307 306 613

where FTi and TMi are the laboratory grades (out of 10) of the corresponding FT and MT analogous students.
From this we established the values of the gain by levels, with their respective confidence interval. The results obtained by this
method are compatible with those obtained by traditional methods.
So, the improvement due to FT was defined as an average individual normalized gain g. The results were compared with the more
commonly used Cohen’s d and the normalized gain g.

3. Results and discussion

3.1. Application of the FT model

Although some studies have indicated that students were reluctant to accept the FT methodology (Chen et al., 2014; Şengel, 2016),
we did not notice any difficulty in adopting this new approach. Although we could not trace how resources had been used in the FT
courses studied (at that time there was no tool available on the PoliformaT platform), there was evidence for the use of the resources.
We found that after the first lab session, the students realised that to be able to carry out the practice correctly they needed to read the
pdf documents and watch the videos in advance. In general, the students following the FT model had a more active attitude during the
class, asking questions or arguing while defending their point of view.

3.2. Statistical analysis of the impact of the methodology

The evolution of the average laboratory grades and the final grade of both courses is shown in Table 3. An increase in the average
laboratory grade is observed when the new methodology is introduced. A student t-test was performed on the students that followed
the complete course and showed no significant differences between the two TM and FT years, when performed independently for
Physics (abbreviated as PHYSPhys) and Electricity (abbreviated as ELEC), as can be seen in Table 3.
In order to join students from different academic years using the same learning methodology, we compared the first- and second-
year groups and found no significant differences. This fact allowed us to joint these two groups in order to increase the total number of
students included in each study group. This means that the sample can be divided into four groups, one for each course and method.
The data of the newly defined groups are shown in Table 4.
We then tried to find significant differences between the groups. As Levene’s test showed the homogeneity of variances of the
laboratory grades of the 4 groups (Levene’s statistic was 2.46), a parametric ANOVA test was applied and revealed significant dif­
ferences between the groups (F(3, 1229) ¼ 7.57 and p < 0.001).
To delve into these differences, orthogonal contrasts were performed. The null hypotheses tested were:

1) The Physics laboratory grade of using TM is equal to the grade using FT (PHYS TM ¼PHYS FT).
2) The Electricity laboratory grade of using TM is equal to the grade using FT (ELEC TM ¼ ELEC FT).
3) The laboratory grade of both courses together using TM is equal to the grade using FT (Grade TM ¼ Grade FT).

The obtained results for these hypotheses made the contribution of the methodology evident:

- There were significant differences in the course of Physics (PHYS FT > PHYS TM; t ¼ 2.44; p < 0.05).
- There were significant differences in the course of Electricity (ELEC FT >ELEC TM; t ¼ 2.94; p < 0.01).
- There were significant differences between both methodologies (FT > TM; t ¼ 3.81; p < 0.001).

Table 3
Average final grade and its standard deviation, average laboratory grade and its standard deviation, t-value and p-value.
Average final grade Standard deviation Average laboratory grade Standard deviation t p-value

PHYS TM (2013/14) 7.17 1.31 7.06 1.31 1.36 0.18

PHYS TM (2014/15) 6.92 1.53 7.32 1.92
PHYS FT (2015/16) 7.20 1.28 7.50 1.58 0.18 0.86
PHYS FT (2016/17) 7.44 1.19 7.47 1.21
ELEC TM (2013/14) 5.51 1.46 6.79 1.49 1.28 0.20
ELEC TM (2014/15) 5.69 1.74 7.02 1.67
ELEC FT (2015/16) 6.13 1.46 7.17 1.53 1.07 0.29
ELEC FT (2016/17) 6.27 1.29 7.36 1.50

6
J.A. G�
omez-Tejedor et al. Computers & Education 144 (2020) 103708

Table 4
Number of students (N), average final grade, its standard deviation and median, average laboratory grade and its standard deviation.
N Average final grade Standard deviation Median final grade Average laboratory grade Standard deviation

PHYS TM 305 7.05 1.42 7.10 7.18 1.63

PHYS FT 299 7.32 1.24 7.30 7.49 1.41
ELEC TM 315 5.60 1.60 5.60 6.90 1.59
ELEC FT 314 6.20 1.38 6.00 7.27 1.51
Total 1233 6.53 1.57 7.21 1.55

This analysis showed the positive impact of the FT methodology on the academic results in both courses as the FT results are
significantly better than those of TM.
This latter result is in line with previous studies in different fields, as for instance in nursing education (Missildine et al., 2013) and
pharmaceutical students (Pierce & Fox, 2012). The same results have been found in the physics field (Aşıksoy & Ozdamlı, € 2016;
Deslauriers et al., 2011; Şengel, 2016), although unlike these studies, our work is focused on the application of this methodology in the
physics lab.

3.3. Interaction effect between the methodology and the subgroups segmented by level

The next step was to determine whether this effect was homogeneous for students with different academic results. For this the
students were classified according to their final grades, using the median shown in Table 3. The sample was divided into eight groups
according to the subject (Physics or Electricity), the methodology (TM or FT), and their final grade. Table 5 shows the characteristics of
theses eight groups, including the average lab grade.
The improvement of the laboratory grades (see Table 5) with FT is also shown in Fig. 3, for both subjects. It can be observed that the
TM results are always lower than those of FT for the same group. It can also be seen that the groups with the highest final grades also
obtained the highest grade in the laboratory, with no interaction between the variables.
To check whether the differences observed were statistically significant, a two-way ANOVA analysis was performed (see Table 6).
The average Physics laboratory grade, that was higher than the grade in Electricity (see Table 3), shows significant differences between
the low- and high-grade subgroups, and also between both methods, while the interaction effect between both variables has no sig­
nificant influence (F(1, 603) ¼ 3.55, p ¼ 0.06).
Similar results were found for Electricity: the grade subgroup and method have a significant influence on the average laboratory
grade, while the interaction between them has no significance (F(1, 628) ¼ 0.50, p ¼ 0.48).
The results obtained allow us to conclude that the method has a clear influence on the students’ laboratory grades. In addition, the
laboratory grade depends on the student’s subgroup grade (high or low). However, there is no interaction between the two factors
(method and subgroup grade).

3.4. Estimation of the improvement of the academic performance by groups of students according to their grade

3.4.1. Cohen’s d and average normalized gain

The results of the Cohen’s d calculations are shown in Table 7. The values obtained confirm that FT method improved the students’
lab performance and therefore in the course, with a medium effect of between 0.3 and 0.5, in the low-grade Physics group (0.321) and
in the high-grade Electricity group (0.394), with a lesser effect on the other two groups.
Table 7 shows the result obtained for g. It is common to categorize the data obtained in three zones in the following way: low, if it is
less than 0.3; medium, if it is between 0.3 and 0.7 and high if it is greater than 0.7 (Hake, 1998). These values also show that although
the net gain is low, all the students are positively affected by the FT method.
In accordance with the Cohens’ d results, the highest average normalized gain is found in the low-grade group for Physics (14.7%),
and in the high-grade group for Electricity (17.6%).

3.4.2. Average of the analogous students normalized gain

The results of gi calculated according to Eq. (3) are shown in Fig. 4. The gi values are mostly between 0 and 0.2, except for the
highest values of the TM scores. In this region, the results are scattered and are different for Physics and Electricity. In the latter subject
the gain of the group with the highest TM scores is higher than that of the lowest TM scores. In Physics the gain does not show a clear

Table 5
Number of students (N), average laboratory grade (ALG) of the eight groups of students.
PHYS TM PHYS FT ELEC TM ELEC FT Total

N ALG N ALG N ALG N ALG N

Low grade 153 6.46 150 6.98 158 6.31 157 6.59 618
High grade 152 7.91 149 8.00 157 7.50 157 7.94 615
Total 305 299 315 314 1233

7
J.A. G�
omez-Tejedor et al. Computers & Education 144 (2020) 103708

Fig. 3. Average laboratory grade for the two methodologies in the two levels, in Physics and Electricity.

Table 6
Results of the ANOVA analysis, where SG stands for the subgroup grade, M stands for methodology, SS stands for sum of squares, DF stands for the
degrees of freedom.
Source SS DF F p

Physics SG 230.91 1 119.07 <0.001

M 14.81 1 7.26 0.007
(SG)*(M) 6.88 1 3.55 0.060
Electricity SG 254.01 1 126.90 <0.001
M 20.29 1 10.12 0.002
(SG)*(M) 1.01 1 0.50 0.480

Table 7
Average and statistical parameter of the Cohen’s d and normalized gain g for each grade
level and course.
Course-Grade level Cohen’s d g

PHYS Low 0.321 0.147

PHYS High 0.081 0.043
ELEC Low 0.144 0.076
ELEC High 0.394 0.176

dependence on the TM scores.

Next, we calculated the average of the individual normalized gain, g for each grade subgroup with its confidence interval. The
results are shown in Table 8, for both subjects. They show that there are significant differences between the low- and high-level groups;
with a higher gain for the low-grade group than for the high-level group. However, in Electricity the high-level group obtains a greater
gain. These results are in complete agreement with those previously obtained for the size effect, as measured by Cohen’s d or the
normalized gain g (Table 7).
These results are in agreement with those previously obtained: the highest gain is obtained in the low-grade group in Physics (17%),
and in the high-grade group in Electricity (20%), which confirms that although the gain is low the students are positively affected by
the FT method.
One of the possible explanations is that the videos used to show the experimental procedure, together with the longer time spent in
discussion and experimental work in the lab, significantly improved all the levels and subjects, especially in the lowest level of physics
students.
On the other hand, while the Electricity laboratory has a higher difficulty and the Physics lab has a lower level of difficulty, physics
performs like a normal subject, with the results already described in the bibliography, in which the lowest-level group has the highest
gain, since the high-level saturates (Nouri, 2016). As the high level does not saturate in Electricity, both levels improve when the
methodology is changed, because the higher-level already has scope for improvement, which does not happen in the physics group.

4. Conclusions

This paper studies the influence of introducing the flipped-classroom model into ICT-based laboratory sessions in the subjects of
Physics and Electricity in the Industrial and Automated Electronics Degree course at the Universitat Polit�ecnica de Val�encia. Data,

8
J.A. G�
omez-Tejedor et al. Computers & Education 144 (2020) 103708

Fig. 4. Individual normalized gain for FT methodology for both courses vs the laboratory grade in the TM.

Table 8
Average normalized gain (g), standard deviation (SD), Std. Error Mean, Confidence interval (CI), student t-test p-values.
Course-Grade level Average normalized gain (g) SD Std. Error Mean CI student t-test p-value

PHYS Low 0.17 0.12 0.010 (0.15, 0.19) <0.001

PHYS High 0.06 0.12 0.005 (0.04, 0.07)

ELEC Low 0.09 0.11 0.009 (0.07, 0.11) <0.001

ELEC High 0.20 0.12 0.009 (0.18, 0.22)

covering the period 2013–2017 following the TM method in the two first years (2013–2014 and 2014–2015) and the FT method in the
other two (2015–2016 and 2016–2017), has been analysed.
The main changes brought about by the FT methodology were that videos were added to the materials available to the students
before the lab session, and that the teachers’ explanation in the lab was replaced by the students’ preparation at home (using the
available material). The students therefore had more time in the lab session to organise, discuss and defend their results.

4.1. Educational implications

The results show that there are no significant differences between the academic results of consecutive academic years in a subject
when the same teaching method is used. However, when the sample is divided into four groups according to the course and the
teaching method, there are statistically significant differences between the academic results, as higher laboratory grades were always
achieved by the FT method. The Cohens’d and normalized gain results show that all the students are positively affected by the FT
method. In the FT methodology, the time devoted to teacher instruction was moved to start working; leaving extra time for more
productive activities (demonstration and activation phases). Therefore, FT methodology application to laboratory practices has
resulted in a significant improvement in academic results.
Once the influence of the methodology had been studied, the next step was to analyse whether this effect was homogeneous for
students with different academic results. After the segmentation analysis of each course, we concluded that, regardless of the statistical
method used, the FT method improves the academic results for both levels. The gain is higher for the high-level Electricity group and in
the low-level Physics group. Our study has thus shown that academic improvement does not affect all the students in the same way, and
it affects to a greater extent the students of the lower level in Physics and students of high level in Electricity. We should point out that
we have introduced an innovative method of finding analogous students to obtain the average individual normalized gain. By doing
this we have obtained more detailed information on the gain than from the normalized gain alone. We propose this methodology to
study similar cases in which methodological changes on the same students cannot be experimented.
As a general conclusion, FT works very well for lab sessions in technical degree courses, confirming the initial hypothesis of the
teachers involved. Students work better with the FT method in the laboratory and get better academic results. It also highlights the role
of the teacher in enhancing students’ autonomous learning. For a progressive application of FT, we recommend starting the experience
with the laboratory practices.

4.2. Limitations and suggestions for future research

Our study was applied to the subjects of Physics and Electricity for the Industrial and Automated Electronics Degree and indicates
that the improvement does not affect both subjects in the same way. The reason for this discrepancy can be interpreted as the dif­
ferences in the difficulty level of both subjects. In the more difficult subject of Electricity, the high level does not saturate, so that both
levels improve when changing the methodology, since the higher-level already has scope for improvement, which does not happen
with the Physics group. This shows up the study’s limitations and encourages us to continue research on this point. However, caution

9
J.A. G�
omez-Tejedor et al. Computers & Education 144 (2020) 103708

should be used in extrapolating the results to non-STEM disciplines.

The good overall results obtained by the FT indirectly show the effectiveness of using computers in education and encourages
research into new ICT-based tools to support the flipped classroom model in an autonomous learning environment.

Funding

This work was supported by the Universitat Polit�

ecnica de Val�encia [Project PIME/2018/B25 Convocatoria de Proyectos de
Innovacio
�n y Convergencia de la UPV].

Acknowledgements

We would like to thank the Instituto de Ciencias de la Educacio

�n (ICE) in the Universitat Polit�ecnica de Val�
encia for their help,
through the Innovation and Educational Quality Program and for supporting the team Innovacio �n en Metodologías Activas para el
Aprendizaje de la Física (e-MACAFI).

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.compedu.2019.103708.

References

Akçayır, G., & Akçayır, M. (2018). The flipped classroom: A review of its advantages and challenges. Computers & Education, 126(January), 334–345. https://doi.org/
10.1016/j.compedu.2018.07.021.
Ardid, M., G� omez-Tejedor, J. A., Meseguer-Due~ nas, J. M., Riera, J., & Vidaurre, A. (2015). Online exams for blended assessment. Study of different application
methodologies. Computers & Education, 81, 296–303. https://doi.org/10.1016/j.compedu.2014.10.010.
Aşıksoy, G., & Ozdamlı,
€ F. (2016). Flipped classroom adapted to the ARCS model of motivation and applied to a physics course. Eurasia Journal of Mathematics, Science
and Technology Education, 12(6), 1589–1603. https://doi.org/10.12973/eurasia.2016.1251a.
Baepler, P., Walker, J. D., & Driessen, M. (2014). It’s not about seat time: Blending, flipping, and efficiency in active learning classrooms. Computers & Education, 78,
227–236. https://doi.org/10.1016/j.compedu.2014.06.006.
Bao, L. (2006). Theoretical comparisons of average normalized gain calculations. American Journal of Physics, 74(10), 917–922. https://doi.org/10.1119/1.2213632.
Bergmann, J., & Sams, A. (2012). Flip your classroom : Reach every student in every class every day. Retrieved from https://www.iste.org/resources/product?ID¼2285.
Bonwell, C. C., & Eison, J. A. (1991). Active learning: Creating excitement in the classroom. In ASHE-ERIC higher education reports. Washington, D.C.: The George
Washington University, School of Education and Human Development.
Chang, S.-C., & Hwang, G.-J. (2018). Impacts of an augmented reality-based flipped learning guiding approach on students’ scientific project performance and
perceptions. Computers & Education, 125, 226–239. https://doi.org/10.1016/j.compedu.2018.06.007.
Chen, Y., Wang, Y., Kinshuk, & Chen, N.-S. (2014). Is FLIP enough? Or should we use the FLIPPED model instead? Computers & Education, 79, 16–27. https://doi.org/
10.1016/j.compedu.2014.07.004.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (Second). New York: Lawrence Erlbaum Associates.
Coletta, V. P., & Phillips, J. A. (2005). Interpreting FCI scores: Normalized gain, preinstruction scores, and scientific reasoning ability. American Journal of Physics, 73
(12), 1172–1182. https://doi.org/10.1119/1.2117109.
Deslauriers, L., Schelew, E., & Wieman, C. (2011). Improved learning in a large-enrollment physics class. Science, 332(6031), 862–864. https://doi.org/10.1126/
science.1201783.
Garrison, D. R., & Vaughan, N. D. (2013). Institutional change and leadership associated with blended learning innovation: Two case studies. The Internet and Higher
Education, 18, 24–28. https://doi.org/10.1016/j.iheduc.2012.09.001.
Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses.
American Journal of Physics, 66(1), 64–74. https://doi.org/10.1119/1.18809.
Hamdan, N., McKnight, P., McKnight, K., & Arfstrom, K. M. (2013). The flipped learning model: A white paper based on the literature review titled a review of flipped
learning. Retrieved from https://flippedlearning.org/wp-content/uploads/2016/07/WhitePaper_FlippedLearning.pdf.
Hung, H.-T. (2015). Flipping the classroom for English language learners to foster active learning. Computer Assisted Language Learning, 28(1), 81–96. https://doi.org/
10.1080/09588221.2014.967701.
Hung, M.-L., & Chou, C. (2015). Students’ perceptions of instructors’ roles in blended and online learning environments: A comparative study. Computers & Education,
81, 315–325. https://doi.org/10.1016/j.compedu.2014.10.022.
Jensen, J. L., Holt, E. A., Sowards, J. B., Heath Ogden, T., & West, R. E. (2018). Investigating strategies for pre-class content learning in a flipped classroom. Journal of
Science Education and Technology, 27(6), 523–535. https://doi.org/10.1007/s10956-018-9740-6.
Jensen, J. L., Kummer, T. A., & Godoy, P. D. d. M. (2015). Improvements from a flipped classroom may simply Be the fruits of active learning. CBE-Life Sciences
Education, 14(1). ar5 https://doi.org/10.1187/cbe.14-08-0129.
Jovanovic, J., Mirriahi, N., Ga�sevi�c, D., Dawson, S., & Pardo, A. (2019). Predictive power of regularity of pre-class activities in a flipped classroom. Computers and
Education, 134, 156–168. https://doi.org/10.1016/j.compedu.2019.02.011.
Kim, M. K., Kim, S. M., Khera, O., & Getman, J. (2014). The experience of three flipped classrooms in an urban university: An exploration of design principles. The
Internet and Higher Education, 22, 37–50. https://doi.org/10.1016/j.iheduc.2014.04.003.
Kohlmyer, M. A., Caballero, M. D., Catrambone, R., Chabay, R. W., Ding, L., Haugan, M. P., et al. (2009). Tale of two curricula: The performance of 2000 students in
introductory electromagnetism. Physical Review Special Topics - Physics Education Research, 5(2), 020105. https://doi.org/10.1103/PhysRevSTPER.5.020105.
Kong, S. C. (2014). Developing information literacy and critical thinking skills through domain knowledge learning in digital classrooms: An experience of practicing
flipped classroom strategy. Computers & Education, 78, 160–173. https://doi.org/10.1016/j.compedu.2014.05.009.
Kong, S. C. (2015). An experience of a three-year study on the development of critical thinking skills in flipped secondary classrooms with pedagogical and
technological support. Computers & Education, 89, 16–31. https://doi.org/10.1016/j.compedu.2015.08.017.
Lage, M. J., Platt, G. J., & Treglia, M. (2000). Inverting the classroom: A gateway to creating an inclusive learning environment. The Journal of Economic Education, 31
(1), 30–43. https://doi.org/10.1080/00220480009596759.
Lo, C. K., Lie, C. W., & Hew, K. F. (2018). Applying “First Principles of Instruction” as a design theory of the flipped classroom: Findings from a collective study of four
secondary school subjects. Computers and Education, 118, 150–165. https://doi.org/10.1016/j.compedu.2017.12.003.

10
J.A. G�
omez-Tejedor et al. Computers & Education 144 (2020) 103708

Marx, J. D., & Cummings, K. (2007). Normalized change. American Journal of Physics, 75(1), 87–91. https://doi.org/10.1119/1.2372468.
Mehring, J., & Leis, A. (2018). In J. Mehring, & A. Leis (Eds.), Innovations in Flipping the language classroom. https://doi.org/10.1007/978-981-10-6968-0.
Merrill, M. D. (2002). First principles of instruction. Educational Technology Research & Development, 50(3), 43–59. https://doi.org/10.1007/bf02505024.
Meyers, C., & Jones, T. B. (1993). Promoting active learning : Strategies for the college classroom. San Francisco, CA: Jossey-Bass.
Missildine, K., Fountain, R., Summers, L., & Gosselin, K. (2013). Flipping the classroom to improve student performance and satisfaction. Journal of Nursing Education,
52(10), 597–599. https://doi.org/10.3928/01484834-20130919-03.
Molina-Mateo, J. (2017). Free fall. Retrieved March 8, 2019, from media.upv.es website: https://media.upv.es/#/portal/video/47f49560-b3ea-11e7-a147-
658397fa8edf.
Nissen, J. M., Talbot, R. M., Nasim Thompson, A., & Van Dusen, B. (2018). Comparison of normalized gain and Cohen’s d for analyzing gains on concept inventories.
Physical Review Physics Education Research, 14(1), 010115. https://doi.org/10.1103/PhysRevPhysEducRes.14.010115.
Nouri, J. (2016). The flipped classroom: For active, effective and increased learning – especially for low achievers. International Journal of Educational Technology in
Higher Education, 13(1), 33. https://doi.org/10.1186/s41239-016-0032-z.
O’Flaherty, J., & Phillips, C. (2015). The use of flipped classrooms in higher education: A scoping review. The Internet and Higher Education, 25, 85–95. https://doi.org/
10.1016/j.iheduc.2015.02.002.
Pierce, R., & Fox, J. (2012). Vodcasts and active-learning exercises in a “flipped classroom” model of a renal pharmacotherapy module. American Journal of
Pharmaceutical Education, 76(10), 196. https://doi.org/10.5688/ajpe7610196.
PoliformaT. (2003). Retrieved from https://poliformat.upv.es/.
Rienties, B., Kaper, W., Struyven, K., Tempelaar, D., van Gastel, L., Vrancken, S., et al. (2012). A review of the role of information communication technology and
course design in transitional education practices. Interactive Learning Environments, 20(6), 563–581. https://doi.org/10.1080/10494820.2010.542757.
Roach, T. (2014). Student perceptions toward flipped learning: New methods to increase interaction and active learning in economics. International Review of
Economics Education, 17, 74–84. https://doi.org/10.1016/j.iree.2014.08.003.
Sahin, A., Cavlazoglu, B., & Zeytuncu, Y. E. (2015). Flipping a college calculus course: A case study. Educational Technology & Society, 18(3), 142–152. Retrieved from
https://www.j-ets.net/ETS/journals/18_3/11.pdf.
Sakai Learning Management System. (2018). Retrieved from https://www.sakailms.org/.
Şengel, E. (2016). To FLIP or not to FLIP: Comparative case study in higher education in Turkey. Computers in Human Behavior, 64, 547–555. https://doi.org/10.1016/
j.chb.2016.07.034.
Staker, H., & Horn, M. B. (2012). Classifying K-12 blended learning. Innosight Institute. Retrieved from Innosight Institute website: https://www.christenseninstitute.
org/wp-content/uploads/2013/04/Classifying-K-12-blended-learning.pdf.
Sun, Z., Xie, K., & Anderman, L. H. (2018). The role of self-regulated learning in students’ success in flipped undergraduate math courses. The Internet and Higher
Education, 36, 41–53. https://doi.org/10.1016/j.iheduc.2017.09.003.
Teo, T. W., Tan, K. C. D., Yan, Y. K., Teo, Y. C., & Yeo, L. W. (2014). How flip teaching supports undergraduate chemistry laboratory learning. Chemistry Education:
Research and Practice, 15, 550–567. https://doi.org/10.1039/C4RP00003J.
Thai, N. T. T., De Wever, B., & Valcke, M. (2017). The impact of a flipped classroom design on learning performance in higher education: Looking for the best “blend”
of lectures and guiding questions with feedback. Computers and Education, 107, 113–126. https://doi.org/10.1016/j.compedu.2017.01.003.
Tho, S. W., & Yeung, Y. Y. (2016). Technology-enhanced science learning through remote laboratory: System design and pilot implementation in tertiary education.
Australasian Journal of Educational Technology, 32(3), 96–111. https://doi.org/10.14742/ajet.2203.
Tomas, L., Evans, N., (Snowy), Doyle, T., & Skamp, K. (2019). Are first year students ready for a flipped classroom? A case for a flipped learning continuum.
International Journal of Educational Technology in Higher Education, 16, 5. https://doi.org/10.1186/s41239-019-0135-4.
Von Korff, J., Archibeque, B., Gomez, K. A., Heckendorf, T., McKagan, S. B., Sayre, E. C., et al. (2016). Secondary analysis of teaching methods in introductory physics:
A 50 k-student study. American Journal of Physics, 84(12), 969–974. https://doi.org/10.1119/1.4964354.
Vo, H. M., Zhu, C., & Diep, N. A. (2017). The effect of blended learning on student performance at course-level in higher education: A meta-analysis. Studies In
Educational Evaluation, 53, 17–28. https://doi.org/10.1016/j.stueduc.2017.01.002.
Willoughby, S. D., & Metz, A. (2009). Exploring gender differences with different gain calculations in astronomy and biology. American Journal of Physics, 77(7),
651–657. https://doi.org/10.1119/1.3133087.
Zacharia, Z. C., & Olympiou, G. (2011). Physical versus virtual manipulative experimentation in physics learning. Learning and Instruction, 21(3), 317–331. https://
doi.org/10.1016/j.learninstruc.2010.03.001.

11