The Malaysian Online Journal of Educational Technology Volume 2, Issue 1

Evaluating the Impact of Technology [1] College of Business,

Universiti Utara Malaysia
Integration in Teaching and Learning [2] folaola@gmail.com
Nafisat Afolake Adedokun-Shittu [1] Abdul Jaleel Kehinde Shittu[2] College of Arts and Science,
Universiti Utara Malaysia²

ABSTRACT

This article reports the impacts of technology integration on teaching and learning
from a study that examines the impact of ICT deployment in teaching and learning
at a University in Nigeria. The survey data were drawn from 593 respondents
(students and lecturers) and the survey instrument employed for both the students
and the lecturers is a 64-item questionnaire that was each subdivided into seven
parts. The study finds many ICT impacts in teaching and learning such as: learning
aid and resourcefulness, comfort with ICT, psychoanalytical and psychosocial aid,
task enabler, interdependence with ICT and learning collaboration. It also
identifies factors that need to be recognized in ICT impact study settings such as:
perception, integration, motivation and challenges.

Impact, technology integration, teaching and learning,

Keywords:
evaluation

INTRODUCTION

This topic is concerned with how the impact of teaching and learning with the various forms of technology can
be evaluated to ensure that the goals of technology integration in education are achieved. This should not be confused
with evaluating the various forms of technology used in teaching and learning. Greenberg (2004) in Adedokun-Shittu
and Shittu (2011) proposed that instead of comparing the effectiveness of varying technologies, efforts should be geared
toward determining the optimal combinations of all; that would best produce excellent learning outcomes for a
particular audience. He further lamented that most researchers fail to control for essential factors such as prior student
knowledge, pedagogical methods techniques, and teachers’ and students’ ability. Thus, this article is focused on
discussing some issues critical to evaluating the impact of technology integration in teaching and learning in order to
respond to this gap identified by Greenberg. The factors researchers must consider while evaluating ICT impact include:
the learning environment, the status of ICT integration in the learning environment, the students’ and teachers’
disposition toward technology, access to technology and training facilities and many others. Several authors (Adedokun-
Shittu & Shittu, 2011; Ecclestone, 2008) have reported that researchers often neglect most important factors such as
students’ and teachers’ technology proficiencies, pedagogical techniques and the peculiarities of learning environment
in impact studies. To exemplify how impact evaluation can be carried out to reflect these factors, this article reports a
study conducted by Adedokun-Shittu (2012) on ICT impact on teaching and learning in higher education. The study finds
many ICT impacts in teaching and learning such as: learning aid and resourcefulness, comfort with ICT, psychoanalytical
and psychosocial aid, task enabler, interdependence with ICT and learning collaboration. It also identifies factors that
need to be recognized in ICT impact study settings such as: perception, integration, motivation and challenges.

The Impact Study Outline

The study addressed in this article studied the impact of ICTs on teaching and learning in a higher institution of
learning in Nigeria. Nigeria being a developing nation recognizes the relevance of ICTs in national development and
particularly in education. Hence the deployment of ICT in Nigerian education generally and specifically in higher
institutions has received considerable attention. The formulation of the National Policy on Computer Education in 1988
contained information on application of computers at various levels of the country’s education with issues related to
basic objectives, hardware and software requirements (Federal Republic of Nigeria, FRN, 1988, cited in Yusuf, 2005). In

23 www.mojet.net
 The Malaysian Online Journal of Educational Technology Volume 2, Issue 1

line with the recommendations of the National policy on IT, the National Universities Commission (NUC), the
government agency responsible for registering and regulating universities has prescribed a minimum level of PC
ownership for universities as follows: one to every four students, one PC to every two lecturers below the grade of
Lecturer 1, one PC to each Senior Lecturer, and one notebook to a Professor/Reader. Many universities in Nigeria have
thus achieved a significant ratio than the prescribed and some have taken steps ahead in building campus-wide area
networking and developing e-learning course deliveries. Even though ICT facilities have been deployed in many learning
institutions, few or no impact studies have been conducted to determine ICTs effect on teaching and learning outcomes.
This is due to the fact that technology integration in higher education in Nigeria is still at a preliminary stage (Adegun,
2007). As such, this study looks into the impacts derived from employing ICTs in teaching and learning in a Nigerian
University through a quantitative means.
The survey instrument employed for both the students and the lecturers is a 64-item questionnaire that was
each subdivided into seven parts. The first part (demographic variable) is a 14 item-question which contains choice and
fill in questions. The next two parts covering perception (21 items) and integration (12 items) are based on the five-
point Likert scale. Motivation (6 items) and constraints (4 items) are rated on a preference scale (1 - least preferred to
5 – most preferred). The problems (3 items) part of the instrument is rated on a frequency scale (1 – Never and 5 -
Always). Constraints and problems were re-coded as challenges. The last part renamed as ICT rate (1 item) carries a
value scale (1 – of no value to 5 – extremely valuable). These four scales are run through linear regression using rate of
ICT value to teaching and learning (ICT rate) as the dependent variable. Both the students’ and lecturers’ survey
instrument display a high reliability (alpha coefficients of .900 and .909) respectively. Linear regression was conducted
to see the interaction of the four scales of items (perception, integration, motivation and challenges) with respect to
the value of ICT in teaching and learning (ICTrate). All the independent factors significantly and positively interact except
integration that has a negative value (-.112). Perception has the highest beta value of .243, while they all have R value
of = .417, and p < .000. The negative relationship revealed by students and lecturers ICT integration in teaching and
learning is a function of the fact that some factors such as access hinder their integration. Thus, ICT impact study cannot
be conducted without taking some factors such as access, constraints, problems, and technical issues into consideration.
The impacts reported in this study (learning aid and resourcefulness, comfort with ICT, psychoanalytical and
psychosocial aid, task enabler, inter-dependence with ICT and learning collaboration) were extracted through factor
analysis.
This study is however limited by the quantitative method it employed because rich and more in-depth findings
could be gathered if supported by qualitative approach. Likewise evaluation studies are better conducted employing
mixed method approach because of its comprehensiveness and in order to achieve valid and well-substantiated
conclusions (Cresswell, 2009; Stufflebeam, Harold, & McKee, 2003). Similarly, studies (Centre for Global Development,
2006; Independent Evaluation Group, 2006; World Bank, 2004) strongly recommend mixed method for impact study
given the lack of credibility flaw identified against many existing impact studies that focus mainly on quantitative
method.
Impacts of Technology in Teaching and Learning
Education technology has been confirmed to have great potentials that impact on teaching and learning. It
motivates and engages students to learn and helps broaden their skills, helps to simulate the workplace experiences
thereby preparing students for the challenges of the labor market. This revolutionalizes the school environment,
facilitates teaching by providing resourceful teaching aids for teachers and connects the school to the outside world.
Trucano (2005) ascertained that technology empowers teachers and learners and promotes the growth of skills
necessary for the 21st century workplace. Wright, Stanford, and Beedle (2007) describe ICTs as giving opportunities for
students to explore, discover, create, communicate effectively and freely with instructors, complete and receive
assignments and feedback online, initiate and participate in online discussions.
Both lecturers and students in the study of discourse in this article agree on the significant impact ICT has on
students and their learning and on teaching and teaching styles. Among the impacts of ICT in teaching and learning
reported were; learning aid and resourcefulness, comfort with ICT, psychoanalytical and psychosocial aid, task enabler,
interdependence with ICT and learning collaboration.
Spector (2008) advocates how student collaboration is achieved through technology-mediated communication
such as e-mail and teleconferencing across space and time in local and wider communities. Kozma (Kozma, 2003; Kozma
& McGhee, 2003) illustrated a student learning approach in which students collaborate with their peers in given
projects. He named this approach the Student Collaborative Research Cluster. These classroom practices support the
development of skills needed by a society focused on sustained economic development and social transformation:
information management skills, communication and collaboration skills, interpersonal and self-directional skills, and
ability to create and innovatively apply new knowledge to solve complex problems. Similarly, King (2005) and Simonson,
Smaldino, Albright, and Zvacek (2003) ascertained that ICTs foster collaborative learning.

24 www.mojet.net
 The Malaysian Online Journal of Educational Technology Volume 2, Issue 1

Considering teachers’ professional development cannot be achieved in isolation, Kozma (2003) also exemplified
how teachers collaborated with students, colleagues in the school and others outside the school such that ideas on
solving classroom problems could be shared and disseminated. Collaboration among lecturers as a benefit of ICT use in
teaching is also found in Abolade and Yusuf (2005); they found that ICT allows for networking with other teachers, thus
connecting teachers and allowing them to exchange ideas, share resources, and improve teaching practices.
The psychoanalytical and psychosocial impacts described in both lecturers and students’ findings in this study
have a backing in the work of Lajbcyier and Spratt (2007). They argue that the social presence develop critical thinking
and cognitive skills and promote higher order learning in a community of learners. Some of these impacts of ICT in
teaching and learning such as interaction and social negotiation of meaning were also affirmed in Madden, Baptista
Nunes, McPherson, Ford, and Miller (2008).
Other impacts in the findings such as learning aid and resourcefulness, task enabler, comfort with ICT are
supported in Abolade and Yusuf (2005) having described ICT as essential tools in any educational system which has the
potential of being used to meet the learning needs of individual students, promote equality of educational
opportunities; offer high quality learning materials, increase self-efficacy and independence of learning among students,
and improve teachers’ professional development. They also affirmed that ICT provides opportunity for connecting
schools to the world, as learning is expanded beyond the classroom; that allows students and teachers to access
information and resources.

Issues in Evaluating Technology Impact in Teaching and Learning

Evaluating technology impact in teaching and learning requires a broad range of issues which are often
undermined when ICT impact researches are being carried out. Thus, the task in this part of the article is to carefully
consider some of them and make recommendations for further researches on technology impact in teaching and
learning. The impact of ICT in education is often difficult to establish especially when other factors that can affect
achievement are isolated.
Critical to evaluating ICT impact in teaching and learning are issues such as: the environment in which teaching
and learning will take place, the status of technology integration in the learning environment, the students’ and
teachers’ technology proficiencies, their disposition towards technology, access to technology and training facilities,
teachers’ teaching methodology, and students’ learning approach. Researchers must focus on each of these issues to
determine if the adoption of technology in teaching and learning produces the desired and maximum impact or
otherwise what factors hinder realization of expected outcomes (Adedokun-Shittu & Shittu, 2011; Newby, Stepich,
Lehman, Russell, & Leftwich, 2011).
The study under discourse in this article finds that technology impact in teaching and learning can be generated
through examining teachers’ and students’ perception of technology use in teaching and learning, assessing their level
of technology integration, determining the motivating factors that propel them and ascertaining the challenges that
restrain them. This is supported by the suggestions given by Kankaanranta (2005) on technology evaluation in schools
such as justifying investment returns by examining if the desired impacts are achieved, assessing technology infusion in
the curriculum to analyze whether the intended curriculum is implemented and ultimately attained and determining if
the pedagogical uses of ICT emphasize how it is employed in the class by the teachers and how it is received by the
students.
Some of the perception of technology in teaching and learning held by the lecturers and students in this study
include; technology changes the nature of student/lecturer interaction, improves higher-order and critical thinking,
improves quality education, transforms the learning environment into a learner-centered one, increases students’
motivation and engagement, increases students’ positive effects on learning, enhances students’ assessment and
independent learning, reduces both students and lecturers’ burden, facilitates learning and enhances performance. It
is also seen as a tool for increased access to resourceful information, improved research output, resource sharing and
student/lecturer collaboration (Adedokun-Shittu, Shittu & Adeyemo, 2013; Jimoh, Shittu & Kawu, 2012).
Evaluating both lecturers’ and students’ perception in this context has thus confirmed that they believe
technology has positive effects on their teaching and learning. Researchers (e.g., Adedokun-Shittu et al., 2013; Kozma,
2005; McNamara, 2003; UNESCO, 2002) have also found ICT as a way to promote educational change, improve learners’
skills and prepare them for the global economy and the information society. However, these positive effects do not
magically occur without proper policy considerations on how to integrate technology in the learning environment. As
such, it is practically crucial to establish how technology is implemented in the learning institution, while conducting
technology impact studies (Adedokun-Shittu & Shittu, 2011).
As mentioned above, the second factor identified in this study is integration. Among the technology integration
aspects evaluated in this study are technology use in classroom teaching and learning, its alignment with the curriculum

25 www.mojet.net
 The Malaysian Online Journal of Educational Technology Volume 2, Issue 1

and students’ assessment and its blend with the existing traditional teaching and learning practices. It was found that
technology integration in teaching and learning across these dimensions has not fully materialized in this university. The
regression analysis employed reveals that integration has a negative correlation (-.112) with the value of technology in
this learning environment. This implies that the level of technology integration in the university is still at an
unprecedented level such that its value could not be attached to established technology impact in teaching and learning.
This is in support of Adegun’s (2007) finding that technology integration in Nigerian education is still at a rudimentary
stage. As a form of recommendation, Kozma (2005) suggests some policy considerations for ICT integration in education
that can help in generating the desired outcome. He recommends that the technology plan should describe how
technology will be coordinated with changes in curriculum, pedagogy, assessment, teacher professional development
and school restructuring.
Following the integration issue is the question on what incentives could motivate teachers and students to
effectively integrate technology to maximize its impact. The motivating factors examined in this study include: training
especially for teachers, adequate access to technology facilities, ease of use of the facilities and compatibility with
teaching and learning needs, relative advantage and usefulness of technology in teaching and learning. Both the
students and lecturers in this study possess an average ICT skill essential for their appreciation of ICT deployment in the
university and a judicious use of limited ICT facilities provided. They also acknowledge a substantial engagement with
ICT for academic purposes though with varying degree based on the faculty, department, lecturers’ teaching style and
course requirement. They have also received some forms of training though insufficient for the desired integration.
The World Bank (2003) demands that teachers need to be lifelong learners to keep abreast of new knowledge,
pedagogical ideas and technology relevant to successful implementation of educational reforms. Similarly, Haddad
(2003) affirms developmental training is required for teachers for successful technological reforms in schools. Thus,
training and mentoring on how to utilize and infuse technology in pedagogy and curriculum is essential for teachers.
With respect to this, Kozma and McGhee (2003) suggest that teachers should collaborate with students, colleagues in
the school and others outside the school on how classroom problems can be solved such that it will serve as mentoring
and motivation for them.
Access to the desired content through ICT facilities can also enhance the quality of students’ learning since this
will provide both the students and the teachers with access to curricular materials and other resources. Haddad (2003)
notes that ICT is relevant in the teachers’ professional development; it is also a source of knowledge given that they
require a large, rich, and easily accessible-knowledge base. Voogt and Pelgrum (2005) and Ololube, Eke, Uzorka,
Ekpenyong, and Nte (2009) also assert that access to ICT can be used to improve the delivery and access to quality
education.
Robinson (2007) and Adedokun-Shittu et al. (2013) however declare that access to technology alone does not
guarantee integration and technology alone will not guarantee students’ learning. Thus, bearing this in mind Robinson
(2007) suggests that technology integration be understood as an integral component of a more comprehensive package
of education reform which will include curriculum, assessment, instruction and other practices within the context of the
entire school. He further asserted that integration is unlikely to happen where access is restricted to specific classrooms;
thus technology integration requires provision of ample access to all and addressing individual actual needs and
perceptual barriers.
This leads to the last factor vital to technology impact evaluation reported in this study. Investigating the
challenges confronting teachers and learners in their engagement with technology in teaching and learning is another
essential factor to evaluate in ICT impact studies. Software, accessibility and technical problems and constraints such as
power failure, internet interruption, and inadequate training were some of the challenges uncovered in this study. These
are all consistent with the description of Ajayi (2002) in Adedokun-Shittu (2012) stating that the status of ICT use in
higher institutions in Nigeria is characterized by poor availability and quality of infrastructure, inadequate institutional
capacity, inadequate human resource capacity, low bandwidth of connectivity, and poor penetration of ICT into higher
institutions. Abolade and Yusuf (2005) also supported this by highlighting as part of the factors militating against
effective ICT integration in Nigerian higher education to include lack of technically experienced lecturers, inadequate
course content for ICT training, limited ICT facilities and infrastructure and electricity supply problems. Wright et al.
(2007) also identified access and connection speed as challenges confronting higher education with the increasing
demand for ICTs in teaching and learning. Other challenges as given by Jimoh et al. (2012) are: lack of access to
computers and software, insufficient time to plan instruction, inadequate technical and administrative support.

26 www.mojet.net
 The Malaysian Online Journal of Educational Technology Volume 2, Issue 1

CONCLUSION

This article identified some of the issues fundamental to evaluating impacts of technology in teaching and
learning such as; teachers and students’ views, technology needs, and proficiencies, technology integration levels in
learning institutions, incentives such as adequate access and training and the imminent challenges of technology in
teaching and learning. It is also recommended that studies evaluating technology impact should employ a combination
of quantitative and qualitative approaches such that each can compensate for the weakness of the other, thereby given
a detailed and credible result. Some of the impacts of ICTs in teaching and learning reported in this article are: ICTs as
learning aid and resourcefulness, ICTs as psychoanalytical and psychosocial aid, ICTs as task enabler, comfort with ICT,
interdependence with ICT and learning collaboration.

REFERENCES

Abolade, A.O. & Yusuf, M. O. (2005). Information and communication technology and Nigerian teacher
education. African Journal of Educational Studies, 3(10), 19-23.

Adedokun-Shittu, N. A. (2012). The deployment of ICT facilities in teaching and learning in higher
education: A mixed method study of its impact on lecturers and students at the University of Ilorin, Nigeria.
Ph.D. thesis, International Islamic University Malaysia.

Adedokun-Shittu, N. A., Shittu, A. J. K., & Adeyemo, A. A. (2013). Impact factors of Information and
Communication Technologies. International Conference on Computing, E-Learning and Emerging Technology
(ICCEET2013) Sydney, Australia. October, 30 – 31, 2013. Retrieved from http://www.icceet.com

Adedokun-Shittu, N. A & Shittu, A. J. K. (2011). Critical issues in evaluating Education Technology. In

M. S. Al-Mutairi & L. A. Mohammad (Eds.), Cases on ICT utilization, practice and solutions: Tools for
managing day- to- day issues (pp. 47–58). IGI Global.

Adegun, O. A. (2007). Managing e-learning to achieve education for all in Nigeria. In Proceeding 12th
Cambridge International Conference on Open and Distance Learning, London.

Ajayi, G. O. (2002). Bridging the digital divide: The Nigerian case study. In Proceedings Developing
Country Access on Online Scientific Publishing Sustainable Alternatives. Trieste, Italy. 4-5 October, 2002.

Centre for Global Development. (CGD). (2006). When will we ever learn? Improving lives through
impact evaluation. Washington DC: Center for Global Development.

Cornford, J., & Pollock, N. (2003). Putting the university online: Information, technology and
organisational change. Buckingham: SRHE.

Creswell, J. W. (2009). Research design qualitative, quantitative and mixed methods approaches (3rd
ed.). Thousand Oaks, CA: Sage.

Ecclestone, K. (2008). The impact of assessment on pedagogy can have damaging consequences. In
Nash I., Jones S., Ecclestone K. & A. Brown (Eds.), Challenge and change in further education: A commentary
by the Teaching and Learning Research Programme. Retrieved from
http://www.tlrp.org/pub/research.html

Forcier, R. C. & Descy, D. E. (2008). The computer as an educational tool: Productivity and problem
solving (5th ed.). Englewood Cliffs, NJ: Pearson.

Greenberg, G. (2004, July/August). The digital convergence: extending the portfolio model’
[online], Educause Review, 39(4), pp. 28–36. Retrieved
from http://net.educause.edu/ir/library/pdf/ERM0441.pdf

27 www.mojet.net
 The Malaysian Online Journal of Educational Technology Volume 2, Issue 1

Haddad, W. D. (2003). Is instructional technology a must for learning? Retrieved from

http://www.techknowlogia.org/TKL_Articles/PDF/455.pdf

Independent Evaluation Group (IEG) (2006). Impact evaluation experience of the independent
evaluation group of the world bank. Washington DC: World Bank.

Jimoh, R. G., Shittu, A. J. K., & Kawu, Y. K. (2012). Students’ perception of computer based test
(CBT) for examining undergraduate chemistry courses. Journal of Emerging Trends in Computing and
Information Sciences, 3(2), 125-134.

Kankaanranta, M. (2005). Innovative Pedagogical Practices in Technology-Enhanced Education -

Finnish Perspective. Retrieved from http://e.finland.fi/netcomm/news/showarticle.asp?intNWSAID=41844

Kozma. R. B. (2005). National policies that connect ICT-based education reform to economic and
social development. Human Technology, 1(2), 117-156. Retrieved from
http://www.humantechnology.jyu.fi/articles/volume1/2005/kozma.pdf

Kozma, R., & McGhee, R. (2003). ICT and innovative classroom practices. In R. Kozma (Ed.),
Technology, innovation, and educational change: A global perspective. Eugene, OR: International Society
for Educational Technology.

Madden, A. D., Baptista Nunes, J. M., McPherson, M., Ford, N., & Miller, D. (2008). Mind the Gap!:
New 'Literacies" Create New Divides. In C. Van Slyke (Ed.), Information Communication Technologies:
Concepts, methodologies, tools, and applications (pp. 2297-2310). Hershey, PA: IGI Global.

Newby, T. J., Stepich, D., Lehman, J., Russell, J. W. & Leftwich, A.T. (2011) Educational Technology
for teaching and learning (4th ed.). Englewood Cliffs, NJ: Pearson.

Ololube, N. P., Eke, P., Uzorka, M. C., Ekpenyong, N. S., & Nte, D. N. (2009). Instructional technology
in higher education: A case of selected universities in the Niger Delta. Asia-Pacific Forum on Science
Learning and Teaching, 10(2).

Simonson, M., Smaldino, S., Albright, M., & Zvacek, S. (2003). Teaching and learning at a distance:
Foundations of distance education (2nd ed.) Upper Saddle River, NJ: Merrill Prentice Hall.

Spector, M. J. (2008). Handbook of research on educational communications and technology. Taylor

& Francis.

Trucano, M. (2005). Knowledge Maps: ICTs in Education. Washington, DC: infoDev. Retrieved
from http://www.infodev.org/ files/1062_file_ Knowledge Maps _ICTs_and_the_Education_MDGs.pdf

Voogt, J., & Pelgrum, H. (2005, October). ICT and curriculum change. An Interdisciplinary Journal on
Humans in ICT Environments, 1(2), 157-175.

World Bank. (2003). Infrastructure services: The building blocks of development. Washington, DC:
World Bank.

World Bank. (2004). Monitoring and evaluation: Some tools methods and approaches. Washington,
D.C. World Bank Group. http://www.worldbank.org/oed/ecd/.

Wright, V. H., Stanford, R., & Beedle, J. (2007). Using a blended model to improve delivery of teacher
education curriculum in global settings. In L. Tomei, Integrating Information and Communications
Technologies into the classroom (pp. 51-61). Hershey, PA: Information Science Publishing.

28 www.mojet.net
 The Malaysian Online Journal of Educational Technology Volume 2, Issue 1

Yusuf, M. O. (2005). Information and communication technology and education. Analysing the
Nigerian national policy for information technology. International Education Journal, 6(3), 316 – 321.
Retrieved from http://iej.cjb.net

Yusuf, M. O. (2010). Higher Educational Institutions and Institutional Information and Communication
Technology (ICT) Policy. In E. Adomi (Ed.), Handbook of research on Information Communication Technology
Policy: Trends, issues and advancements (pp. 243-254). IGI Global.

Yusuf, M. O., & Afolabi, A. O. (2010). Effects of computer assisted instruction (CAI) on secondary
school students’ performance in biology. The Turkish Online Journal of Educational Technology, 9(1), 62-
69. Retrieved from http://www.tojet.net/volumes/v9i1.pdf

29 www.mojet.net