TECHNO INTERNATIONAL NEWTOWN

Newtown, Megacity, Kolkata -700156.

Department of Information Technology

Submission of B. Tech Final Year Project 1- IT792
Engagement Detection in E-learning

Prepared By: Anish kumar

Vikram kumar

Deep kumar Dutta

Abhishek Kumar

Under the Guidance of: Arindam Ghosh

For

Batch:-2020-24 Semester : 7th Year : 2023

Stream:- Information Technology Year of Study: 4th

Affiliated to

MAULANA ABUL KALAM AZAD UNIVERSITY OF TECHNOLOGY, WESTBENGAL

(FORMERLY KNOWN AS WEST BENGAL UNIVERSITY OF TECHNOLOGY)
 ACKNOWLEDGEMENT

We would like to express our sincere gratitude to ARINDAM GHOSH of the department of Information
Technology, whose role as project guide was invaluable for the project. We are extremely thankful for the
keen interest he took in advising us, for the books and reference materials provided for the moral support
extended to us.

Last but not the least we convey our gratitude to all the teachers for providing us the technical skill that
will always remain as our asset and to all non-teaching staff for the cordial support they offered.

Place: Techno India College of Technology, Newtown Megacity

Date: _ _ _ _ _ _ _ _ _ _

NAME ROLL NUMBER SIGNATURE

ANISH KUMAR 18700320008
ABHISHEK KUMAR 18700220019
DEEP KUMAR DUTTA 18700220035
VIKRAM KUMAR 18700220021

Department of Information Technology,

Techno International New Town, Newtown Megacity
Kolkata – 700 156
West Bengal, India.
 Approval

This is to certify that the project report entitled Engagement Detection in E-Learning prepared under my
supervision by (students name) Anish Kumar, Abhishek Kumar, Deep Kumar Dutta, Vikram Kumar
be accepted in partial fulfillment for the degree of Bachelor of Technology in Information Technology
which is affiliated to Maulana Abul Kalam Azad University of Technology, West Bengal (Formerly
known as West Bengal University of Technology).

It is to be understood that by this approval, the undersigned does not necessarily endorse or approve any
statement made, opinion expressed or conclusion drawn thereof, but approves the report only for the
purpose for which it has been submitted.

Project Mentor:

…………………………………………

External Examiner(if any):

……………………………………….

………………………………………….
HOD, Dept. of Information Technology
Techno International New Town
 ABSTRACT

Traditional methods of assessing student engagement, such as observing facial expressions,

are not feasible in online learning environments. To address this challenge, a face
expression-based engagement detection technique is proposed. This technique utilizes
computer vision and machine learning to analyze student facial expressions and track eye
movements. The resulting data is used to create an engagement detection model that can
accurately assess student participation in virtual classrooms.

Keywords: Emotions, Facial expressions, Online learning,

Convolutional Neural Network
CONTENTS

1. INTRODUCTION
2. PROBLEM DEFINITION
3. PROPOSED MODEL
3.1 Block Diagram
3.2 Hardware and Software Description
4. LITERATURE REVIEW
5. IMPLEMENTATION/FULL DESCRIPTION
6. FUTURE SCOPE
7. CONCLUSION
8. REFERENCE
INTRODUCTION

In the world of modern education, the shift from traditional classrooms to online learning
platforms has been swift and transformative. This shift brings with it a pressing need to
understand and gauge a crucial aspect: student engagement within these digital realms.
Engagement, the active involvement, interest, and commitment of students towards their
learning, stands as a cornerstone for successful educational outcomes, especially in the
evolving landscape of e-learning. Imagine learning on a computer or tablet instead of in a
classroom. That's what e-learning is all about! But making sure students are interested and
involved while learning online is really important. That's what we call "engagement" – it
means how much students are paying attention and enjoying their online lessons. This work
is all about figuring out how to know if students are engaged when they're learning online.
We'll look at different ways to understand and measure this engagement. It's like trying to
see if students are really interested in what they're learning or if they're just clicking around
without really paying attention. A learner's emotional condition has a significant impact on
the quality of their learning. First, with the use of multimodal sentiment analysis
techniques, learners' emotional states could be ascertained from their responses and
comments on various social media sites, such as Facebook, Instagram, Twitter, and other
sources. Then, in accordance with this work's concept, students who showed noninteractive
tendencies could receive support or be given recommendations for resources to help them
feel better emotionally. Emotions have a significant impact on how we learn as well.
Additionally, emotion has a significant impact on our capacity for memory, reasoning,
perception, and logical thought. Emotion is important in education since it has a strong
correlation with students' attentiveness. Researchers are always examining the various
kinds of emotions. Basic feelings like fear, rage, disgust, happiness, sadness, and surprise
are among them as well as intellectual feelings like content, bored, enjoy, sluggish,
depressed, excited, proud, tired, and furious. Students' engagements may be impacted by
their emotions. In order to determine the degree of student engagement based on their
feelings, emotion-based models therefore seek to create a relationship between academic
emotions and engagement levels. For improved educational outcomes, it is therefore
essential to comprehend students' emotions and link them to different levels of
participation.
PROBLEM DEFINITION

Detecting the level of engagement typically involves assessing the degree of interest or
interaction in a given context, such as user engagement with a website, application, or
content. In the realm of image processing, this might relate to analyzing visual cues to
determine the level of attention or interest from users. For instance, facial expressions, eye
tracking, or body language in images or videos can be considered as indicators of
engagement. Methods for detecting engagement may include computer vision techniques to
analyze facial expressions for signs of interest, eye gaze tracking to understand where
attention is focused, or even sentiment analysis to gauge emotional responses. Machine
learning models can be trained to recognize patterns associated with varying levels of
engagement. In essence, detecting the level of engagement through image processing
involves extracting meaningful information from visual data to understand how actively
and positively users are interacting with a given system, content, or environment. This is
valuable in applications ranging from user experience design to content optimization,
where tailoring experiences to user engagement levels can enhance overall effectiveness.
The problem at the heart of E-learning engagement detection extends beyond mere
measurement of participation; it encompasses the nuanced understanding of how students
interact, absorb, and contribute to the digital learning environment. This challenge requires
a holistic exploration of the factors influencing engagement, the limitations imposed by the
digital medium, and the development of strategies to bridge the gap between traditional and
online learning experiences. Digital distractions, the lack of physical presence, diverse
learning styles, and technological barriers are key factors contributing to the complexity of
engagement detection in E-learning. Distractions from social media, the absence of face-to-
face interactions, varying learning preferences, and unequal access to technology pose
challenges that must be addressed for effective engagement measurement.
PROPOSED MODEL

Detection of ROI

In the process of
detecting a Region of
Interest (ROI) in image
processing or computer
vision, the system
identifies and isolates a
specific area within an
image for further
analysis. This can be
achieved through either
manual selection or automated methods. Manual selection involves users specifying the
region through tools or software, while automated approaches use algorithms. These
algorithms may employ feature-based methods, object recognition models, or segmentation
techniques to identify regions of interest based on criteria like color, texture, or shape.
Machine learning, both supervised and unsupervised, plays a crucial role, allowing models
to learn and recognize patterns within images, ultimately aiding in the accurate detection of
ROIs. Post-processing steps, such as refinement and filtering, may be applied to enhance
the precision of the detected region, ensuring that the subsequent analysis or processing
focuses on the relevant and meaningful parts of the image.

Feature Extraction

Feature extraction is like finding the most important information in a picture. Imagine you
have a lot of dots (pixels) in an image, and you want to pick out the key things that make it
interesting. It’s like highlighting edges, recognizing colors, or spotting patterns. By doing
this, you make the picture simpler but still keep the important details. It’s kind of like
picking out the unique parts of a photo, so when a computer looks at it, it can understand
what’s going on. This helps in tasks like recognizing objects, understanding shapes, and
making sense of images without having to look at every single pixel.
Classification

In the context of image processing or machine learning, it involves teaching a computer to

recognize and sort images or data into different groups. Think of it like teaching a
computer to tell the difference between cats and dogs in pictures. To do this, we use the
features we extracted earlier—like recognizing shapes, colors, or patterns. We show the
computer many examples of cats and dogs, and it learns the patterns associated with each.
Once trained, when you give it a new picture, it can predict whether it’s more likely to be a
cat or a dog based on what it learned. Classification is a powerful tool used in various
applications, from identifying spam emails to recognizing faces in photos, making
technology smarter and more helpful.

Tracking of ROI

Tracking of Region of Interest (ROI) in a video involves the continuous monitoring and
follow-up of a specific area within successive frames of the video. This is particularly
useful when you want to keep tabs on a moving object or specific region throughout the
video sequence. The process typically begins with the initial detection of the ROI in the
first frame, using techniques like object recognition or manual selection. Once the ROI is
identified, tracking algorithms come into play. These algorithms analyze subsequent
frames, predicting and updating the position of the ROI over time. Various tracking
methods exist, such as correlation filters, Kalman filters, or more advanced deep learning-
based trackers. The goal is to maintain accurate and consistent tracking even when the ROI
undergoes changes in position, size, or appearance. This capability is valuable in
applications like surveillance, video analysis, and object monitoring in real-time
environments. In summary, tracking ROI in a video involves the continuous observation
and updating of a specified region of interest as it moves or changes throughout the video
sequence.

Decision

Decision-making in image processing is like making choices based on what we’ve learned.
After tracking and analyzing a specific part of a video, decisions are made about what’s
happening. For example, if we’re tracking a moving object, the decision might be whether
it’s a person or something else. This step helps computers understand and react to what
they “see” in videos or images.

Detecting the level of engagement means figuring out how interested or involved someone
is when looking at something, like a video or a website. This can involve looking at their
facial expressions, where they’re looking, or even sensing their emotions. In simpler terms,
it’s about understanding how much attention or interest someone has in what they’re
seeing.

IMPLEMENTATION

Parser Module: The parser module in an engagement detection system is a component

designed to extract, interpret, and organize information from diverse data sources. This
module may interact with data from different modalities, such as facial expressions,
gestures, eye tracking, screen capture, or user sensing. It could also process log files, text
data, or any other relevant information available during online learning activities.

Data smoothing module: The data smoothing module in an engagement detection system
serves the purpose of enhancing the quality and interpretability of data by reducing noise,
fluctuations, or outliers. Raw data collected from various sensors or sources may contain
inherent noise or inconsistencies. The data smoothing module employs algorithms or
techniques to filter out this noise, ensuring that the underlying patterns or trends are more
clearly discernible.

Normalization Module: The normalization module in an engagement detection system is

designed to standardize and scale input data, ensuring that different features or modalities
contribute equally to the analysis process. Scaling Features: Raw data collected from
diverse sources may have different units or scales. The normalization module addresses this
by transforming the data so that each feature or modality has a similar scale. This is
essential for algorithms that are sensitive to the magnitude of input variables.

Data binding module: In the context of engagement detection in online learning, a data
binding module could be a component responsible for establishing and managing the
connections between various data sources and the system's user interface or processing
modules. In systems with a user interface component, the data binding module may play a
role in updating the interface in response to changes in the underlying data. This ensures
that the user is presented with relevant and up-to-date information regarding engagement
levels.
Feature Selection Module: The feature selection module aims to identify and retain the
most informative features from the raw data collected from various modalities, such as
facial expressions, gestures, physiological signals, or log files, for the engagement
detection process. Engagement detection systems often deal with high-dimensional data,
including multiple modalities and numerous features.

Classification And Labeling Module: The primary goal of the classification and labeling
module is to categorize learners into different engagement levels based on the features
extracted from the selected modalities. Engagement levels can include categories such as
boredom, confusion, delight, frustration, neutral, and potentially others depending on the
specific goals of the engagement detection system.

Hardware Requirements

 Pentium IV or higher, (PIV-300GHz recommended)

 500 GB HDD

 4GB RAM

Software Requirements

 Pycharm

 Anaconda

 Operating System: Windows 7 /Windows 8/ Windows 8.1

 Web Browser: Microsoft Internet Explorer, Mozilla, Google Chrome or later

ALGORITHMS TO BE USED FOR CLASSIFICATION

Decision Tree: A decision tree is a flowchart-like structure in which each internal node
represents a "test" on an attribute (e.g., whether a coin flip comes up heads or tails), each
branch represents the outcome of the test, and each leaf node represents a class label
(decision taken after computing all attributes). Decision trees are one of the most popular
machine learning algorithms due to their simplicity and interpretability.

Random Forest: A random forest is an ensemble learning method for classification,

regression, and other tasks that operates by constructing a multitude of decision trees at
training time. For classification tasks, the output of the random forest is the class selected
by most trees. For regression tasks, the output is the average prediction of the individual
trees. Random forests are a powerful tool for machine learning because they are able to
combine the strengths of multiple decision trees. By averaging the predictions of many
different trees, random forests are able to reduce the variance of the predictions, which
makes them more accurate. Additionally, random forests are less prone to overfitting than
individual decision trees.

Naive Bayes: Naive Bayes classifiers are a family of supervised machine learning
algorithms that are based on Bayes' theorem. They are widely used for classification tasks,
where the goal is to assign a label to a new instance of data. Naive Bayes classifiers are
known for their simplicity, efficiency, and interpretability. The core assumption of Naive
Bayes classifiers is that the features of a data instance are independent of each other, given
the class label. This means that the probability of observing a particular feature is not
affected by the values of other features. While this assumption may not always hold in real-
world data, it often makes Naive Bayes classifiers computationally efficient and relatively
accurate.

Logistic Regression: Logistic regression is a statistical model that uses a logistic function
to predict the probability of a binary outcome. It is one of the most commonly used
machine learning algorithms for classification tasks. Logistic regression is a type of linear
regression, but instead of predicting a continuous numerical outcome, it predicts the
probability of a binary outcome. The logistic function is used to transform the predicted
value to a probability between 0 and 1. The logistic regression model is defined by a linear
combination of the input variables and a set of parameters. The parameters are estimated
using a maximum likelihood estimation (MLE) algorithm. The MLE algorithm finds the
parameters that maximize the probability of the observed data.

LITERATURE REVIEW

A technique for evaluating an e-learner's understanding of material displayed on screen by

examining their nonverbal behaviour pattern was presented by the authors in [1]. Forty-four
undergraduate students served as the test group. Using a USB front-facing camera, their
nonverbal behaviour was observed while participants completed an on-screen multiple-
choice question and answer test. An ANN is fed the information on their question-answer
scores and behavioural patterns to classify their comprehension states almost instantly.
Future work by the authors is planned to increase the classifier's accuracy. Furthermore,
this investigation might be extended to examine the relationship between behaviour and the
type of question asked. In the future, this technique might also be used to various kinds of
behaviours and mental states. The authors of [2] hope to use machine learning to give
computers the ability to recognize and differentiate between human faces. This might lead
to a variety of applications, including the detection of inattentive drivers. Additionally, it
monitors each user's level of involvement, allowing for the analysis of how attentively
students are following instructions. When a learner's attention wanders, the system
responds by refocusing it. CNN is without a doubt the model that best embodies deep
learning's network topology and serves as a hub for image and speech recognition.
The concentration index for kindergarten is low because distractions are more common
there, whereas primary students are less likely to get sidetracked, and adults perform better
on the concentration index than both in [3], which also calculates the concentration index
and assesses students' focus during the lesson. Because they are present-minded when
learning problems, employees are classified as neutral learners. [4] Knowledge extraction
and rule creation from data are the foundations of knowledge representation. In rough set
theory, knowledge extraction and rule creation are crucial methods. [5] This study suggests
using facial emotions to gauge how learners are feeling about their learning. This is
accomplished by predicting facial emotions with CNN. The network is trained using the
FER2013 dataset. The CNN architecture achieved state-of-the-art performance on image
classification problems, which is why it is used: it can capture and extract relevant and
useful attributes from an image. [6] According to the authors, multi-modal emotion
recognition outperforms single-modal methods in terms of efficiency because it combines
user messages, body gestures, and facial expressions. Typically, the FER models are pre-
trained using individual images from a sizable dataset, like AffectNet. [7] An alternative to
artificial neural networks (ANNs) are recurrent neural networks (RNNs). They can process
sequential data one element at a time and selectively pass information across sequence
steps. RNNs have been proposed to handle sequential data by getting around one of ANN's
main limitations, which is the assumption of independence among data. [8] There are three
subfields in the field of facial expression and emotion recognition: the beginning, apex, and
end. Moreover, the amount of facial movement—also referred to as motion/action unit
moments—can be used to depict facial expressions. [9] In order to assist teachers in
evaluating the efficacy of their chosen teaching strategies, we applied real-time FER to
images of student faces taken in a classroom. This study's main goal was to assess the
effectiveness of active teaching strategies using FER as a tool to predict students' emotions.
[10] The purpose of this study is to comprehend the feelings and levels of engagement that
students experience when taking classes virtually and not in person with their instructors.
The goal is to create an educational application for use by instructors who use online
resources to deliver lectures. [11] The primary objective of this research is to create reliable
facial information models that can describe how people interact in a learning environment.
Specifically, at the behavioural level, where eye-blinking and gross head movement are
employed as attention markers, and at the emotional level, where facial expressions match
attention-related muscle movements.

FUTURE SCOPE
The future scope for work will be integrating the engagement detection Module in e-
learning recommender Systems. Secondly, we will try to predict the Understanding Level
of the e-learner from the engagement Level making it more accurate and efficient.

CONCLUSION

Finally, the transition from traditional classrooms to online learning emphasizes how
critical it is to comprehend and assess student engagement in the context of digital
education. Effective student engagement, which includes both commitment and active
involvement, is critical to successful outcomes in the rapidly changing e-learning
environment. This study delves into the intricacies of measuring engagement,
acknowledging its crucial function in augmenting the effectiveness of virtual learning. It is
acknowledged that emotional states have an impact on the quality of learning experiences,
which is why multimodal sentiment analysis techniques are being proposed as a means of
interpreting learners' emotions from social media responses. Providing emotional support to
individuals who exhibit noninteractive tendencies appears to be a strategy to promote
positive engagement. Furthermore, research on how emotions affect learning emphasizes
how deeply they affect memory, perception, reasoning, and logical thought. The awareness
of different emotional states—both fundamental and intellectual—highlights how difficult
it is to comprehend and improve students' engagement. To improve educational outcomes,
the paper promotes the creation of emotion-based models that link academic emotions and
engagement levels. It emphasizes the importance of understanding and making the
connection between emotions and varying degrees of participation.

REFERENCES

1. Mike Holmes, Annabel Latham, Keeley Crockett, and James D. O’Shea “Near real-time
comprehension classification with artificial neural networks: decoding e-Learner non-verbal
behaviour”
2. Fabri, M., Moore, D.J., Hobbs, D.J “Mediating the Expression of Emotion in Educational
Collaborative Virtual Environments: An Experimental Study”
3. Nour, Nahla & Elhebir, Mohammed & Viriri, Serestina. (2020). Face Expression Recognition
using Convolution Neural Network (CNN) Models. International Journal of Grid Computing &
Applications. 11. 1-11. 10.5121/ijgca.2020.11401.
4. Pawlak, Z.: Rough Sets: Theoretical Aspects of Reasoning about Data. Kluwer Academic,
Dordrecht (1991)
5. Brittany Gilbert, “Online Learning Revealing the Benefits and Challenges,” 2015.
6. J. Shen, H. Yang, J. Li, and Z. Cheng, “Assessing learning engagement based on facial expression
recognition in MOOC’s scenario,”Multimedia Syst., vol. 28, pp. 469–478, 2022.
7. Arkorful V, Abaidoo N (2015) The role of e-learning, advantages and disadvantages of its adoption
in higher education. Int J Instr Technol Distance Learn 12:29–42
8. Boughrara H, Chtourou M, Amar CB, Chen L (2016) Facial expression recognition based on a
mlp neural network using constructive training algorithm. Multimed Tools Appl 75(2):709–731
9. Michel, N.; Cater, J.J., III; Varela, O. Active versus passive teaching styles: An empirical study of
student learning outcomes.Hum. Resour. Dev. Q. 2009, 20, 397–418.
10. V. Trowler, “Student engagement literature review,” The higher education academy, vol. 11, no. 1,
pp. 1–15, 2010.
11. Lilly Dimitrovsky, Hedva Spector, Rachel Levy-Shiff, and Eli Vakil, “Interpretation of facial
expressions of affect in children with learning disabilities with verbal or nonverbal deficits,”
Journal of Learning Disabilities,vol. 31, no. 3, pp. 286–292, 1998.