International Journal of Recent Technology and Engineering (IJRTE)

ISSN: 2277-3878, Volume-8 Issue-6, March 2020

Eye Blink Detection System for Paralyzed

Patients
Kavitha H. S., Suguna G. C.

Abstract: Paralysis of an human being is caused due to the

degeneration of motor neurons which weakens the muscle, so that II. LITERATURE REVIEW
it does not allow patient to move, speak, breathe , and loss in the
voluntary actions. It is an incurable disease. To understand the Joshua et.al, implemented portable Video-Oculography
feelings of a paralyzed patients Brain wave technique and device to evaluate mild ocular motor function and intensity of
Electro-oculography techniques were used. These techniques are head trauma. VCO is of low cost portable eye tracking
afflictive, discomfortable and leads to unconsciousness of the device .The device evaluates the movement of an eye to
paralyzed patient. The real time video oculography system fills the assess higher cortical functioning and to suspect the head
communication gap between the patient and the world. Video
Oculography (VOG) is video-based method of measuring the
trauma. The drawback of this device is unease to test[1].
vertical, torsional and horizontal position components of both the Sudhir et.al , implemented a novel algorithm for the extraction
eye blinks with the help of small cameras placed in the of face , analyze the blinks ,identify the type of the blink and
head-mounted mask .This paper presents different visual convert the same into speech. The analysis of the blinks were
technologies, such as eye blink detection, eye center localization done based on the intensities sclera, edge of the iris , central
and conversion of the eye blink to speech. The video oculography part of the iris and back to the sclera and the diameter of the
could achieve accuracy of 0.968. iris[2]. Masaru et.al ,has implemented pupil tracking
algorithm which determines pupil area by the selection of
Keywords : VOG, Eye Blink detection
reliable contour points, verified the same with former frames,
evaluated the pupil area detection over obstacles and
I. INTRODUCTION
estimated of pupillary light-reflex parameters like precision,
Paralysis is a metabolic disturbances caused by the recall and F-measures[3]. Jansen et.al , have designed and
impairment of voluntary muscules. This is due to the implemented a ample VOG system based on the torsional eye
degeneration or loss of group of motor neurons. Due to the movement detection algorithm. Region of interest (ROI) of
motor neuron disease paralyzed patient cannot communicate the image is pre-processed, and enhanced. To determine the
with the external world. Eye organ could be used as effective suitable horizontal and vertical translation and to calculate
tool for communicating with the external world. The eye the degrees of rotation, one of the ROI is shifted with respect
movement of a paralyzed patient is analyzed to determine the to the other ROI[4]. Akshat et.al , developed a VOG system
action. The acquisition and analysis of eye movements of in which camera focused on the eye detects and records the
blink that matches the required threshold intensity. Pre
paralyzed patients is presented in the paper. An eye blink
processing of the recorded image is carried out with erosion
sensor is used for detecting eye movement thereby reducing
and dilation techniques. Eye region in the image was
the occurrence of artifacts and a cost effective simple circuitry
extracted by segmentation technique. The recorded
is designed for implementation of signal processing. movement of eye is converted to speech [5]. Jacek et.al , has
In this paper, small video cameras are mounted on head and developed a VOG system based on a web camera which
IR illuminations are used to capture an image of an eye. captures minute information about head shaking and eye
Algorithms are developed for extracting vertical and position of a Parkinson's disease patients.
horizontal movements of eye from video images. Developing An algorithm was implemented for face detection, facial
a method for determining eye blinks is more complex. Low feature localisation and tracking, Iris localisation, Iris
contrast images can also be tested by this algorithm using position registration[7].
IR-wavelengths for illumination. This paper presents design
and implementation of an algorithm detecting torsional robust III. PROPOSED WORK
movement of eye for VOG. The algorithm used here uses new
Video Oculography (VOG) is video-based method of
approach for measuring torsional movement of eye and it is
measuring the vertical, torsional and horizontal position
suitable for videos which are of low contrast. It is
components of both the eye blinks with the help of small
implemented and compared with the performance of other
cameras placed in the head-mounted mask . There are several
techniques.
techniques used for detecting closing and opening of eyes and
eye movements. The technique used is simple and less time
consuming. Blinks are converted to sentence or words, which
is not been used before. The camera which is focused to the
Revised Manuscript Received on February 08, 2020.
Mrs. Kavitha H. S., Assistant Professor and Research Scholar, face of the user records the blinks and is considered as video
Electronics and Communication Engineering, JSS Academy of Technical sequence. It is then considered as the input for detection of
Education, Bengaluru. eye blinks and code counting.
Mrs.Suguna G. C., Assistant Professor and Research Scholar,
Electronics and Communication Engineering, JSS Academy of Technical
Education, Bengaluru.

Published By:
Retrieval Number: F7368038620/2020©BEIESP Blue Eyes Intelligence Engineering
DOI:10.35940/ijrte.F7368.038620 & Sciences Publication
333
 Eye Blink Detection System for Paralyzed Patients

Calculating the number of eye blinks is done and the stages depend upon the desired accuracy of extraction. Such
same corresponds to meaningful sentence according to the cascaded stages of classifiers are called Haar cascade
count. It works with less number of specifications. This also classifiers
helps in cost reduction. For detection of eye blinks and eye
blinks there are various methods involved. The method which 3. Blink Detection and conversion of gestures to voice.
is used in this paper reduces time for detection of blinks and For Blink detection , Eye Aspect Ratio (EAR) is computed
lessens the cost by large margin. Figure 1 shows the overview using six coordinates placed on the eye, These co-ordinate
of the proposed system from camera for recording to the points gives us vertical and horizontal lengths using which
speech output. EAR is calculated.

EAR = VL/HL

For Eye movement monitoring, Eye Gaze ratio (EAR) is

computed by processing white and grey areas of eye
ball.Eye movement in our case is divided into Right ,
Left and Center. Eye gaze values are divided into sections
to detect Left Right and center position of eye ball.
In our case , Gaze Ratio values between 0 to 1
corresponds to position Right 1 to 3 corresponds to position
Figure 1 Proposed system Center and more than 3 corresponds to position Left of the
Eye ball. Now both EAR and EGR parameters are combined
The more useful organ in human is eye which helps in to obtain several combination of gestures. Each gesture then
visualizing the outside world. With the analysis of eye corresponds to a voice command which is played out
movement, lot of knowledge about human is disclosed. louder to get the attention of the care taker and to convey
Analysis of eye movement is used for applications like the necessary vital information.
disease diagnosing, state of mind recognition, recognition of
Real time Blink Detection basically employs 2 main
activity, identification of a person etc. An algorithm which
algorithms:
helps in detecting movements of eyes such as blinks which
can be used for communication.  Facial Landmark Prediction Algorithm
This technique which uses eye blink for communication is  Eye Aspect Ratio (EAR) Algorithm
useful for people who have motor neuron disease where the Steps followed to detect facial landmarks in an image is
patient feels hard in communicating with the world. The eye given below:
blinks can be converted to transmit messages to Morse code, 1. Localization of a face in an image is identified using Haar
in which eye blink is represented as dash or dot. For cascades or HOG + Linear SVM detectors .
accomplishing this, an algorithm is presented which identifies 2. Obtain the 2D - coordinates of the face region ROI by
first the facial region. and then identifies eye blinks with the applying the shape predictor, specifically a facial landmark
help of image processing. The algorithm designed and detector.
developed here for FPGA Spartan 3e and then it is tested Facial landmarks are nothing but face part extraction like
using software MATLAB. nose,eyes, mouth, jawline, etc., Facial landmarks can be
used for facial alignment , head pose estimation, face
IV. METHODOLOGY swapping, and blink detection .
Fast closing and reopening of a human eye could be
Figure 2 explains the methodology followed in implementing called as eye blink. The pattern of blinking differs from
the proposed work. person to person. The pattern may differ in the speed of
closing, opening , a degree of squeezing the eye and the time
1. Conversion of obtained video feed of the subject/patient taken to blink. The eye blink lasts approximately 100-400 ms.
into set of frames of images. One may use the facial landmark detectors to localize the eyes
A simple Webcam is used to capture and relay the video
and eyelid contours. The Eye Aspect Ratio (EAR) from the
feed. Python code along with OpenCV libraries are employed
landmarks detected image is used to estimate eye open state
to convert analog video feed into discrete set of image
or close state. Per frame EAR may not necessarily recognize
frames. In our case, sampling rate is set as 10
frames/s.Then obtained set of color images are converted into the eye blinks correctly, large temporal window of a frame
greyscale images to facilitate processing operations. into account is trained by the classifier. For every video
frame, the eye landmarks are detected. The eye aspect ratio
2. Extraction of facial and eye features. (EAR) between height and width of the eye is computed using
Haar wavelets i.e rectangular filters of different window the below given expression.
sizes are evaluated over the obtained images resulting in
edge, line and feature detection of the input face. Since
evaluating all parts of face real-time results in computational
complexity, Ada boost algorithm is made use to predict and
evaluate only desired parts of face , in our case only regions
around eyes. Haar classifiers are evaluated in stages , no of

Published By:
Retrieval Number: F7368038620/2020©BEIESP Blue Eyes Intelligence Engineering
DOI:10.35940/ijrte.F7368.038620 & Sciences Publication
334
 International Journal of Recent Technology and Engineering (IJRTE)
ISSN: 2277-3878, Volume-8 Issue-6, March 2020

landmarks is computed by the numerator and denominator

gives the distance between horizontal eye landmarks. Using
this EAR one can avoid image processing techniques to detect
blinking. The constant EAR represents the eye open and is
falls below the threshold while closing an eye. The EAR of
both eyes is averaged because both the eyes blink
simultaneously.

Figure 4 (a) Facial landmarks points for Eye Open

Figure 2 Methodology

Figure 4 (b) Facial landmarks points for Eye Close

V. RESULTS

Figure 5 EAR Values for eyes closed and open

EAR Value for an open eye must be 0.4 and for a closed
eye, it must be 0.00.The experimental results in Figure 5,
show that EAR value is around 0.3 for an open eye and for a
closed eye, the EAR Value is as low as 0.05.

Figure 3 Visualizing the 68 facial landmark coordinates

|| P 2  P 6 ||  || P3  P5 ||
EAR=
2 || P1  P 4 ||

P1, P2, P3, P4, P5, P6 are the 2D facial landmark locations
as shown in Figure 3.The distance between the vertical eye

Published By:
Retrieval Number: F7368038620/2020©BEIESP Blue Eyes Intelligence Engineering
DOI:10.35940/ijrte.F7368.038620 & Sciences Publication
335
 Eye Blink Detection System for Paralyzed Patients

International Conference on Advances in Electrical Engineering

(ICAEE). IEEE, 2014.
3. Kiyama, Masaru, Hitoshi Iyatomi, and Koichi Ogawa. "Development
of robust video-oculography system for non-invasive autonomic nerve
quantification." 2012 IEEE-EMBS Conference on Biomedical
Engineering and Sciences. IEEE, 2012.
4. Jansen, S. M. H., et al. "A torsional eye movement calculation
algorithm for low contrast images in video-oculography." 2010 Annual
International Conference of the IEEE Engineering in Medicine and
Biology. IEEE, 2010.
5. Gambhir, Akshat, et al. "Video Oculographic System using Real-Time
Video Processing." (2015).
6. Karthik, K. P., and S. Basavaraju. "Design and Implementation of
Unique Video Oculographic Algorithm Using Real Time Video
Figure 6(a) User Interface for Eye movement and Blink Processing on FPGA."
7. Naruniec, Jacek, et al. "Webcam-based system for
Detection (Center of the Eye) video-oculography." IET Computer Vision 11.2 (2016): 173-180.
8. Joseph K George , Subhin K B, et.al , “Eye Controlled
Home-Automation For Disables” IOSR Journal of Electrical and
Electronics Engineering (2017): 06-08.

AUTHORS PROFILE

Mrs. Kavitha H. S., Assistant Professor and Research

Scholar, Electronics and Communication Engineering,
JSS Academy of Technical Education, Bengaluru.
Published one international journal and two
international conference papers and two national
conference papers.

Figure 6(b) User Interface for Eye movement and Blink Mrs.Suguna G. C., Assistant Professor and Research
Detection (Right Eye) Scholar, Electronics and Communication Engineering,
JSS Academy of Technical Education, Bengaluru.
VI. CONCLUSION Published three international journal and two
international conference papers.
The proposed method provides efficient and accurate eye
detection. When an artificial light is used ,the overall and
detection accuracy are 98% and 100% respectively for a
distance equal to 35 cm. Each frame takes an average time of
71 ms for execution which is very efficient for real time
application. The proposed system is able to bring out a
solution for the paralyzed patients to communicate with the
external world effectively. The developed system is different
from previously developed prototypes because none of the
components are in direct contact with the patient’s body hence
it will prove to be safer. Use of simple facial landmark
technique reduces the use of image processing. Use of
Raspberry pi is simple and developing tremendously in the
market today. The tool had advantages over the older
conventional tools. the micro-controller is interfaced with an
LCD module to display the speech output as text
simultaneously.

ACKNOWLEDGMENT
I would like to thank my parents and my family who has
helped me to carry out the work successfully, its my pleasure
to address my guide who assists me with the full support in
carrying the research work successfully.

REFERENCES
1. Fischer, Joshua D., and Dawie J. van den Heever. "Portable
video-oculography device for implementation in sideline concussion
assessments: prototype." 2016 38th Annual International Conference
of the IEEE Engineering in Medicine and Biology Society (EMBC).
IEEE, 2016.
2. Rupanagudi, Sudhir Rao, et al. "Novel methodology for blink
recognition using video oculography for communicating." 2014

Published By:
Retrieval Number: F7368038620/2020©BEIESP Blue Eyes Intelligence Engineering
DOI:10.35940/ijrte.F7368.038620 & Sciences Publication
336