Received May 20, 2019, accepted June 11, 2019, date of publication July 10, 2019, date of current

version August 2, 2019.

Digital Object Identifier 10.1109/ACCESS.2019.2927755

EOG-Based Eye Movement Classification and

Application on HCI Baseball Game
CHIN-TENG LIN 1 , JUANG-TAI KING2 , PRIYANKA BHARADWAJ1 , CHIH-HAO CHEN3 ,
AKSHANSH GUPTA4 , WEIPING DING 5 , AND MUKESH PRASAD 1
1 Centre for Artificial Intelligence, School of Computer Science, FEIT, University of Technology Sydney, NSW 2007, Australia
2 Brain Research Center, National Chiao Tung University, Hsinchu 30010, Taiwan
3 Institute of Imaging and Biomedical Photonics, National Chiao Tung University, Hsinchu 30010, Taiwan
4 School of Computational and Integrative Sciences, Jawaharlal Nehru University, New Delhi 110067 , India
5 School of Information Science and Technology, Nantong University, Nantong 226019, China

Corresponding author: Mukesh Prasad (mukesh.nctu@gmail.com)

This work was supported in part by the Australian Research Council (ARC) under discovery Grant DP180100670 and Grant DP180100656,
in part by the Army Research Laboratory and was accomplished under Cooperative Agreement Number W911NF-10-2-0022 and
W911NF-10-D-0002/TO 0023, and in part by the Taiwan Ministry of Science and Technology MOST 106-2218-E-009-027-MY3.

ABSTRACT Electrooculography (EOG) is considered as the most stable physiological signal in the
development of human–computer interface (HCI) for detecting eye-movement variations. EOG signal
classification has gained more traction in recent years to overcome physical inconvenience in paralyzed
patients. In this paper, a robust classification technique, such as eight directional movements is investigated
by introducing a concept of buffer along with a variation of the slope to avoid misclassification effects in EOG
signals. Blinking detection becomes complicated when the magnitude of the signals are considered. Hence,
a correction technique is introduced to avoid misclassification for oblique eye movements. Meanwhile,
a case study has been considered to apply these correction techniques to HCI baseball game to learn
eye-movements.

INDEX TERMS Eye movement classification, HCI, baseball game, EOG.

I. INTRODUCTION A. VERGENCE
The importance of eye movement tracking along with human- Vergence eye movements are considered as, ‘‘slow discon-
computer interaction (HCI) has been investigated in this jugate eye movements that allow the visual system to fuse
paper. This approach has remained a promising method which targets moving in depth, giving a person the ability to perceive
is used in recent years to detect and analyze eye movements. the world in all three dimensions’’ [3].
Electrooculography (EOG) is an inexpensive technique used
in recent years to record eye movements [1]. EOG signal B. PURSUIT MOVEMENTS
classification is considered as the most useful control sig- Pursuit movement occurs while the eye tracks a moving
nals for human-computer interface [2]. Eight directional eye object. It means that the image of an object can maintain focus
movement classification algorithm is an effective way to ana- on the fovea.
lyze the aftermath effect of noise in EOG signals. However,
a thorough understanding of various characteristics of eye C. SACCADE
movements leads to a better understanding of eye-movement Saccades are classified as rapid eye movements where these
detection algorithm. eye movements observe the world without an externally
Following types of eye movements can be detected through driven feedback system [3]. Saccades are faster than Vergence
EOG signals. and Pursuit eye movements.

D. BLINK
The associate editor coordinating the review of this manuscript and Blink can be described as a rapid eyelid movement which
approving it for publication was Mohammad Zia Ur Rahman. has a stimulant to the surrounding environment such as

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see http://creativecommons.org/licenses/by/4.0/
96166 VOLUME 7, 2019
C.-T. Lin et al.: EOG-Based Eye Movement Classification and Application on HCI Baseball Game

temperature, relative humidity, and brightness. Blink rate

is directly associated with mental state, physical activity,
or fatigue [4], [5].

E. FIXATION
Fixations are the stationary state of eyes. Visual gaze is
maintained in a single location during fixation state. Fixations
are the events that occur between two saccades. The average
fixation time ranges from 100ms to 200ms [6].
In recent years, several eye tracking techniques have
evolved which allow the detection and monitoring of eye
movements. One of them is Infrared oculography (IR), which
is generally used to quantify the difference between the
amounts of infrared light reflected by the sclera and sen-
sor (phototransistor) pair [7]. However, IR is not a reason-
able technology to measure pursuit or saccades because of FIGURE 1. System Overview for extracting EOG signals.
the nonlinearity problem. Many other techniques such as
search eye coil [8], [9], video images [10], [11] and EOG
have been proposed to track eye movements [12], [13].
EOG has been very popular due to its ease of signal
acquisition approach. However, studies show that hybrid
brain-computer interface utilizing hybrid signal are in prac-
tice [12], [14], [15], these papers concentrate on EOG based
eye movement analysis. EOG measurement is based on the FIGURE 2. Schematic diagram of electrodes placement.
potential difference between electrodes from the skin it is
placed. Human eyes act like a dipole with cornea acting
as positive side and retina as a negative side. When eye- eye-movement detections, and the output is represented by
balls are rotated, the inner dipoles also move consequently. a graphical user interface.
These movements of eye dipoles make electrical potential
slightly change around the eyes. Thus the potential difference A. EOG MEASURING DEVICE
assessing eyeball rotation can be measured. Because of these An EOG Mindo device from National Chiao Tung University
characteristics, EOG signals are considered as an appropriate Brain Research Center has been used to measure EOG signals
approach to develop human-computer interface (HCI). It also from subjects. Electro-physiological signals are measured by
aids in translating eye movements into human understandable placing electrodes around eyes as shown in the figure 2.
commands. Electrode placed on the forehead is a reference signal. Four
EOG has become a preliminary eye movement detect- channels are read by placing electrodes around eyes, where
ing technique in developing HCI systems such as voice Ch1 and Ch2 collect horizontal signals, and Ch3 and Ch4 col-
recognition [16], [17], visual information [18], gesture con- lect vertical signals.
trol [19], [20], methods based on brain signals, infrared
head-operated joysticks [21] and many other medical B. SIGNAL ACQUISITION
usages. Extensive research is being carried out in terms The proposed wireless EOG signal acquisition device was
of non-medical applications such as gaming [22]–[25], and approximately 45 × 32 × 8 mm3 in size. A Bluetooth
browsing internet [26]. However, this paper aims to utilize module was employed to transmit the EOG signals wire-
EOG based classification in gaming applications for practical lessly. The Bluetooth module BM0203 provided a sufficient
consumption. It discusses an approach to have high accuracy transfer band rate (115 200 b/s) and was compliant with the
and low computation for an EOG-based HCI baseball game. computer’s Bluetooth v2.0 with enhanced data rate (EDR)
specification. Power was supplied by a lithium battery with an
II. MATERIALS AND METHODS output voltage of 3V. A commercial 750 mA·h Li-ion battery
Figure 1 gives the overview of the proposed BCI system. has been used to supply power to the EOG acquisition circuit,
BCI system focuses on aspects of extracting EOG sig- which has capacity to operate continuously for 12 hours. EOG
nals. An EOG measuring device will be used to record the signals are measured by the wet or dry sensors which are
eye-movements from the subjects. A signal acquisition sys- firstly amplified by the preamplifier unit. The preamplifier
tem is used to collect EOG signals from the devices and the amplifies the voltage difference between the reference sig-
processed signals are transmitted to personal devices with nals and those of the EOG electrodes, while simultaneously
the aid of Bluetooth devices. Thereby, HCI computations rejecting common-mode noise (i.e., the power line noise).
are carried out. Classification algorithms are applied for An instrumentation amplifier (INA2126, Texas Instruments,

VOLUME 7, 2019 96167

 C.-T. Lin et al.: EOG-Based Eye Movement Classification and Application on HCI Baseball Game

FIGURE 3. A structural overview of classification algorithm.

FIGURE 4. Signal with 60 Hz noise before moving average method.

Dallas, TX, USA) was used for its extremely high input
impedance and high common-mode rejection ratio (CMRR) Vertical Signal = Ch3Signal (Vertical +)
(∼90 dB) [27]. − Ch4Signal (Vertical −) (2)
Instrumentation amplifiers have the ability to improve
CMRR and amplify the EOG signals to a degree, where the a: SIGNAL SMOOTHING
minute voltage levels can also be detected. Gain of the pream- Some high frequency noise still could corrupt the signal in
plifier unit was set to 5.5 V/V. The cutoff frequency was reg- an unexpected way. Thus, to solve this problem, a filtering
ulated at 0.1 Hz by using a high-pass filter. Microcontroller process in the firmware level is introduced. A moving average
program which is controlling preamplifier and filter stage has method is utilized, to fit the limitation of the hardware. Mov-
reduced the 60 Hz noise in the EOG signals employing a mov- ing average also called rolling average, is the basic type of
ing average. In addition, a 12-bit resolution ADC has been FIR filter in DSP domain. Moving average is most commonly
used to digitize the EOG signals. A microcontroller unit was used with time series data to smooth out short-term fluctu-
also used to digitize the EOG signals, with a sampling rate ations and highlight long-term trends or cycles. The choice
of 256 Hz. The sync filter removed signals with frequencies between short-and long-term, and setting of moving aver-
higher than 62.5 Hz. After removing the noise and amplifying age parameters depends on the requirement of application.
the EOG signals, the data was transmitted to the computer Mathematically, the moving average is a type of convolution
interface via a wireless module. and similar to a low-pass filter used in signal processing.
The moving average filter is optimal for a common task:
C. SIGNAL CLASSIFICATION ALGORITHM reducing random noise while retaining a sharp step response.
EOG Classification algorithm is designed to reduce the This makes it as the premier filter for time domain signals.
overall calculation time and it also does not require signal Now considering an M -point sequence x[n], it needs to be
down sampling. The structural overview of the classifica- transformed to a new sequence y[n] through an N -point mov-
tion algorithm is as shown in figure 3. A software program ing average for this sequence. It means that the each element
gathers four channels transmitted from a Bluetooth device. of output y[n] is the average of N values in order of input
System reduces the common mode noise caused by elec- sequence x[n]. Its input-output relation can be represented in
tromyography (EMG) and environmental noise. Raw signals equation (3).
are obtained in horizontal and vertical form. In order to
1
extract features from the eye-movement, raw signals need to y [n] = (x [n] + x [n + 1] + . . . + x [n + N − 1])
be smoothened. Calculation amount of the signal has been N
1 XN −1
reduced by introducing buffer in the classification phase. = x [nk] (3)
N k=0

1) RAW EOG SIGNAL As mentioned above, the recorded signals are easily inter-
Electrodes are placed around the eyes to record EOG signals. fered by 60Hz noise, especially when the acquisition circuit
During this process traces of EMG signals are found due gets closer to the electric appliances. It has been showed in the
to facial contact of electrodes. This paper intends to discuss figure 4, that the original sine wave had been contaminated by
extracting only the EOG signals. Hence, EMG signals needs 60Hz power-line noise. After applying the moving average
to be removed from the raw signals. Equation (1) and (2) filter with a 5-point moving window, the moving average
demonstrates the subtraction of channel 2 from channel 1 and could be effectively removed by power-line noise, as shown
channel 3 from channel 4. The signal processing is done by in the figure 5.
using these equations. Given a continuous noise signal x(t) with frequency F Hz,
it is apparently that the integral within 1/F sec is equal to zero.
Horizontal Signal = Ch1signal (Horizontal +) A digital situation is demonstrated here. Equation (3) can be
− Ch2signal (Horizontal −) (1) extended to digital form. That means the summation of all

96168 VOLUME 7, 2019

C.-T. Lin et al.: EOG-Based Eye Movement Classification and Application on HCI Baseball Game

FIGURE 5. Signal with 60 Hz noise after moving average method.

FIGURE 6. Special blinking types using magnitude classification.

discrete signals with one period is equal to zero as shown in

equations (4) and (5).
Z 1/x
x(t) = 0 (4)
0
XAll signals with one period
X [n + K ] = 0 (5)
k=0
The moving window size is decided by both sampling rate
and the noise frequency as shown in equation (6).
Sampling Rate S
Moving Window Size = (6)
Noise Frequency F
FIGURE 7. Feature of look-up saccade and blinking.
b: BUFFER
Computational expense of the system can be reduced by
introducing buffer which is employed to retrieve temporary look-up saccade when compared with the look-up saccade
data. Computation occurs only when the buffer is full. Hence, in figure 7 with the special blinking #2 in figure 6. It is
it avoids the unnecessary computation there by increasing the discovered that their magnitude is both around 1000µV but
efficiency of classifier unit. the look-up saccade has longer duration than special blinking
#2. That means the slope variation of the special blinking
2) FEATURE EXTRACTION #2 is still larger than the look-up saccade. The slope variation
In order to analyze the eye-movements from EOG, mean- method increases the efficiency to classify blinks from other
ingful features needs to be recognized and extracted. Dis- eye-movements.
tinguishable patterns present in saccades makes it easy to be
classified further. Primarily, blinks and saccades needs to be a: PEAK DETECTION
segregated. Secondly, more than one eye movement needs to Peak detection is a method designed to reduce the calculation
be identified based on this study. time and the number of misclassified cases by detecting the
peak values of the vertical and horizontal signal. Classifica-
3) CLASSIFIER tion algorithm will find peak values of the differentiated sig-
Differentiation and peak detection play an important role in nals. The peak value detection is utilized to identify various
the classification algorithm. Differentiation is used to observe types of eye-movements.
the variation of the slopes which can distinguish blinking and
other eye-movement efficiently. Figure 6 demonstrates eye- b: BLINKING DETECTION AND REJECTION
movement classification based on magnitude variation tech- There is a need to overcome misclassification which might
nique. However, this approach is not used to detect certain adversely affect the specific eye-movement detection. Blinks
eye-movements. in the signal are identified and removed in order to avoid
Hence, signal classification requires a novel approach the interference of blinks with horizontal and vertical signals.
to identify blinks in a comprehensive manner, and which Interference with horizontal and vertical signal will result in
can also decrease the correction rate. In this paper, a slope misclassification.
variation technique is used to distinguish blinks from other A novel method has been introduced to overcome misclas-
eye-movements. Figure 7 shows the slope variation of a sification caused by blinks. An efficient way of classifying
look-up saccade and the slope variation of a blinking. The eye-movement is to differentiate signals and to extract peak
slope variation of the blinking is apparently larger than the value of signals is shown in figure 8. Once the peak values

VOLUME 7, 2019 96169

 C.-T. Lin et al.: EOG-Based Eye Movement Classification and Application on HCI Baseball Game

FIGURE 11. Representation of a look-up-and-left saccade.

FIGURE 8. Differentiating blink and saccade.

FIGURE 9. Process of rejecting the blinks.

FIGURE 12. Setting interface.

its threshold value. Combination of look-up and look-left

saccades forms a look-up-and-left saccade eye-movement
as shown in figure 11. However, both look-up saccade and
FIGURE 10. Representation of a look-up saccade.
look-left saccade have to occur at the same time. A misclassi-
fication is created when there is a mismatch in the occurrence
of two signals. This misclassification can be removed by
has been verified, blinks can be easily rejected based on applying the exception correction.
their threshold values. The eye-movement marked beyond
their threshold values after the peak values are recognized are D. GRAPHICAL USER INTERFACE DESIGN
classified as blink. Once the blink has been identified, they This paper aims to present classification results on a HCI
are rejected to extract saccades. The system searches for peak baseball game platform. An initial baseball game interface
values, and then the left signal of figure 8 is decided as a blink. is shown in the figure12. A time range is set up to display
System does not identify center signal, hence it is marked as a the data, file and name. Once all the required information is
saccade. The blinking rejection process is shown in figure 9. gathered, device is paired with a Bluetooth device to stimu-
late interface. Figure 13. Simulating interface is activated by
c: PATTERN RECOGNITION pressing start button and it will guide user through different
Various eye-movement detection is done by observing the steps. This will aid us to record user reaction and to recognize
peak values of the signal. Figure 10 illustrates that the peak various eye-movements. The total number of eye-movements
value of the vertical signal is marked above the upper thresh- occurred during this session can also be registered.
old and hence the system considers it as a look-up saccade.
Four other eye-movements identified are look-up-and-left, III. EXPERIMENTAL SETUP
look-up-and-right, look-down-and-left, and look-down-and- Three aspects of experimental set up has been discussed in
right as oblique saccades. System identifies a look-up saccade this paper. First experiment set up is to verify the classifi-
when the peak of vertical signal is marked beyond the upper cation working capability by considering normal scale and
threshold value. Similarly system can identify a look-left cues. Second experiment setup tests the capability of the
saccade when it encounters horizontal signal marked beyond classification by eliminating cues and using the same scale

96170 VOLUME 7, 2019

C.-T. Lin et al.: EOG-Based Eye Movement Classification and Application on HCI Baseball Game

FIGURE 15. Experiment environment.

FIGURE 13. Stimulating interface and the user steps.
TABLE 1. Angle of view.

FIGURE 14. Calibration interface and using procedure.

as the first test experiment. Third experiment is to test the

classification functionality on a tablet by reducing original
scale size to half of its size as to make it work on a tablet while
considering the cues. Eye-movement is detected based on
the horizontal and vertical threshold values of EOG signals.
FIGURE 16. Color code representation for experiment procedure
A Matlab based approach has been utilized to analyze the with cues.
recorded EOG signal. The calibration interface utilized in this
project can distinguish various eye-movements based on the
threshold values. and the scale of screen. Table 1 illustrates the calculated angle
Figure 14 shows a simple and effective calibration inter- of view. The above equation can be extended to digital form.
face system. Initially user needs to press ‘‘Start calibration’’ That means the summation of all discrete signals with one
button, and the calibration will show the cue in the center period is equal to zero.
of the frame. Then the up-right red dot will show up, now
the user will have two seconds to move their eyes to the A. EXPERIMENT PROCEDURE WITH CUES
up-right position. Similarly, experiments will be repeated In day-to-day activities, eight directional saccades and fix-
for down-right position. This experiment position will be ation are observed. Different color code is assigned for
repeated for 10 times for the system to collect sufficient data respective eye-movements as shown in figure 16. Look-up,
to set up an appropriate threshold value. look-down, look-right, look-left, look-right-up, look-right-
An experimental environment is set up to mimic the down, look-left-up and look-left-down are represented by red,
day to day computer usage. Hence, a distance of 50cm is orange, green, yellow, blue, aqua blue, violet and navy blue
maintained between the viewer and the monitor. Look-up respectively.
and look-down distances from eyes and monitor are main-
tained at 11cm. Look-right and look-left distance from eye B. EXPERIMENT PROCEDURE WITHOUT CUES
to monitor is 13cm. This experiment is set up on a 22’’ This experiment is designed to simulate an intuitive technique
monitor. Figure 15 shows the experiment set up. Since the while using the EOG application. Cues have been elimi-
magnitude and accuracy [28] of EOG signal depends on the nated so that the user don’t have to limit their eye moves
angular velocity, distance is transferred into the angle which in a particular direction. Process of this experiment is empty
is convenient to establish the relation between EOG signal for initial 2 seconds. For the next 5 seconds the subject is

VOLUME 7, 2019 96171

 C.-T. Lin et al.: EOG-Based Eye Movement Classification and Application on HCI Baseball Game

FIGURE 17. Experiment procedure without cues.

FIGURE 19. Calibration interface.

TABLE 2. Results of experiment procedure with cues.

FIGURE 18. Computer screen and tablet screen.

asked to move eyes in any direction. Color code represents

the respective eye-movement as described in the previous
section. Figure 17 shows the experimental set up without
cues. Primarily, this experiment is intended to provide a
natural approach to play HCI game by allowing user to have
an independent eye-movement.

C. EXPERIMENT PROCEDURE WITH CUES USING

SMALL SCALE
This experiment is repeated similar to the previous set up by
narrowing down the scale. A challenge has been encountered
while narrowing the scale, as the scale is narrowed the dis- IV. RESULTS
tance between eye and the monitor is also reduced. This will EOG signal is considered in this study to differentiate various
cause the signal to be smaller in amplitude and it becomes eye-movements of the subjects. A classification technique is
difficult to classify the signal. It will also raise misclassifi- provided which removed 90% of blinks along with extracting
cation due to the signal direction being deviated from the required saccades. Hence, it is effective in removing blinks.
expected direction. In order to use this EOG classification Overall computational time has been reduced by eliminating
algorithm on a tablet, the scale is narrowed about half of the down sampling of the EOG signals. This has increased effi-
original size. Figure 18 shows that as we deflate the scale to ciency of the classification system.
6 cm X 6.5 cm, it allows users to see the tablet from 41.7 cm
distance. A. RESULTS OF EXPERIMENT PROCEDURE WITH CUES
Now the shrinking scale will change the threshold that Experiment procedure with cues result in high correct rate.
classifies eye-movements because the distance and the angle The current classification technique yields higher accuracy
of view are smaller. A calibration interface has been designed when compared with the historical data and classification
to fit the screen size. This has been stimulated on the PC techniques. It is evident from the comparison results listed
as shown in figure 19. As shown in figure 19 each of them in table 2 and 3. The classification result is more stable for
has three red points. First user needs to focus on the center number 2, number 4, number 6 and number 8. Number 1,
red point, after the cue vanishes, the user will now have number 3, number 7, number 9 have resulted in stable oblique
two seconds to make an eye-movement. The user is asked eye-movement.
to look at the right-up red point for five times. Each time the
user is given two seconds to look at the point. Later, the user is B. RESULTS OF EXPERIMENT PROCEDURE
asked to follow the similar pattern in the right-down direction. WITHOUT CUES
The system will acquire required information from these In this experiment procedure without cues, the correct rate
eye-movements. decrease apparently. Results showed in table 4 indicates that

96172 VOLUME 7, 2019

C.-T. Lin et al.: EOG-Based Eye Movement Classification and Application on HCI Baseball Game

TABLE 3. Results of previous classification. TABLE 5. Results of experiment procedure with cues SSD.

TABLE 4. Results of experiment procedure without cues. TABLE 6. Results of application on HCI baseball game.

correct rates are slightly deteriorated from that procedure

with cues. The correct rate of the number 5 has lowered
significantly.
FIGURE 20. HCI game processing.
C. RESULTS OF EXPERIMENT PROCEDURE WITH CUES
ON SMALL SCALE DEVICE (SSD)
Result obtained by procedure with cues on a small scale light up and it will blink. While the number is blinking,
device show that there is a decrease in the correct rate. we move the eyes towards the blinking number, from the
Table 5 shows that the correct rates of number 2, number 4, center of panel. If the blinking number is 5, eyes still stand
number 6 and number 8 have increased from that of previ- on the center of the panel.
ous results. It signifies that the proposed classification tech- Accuracy rate as shown in figure 21. Since every run has
niques works appropriately for small scale screens. However, 10 trials, each run of the interface will show a number and
the correct rate of number 1, number 3, number 7, number the user repeats the task 10 times. A correct rate is obtained
9 are considerably low. This classification can fit the small by dividing it by ten trials. The correct rate has increased and
scale, it can be applied on the tablet. hence this EOG classification can be leveraged into real life
scenario.
D. RESULTS OF APPLICATION ON HCI BASEBALL GAME
The setting up for the HCI Baseball game is as shown in V. DISCUSSION
figure 20. Firstly, press the ‘‘START’’ button to enter the HCI Experimental results have demonstrated that the proposed
Baseball game. A translucent panel with the numbers will classification techniques provide high accuracy and have
show up. Number 5 in the center on the panel is brighter than improved the fluency of HCI game interpretation methods.
other numbers. Subsequently, the next number will randomly Stable classification is obtained by conducting experiments

VOLUME 7, 2019 96173

 C.-T. Lin et al.: EOG-Based Eye Movement Classification and Application on HCI Baseball Game

FIGURE 22. The electrode placement with the oblique eye-movement.

FIGURE 21. Look-up saccade and look-right saccade compared with

look-up-right saccade.

with cues. Most of the blinks were removed during this clas- FIGURE 23. Two look-up saccades without differentiation.
sification technique and the oblique eye-movements are well
classified with the above method. When the experiment was
conducted without cues, blinks were not removed effectively
due to processing time. Hence, a buffer was implemented
which aided in classifying eye-movements. This system will
split the signal when it encounters a blink before passing it
through buffer. This will cause misclassification. This factor
explains the decrease in correctness rate for experiment pro-
cedures without cues for number 5.
The average correct rate of the result for experiment with
cues in the small scale is lower than the average correct rate
of the result for experiment with cues. This can be observed FIGURE 24. Two look-up saccades with differentiation.
for the correct rate of number 1, number 3, number 7 and
number 9. This circumstance will explain that the angle of
view is smaller, which can make the EOG signal smaller classification applies differentiation. This will shrink the
and the EOG signal is proportionate with the angle of view. magnitude of signals which makes deviation smaller.
When the oblique eye-movement distance is longer from the Figure 23 shows two saccades without differentiation,
screen, the signal of the vertical and the horizontal are smaller and the deviation is 293 micro-volt. In figure 24 we can
than the up, down, right and left eye-movements. observe two saccades with differentiation, and the deviation
It is evident from figure 21 that the oblique eye-movement is 15 units. When a threshold is set by the calibration, the error
signal is smaller than the look-up saccade or look-right sac- probability of the two saccades without differentiation is
cade. This occurrence demonstrates that the signal scale is higher than two saccades with differentiation. It aids differen-
about ten times smaller than the original signal and it is tiation to shrink the scale of the signals and this can shrink the
caused by electrode displacement. deviation at the same time which in turn decreases the error
Figure 22 explains a look-up-left saccade. For look-up-left probability.
saccade signal is captured by channel 2 and channel 3. If there
is only a look-up saccade, signal is captured by channel 3. VI. CONCLUSION
Channel captured for look-up saccade is clear and hence It is evident from the HCI Baseball game that the classifica-
appear large. When an oblique eye-movement occur, the left tion can be utilized in everyday life. Usability and simplicity
eye will not directly approach the channel 2 or channel 3. of the classification is made efficient due to online compu-
Therefore, vertical and the horizontal signal of the oblique tation. The performance accuracy of the system has been
eye-movement are smaller than the up, down, right and left improved by scaling down the measurement to fit a tablet.
eye-movements. Small scale has the smaller angle of view The proposed method has established that by utilizing eight
than the normal scale, apparently the signal in small scale eye-directional movement the accuracy and performance of
is smaller than normal scale. The other key point is that the system can be increased. Research conducted based on
if there is a slight disturbance while using the tablet, this procedures without cues and small scale measurements calls
classification can tolerate a bit of deviation. That is because, for a further study in terms of improving the accuracy.

96174 VOLUME 7, 2019

C.-T. Lin et al.: EOG-Based Eye Movement Classification and Application on HCI Baseball Game

In future, we focus on developing descriptive alternatives [23] R. Krepki, B. Blankertz, G. Curio, and K. R. Müller, ‘‘The Berlin brain-
for all directions and even smaller scale eye-movements clas- computer interface (BBCI) —towards a new communication channel for
online control in gaming applications,’’ Multimedia Tools Appl., vol. 33,
sification and also on the implementation of a stable classifi- no. 1, pp. 73–90, Apr. 2007.
cation on the circuit board. This EOG device can work freely [24] D. P.-O. Bos, B. Reuderink, B. van de Laar, H. Gürkök, C. Mühl, M. Poel,
like a remote controller or a joystick. A. Nijholt, and D. Heylen, ‘‘Brain-computer interfacing and games,’’ in
Brain-Computer Interfaces. London, U.K.: Springer, 2010, pp. 149–178.
[25] S.-A. Chen, C.-H. Chen, J.-W. Lin, L.-W. Ko, and C.-T. Lin, ‘‘Gaming
REFERENCES controlling via brain-computer interface using multiple physiological sig-
[1] A. López, F. Ferrero, D. Yangüela, C. Álvarez, and O. Postolache, ‘‘Devel- nals,’’ in Proc. IEEE Int. Conf. Syst., Man, Cybern. (SMC), Oct. 2014,
opment of a computer writing system based on EOG,’’ Sensors, vol. 17, pp. 3156–3159.
no. 7, p. 1505, Jun. 2017. [26] S. He, T. Yu, Z. Gu, and Y. Li, ‘‘A hybrid BCI Web browser based on EEG
[2] S. Aungsakun, A. Phinyomark, P. Phukpattaranont, and C. Limsakul, and EOG signals,’’ in Proc. 39th Annu. Int. Conf. IEEE Eng. Med. Biol.
‘‘Development of robust electrooculography (EOG)-based human- Soc. (EMBC), Jul. 2017, pp. 1006–1009.
computer interface controlled by eight-directional eye movements,’’ Int. [27] B. Koo, Y. Nam, and S. Choi, ‘‘A hybrid EOG-P300 BCI with dual
J. Phys. Sci., vol. 7, no. 14, pp. 2196–2208, Mar. 2012. monitors,’’ in Proc. Int. Winter Workshop Brain-Comput. Interface (BCI),
[3] C. Yaramothu, E. M. Santos, and T. L. Alvarez, ‘‘Effects of visual distrac- Feb. 2014, pp. 1–4.
tors on vergence eye movements,’’ J. Vis., vol. 18, no. 6, p. 2, Jun. 2018. [28] A. N. Belkacem, S. Saetia, K. Zintus-Art, D. Shin, H. Kambara,
[4] R. Schleicher, N. Galley, S. Briest, and L. Galley, ‘‘Blinks and saccades as N. Yoshimura, N. Berrached, and Y. Koike, ‘‘Real-time control of a video
indicators of fatigue in sleepiness warnings: Looking tired?’’ Ergonomics, game using eye movements and two temporal EEG sensors,’’ Comput.
vol. 51, no. 7, pp. 982–1010, 2008. Intell. Neurosci., vol. 2015, Jul. 2015, Art. no. 653639.
[5] A. Sammaiah, B. Narsimha, E. Suresh, and M. S. Reddy, ‘‘On the per-
formance of wavelet transform improving eye blink detections for BCI,’’
in Proc. Int. Conf. Emerg. Trends Elect. Comput. Technol., Mar. 2011,
pp. 800–804.
CHIN-TENG LIN received the B.S. degree from National Chiao-Tung Uni-
[6] B. R. Manor and E. Gordon, ‘‘Defining the temporal threshold for ocu-
versity (NCTU), Taiwan, in 1986, and the master’s and the Ph.D. degrees
lar fixation in free-viewing visuocognitive tasks,’’ J. Neurosci. Methods,
vol. 128, nos. 1–2, pp. 85–93, 2003. in electrical engineering from Purdue University, USA, in 1989 and 1992,
[7] A. Kumar and G. Krol, ‘‘Binocular infrared oculography,’’ Laryngoscope, respectively. He is currently the Chair Professor of Faculty of Engineering
vol. 102, no. 4, pp. 367–378, Apr. 1992. and Information Technology, University of Technology Sydney, Chair Pro-
[8] D. A. Robinson, ‘‘A method of measuring eye movemnent using a scieral fessor of Electrical and Computer Engineering, NCTU, International Faculty
search coil in a magnetic field,’’ IEEE Trans. Bio-Med. Eng., vol. 10, no. 4, of University of California at San-Diego (UCSD), and Honorary Professor-
pp. 137–145, Oct. 1963. ship of University of Nottingham. He was elevated to be an IEEE Fellow
[9] R. V. Kenyon, ‘‘A soft contact lens search coil for measuring eye move- for his contributions to biologically inspired information systems in 2005,
ments,’’ Vis. Res., vol. 25, no. 11, pp. 1629–1633, 1985. and was elevated an International Fuzzy Systems Association (IFSA) Fellow
[10] C.-S. Lin, C.-C. Huan, C.-N. Chan, M.-S. Yeh, and C.-C. Chiu, ‘‘Design in 2012. He has been elected as the Editor-in-Chief of the IEEE TRANSACTIONS
of a computer game using an eye-tracking device for eye’s activity reha- ON FUZZY SYSTEMS, since 2011. He also served on the Board of Governors
bilitation,’’ Opt. Lasers Eng., vol. 42, no. 1, pp. 91–108, Jul. 2004. with the IEEE Circuits and Systems (CAS) Society in 2005-2008, the IEEE
[11] Z. Zhu and Q. Ji, ‘‘Eye gaze tracking under natural head movements,’’ in Systems, Man, Cybernetics (SMC) Society in 2003-2005, the IEEE Com-
Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. (CVPR), putational Intelligence Society (CIS) in 2008-2010, and Chair of the IEEE
vol. 1, Jun. 2005, pp. 918–923.
Taipei Section in 2009 – 2010. He is the Distinguished Lecturer of the
[12] S. M. Hosni, H. A. Shedeed, M. S. Mabrouk, and M. F. Tolba, ‘‘EEG-
IEEE CAS Society, from 2003 to 2005, and CIS Society, from 2015 to
EOG based virtual keyboard: Toward hybrid brain computer interface,’’
Neuroinformatics, vol. 17, no. 3, pp. 323–341, Jul. 2018. 2017. He served as the Deputy Editor-in-Chief for the IEEE TRANSACTIONS
ON CIRCUITS AND SYSTEMS-II, in 2006 – 2008. Prof. Lin was the Program Chair
[13] S. N. Abdulkader, A. Atia, and M.-S. M. Mostafa, ‘‘Brain computer
interfacing: Applications and challenges,’’ Egyptian Inform. J., vol. 16, of the IEEE International Conference on Systems, Man, and Cybernetics
pp. 213–230, Jul. 2015. in 2005 and General Chair of 2011 IEEE International Conference on Fuzzy
[14] K.-S. Hong and M. J. Khan, ‘‘Hybrid brain-computer interface techniques Systems. He is the coauthor of Neural Fuzzy Systems (Prentice-Hall), and
for improved classification accuracy and increased number of commands: the author of Neural Fuzzy Control Systems with Structure and Parameter
A review,’’ Frontiers Neurorobot., vol. 11, p. 35, Jul. 2017. Learning (World Scientific). He has published more than 200 journal papers
[15] J. Jiang, Z. Zhou, E. Yin, Y. Yu, and D. Hu, ‘‘Hybrid brain-computer (Total Citation: 20,155, H-index: 53, i10-index: 373) in the areas of neu-
interface (BCI) based on the EEG and EOG signals,’’ Biomed. Mater. Eng., ral networks, fuzzy systems, multimedia hardware/software, and cognitive
vol. 24, no. 6, pp. 2919–2925, 2014. neuro-engineering, including approximately 101 IEEE journal papers.
[16] S. Gamm, R. Haeb-Umbach, and D. Langmann, ‘‘The development of
a command-based speech interface for a telephone answering machine,’’
Speech Commun., vol. 23, nos. 1–2, pp. 161–171, Oct. 1997.
[17] C. A. F. Jorge, A. C. de Abreu Mól, P. Claudio, A. Mauricio, and
D. V. Nomiya, ‘‘Human-system interface based on speech recognition: JUNG-TAI KING received the B.S. degree in psychology from National
Application to a virtual nuclear power plant control desk,’’ Prog. Nucl. Cheng-Chi University in 1998, the M.S. degree in criminology from National
Energy, vol. 52, no. 4, pp. 379–386, May 2010. Chung-Cheng University in 2001, and the Ph.D. degree in neuroscience
[18] S. Funck, ‘‘Video-based handsign recognition for intuitive human- from National Yang-Ming University (NCTU) in 2010. He is currently an
computer-interaction,’’ presented at the Proc. 24th DAGM Symp. Pattern Assistant Research Fellow with Brain Research Center, NCTU, Taiwan. His
Recognit., 2002. research interests include psychophysiology, cognitive, and social neuro-
[19] S. Mitra and T. Acharya, ‘‘Gesture recognition: A survey,’’ IEEE Trans. science and neuro-marketing.
Syst., Man, Cybern. C, Appl. Rev., vol. 37, no. 3, pp. 311–324, May 2007.
[20] F. Loewenich and F. Maire, ‘‘Motion-tracking and speech recognition
for hands-free mouse-pointer manipulation,’’ in Speech Recognition,
F. Mihelic and J. Zibert, Eds. Rijeka, Croatia: InTech, pp. 427–434, 2008.
[21] D. G. Evans, R. Drew, and P. Blenkhorn, ‘‘Controlling mouse pointer PRIYANKA BHARADWAJ received the B.S. degree from Visvesvaraya
position using an infrared head-operated joystick,’’ IEEE Trans. Rehabil. Technological University, Belgaum, Karnataka, India, in 2012, and master’s
Eng., vol. 8, no. 1, pp. 107–117, Mar. 2000. degree from the State University of New York, USA, in 2014. Additionally,
[22] Y. Yang, J. Wiart, and I. Bloch, ‘‘Towards next generation human-computer she was a Software Engineer in the USA and Singapore before joining
interaction–brain-computer interfaces: Applications and challenges,’’ in UTS as a Visiting Fellow. Her research interests include signal processing,
Proc. 1st Int. Symp. Chin. CHI (Chin. CHI), 2013, pp. 1–3. machine learning, and data analytics.

VOLUME 7, 2019 96175

 C.-T. Lin et al.: EOG-Based Eye Movement Classification and Application on HCI Baseball Game

CHIH-HAO CHEN received the master’s degree from Institute of Imag- coauthored more than 60 papers in top journals and prestigious conference
ing and Biomedical Photonics, National Chiao Tung University, Hsinchu, proceedings. He was a recipient of the National Natural Science Young Foun-
Taiwan. His research interests include brain computer interface, machine dation of China in 2013. He was awarded as High-Level Talent (Six Talent
learning, and signal processing. Peak) of Jiangsu Province in 2016, and a Middle-aged and Young Academic
Leaders (Qing Lan Project) of Jiangsu Province in 2019. Dr. Ding currently
serves on the Editorial Advisory Board for Knowledge-Based Systems and
AKSHANSH GUPTA received the master’s and Ph.D. degrees from the
Editorial Board of Information Fusion. He serves/served as an Associate
School of Computer and Systems Sciences, JNU, in 2010 and 2015, respec-
Editor for several prestigious journals, including the IEEE TRANSACTIONS
tively. He is currently a Postdoctoral Research Fellow with the School
ON FUZZY SYSTEMS, Information Sciences, Swarm and Evolutionary Com-
of Computational and Integrative Sciences, Jawaharlal Nehru University
putation, IEEE ACCESS and Journal of Intelligent and Fuzzy Systems,
(JNU), New Delhi, India. His research interests include signal processing,
Co-Editor-in-Chief of Journal of Artificial Intelligence and Systems, as well
brain-computer interface, cognitive science, and healthcare.
as the leading guest editor in several international journals. He serves/served
as a program committee member for several international conferences and
WEIPING DING received the M.S. degree in software engineering from workshops.
Soochow University, Suzhou, China, in 2005, and the Ph.D. degree in com-
puter application from Nanjing University of Aeronautics and Astronautics,
Nanjing, China, in 2013. His current research interests include granular com-
puting, data mining, machine learning, and their applications in medicine. MUKESH PRASAD received the Ph.D. degree in computer science from
He was a Visiting Researcher with the Department of Mathematics and Com- National Chiao Tung University (NCTU), Hsinchu, Taiwan, in 2015 and
puter Science, University of Lethbridge, Alberta, Canada, in 2011. In 2014, master’s degree in computer application from Jawaharlal Nehru Univer-
he was a Postdoctoral Researcher with the Brain Research Center, National sity, New Delhi, India, in 2009. He was a Principle Engineer (Research
Chiao Tung University (NCTU) with Professor Chin-Teng Lin, Hsinchu, and Development) with Taiwan Semiconductor Manufacturing Company,
Taiwan. In 2016, he was a Visiting Scholar with National University of Hsinchu, Taiwan. He is currently a Lecturer with the School of Computer
Singapore (NUS), Singapore. From 2017 to 2018, he was a Visiting Professor Science, Faculty of Engineering and Information Technology, University of
with University of Technology Sydney (UTS), Ultimo, NSW, Australia. He is Technology Sydney, Australia. He has published 35 peer reviewed journals
currently an Associate Professor with the School of Information Science and and 40 conference articles. His current research interests include machine
Technology, Nantong University, Nantong, Jiangsu, China. He is a Senior learning, data analytics, pattern recognition, fuzzy systems, neural networks,
Member of CCF. He is Chair of Task Force on Granular Data Mining for artificial intelligence, and brain computer interface.
Big Data, the IEEE Computational Intelligence Society. He has authored or

96176 VOLUME 7, 2019