See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/342957549

GRAYSCALE IMAGE ENHANCEMENT FOR ENHANCING FEATURES DETECTION

IN MARKER-LESS AUGMENTED REALITY TECHNOLOGY

Article  in   Journal of Theoretical and Applied Information Technology · July 2020

CITATIONS READS

0 212

8 authors, including:

Devi Willieam Anggara Aris Rahmansyah

Universiti Teknologi Malaysia Telkom University
3 PUBLICATIONS   1 CITATION    5 PUBLICATIONS   0 CITATIONS   

SEE PROFILE SEE PROFILE

Mohd Shafry Mohd Rahim

Universiti Teknologi Malaysia
144 PUBLICATIONS   990 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

THE INFLUENCE OF POPULAR CULTURE ON ASTRAJINGGA COSTUME CREATIVITY View project

A new framework for pigment spots extraction on the iris surface View project

All content following this page was uploaded by Devi Willieam Anggara on 15 July 2020.

The user has requested enhancement of the downloaded file.

 Journal of Theoretical and Applied Information Technology
15th July 2020. Vol.98. No 13
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

GRAYSCALE IMAGE ENHANCEMENT FOR ENHANCING

FEATURES DETECTION IN MARKER-LESS AUGMENTED
REALITY TECHNOLOGY
1
DEVI WILLIEAM ANGGARA, 2MOHD SHAFRY MOHD RAHIM, 3AJUNE WANIS ISMAIL,
4
RUNIK MACHFIROH, 5 ARIF BUDIMAN, 6ARIS RAHMANSYAH,
7
DAHLIYUSMANTO, 8NOOR AZEAN ATAN
1
School of Computing, Faculty of Engineering, Universiti Teknologi Malaysia, Malaysia
2, 3
UTM-IRDA Digital Media Centre, Media and Game Innovation of Excellence (MaGICX), Institute of
Human Centered Engineering (iHumEn), Universiti Teknologi Malaysia, Malaysia
4,5,6
Design Communication Visual, Creative Industries School, Telkom University, Indonesia
7
Department of Electrical Engineering, Universitas Riau, Indonesia
8
Faculty of Social Sciences and Humanities (FSSH), Universiti Teknologi Malaysia, Malaysia
E-mail: 1wadevi@graduate.utm.my, 2shafry@utm.my, 3ajune@utm.my
4
runikmachfiroh@telkomuniversity.ac.id, 5ariefiink@telkomunivercity.ac.id,
6
arisrahmansyah@telkomuniversity.ac.id, 7 dahliyusmanto@lecturer.unri.ac.id, 8azean@utm.my

ABSTRACT

Tracking is a fundamental task in Augmented Reality (AR) technology which requires robust real-time to
properly adjust real and virtual objects in a single alignment, so that, both objects appear to coexist in the
same world. Marker-less tracking has been explored to overcome the limitations of conventional marker-
based tracking in AR. By capturing real surroundings to produce the features, the marker-less tracking will
recognize these features to overlay the virtual objects on the top of the captured features. The features have
been tracked in real-time by the display device, based on the real environment. Therefore, this article aimed
to explain the features detection using Features Accelerated Segment Test (FAST) to detect corner features.
Related works were reviewed and the features extraction for AR framework using Grayscale Image
Generation (GIG) were presented. In addition, to enhance details of grayscale images, a comprehensive study
was performed on the three techniques of Contrast Enhancement (CE), namely, Colormap, HE and CLAHE
to determine the best method for robust features detection. The findings showed Colormap to be the best
technique, compared to HE and CLAHE, in terms of noise, the accuracy of the corner, distributed histogram
and amount of features.
Keywords: Augmented Reality, Contrast Enhancement, Template Matching, FAST Corner Detector, GIG

1. INTRODUCTION accurate and robust tracking by camera [5]. Fiducial

markers basically use frames or other landmarks to
Augmented Reality (AR) is a technology which recognize an object and pose estimation of features
blends virtual contents into the real environment [1]. for displaying virtual objects.
In other words, it displays digital data in the real
world. In addition, a variety of data can be displayed, In marker-less tracking, Natural Feature Tracking
such as image, text, audio and video [2]. Marker- (NFT) approaches, such as SIFT, SURF, Ferns and
based tracking and marker-less tracking are two FAST use natural features, for example, corners,
tracking systems that are commonly used in AR [3]. edges, and blobs that exist in the real world [6]. NFT
Marker-based tracking uses a black and white requires a vision-based tracking technique which
marker, for example, ARToolkit and a fiducial applies image processing to use features on the
marker, including ARTag or ARToolkitplus [4]. A captured images to calculate the camera’s pose [6].
fiducial marker is a point of reference of the real The images become unseen or hidden from user
location. A marker is designed with the aim of being when it uses natural features as a marker. Users

2671
 Journal of Theoretical and Applied Information Technology
15th July 2020. Vol.98. No 13
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

cannot see the features through their naked eyes. the marker-less tracking is further divided into three
This condition has turned this features-based categories, edge-based, keypoint-based and interest
tracking to become marker-less. Besides features- point-based. Edge-based tracking uses edges as
based tracking, the location-based tracking is also features or known as points sampled. Interest point-
commonly known as marker-less tracking. It uses the based tracking uses keyframe from camera pose,
GPS to register the 3D content. With marker-less while keypoint-based tracking uses natural features,
tracking, user does not need to worry about the such as corners. Keypoint-based tracking is also
location of the virtual objects. It does not require an known as features-based tracking. Based on the
anchor to the real world [7]. principles of features tracking, there are three
approaches highlighted which can affect tracking.
Markers are defined as a sign or image that can be
detected by a camera, using pattern recognition, First is the black and white ratio on fiducial
image processing, and computer vision techniques marker, and this approach can be considered as
[8]. According to [9], there are five types of marker an easy detection in various lighting conditions. This
in AR, namely, template marker, circular marker, ratio can affect a detection in marker-based tracking
imperceptible marker, features marker, and infrared [11]. Along with black and white markers, the use
marker. Feature is a natural marker type, which uses of fiducial markers is a common practice in AR,
natural features. Imperceptible marker and infrared where it can be divided into two types, the template
marker are sensor marker types. Template marker marker and the circular marker. Meanwhile in the
and circular marker are fiducial marker types [9]. grayscale approach, such as FAST, ORB, and
Features marker typically uses templates or features BRISK, the intensity pixel of grayscale images is
matching for its recognition. Implementations of used to detect features. Grayscale Image Generation
features marker have an advantage in AR (GIG) method is used in marker-less AR framework
application, because it can work without changing [16-17], where it enables the intensity of the
the environment in an existing real environment [9]. grayscale image to affect features detection. The
The marker must have a different feature to features marker, also known as natural marker is
distinguish one another. Some features tracking uses derived from the grayscale approach. The
four corners of a square image as features to combination of marker-based and marker-less
recognize the images in AR. tracking is known as hybrid tracking, which usually
uses the sensor output as a marker [14], for example,
Based on Figure 1, there are five types of trackers, imperceptible marker and infrared marker. Edge-
namely, magnetic tracker, inertial tracker, optical based and interest point-based tracking use the last
tracker, ultrasonic tracker and mechanical tracker calculated pose as an estimate for the current pose
[10]. This study focused only on the optical tracker. (recursive), while the keypoint-based tracking uses
A tracker is a device used to track or capture images. the current frame to estimate the pose (non-
There are two divisions in AR, namely, marker- recursive) [15].
based tracking and marker-less tracking. In general,

Figure 1. Tracking system based on type of trackers, features tracking techniques, and type of markers

2672
 Journal of Theoretical and Applied Information Technology
15th July 2020. Vol.98. No 13
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

Features detection mainly aims to produce removal [26] to design high quality markers.
features for robust object detection. With certain Tracking a marker in AR has three problems,
features in the images, hidden information can be namely, inter-marker confusion, true detection, and
added, such as steganography techniques [16] and false detection. The severity of these problems
watermarking techniques [17]. During features depend on the available extraction features and
extraction process, the process cannot extract matching features [27]. Certain images do not have
features precisely, due to the noise and image many detectable features, and according to [28],
quality. The more features can be detected much contrast value, brightness and colour image
robust the markers can be tracked and recognized manipulation can affect the quality of image marker
[18]. Therefore, to identify and recognize the object in AR.
or marker on a digital image, image processing is
necessary [19]. Sometimes, the marker used does not Features detection using natural features have
have enough features, such as the lack of a good the same problems. As such, good extraction
texture and noise and lighting problems, especially features and matching features are needed to resolve
if it is directly captured using a camera. This makes the inter-marker confusion, true detection, and false
the features difficult to detect or encounter detection problems. To detect the natural features,
mismatches [18]. two software libraries have been compared between
ARToolkit and Vuforia. Based on the findings in
To improve features tracking, such as problems [18], Vuforia detects fewer features, compared to
that occur in [18], the tracking accuracy will be ARToolkit, and the amount of features detected can
increased with the more features which have been increase tracking accuracy and robustness of the
detected. In this research, GIG was used as the marker recognition.
method to extract features from a natural marker in
AR. According to [20] and [21] features extraction Based on related works, researchers have
in AR can use GIG method to convert the RGB attempted to produce features detection in template
images into grayscale images before producing the matching better and to recognize features in objects
features. Converting RGB images into grayscale more quickly and accurately. However, our study
images can cause a loss of some details [18-19]. A focused only on natural features marker for marker-
method was proposed in this study to improve the less AR.
details on grayscale images for a precise detection of
3. FUNDAMENTAL OF TRACKING
the features. It was revealed that, the details of
FEATURES
grayscale images could be enhanced using Contrast
Enhancement (CE) method, such as CLAHE, HE
There are five types of tracking system as
and Colormap [22]. Experiments were carried out to
mentioned above. The magnetic tracker uses the
compare the three techniques in terms of features
magnetic field as a source to estimate orientation,
tracking, noise, the accuracy of the corner,
while an ultrasonic tracker, also known as an
distributed histogram and the amount of features.
acoustic tracking system, uses sound wave to
2. RELATED WORKS determine the position and orientation. Inertial
tracker uses Inertial measurement units (IMU) with
Several researchers have studied the reliability of a combination of multi-sensors, such as
marker recognition. A previous work by [23] accelerometer, gyroscope, and magnetometer to get
developed a tracking technique to detect the features orientation. Mechanical tracking systems rely on
of fiducial marker underwater, using generated physical connections between targets and fixed
synthetic images for robust detection in these reference points. The last fifth type of the tracking
environments, while [24] developed a new marker system, namely, optical tracker, also known as
detection known as mono-spectrum marker which Optical Tracking System (OTS), depends on two
can accurately detect blurred and defocused images components, light source and optical sensor [10].
in real time. A black and white fiducial marker which Virtual Reality (VR) and AR are included in the
can positively affect detection is used to increase the optical trackers category [29].
robustness of the marker recognition[11]. Designing
highly reliable fiducial markers can solve interest Corner is a characteristic of the image for a robust
point detection and matching problems, as agreed by detection [30], and it carries an extensive data about
[25]. Fiducial Marker Optimization (FMO) method picture composition and invariant characteristics
has been developed using image processing under many geometric transformations into pictures
techniques, which are edge sharpness and noise [31]. The corner detector has been widely used as a

2673
 Journal of Theoretical and Applied Information Technology
15th July 2020. Vol.98. No 13
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

feature point detectors, because each corner in the indicates a little dynamic range which corresponds
image can distinguish information from the image to an image having a low contrast [38] and Vuforia
[32]. Features detection algorithms try to identify transforms the images into grayscale. When the
features directly, based on the intensity patterns in images do not have enough contrast, the marker will
the marker. The selection of features detection can produce very less or small amount of features, and
affect the overall system performance [33]. Some the histogram looks narrow and spiky [39]. Non-
techniques, such as SUSAN (Smallest Univalue maximum suppression is used for smaller patterns to
Segment Assimilating Nucleus), Harris corner, and better preserve the locality constraint of a corner
FAST corner (Features from Accelerated Segment [40]. Distinctiveness, repeatability, efficiency, and
Test) are used for tracking corners. Among these, invariance tests are the next experiment to be carried
FAST corner is the most stable [8]. FAST corner out for real-time tracking to improve the features
uses grayscale images to find features [34]. The ratio extraction in marker-less AR. This study proposed
of darker and brighter areas in a grayscale image, is CE method to improve grayscale images for
very important to find a candidate corner using enhancing features detection in marker-less AR.
FAST corner detector. FAST corner with rotated
4. CONTRAST ENHACEMENT METHOD
BRIEF or known as ORB is also suitable to be used
in AR mobile device, because it is fast, with no
According to [22], CE method can enhance low
heavy computational load needed [35] and mobile
contrast grayscale images to improve the details of
phones and tablets are increasingly becoming
images. An experiment has been conducted before,
common media for AR, which provide a significant
using Image Adjust Intensity Values or Colormap
growth, because they are oriented towards consumer
[41] to enhance old pictures in a museum as marker
experiences [36].
in AR. In this section, the three types of CE method
which can be used to increase the details of grayscale
There are eight important qualities which must
images, are described. The following subsections
be had for ideal features, namely, distinctiveness,
explain the three types of CE method, namely,
locality, quantity, accuracy, efficiency, repeatability,
Histogram Equalization, Contrast Limited Adaptive
invariance, and robustness [37]. Distinctiveness is
Histogram Equalization and Colormap.
the condition where detected features must have
variations which can be used to differentiate each 6.1 Histogram Equalization
feature and match. Locality means, the features
should be local to reduce the chances of getting Histogram equalization is the act of modifying the
occluded and allow simple estimations of geometric intensity distribution that adjusts the contrast of an
and photometric deformations. Quantity is the total image-based on a histogram to provide a linear trend
amount of detected features which must be enough to the cumulative probability function associated
to reflect the object. Accuracy is the detected with the image to produce an image with an even
features which must also be located accurately from histogram. The even histogram will increase the
the various locations of pixels, shapes, and scales. details of the image achieved, as agreed by [42]. The
Efficiency means, the features detected do not take histogram of an image is a discrete function formed
much time to be identified and are suitable for real- by counting the number of pixels in an image that has
time applications. Repeatability is the feature a certain grey value, so that, the histogram of the
detected in the same object, which must have output image approximately matches the specified
similarity features when tested in different display histogram. Even distribution of regional histograms
conditions. Invariance means, each feature detected in images can increase the visibility of local details
is always matched if the image is transformed. in images [22].
Robustness is the deformation resistance from image
noise, discretization effects, artifact compression, 6.2 Contrast Limited Adaptive Histogram
blur, etc.
CLAHE is widely used in medical image
According to these ideal features, improvements processing. The CLAHE algorithm divides the
were made on the details of grayscale images for image into rectangular areas, applying several
thresholds and equality locally in each region. After
benchmarking purposes, in terms of noise for
robustness, accuracy, quantity, and distributed setting a threshold for the grey level, as in the above
histogram. The shape of the histogram of an image event, this threshold is truncated to minimize
gives information about the possibility for a contrast saturation, followed by a uniform and recursive
enhancement. A histogram of a narrow shape redistribution throughout the local histogram [43].

2674
 Journal of Theoretical and Applied Information Technology
15th July 2020. Vol.98. No 13
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

6.3 Colormap result of the original image. In the second and third
steps, the acquisition of the image applied image
Colormap, also known as the Image Adjust
enhancement technique, using each of the three types
Intensity Values, is a colour transformation, from the
of CE method to get the results, which relied on
colours of one source image to another target image
image noise, accuracy corner, distributed histogram
[44]. Colormap is also known as an algorithm which
and features amount.
can change the colour of an image or an algorithm
that creates and produces an image mapping.
Colormap is a merger of 3 matrices (m-by-3) real
numbers between 0.0 and 1.0. The RGB colour in
each row determines the colour transformed from the
image having the RGB to grayscale image. Colour
kith is defined in kith lines, with the specific
intensity of the colours in red, green and blue, map
(k, :) = [r (k) g (k) b (k)]) [44].

Image enhancement techniques are used to

improve an image [45]. Intensity adjustment is an
image enhancement technique that can be used to
map and modify image values using the imadjust
method in MATLAB. This is a method to increase
the contrast of the output image, where the range of
intensity values can be defined in the output image
[46]. Colormap can strengthen the quality of images
which will improve and increase the contrast in low-
contrast grayscale image by remapping data values.
By definition, 1 percent of data is saturated with a
low and high output data strength [22].
5. DATASET Figure 3. Research Framework

An appropriate dataset is needed to get an accurate 7. TOOLS AND HARDWARE

corner position result. This experiment used a
grayscale dataset to prove that, enhancing details can The tools used to enhance grayscale images and
affect corner detection. Experiments were carried out develop AR, based on mobile platform (Android) in
by using a dataset from Guo Chengung, usually used our experiment were as follows:
to test detection of corner accuracy [47]. The dataset  MATLAB as image processing tools, using CE
from Guo Chengung is as shown in Figure 2. method and OpenCV to accelerate machine
perception for features tracking and matching.
 Unity 3D and Vuforia SDK as a media for
designing and building AR applications.
The quality of marker in Vuforia SDK was
compared with that of our code using OpenCV.

The hardware required for AR is as presented in

Table 1, which lists the standard requirement and
specification for RAM, Processor and VGA.
Figure 2. The dataset used for keypoint detection Table 1. Hardware Specification
6. RESEARCH FRAMEWORK Hardware Specification
Figure 3 shows the proposed research framework. Processor Intel Core i7 2670QM @ 2.20GHz
The original image is compared against the target
image in each CE method used. The original image RAM 8.00GB Dual-Channel DDR3 797MHz
was acquired, followed by the tracking of the
VGA 3071MB NVIDIA GeForce GT 555M
features using a FAST corner detector to get the

2675
 Journal of Theoretical and Applied Information Technology
15th July 2020. Vol.98. No 13
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

8. ENHANCE GRAYSCALE IMAGE USING 9. FEATURES DETECTION

CONTRAST ENHANCEMENT METHOD
Some AR SDK use corner as the features to
Grayscale image is a variant form of black and recognize the marker in AR, such as Vuforia, which
white. Grayscale is a scale of black and white in has applied FAST corner detection [48] and which
digital image technology. It measures the intensity of has been patented by Qualcomm [49]. To recognize
light, reflected on an area or point of surface area and objects on the camera, features on objects or markers
subsequently defines each pixel as a computer byte are stored first in the database as a template or
value. Grayscale image is included in the Histogram references. Following this, the features stored do the
Stretching (HS) category. The operations will cause matching with the object or marker tracked by the
the pixels to be rounded to the highest or lowest camera. If the features match with the marker, the
grayscale values, causing the image to lose some virtual object will be shown.
details [12]. To enhance the grayscale images in this
experiment, CE method comprising three techniques Based on the review on features tracking with the
stated in the previous section was used in this study. FAST corner technique, the ratio of the darker pixels
The marker quality was improved in each of the to the brighter pixels in grayscale image is very
three techniques used, which results were later important. FAST tracking comparing 16 pixels with
compared against each other. The three types of CE darker and brighter pixels to determine features
algorithm can be seen in Algorithm 1, Algorithm 2, corner and the intensity is very influential. FAST
and Algorithm 3. corner is ten times faster than the SIFT and SURF
algorithms and until now, it is still used as the fastest
Algorithm 1 Histogram Equalization (HE) detector [50]. Feature matching based on ORB
Input : Original image (Oriented FAST and Rotated BRIEF) is better than
Output : Enhance image with HE other feature matching methods, such as SIFT,
1: Read image SURF, KAZE, AKAZE, and BRISK [51]. ORB is
Image  imread(“Location image”) suitable for real-time recognition in mobile devices,
2: Grayscale Image based on accuracy and time [35]. Summary of a
Image  rgb2gray (Image) features extraction method for feature detectors and
3: Enhance image using HE technique descriptors is as presented in Table 2. The intensity
Image histeq(Image) of image pixel is the foundation of the FAST,
Algorithm 1. Histogram Equalization Algorithm BRIEF, ORB, BRISK and FREAK methods, which
all use intensity pixel of candidate corner as the
Algorithm 2 Contrast-limited Adaptive centroid to measure orientation [52]. It shows that,
Histogram Equalization (CLAHE) the intensity of images strongly influences the
Input : Original image detection of features. Different from Harris corner,
Output : Enhance dataset with CLAHE SIFT and SURF use histograms of gradient
1: Read image computations to find the candidate corner [52].
Image  imread(“Location image”)
2: Grayscale Image Table 2. Features Extraction Method [53].
Image  rgb2gray (Image) Foundation
3: Enhance image using CLAHE technique Algorithm Detector Descriptor of the
Image  adapthisteq (Image) Method
Algorithm 2. CLAHE Algorithm Harris Harris - Gradient
Corner Corner
Algorithm 3 Color Map (Image Adjust Intensity SIFT DoG SURF Gradient
Values) SURF Fast SURF Gradient
Hessian
Input: Original image
FAST FAST - Intensity
Output: Enhance dataset with Color Map
BRIEF - BRIEF Intensity
1: Read image ORB oFAST rBRIEF Intensity
Image  imread(“Location image”) BRISK FAST BRISK Intensity
2: Grayscale Image FREAK - FREAK Intensity
Image  rgb2gray (Image)
3: Enhance using Color Map technique FAST was developed by Edward Rosten and Tom
Image  imadjust (Image) Drummond [54], known for its high-speed algorithm
Algorithm 3. Colormap Algorithm for corner detection [55]. This algorithm detects the

2676
 Journal of Theoretical and Applied Information Technology
15th July 2020. Vol.98. No 13
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

pixel value on the circle formed around the candidate 10. RESULTS AND DISCUSSION
point, as shown in Figure 4. Neighbours of pixel
The dataset from Guo Chengung [47] applied all
must have at least nine pixels to be identified as
the three techniques of CE method. Each technique
corner, all brighter or darker than the threshold.
was performed to get the best method for enhancing
Decision trees are trained to test several pixels to
the quality of marker and also to compare the marker
classify prospective pixels as interest points or
with the original images. Comparisons were made
otherwise.
based on image noise, accuracy corner, histogram
and features amount.
(1)
Noise comparison of the three techniques can be
seen in the following Figure 5.
S+ is the subset of pixels on the circle that are
brighter than p (by threshold) and S_is the subset of
pixels that are darker than p (by threshold). The point
of candidate p is a black point. The discretized
approximation of the circle around p is used by 16
grey points p [55]. Figure 4 shows how FAST
determines a candidate corner.
(a) (b) (c) (d)
Figure 5. Comparison of original image with grayscale
image obtained using the three techniques of CE Method

The results of enhancement on grayscale images

using CE method can be seen in Figure 5. The first
image is the original image (as in Figure 5 (a)), while
Figure 4. FAST Corner Technique Figure 5 (b) is the original grayscale image, using
FAST corner detector will get more threshold Colormap technique, which shows more details ,
responses in highly contrasted areas and even more compared to Figure 5 (a). The image that uses HE
in the homogeneous region of the image to count the technique (as in Figure 5 (c)) can be seen with noise
total amount of corners and matching corners. To in the image , and Figure 5 (d) shows the image using
detect corners using FAST with a fixed threshold τ = CLAHE technique. The details of image that uses the
25 [56]. This study also uses Vuforia to measure the Colormap technique are much better, compared to
quality of the marker and to compare with FAST those using the other techniques, namely HE and
corner tracking on OpenCV. The FAST algorithm is CLAHE, which recorded more noise.
defined in Algorithm 4.
For accuracy of the corner features, FAST corner
Algorithm 4 FAST Corner (FAST) detector algorithm was used for features tracking in
Input: Original & Enhance dataset using CE the markers, which can be seen in Table 3.
method Table 3. FAST corner tracking for features tracking
Output: Features detection
1: Read image Marker Tracking Corner Type
Image
Image  imread (“Location image”)
2: Initiate FAST object with default values
fast  cv2.FastFeatureDetector_create
Original
(threshold=25) Image
3: Find keypoints
kp  fast.detect(img,None)
4: Draw keypoints
lmg  cv2.drawKeypoints(img, kp,
None,color=(255,0,0))
5: Show image Histogram
plt.imshow(img2) Equalization

Algorithm 4. FAST Corner Algorithm

2677
 Journal of Theoretical and Applied Information Technology
15th July 2020. Vol.98. No 13
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

Figure 8 improves the corner accuracy, comparedto

Contrast- the original image shown in Figure 7.
limited
Adaptive
Histogram
Equalization
/ CLAHE

Colormap

Figure 8. Image marker using Colormap

Table 3 shows the images using HE technique and
The high speed test of FAST corner can be
the CLAHE technique, where more mismatched
performed by eliminating a candidate point which
positions of the corner are found. Obviously,
does not have the values of intensities pixel 3 by 4
Colormap is the best technique to detect the position
greater or less than the candidate point, where the
of the corner.
number of intensities of pixel is 1,5,9 and 13 [54].

To prove the accuracy of features detection,

features in Vuforia were also compared, using FAST
corner which has been improved to measure features
of corners.

Figure 6. Testing using Vuforia SDK

Marker rating using Vuforia SDK can be seen in
Figure 6. The results showed that, the marker rating
increased using the colormap technique. The Figure 9. Values of grayscale pixel before using
technique doubles the original image rating in Colormap
Vuforia SDK database from two stars to four stars.
Vuforia has a rating range between 0 to 5 stars. 1 and
2 stars indicate a low rating, while 4 or 5 stars
indicate the best results.

Figure 10. Values of grayscale pixel using Colormap

Figure 9 presents the original image and Figure 10

shows the results after the Colormap technique has
Figure 7. Image marker before using Colormap been applied. It can be seen in Figure 9 that, the point
marked by the arrow detected was not a corner,
The Colormap technique enables the number of
because there were 4 pixels greater than the
corners previously not detectable to be seen. The
candidate pixel. Figure 10 shows that, the point
marker which uses the Colormap technique as in
marked by the arrow is a corner, because 3 out of 4
pixels are greater than the candidate pixel.

2678
 Journal of Theoretical and Applied Information Technology
15th July 2020. Vol.98. No 13
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

Marker quality can also be measured through a the quality of the image details. Table 4 shows a
histogram on the image, as listed in the Vuforia histogram of each image using CE method and the
website [49]. According to [42], the image details original image. The evenly distributed histogram can
can be seen from the histogram taken. be seen on the type of image using colormap
The more evenly distributed the histogram, the better technique.

Table 4. Histogram images before and after using Contrast Enhancement Method

Marker Tracking Corner Type Image

Original Image

Histogram Equalization

Contrast-limited
Adaptive Histogram
Equalization / CLAHE

Colormap

2679
 Journal of Theoretical and Applied Information Technology
15th July 2020. Vol.98. No 13
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

In the next experiment carried out to count the The amounts of features using non maximum
features amount, a problem was encountered in suppression and without non maximum suppression
detecting multiple corner points on different are as shown in Table 5, and it can be seen that,
locations. Non maximum suppression remove image with the colormap technique detects more
corners which have an adjacent corner [54]. corners.

Table 5. Tracking corner using FAST corner Algorithm

Non maximum Without non maximum
Marker Type of marker
suppression suppression

Original image 49 Corner 337 Corner

Using Colormap 66 Corner 492 Corner

Upon completion of the features tracking and the The brute-force-matcher approach was used for
corner counting process, feature matching was the feature matching [57]. It was used to match
carried out, aimed to test whether each corner corners, before and after enhancement with
detected matches what is in the initial database or threshold=25. Figure 11 shows that, the features
initial tracking. matching using colormap technique are better.

Figure 11. Matching original image & enhanced image using Colormap

2680
 Journal of Theoretical and Applied Information Technology
15th July 2020. Vol.98. No 13
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

11. CONCLUSION & FUTURE WORK “Magic Boosed” an Elementary School

Geometry Textbook with Marker-Based
Enhanced grayscale images help features detector
Augmented Reality”, TELKOMNIKA
to detect features corner better. CE method can be
(Telecommunication Computing Electronics
used to enhance grayscale images, and out of the
and Control). 2019, Vol.17, p. 1242.
three techniques used, Colormap technique was
[4] Devedzic G., G. Klinker, V. Lukovic,“An
found to be the best. The results obtained using HE
Interactive Augmented Reality Platform for Cad
and CLAHE from Guo Chenguang dataset showed
Education”, International Conference of
many mismatches of the corner positions, while
Production Engeneering - ICPE, 2013.
more noise was recorded. These mismatches proved
[5] Jumarlis M., M. Mirfan, “Implementation of
that CLAHE and HE are not suitable to be used for
Markerless Augmented Reality Technology
enhancing the details of grayscale images to improve
Based on Android to Introduction Lontara in
the detection of features corner, based on FAST
Marine Society”, IOP Conference Series: Earth
corner detector. The accuracy of corner using image
and Environmental Science. 2018, Vol.156, p.
adjust intensity values or Colormap was found to be
012017.
better, compared to that using HE and CLAHE, with
[6] Wagner D., G. Reitmayr, A. Mulloni, T.
more evenly distributed histograms being produced.
Drummond, D. Schmalstieg, “Pose Tracking
The amount of features also increased using
from Natural Features on Mobile Phones”,
Colormap, and the matching was found to be a lot
International Symposium on Mixed and
better.
Augmented Reality. 2008, pp. 125–134.
The matching keypoint is usually used as ID
[7] Ismail A.W., M. Billinghurst, M.S. Sunar,
marker in AR. Colormap method is suitable for
“Vision-Based Technique and Issues for
improving details of images to find keypoint or
Multimodal Interaction in Augmented Reality”,
features corner using FAST corner tracking. The
ACM International Conference Proceeding
overall results showed Colormap to be better than
Series, 2015, pp. 75–82.
HE and CLAHE, in terms of noise, the accuracy of
[8] Siltanen S., “Theory and Applications of
the corner, the distributed histogram and the amount
Marker-Based Augmented Reality”, VTT
of features.
Science Series 3, 2012.
As for future works, this method can be used
[9] P.Chauhan V., M.M. Kayasth, “Augmented
during features extraction before the features are
Reality Markers, It’s Different Types, Criterion
stored in databases. In addition, this method can
for Best Fiducially Marker and Necessary
enhance features extraction in AR based template
Requirements to Selecting Application Oriented
matching.
Markers.”, International Journal of Engineering
Sciences & Research Technology, 2015,Vol.4,
12. ACKNOWLEDGEMENT
pp. 550–559.
[10] Enayati N., E. De Momi, G. Ferrigno, “A
This work was sponsored under a research grant
Quaternion-Based Unscented Kalman Filter for
by Universiti Teknologi Malaysia, grant numbers:
Robust Optical/Inertial Motion Tracking in
R.J130000.7308.4B429, Q.J130000.3051.01M37
Computer-Assisted Surgery”, IEEE
and Q.J130000.3008.01M88, in collaboration with
Transactions on Instrumentation and
Telkom University, Indonesia, grant number:
Measurement, 2015,Vol.64, pp. 2291–2301.
KWR.0039/LIT4/LIT/2019,.
[11] Khan D., S. Ullah, I. Rabbi, “Optimality of
Black to White Ratio and Information
REFERENCES:
Complexity for Robust Marker Recognition”,
2014 4th International Conference on Image
[1] Azuma R.T., “Making Augmented Reality a
Processing Theory, Tools and Applications,
Reality”, Optics InfoBase Conference Papers.
IPTA, 2014, pp. 1–6.
2017.
[12] Mcandrew A., “An Introduction to Digital
[2] Sharifara A., M.S.M. Rahim, M. Bashardoost,
Image Processing with Matlab Notes for
“A Novel Approach to Enhance Robustness in
SCM2511 Image Processing 1”,School of
Digital Image Watermarking using Multiple
Computer Science and Mathematics, Victoria
Bit-Planes of Intermediate Significant Bits”,
University of Technology, 2004.
Proceedings - International Conference on
[13] Saravanan C., “Color Image to Grayscale Image
Informatics and Creative Multimedia, ICICM,
Conversion”,2010 Second International
2013, pp. 22–27.
Conference on Computer Engineering and
[3] Andrea R., S. Lailiyah, F. Agus, R. Ramadiani,

2681
 Journal of Theoretical and Applied Information Technology
15th July 2020. Vol.98. No 13
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

Applications, 2010,Vol.2, pp. 196–199. Markers: An Optimization Tool for

[14] Amin D., S. Govilkar,“Comparative Study of ARToolKit”, IEEE Access, 2018, Vol.6, pp.
Augmented Reality Sdk’s”,International 22421–22433.
Journal on Computational Science & [27] Kurniawan F., M.S.M. Rahim, D. Daman, A.
Applications, 2015,Vol.5, pp. 11–26. Rehman, D. Mohamad, S.M. Shamsuddin,
[15] Lima J., F. Simões, “Model Based 3D Tracking “Region-Based Touched Character
Techniques for Markerless Augmented Segmentation in Handwritten
Reality”,SBC Journal on 3D Interactive Words”,International Journal of Innovative
Systems, 2010,Vol.1, pp. 37–47. Computing, Information and Control, 2011,
[16] Taha M.S., M.S.M. Rahim, S.A. Lafta, M.M. Vol.7, pp. 3107–3120.
Hashim, H.M. Alzuabidi, “Combination of [28] Mandarani P., N.A. Suhendri, “Pengaruh
Steganography and Cryptography: A Short Manipulasi Nilai Kontras, Kecerahan Dan
Survey”, IOP Conference Series: Materials Warna Gambar Terhadap Rating Image Marker
Science and Engineering, 2019, Vol.518, p. Pada Augmented Reality”, Jurnal Teknoif.
052003. 2018, Vol.6, pp. 120–127.
[17] Bashardoost M., M.S.M. Rahim, T. Saba, A. [29] Dorfmüller K., “An Optical Tracking System
Rehman, “Replacement Attack: A New Zero for VR/AR-Applications”, IEEE Access, 1999,
Text Watermarking Attack”, 3D Research, pp. 33–42.
2017, Vol.8, p. 8. [30] Rad A.E., M.S.M. Rahim, H. Kolivand, I. Bin
[18] Blanco-pons S., B. Carrión-ruiz, J.L. Lerma, Mat Amin, “Morphological Region-Based
“Augmented Reality Application Assessment Initial Contour Algorithm for Level Set
for Disseminating Rock Art”, 2019, pp. 10265– Methods in Image Segmentation”, Multimedia
10286. Tools and Applications. 2017,Vol.76, pp. 2185–
[19] Rakhmadi A., N.Z.S. Othman, A. Bade, M.S.M. 2201.
Rahim, I.M. Amin, “Connected Component [31] Han S., “An Improved Corner Detection
Labeling Using Components Neighbors-Scan Algorithm Based on Harris”, Chinese
Labeling Approach”,Journal of Computer Automation Congress (CAC), 2018, pp. 1575–
Science, 2010, Vol.6, pp. 1099–1107. 1580.
[20] Lanka S., C. Science, S. Ehsan, A. Ehsan, “A [32] Hedley M.,“Fast Corner Detection”, Elsevier-
Review of Research on Emerging Technologies Image and Vision Computing. 2006, Vol.16, pp.
of the Internet of Things and Augmented 75–87.
Reality”, International conference on I-SMAC [33] Liu Q., R. Li, H. Hu, D. Gu, “Extracting
(IoT in Social, Mobile, Analytics and Cloud) (I- Semantic Information from Visual Data: A
SMAC), 2017, pp. 770–774. Survey” Robotics, 2016, Vol.5.
[21] Parmar J.K., A. Desai, “Feature Extraction in [34] Wang C., “Real Time Non-Rigid Surface
Augmented Reality”,arXiv:191109177v1 Detection Based on Binary Robust Independent
[csGR], 2019. Elementary Features”,Revista Mexicana de
[22] Mittal N., “Automatic Contrast Enhancement of Trastornos Alimentarios, 2015, Vol.13, pp.
Low Contrast Images Using MATLAB”, 297–304.
International Journal of Advanced Research in [35] Li J., “Real-Time Image Recognition Algorithm
Computer Science, 2012,Vol.3, pp. 333–338. and System Design of Android Mobile
[23] Čejka J., F. Bruno, D. Skarlatos, F. Liarokapis, Devices”, International Journal of Computers
“Detecting Square Markers in Underwater and Applications, 2020, pp. 1–9.
Environments”, Remote Sensing, 2019. [36] Goh E.S., M.S. Sunar, A.W. Ismail, “3D Object
[24] Toyoura M., H. Aruga, M. Turk, X. Mao, Manipulation Techniques in Handheld Mobile
“Detecting Markers in Blurred and Defocused Augmented Reality Interface: A Review”, IEEE
Images”, Proceeding International Conference Access, 2019, Vol.7, pp. 40581–40601.
on Cyberworlds, CW, 2013, pp. 184–190. [37] Tuytelaars T., K. Mikolajczyk, “Local Invariant
[25] Khan D., S. Ullah, I. Rabbi, “Factors Affecting Feature Detectors: A Survey”, Foundations and
the Design and Tracking of ARToolKit Trends in Computer Graphics and Vision,
Markers”, Computer Standards and Interfaces, 2007,Vol.3, pp. 177–280.
2015,Vol.41, pp. 56–66. [38] Dadwal J., B. Sharma, “Image Processing
[26] Khan D., S. Ullah, D.M. Yan, I. Rabbi, P. Technique Used for Enhancement Image
Richard, T. Hoang, et al., “Robust Tracking Application Process in Electronics
Through the Design of High Quality Fiducial Engineering”, International Journal of

2682
 Journal of Theoretical and Applied Information Technology
15th July 2020. Vol.98. No 13
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

Advances in Scientific Research, 2015,Vol.1, [47] Chenguang G., L. Xianglong, Z. Linfeng, L.

pp. 356–358. Xiang, “Application A Fast and Accurate
[39] Rodriguez-Pardo C., S. Hernandez, M.Á. Corner Detector Based on Harris Algorithm”,
Patricio, A. Berlanga, J.M. Molina, “An International Symposium on Intelligent
Augmented Reality Application for Learning Information Technology, 2009, Vol.2, pp. 50–
Anatomy”, Springer International Publishing, 53.
2015, pp. 359–368. [48] Ratnasari A., E. Mulyanto, U.D. Nuswantoro,
[40] Mair E., G.D. Hager, D. Burschka, M. Suppa, F.I. Komputer, U.D. Nuswantoro, “Augmented
G. Hirzinger, “Adaptive and Generic Corner Reality Struktur Dan Fungsi Tumbuhan Sebagai
Detection Based on the Accelerated Segment Media Pembelajaran Berbasis Platform
Test”, Lecture Notes in Computer Science Android”, 2012, pp. 1–16.
(including subseries Lecture Notes in Artificial [49] Vuforia Website, " The histogram shows the
Intelligence and Lecture Notes in quality of images"
Bioinformatics), 2010, Vol.6312, pp. 183–196. See https://library.vuforia.com/articles/Solution
[41] Dahliyusmanto, M. Nizam, D.W. Anggara, E. /Optimizing-Target-Detection-and-Tracking-
Hamdani, B. Anto, Z. Laili, “Marker Quality Stability.html, 2020.
Improvement in Augmented Reality Using [50] Saipullah K., N.A. Ismail, A. Anuar,
Grey- Scale Method”, International Journal of “Comparison of Feature Extractors for Real-
Electrical, Energy and Power System Time Object Detection on Android
Engineering, 2019,Vol.2, pp. 17–23. Smartphone”, Journal of Theoretical and
[42] Onyedinma E., I. Onyenwe, H. Inyiama, Applied Information Technology, 2013, Vol.47,
“Performance Evaluation of Histogram pp. 135–142.
Equalization and Fuzzy Image Enhancement [51] Saleem Z., “A Comparative Analysis of SIFT ,
Techniques on Low Contrast Images”, SURF , KAZE , AKAZE , ORB , and BRISK”,
International Journal of Computer Science and International Conference on Computing,
Software Engineering (IJCSSE), 2019, Vol.8, Mathematics and Engineering Technologies
pp. 144–150. (iCoMET), 2018, pp. 1–10.
[43] da Rocha D.A., A.B.L. Barbosa, D.S. [52] Rublee E., V. Rabaud, K. Konolige, G. Bradski,
Guimarães, L.M. Gregório, L.H.N. Gomes, L. “ORB: An Efficient Alternative to SIFT or
da Silva Amorim, et al., “An Unsupervised SURF”, IEEE International Conference on
Approach to Improve Contrast and Computer Vision, IEEE, 2011. pp. 2564–2571.
Segmentation of Blood Vessels in Retinal [53] Lee H., S. Jeon, I. Yoon, J. Paik, “Recent
Images Using CLAHE, 2D Gabor Wavelet, and Advances in Feature Detectors and Descriptors :
Morphological Operations”, Research on A Survey”, IEIE Transactions on Smart
Biomedical Engineering, 2020, Vol.36, pp. 67– Processing and Computing, 2016,Vol.5, pp.
75. 153–163.
[44] Al J., H. Alhatamleh, Z. A., M. Khalil, [54] Rosten E., R. Porter, T. Drummond, “Faster and
“Creating a Color Map to Be Used to Convert a Better: A Machine Learning Approach to
Gray Image to Color Image”, International Corner Detection”, IEEE Transactions on
Journal of Computer Applications. 2016, Pattern Analysis and Machine Intelligence,
Vol.153, pp. 31–34. 2010,Vol.32, pp. 105–119.
[45] Lakshmanan R., M.S. Nair, M. Wilscy, R. [55] Rachmawati E., I. Supriana, M.L.
Tatavarti, “Automatic Contrast Enhancement Khodra,“FAST Corner Detection in Polygonal
for Low Contrast Images: A Comparison of Approximation of Shape”, International
Recent Histogram Based Techniques”, Conference on Science in Information
International Conference on Computer Science Technology: Theory and Application of IT for
and Information Technology, ICCSIT , 2008, Education, Industry and Society in Big Data
pp. 269–276. Era, ICSITech, 2017. pp. 166–170.
[46] Abdullah-Al-Wadud M., M.H. Kabir, M.A.A. [56] Florentz G., E. Aldea, “SuperFAST : Model-
Dewan, O. Chae, “A Dynamic Histogram Based Adaptive Corner Detection for Scalable
Equalization for Image Contrast Enhancement”, Robotic Vision”, IEEE, 2014.
Digest of Technical Papers - IEEE International [57] Jakubović A., J. Velagić, “Image Feature
Conference on Consumer Electronics. 2007, Matching and Object Detection Using Brute-
Vol.3, pp. 7–8. Force Matchers”, International Symposium
Electronics in Marine, IEEE, 2018, pp. 83–86. 

2683

View publication stats