Real-time Moving Vehicle Detection, Tracking, and

Counting System Implemented with OpenCV

Da Li1, 2 Bodong Liang1, 3 Weigang Zhang2
1. School of Automotive and Transportation Engineering, Shenzhen Polytechnic, Shenzhen 518055, China
2. School of Information Engineering, Chang’an University, Xi’an 710064, China
3. Corresponding author, Email: liangbodong@szpt.edu.cn

estimate a robust background to deal with the changing

Abstract – Moving vehicle detection, tracking, and counting are
very critical for traffic flow monitoring, planning, and object. For this reason, in the case of vehicle detection on
controlling. Video-based solution, comparing to other road, an adaptive rather than static background is needed for
techniques, does not disturb traffic flow and is easily installed. real-time road situations [3].
By analyzing the traffic video sequence recorded from a video Regarding to real-time vehicle tracking system, the
camera, this paper presents a video-based solution applied with crucial issue is initiating a track automatically. Here we
adaptive subtracted background technology in combination with describe two systems in which the problem is attacked quiet
virtual detector and blob tracking technologies. Experimental differently. First one is virtual detector constructed with a
results, implemented in Visual C++ code with OpenCV set of rectangular regions in each frame [4][3]. Because the
development kits, indicate that the proposed method can detect,
camera is fixed, the virtual detector can be chosen to span
track, and count moving vehicles accurately.
each lane and the system then monitors changes in area of
Index Terms – vehicle detection and tracking, video analysis, virtual detector that indicate the presence of a vehicle. The
background estimation, virtual detector, blob tracking second one is blob tracking [5][6]. In this algorithm, a
background model is generated for the scene. For each input
I. INTRODUCTION image frame, the absolute difference between the input image
Expressways, highways and roads are becoming and the background image is processed to extract foreground
overcrowded with increasing of large number of vehicles. blobs corresponding to the vehicles on the road. Both above
Intelligent transportation systems (ITS), applied to collect, two approaches have difficulty tackling shadows, occlusions,
cognize, and manage information about transportation flows and large vehicles (e.g., trucks, trailers), all of which cause
from various sources, are emerging worldwide to make multiple vehicles to appear as a single region.
transportation more efficient, reliable, cleaner and safer. The To overcome the previous limitations in vehicle detection
requirement to detect, track, and count the moving vehicle is and tracking, we present an improved method in this paper to
getting very important for traffic flow monitoring, planning, accurately separate the vehicle foreground from the adaptive
and controlling. background model through a combination of Otsu’s
The vehicle detections can be traditionally achieved thresholding method [7][8] and moving cast shadow detection
through inductive loop detector, infrared detector, radar method [9][10].
detector or video-based solution. Compared to other This paper is organized as the following. Section II
techniques, the video-based solutions based on surveillance describes in detail the adaptive categorizing of background
camera mounted outdoor are easily influenced by and foreground algorithm bundling with Otsu’s method and
environments such as weather, illumination, shadow, etc. shadow detection method. Section III and IV present the
However, because video-based systems can offer several method of virtual detector and blob tracking, respectively.
advantages over other methods such as traffic flow Section V shows experimental results implemented with
undisturbed, easily installed, conveniently modified, etc., they Intel’s OpenCV (Open Source Computer Vision Library)
have drawn significant attention from researchers [1][2] in the [11][12] to illustrate the performance of our solution.
past decade. Section VI gives conclusion.
A traditional computer vision method for moving objet
detection in video-based system is so-called “background II. ADAPTIVE CATEGORIZING OF BACKGROUND AND
subtraction”, or computing the difference between a FOREGROUND ALGORITHM
background model and current frame, which demands to
The system software deals with image frames read from
The work described in this paper was partially supported by grants from video sequences, so that we can make image binarization.
Shenzhen Polytechnic (No. 2210K3100042 and No. 2613K3100021). The whole processing flowchart is shown in Fig. 1.
____________________________________
978-1-4799-4808-6 /14/$31.00 ©2014 IEEE


A. Adaptive background generation in gray scale After having original background, we apply Gaussian
filters to smooth the following image frames and convert all
Firstly, our algorithm reads image frames from video, RGB color images to gray ones. The background gray
decomposes the initial 50 image frames into RGB channels, image and the current grayscale image are shown in Fig. 2(a)
weights average of the top 50 frames, and gets an original and Fig. 2(b), respectively.
background picture. This original background could be Following software calculates the difference between the
erroneous if there are moving objects present in the field of grayed image of background and the current image frame, that
view, but it will converge to a robust background after means we can extract the foreground presenting the moving
handling more frames thanks to the operation of “Background objects shown in Fig. 2(c).
update”. At the same time, the system updates in real-time the
background image in the above difference calculation
Read Video frame processing. The latest background image can be defined as a
weighted sum of the current background and the extracted
foreground.
YES
Frame>50 B. Binarization with shadow removal

NO At following this software will reduce the gray level of

The frame image is extracted foreground to a binary image, in which we adopt the
decomposed into image RGB
three channels Otsu’s algorithm to obtain the threshold value. Otsu’s
The weighted algorithm assumes that the image to be thresholded contains
average of top 50 Gaussian filter two classes of pixels or bi-modal histogram (e.g. foreground
frames
Background update and background) then calculates the optimum threshold
Initial background dividing those two classes so that their intra-class variance is
minimal. Otsu’s method is considered to be the best one to
get threshold value in image segmentation algorithm, and is
Gray not affected by the image brightness and contrast, therefore it
has been widely used in digital image processing.
After getting binary image by thresholding, we see some
errors due to shadow appearance in the video. Therefore
Median filter
HSI shadow removal algorithm is used to segment the moving
vehicle according to the characteristic of cast shadows in HSI
color space. Our solution uses morphology filtering on
The matrix of current
frame and background image to remove the shadow, including steps in combination
of corrosion and expansion. In real processing we firstly do
corrosion and following by expansion, which is usually called
The difference of current open operation. It can eliminate small objects, separate
gray frame and background
boundary of the object in the fine points, smooth the boundary
of the bigger objects but not significantly alter area. The
binary image removing shadow is shown in Fig. 2(d).
Otsu’s threshold
C. Filling in contour

However, sometimes we find other problems like as

over-segmentation, in which one vehicle is divided into two
Binarization
parts, and weak boundary, which is hard to detect. Therefore
we also need to adopt algorithm to fill the current binary
image.
HSI shadow
removal The specific process is as follows: firstly making edge
detection based on the extracted foreground gray level image,
secondly adding the binary image removing shadow, and
Open operation morphological finally finding a closed contour and filling the region in
filtering
contour. The three results are shown in Fig. 2(e), Fig. 2(f),
and Fig. 2(g), respectively. It is clear that the binary image
Image filling
after filling the contour of moving objects has greatly
improved over the original shadow removal binary image.
Then we will make process based on this filled image in
Fig. 1 Flowchart of algorithm processing.
later sections.


 III. VIRTUAL DETECTOR detector in running system is highlighted with the thin red
rectangular box and shown in Fig. 3(a). While vehicles pass
Here we shall present a technique called “Virtual through the virtual detector, pixel changes in histogram can be
Detector” to count the moving vehicle in each lane. found as shown in Fig. 3(b).
The traditional and popular technology to detect the
vehicle on road is based on principle of electromagnetic IV. BLOB TRACKING
induction. Its sensor is a real inductive loop being buried
under the road. Once a vehicle enters, passes through, and To track a moving object is to detect the movement of the
leaves the lane area embedded inductive loop below, induced object, number the moving object, and obtain its trajectory.
current will change and indicate the presence of vehicle. In our method we adopt the blob tracking technology to track
However, this technology in practical application will and number the vehicles moving in the field of view from the
encounter some challenges such as difficult construction, recorded video camera.
expensive cost, short lifetime, etc. The whole process of blob tracking can be divided into
In this paper, we propose a method in combination of several modules at following: foreground detection module,
vehicle detection and virtual detector to count the number of new blob detection module, blob tracking module, path
vehicles on road in real time. It draws a set of rectangular generation module, and trajectory of post-processing module.
regions of interest (ROIs) in each lane as the virtual detector The processing flow is shown as Fig. 4 and all modules are
in image space, and determines the presence of vehicles by described in detail at below:
monitoring changes in area of virtual detector. The virtual a) Foreground detection module: To determine each
pixel in image space belongs to foreground or
background. This part of processing can be resulted
from Section II;
b) New blob detection module: Using the latest
foreground detected previously to check new blob
corresponding to the vehicle just entering into the
field of view on the road;
c) Blob tracking module: Monitoring the new blobs
detected previously to initialize the tracking to all
(a) Grayed background (b) Grayed current input frame existing blobs;
d) Path generation module: To collect all trajectories of
being tracked blobs and save the corresponding
trajectory at the end of each trace (such as tracking
lost, leaving scene, etc.);
e) Trajectory of post-processing module: Mainly to
smooth the trajectory.

V. EXPERIMENTAL RESULTS BASED ON OPENCV

(c) Extracted foreground (d) Binary image
In this part, we shall apply above all steps from video
sequences to detect, track, and count the moving vehicle on
road. We shall compare the results from two solutions:
virtual detector and blob tracking. All software programs
are developed in Visual C++ code with OpenCV image
development kits. OpenCV, standing for Open Source
Computer Vision Library, is a library of programming

(e) Edge of current frame (f) Combined image

(a) (b)

(g) Binary image after filling Fig. 3 Virtual detector. (a) Be highlighted with red color of
rectangular region in image space. (b) Pixels change in histogram
Fig. 2 Example of algorithm processing. while vehicle traversing


 Experimental results, implemented with OpenCV,
Video
indicate that the proposed method is effective to detect, track,
frame Foreground
detection
New blob
detection
Blob
tracking
Trajectory of
post-processing
Blob˄IDˈ and count moving vehicles accurately.
POSˈSize˅
module module module module
REFERENCES
Path
Blob position generation [1] S. Sivaraman and M. Trivedi, “Looking at vehicles on the road: A
correction module survey of vision-based vehicle detection, tracking, and behavior
analysis”, IEEE Transactions on Intelligent Transportation Systems,
vol. 14, no. 4, pp. 1773-1795, 2013.
Fig. 4 Flowchart of blob tracking. [2] Y. Liu, B. Tian, S. Chen, F. Zhu, and K. Wang, “A survey of
vision-based vehicle detection and tracking techniques in ITS”, 2013
IEEE International Conference on Vehicular Electronics and Safety
functions mainly aimed at real-time computer vision. (ICVES), pp. 72-77.
We mount a video camera on footbridge crossing a main [3] M. Lei, D. Lefloch, P. Gouton, and K. Madani, “A Video-Based
street to monitor three lanes simultaneously and avoid Real-Time Vehicle Counting System Using Adaptive Background
erroneous counting caused by vehicle overlapping in image. Method”, 2008 IEEE International Conference on Signal Image
Technology and Internet Based Systems, pp. 523-528.
The results of two different approaches and their respective [4] G.D. Sullivan, K.D. Baker, A.D.Worrall, C.I. Attwood, and P.M.
accuracy rate are listed in Table 1. The accuracy rate Remagnino, “Model-based vehicle detection and classification using
reaches to 97.1% for virtual detector method, and 98.4% for orthographic approximations”, Image and Vision Computing, vol. 15,
blob tracking method. From the table, we observe that this no. 8, pp. 649-654, 1997.
[5] S. Gupte, O. Masoud, R.F.K. Martin, and N.P. Papanikolopoulos,
system is able to detect, track, and count most vehicles “Detection and classification of vehicles”, IEEE Transactions on
successfully. Intelligent Transportation Systems, vol. 3, no. 1, pp. 37-47, 2002.
[6] S.C. Lee and R. Nevatia, “Speed Performance Improvement of Vehicle
VI. CONCLUSION Blob Tracking System”, in Multimodal Technologies for Perception of
Humans, Lecture Notes in Computer Science, vol. 4625, pp: 197-202,
Due to increasing demands in ITS, there is a huge amount 2008.
of potential applications of detecting, tracking, and counting [7] N. Otsu, “A threshold selection method from gray-level histograms”,
IEEE Transactions on Systems, Man, and Cybernetics, vol. 9, no. 1, pp.
the moving vehicles on road in real time. 62-66, 1979.
In this paper we presented unitized techniques to achieve [8] P.S. Liao, T.S. Chen, and P.C. Chung, “A Fast Algorithm for Multilevel
the goal. In vehicle detection, we applied a method to Thresholding”, Journal of Information Science and Engineering, vol.
accurately separate the vehicle foreground from the adaptive 17, no. 5, pp. 713-727, 2001.
[9] B. Sun, S. Li, “Moving Cast Shadow Detection of Vehicle Using
background model through a combination of Otsu’s Combined Color Models”, 2010 Chinese Conference on Pattern
thresholding method and moving cast shadow detection Recognition (CCPR), pp. 1-5.
method. In vehicle tracking and counting, we applied two [10] A. Sanin, C. Sanderson, and B.C. Lovell, “Shadow detection: A survey
methods (virtual detector and blob tracking) to double check and comparative evaluation of recent methods”, Pattern Recognition,
vol. 45, no. 4, pp. 1684-1695, April 2012.
the accuracy and maturity of our proposed method. [11] OpenCV User Site: http://opencv.org/
[12] OpenCV on SourceForge: http://sourceforge.net/projects/opencvlibrary/

TABLE 1 EXPERIMENTAL RESULTS BASED ON OPENCV

Lane Total number Counting number Accuracy rate

Number of
of vehicles in
input frames 1 2 3 ground truth Virtual detector Blob tracking Virtual detector Blob tracking
60 1 0 1 2 2 2 100% 100%
300 3 1 2 6 6 6 100% 100%
450 6 3 8 17 18 18 94.4% 100%
876 11 9 14 34 35 33 97.1% 94.2%
1401 15 15 18 48 50 49 96.0% 98.0%
1812 23 22 30 85 88 84 96.6% 95.4%
2512 41 31 50 122 126 124 96.0% 98.4%