See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/327253343

Advanced Driver-Assistance Systems: A Path Toward Autonomous

Vehicles

Article in IEEE Consumer Electronics Magazine · August 2018

DOI: 10.1109/MCE.2018.2828440

CITATIONS READS
211 21,785

4 authors, including:

Vipin Kumar Kukkala Sudeep Pasricha

NVIDIA Colorado State University
17 PUBLICATIONS 313 CITATIONS 324 PUBLICATIONS 4,186 CITATIONS

SEE PROFILE SEE PROFILE

Thomas H Bradley
Colorado State University
139 PUBLICATIONS 4,553 CITATIONS

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Optimal Fuel Economy in Vehicles View project

Electrified Vehicle In-Motion Wireless Charging View project

All content following this page was uploaded by Vipin Kumar Kukkala on 14 March 2019.

The user has requested enhancement of the downloaded file.

 Advanced
Driver-Assistance Systems
A path toward autonomous vehicles.

By Vipin Kumar Kukkala, Jordan Tunnell, Sudeep Pasricha, and Thomas Bradley

A
dvanced driver-assistance systems (ADASs) vehicles, passive safety systems that have been under contin-
have become a salient feature for safety in modern uous development for many decades have been augmented by
vehicles. They are also a key underlying technolo- active safety systems, which seek to prevent a crash from
gy in emerging autonomous
vehicles. State-of-the-art
ADASs are primarily vision based,
but light detection and ranging (lidar),
radio detection and ranging (radar),
and other advanced-sensing technolo-
gies are also becoming popular. In this
article, we present a survey of differ-
ent hardware and software ADAS
technologies and their capabilities and
limitations. We discuss approaches
used for vision-based recognition and
sensor fusion in ADAS solutions. We
also highlight challenges for the next
generation of ADASs.

OVERVIEW OF AUTOMOTIVE
SYSTEM SAFETY
Safety in automotive systems has been
a major concern since the early days
of on-road vehicles. Several original
equipment manufacturers (OEMs)
have attempted to address this issue
by developing various safety systems
to protect occupants within a vehicle
as well as prevent injuries to people
outside the vehicle. These systems are
mainly classified into two types: 1)
passive (or reactive) and 2) active (or
proactive). Passive safety systems pro-
tect vehicle occupants from injuries
after a crash, e.g., seat belts, air bags,
and padded dashboards. Due to a con-
sistent consumer demand for safer

Digital Object Identifier 10.1109/MCE.2018.2828440

Date of publication: 9 August 2018

18 IEEE Consumer Electronics Magazine ^ september 2018 2162-2248/18©2018IEEE

happening altogether. Active systems are one of the main cameras, lidar, radar, and ultrasonic sensors, enable a variety of
areas of interest and have seen major growth in today’s vehi- different ADAS solutions. Among them, the vision-based
cles. Examples of such systems include lane keeping, auto- ADAS, which primarily uses cameras as vision sensors, is pop-
matic braking, and adaptive cruise control. These systems are ular in most modern-day vehicles. Figure 1 shows some of the
commonly known as ADASs and are becoming increasingly state-of-the art ADAS features and the sensors used to imple-
popular as a way for automotive manufacturers to differenti- ment them.
ate their offerings while promoting consumer safety. Modern-day ADASs are also key technologies to realize
Recent studies from the World Health Organization indicate autonomous vehicles [3]. But several challenges with the
that 1.25 million deaths occur every year due to road traffic design, implementation, and operation of ADASs remain to be
accidents [1]. Moreover, such accidents in recent years have an overcome. Some of these challenges include minimizing energy
annual global cost of US$518 billion, which takes away consumption, reducing response latency, adapting to changing
approximately 1–2% of gross domestic product from all of the weather conditions, and security. In this article, we provide a
countries in the world [2]. These high fatality rates, monetary synopsis of the landscape of ADAS research and development
losses, and increasing customer demand for intelligent safety to address these challenges.
systems are some of the key reasons for OEMs to develop
ADASs. Moreover, with the increasing number of electronic ADAS TAXONOMY
control units and integration of various types of sensors, there We propose a taxonomy of ADASs based on the type of sen-
are now sufficient computing capabilities in vehicles to support sors they use (Figure 2), as discussed next.
ADAS deployments. The different types of sensors, such as
VISION SENSORS
Cameras are the most commonly used vision sensors in vehi-
cles. Vision-based ADAS uses one or more cameras to cap-
ture images and an embedded system to detect, analyze, and
track different objects in them. In high-end ADAS, cameras
are used to monitor both the inside and outside of the vehicle.
Camera integration in modern vehicles is becoming more
common because of its low cost and easy installation. At the
2018 Consumer Electronics Show, Mobileye stated that it is
introducing smart cameras in millions of cars hitting the
streets in 2018. In addition, laws such as [4] (that mandate all
vehicles manufactured from 1 May 2018 onward use vision-
based ADAS) will further aid in camera integration. Cameras
capture information such as color, contrast, and texture,
which gives them a unique advantage over other sensors. Two
types of cameras are often used in vision-based ADAS: 1)
monocular and 2) stereo.

MONOCULAR CAMERAS
These camera systems have only one lens. As these systems
have only one image output at any point of time, they have
low image-processing requirements compared to those of
other camera types. These cameras can be used for multiple
applications, such as the detection of obstacles, pedestrians,
lanes, and traffic signs [5]. They can also be used for moni-
toring the driver inside a vehicle, e.g., for face- and eye-
detection and head-pose analysis [23]. But monocular camera
images lack depth information and are, therefore, not reliable
sensors for distance estimation. Some techniques [5] allow
approximating distance by identifying key features in the
captured image frame and tracking their position when the
©iStockphoto.com/metamorworks

camera is in motion.

STEREO CAMERAS
These systems consist of two or more lenses, each with image
sensors, separated by a certain distance (known as stereo
base). Stereo cameras are useful in extracting three-dimensional

september 2018 ^ IEEE Consumer Electronics Magazine 19

 Surround View

Blind-Spot
Traffic Sign Detection
Recognition

Rear Collision Warning

Cross-Traffic Alert

Park Assist

Park Assist
Adaptive Automatic Braking
Surround
Cruise Pedestrian Detection
View
Control Collision Avoidance

Lane-
Keeping
Blind-Spot
System
Detection
Surround View

Long-Range Radar Lidar Camera Short-/Medium-Range Radar Ultrasonic

FIGURE 1. The state-of-the-art ADAS sensors used.

ADAS

Vision Lidar Radar Ultrasonic Others

Short-/
Monocular Stereo Infrared Long-Range
Medium-Range PMD IMUs GPSs
Camera Camera Camera Radar
Radar

FIGURE 2. The taxonomy of an ADAS. PMD: photonic mixer device; IMUs: inertial measurement units; GPSs: global positioning systems.

(3-D) information from two or more two-dimensional images capture the reflected light. Passive IR cameras use an IR sen-
by matching stereo pairs (images from left and right sensors) sor, where every pixel on the IR sensor can be considered as a
and using a disparity map to estimate the relative depth of a temperature sensor that can capture the thermal radiation emit-
scene. These cameras can be used for a variety of applications, ted by any material. Unlike active IR cameras, passive IR cam-
such as traffic sign recognition, lane, pedestrian, and obstacle eras do not require any special illumination of the scene. Still,
detection as well as distance estimation, with much greater popular night-vision solutions mainly use active IR cameras to
accuracy compared to monocular cameras. assist the driver by displaying video data on a screen during
Stereo systems can be relied upon for accurate distance low light conditions.
(depth) estimation over short distances, up to 30 m. In most
production vehicles with stereo cameras, the cameras are locat- LIDAR
ed inside the vehicle, behind the rear-view mirror, angled slight- Lidar works by firing a laser beam at an object and then mea-
ly downward, and facing the road. suring the time taken for the light to bounce back to the sen-
sor, to calculate the distance of an object. These systems can
IR CAMERAS achieve high-resolution 3-D images and operate at longer
There are two main types of IR cameras. Active IR cameras ranges than camera systems. Some of the lidar scanners sup-
use a near-IR light source (with wavelengths from 750 to 1,400 nm) port surround-view sensors (that fire laser beams continuously
built in the vehicle to illuminate the scene (which cannot be in all directions), which can generate a 360° 3-D image of the
seen by the human eye) and a standard digital camera sensor to surroundings with extremely accurate depth information (as

20 IEEE Consumer Electronics Magazine ^ september 2018

shown in Figure 3). Lidar is becoming very popular in autono- COMPUTER VISION DATA FLOW FOR ADASs
mous vehicles. Several prototype vehicles [8], [9] have dem- Figure 4 shows the steps involved in a vision-based system,
onstrated the advantages of using lidar in autonomous driving. each of which is discussed.
Lidar is useful for systems implementing automatic braking,
object detection, collision avoidance, and more. Depending on IMAGE ACQUISITION
the type of sensor, lidars for cars can have a range of up to This refers to the process of capturing a frame from a video.
60 m. Despite the aforementioned advantages, lidars are The frame is often represented as a matrix of pixel data where
heavy, bulky in size, and expensive. Moreover, atmospheric each frame contains three channels of information, e.g., red,
conditions such as rain or fog can impact the coverage and green, and blue (RGB) sets of pixels. Typical frame rates in
accuracy of these systems. Emerging solid-state lidars [10] ADASs range from five frames per second (fps) to 60 fps
have opened the possibility of powerful lidars that are signifi- depending on the application. Applications that involve detec-
cantly smaller and relatively inexpensive. tion of vehicle proximity need a higher frame rate due to the
rapid change in distance for cars on the road. In contrast, traf-
RADAR fic sign detection does not demand a higher frame rate
Radar systems emit microwaves and estimate the speed and because only one frame of the sign needs to be captured for
distance of an object by measuring the change in the frequen- the sign to be detected.
cy of the reflected wave as per the Doppler effect. Due to the
longer wavelength of microwaves, they can travel much far- PREPROCESSING
ther than optical light (e.g., with lidar) and can detect objects There are several common preprocessing steps needed to pre-
at a longer distance. Unlike lidar, radar is not affected by pare an image for various computer vision algorithms, e.g.,
foggy or rainy weather conditions and is relatively inexpen- denoising, color enhancement, color space conversion, and
sive. Depending on their operating distance range, radar sys- image stabilization. A typical example of color space conver-
tems can be classified as short range (0.2–30 m), medium sion is to convert the RGB color space to hue, saturation, and
range (30–80 m), or long range (80–200 m) [11]. Cross-traffic value to separate color from the intensity. Moreover, the hue
alerts and blind-spot detection are some of the applications of channel is often used to separate out adverse effects (e.g.,
short-/medium-range radars. These systems are often located shadows, uneven lighting, and over- and underexposure) in the
at the corners of a vehicle. Adaptive cruise control is a long- image to make tracking and detection easier.
range radar application, with the system located behind the
front grill or under the bumper. Researchers have been devel- SEGMENTATION
oping algorithms to improve the performance of radar and This is the process of separating features from a frame. In
reliability all while attempting to reduce the cost and power of analyzing an image, it is helpful to partition it into recogniz-
the system [6]. able objects, e.g., identifying the road and sky in a frame as

ULTRASONIC SENSORS
Ultrasonic sensors use sound waves to measure the distance
to an object. These sensors are mainly used for detecting
objects very close to the vehicle. Some example applications
include automatic parking and parallel parking assist. These
sensors are mainly located under the front and rear bumper of
the vehicle.

OTHERS
A few other sensors are used to complement and improve the
functionalities of those discussed earlier. For instance, pho-
tonic mixer device (PMD) cameras consist of an array of FIGURE 3. The Google self-driving-car-generated 3-D image of its
smart sensors that enable fast optical sensing and demodula- surroundings using lidar [9].
tion of incoherent light signals simultaneously [12]. PMDs
can support parallel target pixel-wise distance measurement
without scanning, thus resulting in faster imaging, high later- Image Preprocessing Segmentation
al resolution, and depth information. IMUs and GPSs are Acquisition
examples of systems that help improve the distance measure-
ments with lidar and radar.
Object Detection Depth
System Control
VISION-BASED ADASs and Tracking Estimation
Vision-based ADASs rely on images from cameras and use
computer vision principles to extract useful information. FIGURE 4. The vision data flow for the ADAS used.

september 2018 ^ IEEE Consumer Electronics Magazine 21

 vision applications include Caffe, Darknet, and MATLAB. An
application of a CNN for object tracking is discussed in [14].
IMUs and GPSs are examples of Kalman-filter-based object tracking is proposed in [15], where
systems that help improve the the filter tracks the object’s velocity.
distance measurements with
DEPTH ESTIMATION
lidar and radar. This step involves estimating the distance of an object in the
image frame relative to the camera. There are two common
two different features. Various thresholding techniques are methods for depth estimation: 1) the use of a stereo camera to
used to filter one class of pixels (e.g., the road) from another create a stereo pair and develop them to make a depth map
(e.g., the sky). One of the methods, e.g., exploits color infor- and 3-D point cloud that allow a real-world reconstruction of
mation to detect a stop sign, where an algorithm may look for the scene [16]; and 2) the use of a monocular camera and sev-
red in the image (typical for stop signs in the United States). eral state-of-the art techniques that use a subset of optical
Any pixels in that red range will be turned white, and any- flow, calibration, and least squares techniques [17].
thing that is not will be turned black, as shown in Figure 5(a).
This results in a binary image that is often used as a mask for System control
finding the area of interest on the original image. This is the last step in the vision data flow, which involves
interpretation of the outputs from previous layers, as shown
OBJECT DETECTION AND TRACKING in the vision data flow diagram in Figure 4. This step requires
This is the process of classifying an object in an image (e.g., a weighing of each layer in the vision pipeline to come up
determining if an object ahead is a vehicle, sign, or pedestrian) with a confidence value that can be used to make decisions. A
and predicting its movement. It is often accomplished with var- major challenge at this step is a false detection with high con-
ious machine-learning (ML) algorithms. ML algorithms are fidence that would take priority over other information
provided large training data sets (thousands of images) to learn obtained from the previous layers. Thus, training with data
and differentiate between vehicles and common objects found that is correct and contains many orientations of the object to
around them. An example of an object detection method is be classified is crucial to achieve high accuracy.
called the cascade classifier, which was first presented in [13]
for face detection, on low-performance hardware systems. OUTDOOR MONITORING
Another common technique to train and classify images is In this section, we will discuss the classification of objects that
using a convolutional neural network (CNN), which typically are outside a vehicle, e.g., pedestrians, vehicles, and roads.
consists of an input layer, multiple hidden layers, and an output
layer. The hidden layers consist of convolution and pooling PEDESTRIAN DETECTION
layers that are used for feature extraction and a fully connected Detecting pedestrians is done using various classifiers, e.g.,
layer for classification. Examples of CNN frameworks used for [18]. Often more than one classifier is used for detecting peo-
ple because of the varying orientation
and configuration in which pedestrians
may appear. Deep-learning networks
Confidence = 0.921278 String = STOP such as CNNs have been helpful to not
only identify pedestrians but also clas-
sify their actions.

VEHICLE DETECTION
Confidence = 0.907121 String = STOP Vehicle detection is a major focus of
object detection in ADASs. The fact
that many vehicles share common fea-
tures, such as having tires, brake lights,
and license plates, allows the detection
Confidence = 0.889147 String = STOP of these objects to indicate the pres-
ence of a car. These features are all
used to distinguish the vehicle from
other objects, such as signs, roads, and
other miscellaneous objects. In Figure 6,
(a) (b) an example of vehicle detection is shown,
using a CNN framework (Darknet)
FIGURE 5. The stop sign detection: (a) a binary stop sign and (b) stop sign classification and a real-time detection system, You
using optical character recognition. Only Look Once [19]. The orientation

22 IEEE Consumer Electronics Magazine ^ september 2018

of vehicles can cause some issues with their identification. A
vehicle being viewed from the front contains a different set of
features than a vehicle from the side or back. Often vehicle
One of the major problems
classifiers consider various classes of vehicles, such as cars, with today’s ADASs is that the
trucks, and semis that are trained with many orientations. performance of the system
SIGN DETECTION is significantly impacted by
Many ADASs are beginning to support traffic sign detection. changing environmental
The most common use case is determining the speed limit on
the road by reading a speed sign (an ADAS would alert the
and weather conditions.
driver if the vehicle speed is over the limit). For instance,
color thresholds can be used to find the location of a sign and SENSOR FUSION
optical character recognition to determine what that sign dis- Sensor fusion refers to combining information from multiple
plays [as shown in Figure 5(b)]. Other methods include using homogenous or heterogeneous sensors to find a single best
CNNs and hybrid techniques, such as [20]. estimation of the state of the environment. Fusion helps sen-
sors complement each other’s limitations and offers greater
LANE DETECTION leverage to the system compared to a system with individual
Another ADAS feature used in a few production vehicles is sensors. Sensor fusion offers high precision, reliability,
the ability to keep the vehicle within the lane lines on the robustness to uncertainty, extended spatial and temporal cov-
road (illustrated in Figure 6). However, lane lines are one of erage, and improved resolution, which are crucial in safety-
the hardest road features to detect because of their inconsis- critical systems, such as vehicles. Although this comes at a
tencies, such as being different colors, faded, and sometimes higher computation cost, the computation power available in
not even present. Current methods to detect lane lines often modern-day cars and the reducing cost of the sensors are
use a Canny transform to find the edges in the image. Once facilitating the widespread integration of these systems.
the edges are found, a Hough transform is used to compare The classification of different levels of sensor fusion along
the lines to a single slope to determine if they are indeed lane with the most commonly used techniques for fusing data are
lines [21]. The use of CNNs is also becoming popular for lane discussed in [25]. The growing interest in deep learning and
line detection. other ML methods in recent years has driven researchers
toward exploring more efficient and intelligent techniques
COLLISION AVOIDANCE that enhance ADASs with sensor fusion capabilities.
ADASs are beginning to incorporate automatic braking and
collision avoidance. This is done by combining many features V2X COMMUNICATION
discussed earlier, such as object tracking, vehicle detection, V2X communication represents a class of communication
and distance estimation [14]. With this combination of data, a systems that provides the vehicle with an ability to exchange
vehicle can predict a collision and stop it from happening by information with other systems in the environment. Examples
braking or even steering out of the way. include vehicle-to-vehicle (V2V) for collision avoidance,
vehicle-to-infrastructure (V2I) for traffic signal timing, vehicle-
INDOOR MONITORING to-network for real-time traffic updates, and vehicle-to-
In a study conducted by the National Highway Traffic pedestrian for pedestrian signaling. State-of-the-art V2X
Security Administration [22], it was observed that driver communication is based on either dedicated short-range com-
fatigue, drowsiness, or distraction are the causes of 80% munications (DSRC) or cellular networks [26]. The IEEE
of vehicle accidents. As ADAS becomes prevalent in pro- 1609 family of standards for Wireless Access in Vehicular
duction vehicles, there has been an increase in focus on Environment (WAVE), which is developed based on the IEEE
monitoring the driver using a camera pointed at him or 802.11p standard, defines an architecture and a set of services
her. If the driver accesses a phone or does not look at the and interfaces to enable DSRC-based secure V2V and V2I
road for a specific time duration, an alert or attempt to get communication [27].
off the road will be made [23]. Drowsiness-fatigue-detec-
tion systems have also included the ability to detect if the
driver has fallen asleep and can attempt to alert the driver
though a sequence of seatbelt vibrations and speaker
alerts [24].

NEXT-GENERATION ADASs
Next-generation ADAS solutions are beginning to use sensor
fusion and other advanced communication systems, such as
vehicle-to-everything (V2X). FIGURE 6. The object (lane, vehicle, and sign) detection.

september 2018 ^ IEEE Consumer Electronics Magazine 23

 CHANGING ENVIRONMENTAL CONDITIONS
One of the major problems with today’s ADASs is that the per-
Modern vehicles are becoming formance of the system is significantly impacted by changing
increasingly connected with environmental and weather conditions. For example, vision-
a lot of different systems, based ADASs have issues with sensing during rainy and
extreme lighting conditions (too dark and/or too bright) [30].
such as Wi-Fi, near-field One of the possible solutions to this problem includes sensor
communication, and V2X. fusion, by relying on other sensor data depending on the
weather conditions, e.g., relying on the camera and radar dur-
ing low light conditions while using the camera and lidar dur-
AUTONOMOUS VEHICLES ing other times for accurate distance estimation. The inclusion
Next-generation ADASs using sensor fusion and V2X commu- of V2I and developing cost-effective smart roads could help
nication are paving the way for autonomous driving. The Soci- mitigate this issue.
ety of Automotive Engineers (SAE) J3016 standard [28] defines
six different levels of driving automation for on-road vehicles RESOURCE-CONSTRAINED SYSTEM
(see Table 1). A vehicle is categorized as level zero if there are Embedded systems used in ADASs have a requirement for
no ADASs assisting the driver in handling steering and acceler- low power consumption. This is because ADASs involve run-
ation/deceleration and everything is handled manually by the ning several complex algorithms that result in high power
driver. Level one vehicles consist of ADASs assisting the driver consumption and thermal dissipation. Due to the limited av­­­­
in handling either steering or acceleration/deceleration under ailability of energy in vehicles, it is essential to minimize the
certain cases with human driver input. ADASs in level two power consumption of the embedded system used in ADASs.
vehicles handle both steering and acceleration/deceleration Using more energy-efficient hardware than conventional gen-
under certain environments with human driver input. In general, eral-purpose central processing units is important, which is
in lower-level vehicles (levels zero to two), the driver monitors why emerging ADAS hardware must rely on graphics pro-
the driving environment. In contrast, ADAS monitors the driv- cessing units, digital signal processors, image signal proces-
ing environment in higher-level (levels three to five) vehicles. sors, etc., that are customized to reduce power consumption
Modern vehicles with the top-of-the-line ADASs, such as the for ADAS applications. Moreover, as the embedded systems
2016 Tesla model S, are level three, where multiple safety sys- for ADAS operate in real time, they have strict timing con-
tems are handled by the system, but the driver intervenes when straints, which establishes a latency minimization require-
needed. Level four vehicles handle multiple safety systems and ment. Hence, optimized hardware and software that results in
operate in a wider range of environments. Level five automation minimal power consumption and greater performance (lower
is the end goal of autonomous driving, where all of the systems latency) predictability are highly desired in an ADAS.
in the car are operated by the ADAS, under all driving condi-
tions (such as snow-covered roads and unmarked dirt roads) SECURITY
and would not require any human intervention. This, however, Modern vehicles are becoming increasingly connected with a
still requires significant advances in multiple areas, such as sen- lot of different systems, such as Wi-Fi, near-field communica-
sor technology, computing systems, and automotive networks. tion, and V2X. This enables the vehicle to sense and receive a
variety of information but also makes it more vulnerable to
CHALLENGES WITH ADASs attacks. Many vehicle hacks have been demonstrated, e.g.,
Despite significant advances in the field of ADASs, several researchers in [7] used onboard diagnostics (OBD-II) to hack a
important challenges remain to be overcome. GM vehicle. In [29], the telematics system in a Jeep Cherokee
was hacked to accelerate, brake, and kill the engine. This prob-
Table 1. SAE J3016 levels of driving lem is aggravated in ADASs and autonomous driving. Prevent-
automation [28]. ing hackers from gaining access to connected vehicles is
becoming increasingly important. This involves securing both
Monitoring of in-vehicle networks and external communication.
SAE Level Definition Driving Environment

Zero No automation
GEOSPATIAL CONSTRAINTS
Many of the modern ADAS solutions being developed are
One Driver assistance Human driver tested within a geographic location or a group of locations
Two Partial automation where they are sold. This limits the ADAS to one or a certain
group of geographical locations. This is because not all coun-
Three Conditional automation
tries (or some states in a country) adhere to the same sign and
Four High automation System road conventions uniformly, which makes ADAS algorithms
that are often trained under one location hard to work
Five Full automation
efficiently in other locations. There is a need to improve

24 IEEE Consumer Electronics Magazine ^ september 2018

­algorithms,
e.g., by exploiting V2X technology deployments [11] NXP. (2017, May 24). RADAR, camera, LiDAR and V2X for
to overcome variations in road sign conventions. autonomous cars. [Online]. Available: https://blog.nxp.com/automotive/
radar-camera-and-lidar-for-autonomous-cars
[12] D. I. B Hagebeuker. (2007). A 3D time of flight camera for object
CONCLUSION detection. PMD Technologies GmbH. Siegen, Germany. [Online]. Avail-
In this article, we presented a detailed survey of different types of able: https://www.ifm.com/obj/O1D_Paper-PMD.pdf
ADAS variants and an overview of the sensors used in these vari- [13] P. Viola and M. J. Jones, “Robust real-time face detection,” Int. J.
ants. We described a classification of ADASs based on the different Comput. Vision, vol. 57, no. 2, pp. 137–154, 2004.
types of sensors used and discussed outdoor and indoor monitoring [14] J. Pyo, J. Bang, and Y. Jeong, “Front collision warning based on
vehicle detection using CNN,” in Proc. IEEE SoC Design Conf. (ISOCC),
with vision-based ADASs. The importance of sensor fusion tech- 2016, pp. 163–64.
niques and advanced communication systems, such as V2X, and [15] Y. Huang, Y. Zhang, Z. Wu, N. Li, and J. Chambers, “A novel adap-
their importance for emerging autonomous vehicles was also dis- tive Kalman filter with inaccurate process and measurement noise covari-
cussed. Finally, we presented some important unresolved challeng- ance matrices,” IEEE Trans. Autom. Control, vol. 63, no. 2, pp.
es with ADASs that must be addressed going forward. 594–601, 2018.
[16] S. Nedevschi, R. Danescu, D. Frentiu, T. Marita, F. Oniga, C. Pocol,
R. Schmidt, and T. Graf, “High accuracy stereo vision system for far dis-
ABOUT THE AUTHORS tance obstacle detection,” in Proc. IEEE Intelligent Vehicles Symp., pp.
Vipin Kumar Kukkala (vipin.kukkala@colostate.edu) is a 292–297, 2004.
Ph.D. student in the Electrical and Computer Engineering [17] S. Diamantas, S. Astaras, and A. Pnevmatikakis, “Depth estimation in
Department at Colorado State University, Fort Collins. still images and videos using a motionless monocular camera,” in Proc.
IEEE Int. Conf. Imaging Systems Techniques (IST), 2016, pp. 129–134.
Jordan Tunnell (Jordantunnell@gmail.com) is pursuing [18] D. Geronimo, A. M. Lopez, A. D. Sappa, and T. Graf, “Survey of
his M.S. degree in the Electrical and Computer Engineering pedestrian detection for advanced driver assistance systems,” IEEE
Department at Colorado State University, Fort Collins. Trans. Pattern Anal. Mach. Intell., vol. 32, no. 7, pp. 1239–1258, 2010.
Sudeep Pasricha (sudeep@colostate.edu) is a Monfort [19] J. Redmon and A. Farhadi, “YOLO9000: Better, faster, stronger,” in
and Rockwell-Anderson professor in the Electrical and Com- Proc. IEEE Conf. Computer Vision Pattern Recognition (CVPR), 2017,
pp. 6517–6525.
puter Engineering Department at Colorado State University, [20] F. Parada-Loira and J. L. Alba-Castro, “Local contour patterns for
Fort Collins. fast traffic sign detection,” in Proc. IEEE Intelligent Vehicles Symp. (IV),
Thomas Bradley (thomas.bradley@colostate.edu) is an 2010, pp. 1–6.
associate professor of mechanical engineering at Colorado [21] Y. Li, L. Chen, H. Huang, X. Li, W. Xu, L. Zheng, and J. Huang,
State University, Fort Collins. “Nighttime lane markings recognition based on canny detection and
Hough transform,” in Proc. IEEE Int. Conf. Real-Time Computing Robot-
ics (RCAR), 2016, pp. 411–415.
REFERENCES [22] S. G. Klauer, T. A. Dingus, V. L. Neale, J. D. Sudweeks, and J. D.
[1] World Health Organization. (2015). Global status report on road safe- Ramsey. (2004, Jan.). The impact of driver inattention on near-crash/
ty. WHO. Geneva, Switzerland. [Online]. Available: http://www.who.int/ crash risk: An analysis using the 100-car naturalistic driving study data.
violence_injury_prevention/road_safety_status/2015/en/ U.S. Nat. Highway Traffic Safety Admin. Washington, D.C. [Online].
[2] Association for Safe International Road Travel. (2018). Annual global Available: https://www.nhtsa.gov/DOT/NHTSA/NRD/Articles/HF/
road crash statistics. ASIRT. Potomac, Maryland. [Online]. Available: Reducing%20Unsafe%20behaviors/810594/810594.htm
http://asirt.org/initiatives/in-forming-road-users/road-safety-facts/road- [23] P. Morignot, J. P. Rastelli, and F. Nashashibi, “Arbitration for bal-
crash-statistics ancing control between the driver and ADAS systems in an automated
[3] B. Markwalter, “The path to driverless cars,” IEEE Consum. Electron. vehicle: Survey and approach,” in Proc. IEEE Intelligent Vehicles Symp.,
Mag., vol. 6, no. 2, pp. 125–126, 2017. 2014, pp. 575–580.
[4] W. Cunningham. (2014, May 31). US requiring back-up cameras in [24] M. I. Chacon-Murguia and C. Prieto-Resendiz, “Detecting driver
cars by 2018. Roadshow. [Online]. Available: www.cnet.com/news/u-s- drowsiness: A survey of system designs and technology,” IEEE Consum.
requiring-back-up-cameras-in-cars-by-2018/ Electron. Mag., vol. 4, no. 4, pp. 107–119, 2015.
[5] I. Gat, M. Benady, and A. Shashua, “A monocular vision advance [25] F. Castanedo. (2013). A review of data fusion techniques. Sci. World
warning system for the automotive aftermarket,” SAE, Warrendale, PA, J. [Online]. Available: http://dx.doi.org/10.1155/2013/704504
Tech. Rep. 2005-01-1470, 2005. [26] K. Abboud, H. A. Omar, and W. Zhuang, “Interworking of DSRC
[6] S. Saponara. (2016). Hardware accelerator IP cores for real time radar and cellular network technologies for V2X communications: A survey,”
and camera-based ADAS. J. Real-Time Image Processing. [Online]. pp. IEEE Trans. Veh. Technol., vol. 65, no. 12, 2016, pp. 9457–9470.
1–18. Available: https://doi.org/10.1007/s11554-016-0657-0 [27] N. Vivek, S. V. Srikanth, P. Saurabh, T. P. Vamsi, and K. Raju, “On
[7] K. Koscher, A. Czeskis, F. Roesner, S. Patel, T. Kohno, S. Checko- field performance analysis of IEEE 802.11 P and WAVE protocol stack
way, D. McCoy, B. Kantor, D. Anderson, H. Shacham, and S. Savage, for V2V & V2I communication,” in Proc. IEEE Int. Conf. Information
“Experimental security analysis of a modern automobile,” in Proc. IEEE Communication Embedded Systems (ICICES), 2014, pp. 1–6.
Security Privacy, 2010, pp. 447–462. [28] SAE International. [Online]. Available: http://www.sae.org/misc/
[8] S. Thrun, M. Montemerlo, H. Dahlkamp, D. Stavens, A. Aron, J. pdfs/automated_driving.pdf
Diebel, P. Fong, J. Gale, M. Halpenny, G. Hoffmann, and K. Lau, “Stan- [29] C. Valasek and C. Miller. (2015). Remote exploitation of an unal-
ley: The robot that won the DARPA grand challenge,” J. Field Robotics, tered passenger vehicle. IOActive. Seattle, WA. White Paper. [Online].
vol. 23, no. 9, pp. 661–692, Sept. 2006. Available: https://ioactive.com/pdfs/IOActive_Remote_Car_Hacking.pdf
[9] E. Guizzo, “How Google’s self-driving car works,” IEEE Spectr., vol. [30] C. Schwarzlmuller, F. Al Machot, A. Fasih, and K. Kyamakya, “Adap-
18, no. 7, pp. 1132–1141, 2011. tive contrast enhancement involving CNN-based processing for foggy
[10] Velodyne. (2017). Velodyne LiDAR announces new velarray LiDAR weather conditions & non-uniform lighting conditions,” in Proc. IEEE Int.
sensor. Business Wire. [Online]. Available: http://www.businesswire Symp. Theoretical Electrical Engineering (ISTET), 2011, pp. 1–10.
.com/news/home/20170419005516/en/ 

september 2018 ^ IEEE Consumer Electronics Magazine 25

View publication stats