On-Road Animal Vehicle Collision Mitigation

System using Convolutional Neural Networks

Project Members:
1.Promodram .S(312215106080)
2.Thevin Arokiaraj L(312215106120)
3. V. M. Kumar(312215106127)
Project Guide:
Dr W. Jino Hans
Associate Professor
SSN College of Engineering
1
 Presentation Outline
 Project Motivation
 Project Objectives
 Research Area
* Computer Vision
 Software requirement
 Introduction
 Problem Statement
 Literature Survey
 Methodology
 Implementation
 Results and Discussions
 Conclusions and Future work
 References

2
 Project Motivation
• Current Scenario
– Animal vehicle collision (AVC) is constantly a major safety issue in India.
– AVCs are on the rise in rural areas due to abundance of cattle density.
– Dataset availability for detecting cattle AVCs is minimal.
– However the major contributor to AVCs is deer collisions.
– Hence there is meager research done in Cattle AVCs

• Severity of Problem
– About 35,000 AVCs occur yearly in India
– More than 98% of AVCs are single-vehicle crashes
– As cattle is heavy sized, collision with a vehicle is severe.
– Least importance is given to animals in roadways compared to paedestrian in India.

3
 Project Objective
• To develop Dataset for animal primarily cattles detection and collision avoidance system to employ in
Advanced Driver Assistance System.

• To develop efficient algorithm for cattle detection using deep learning.

• To employ state-of-the-art Active AVC detection equipment to assist in detection of heavy-sized

animals(such as Cows, Elephants) and hence prevent collisions.

4
 Research Area
• Computer Vision (Neural Networks)
– Class of deep, feed-forward artificial neural networks, most commonly applied to analyzing visual
imagery
– Machine learning algorithm for deep learning of robust datasets
– Gaining high-level understanding from digital images for Heavy sized animal detection

5
 Software Requirements
• Python and TensorFlow
– Object-oriented language for general-purpose programming
– TensorFlow-An open-source software library for dataflow programming; written in Python
– Designed to run on select CPUs and GPUs(with nVIDIA CUDA capability); used for machine
learning applications such as Convolutional Neural Networks

6
 Introduction
 Intelligent Transport Systems
Provision of innovative services relating to different modes of transport and traffic management; One
domain of ITS being ADAS
Advanced Driver Assistance System
1. Systems developed to automate, adapt and enhance vehicle systems for safety and better
driving
2. Adaptive features may automate lighting, provide adaptive cruise control, lane departure
warning, and collision avoidance.(particularly AVC detection)
Collision Detection and Avoidance System include pedestrian protection system, and forward
collision system of which detection of animals to prevent AVC is considered recently.
1. AVC detection systems designed for deer called DVS; for wildlife, WVS(Wildlife
vehicle detection system)
2. No extensive database exists for Heavy sized animals(eg cow, elephant etc.,) hence an
AVC-ADAS system for such domestic animals is demanding(especially in India which
again depend on terrain).

7
 Problem Statement

Detection Algorithm

Dataset
Acquisition Annotation Feature Deep
Detection
Extraction Learning
 Literature Survey
Animal Detection

Active
Passive Approach
Approach

Sensor Camera
Based Based
Ultrasonic Noise (MA
Light detection
sharaf sallah et al
&Ranging(V Mitra
Evaluation of animal
et al Network for Knapp et
warning system
LIDAR
al.,Deer-vehicle
Micro Doppler countermeasure
signal Tahmoush toolbox
et al modelled gait
variations
 Literature survey
Literature Survey 1 (Abdelhamid Mammeri, Depu Zhou, and Azzedine Boukerche)
TITLE: Animal–Vehicle Collision Mitigation System for Automated Vehicles
ABSTRACT
Detecting large animals on roadways using automated systems such as robots or vehicles is a vital task.
This can be achieved using conventional tools such as ultrasonic sensors, or with innovative technology
based on smart cameras. In this paper, we investigate a vision-based solution. We begin the paper by
performing a comparative study between three detectors: 1) Haar-AdaBoost; 2) histogram of oriented
gradient (HOG)-AdaBoost; and 3) local binary pattern (LBP)- AdaBoost, which were initially developed
to detect humans and their faces. (Moose)
INFERENCE
Based on the evaluation results, The paper propose a two-stage architecture that out performs the
schemes Haar-AdaBoost, HOG-AdaBoost, LBP-AdaBoost, and HOG-SVM. All of these detectors were
evaluated and tested in different daytime and nighttime conditions. Our two-stage architecture exhibits
good performance in conditions; however, at nighttime, it is less efficient. In night time LBP-AdaBoost
and HOG-SVM has shown limited capabilities.
Literature Survey 2 (Debao Zhou, IEEE Member, Matt Dillon and Eil Kwon)
TITLE: Tracking-Based Deer Vehicle Collision Detection Using Thermal Imaging
ABSTRACT
Deer vehicle collision (DVC) is constantly a major safety issue for the driving on rural road. It is estimated
that there are over 35,000 DVCs yearly By analyzing the infrared thermal images which are independent of
visible light, the presence of an animal can be determined in either night or day time through pattern
recognition and matching.(deer)
INFERENCE
• In order to reduce the accidents of vehicle and deer, an automatic deer detection device with tracking
function using thermal imaging has been developed in this research.
• The results using the thermal camera showed the feasibility of such a method.
• In the future, we will focus on the pattern matching algorithm to make the system capable of detecting
the presence of deer, or to distinguish between deer and different animals.
Literature Survey 3 (Mohammad (Ashkan) Sharafsaleh, P.E.,Marcel Huijser, PhD,
Christopher Nowakowski, Mark C. Greenwood, PhD, Larry Hayden, Jonathan Felder,
and May Wang)
TITLE: Evaluation of an Animal Warning System Effectiveness Phase Two - Final Report
ABSTRACT
The study was conducted to achieve two objectives:
a)study the effectiveness of animal detection systems
b) Measure driver’s response to the system
INFERENCE
The results of the survey indicate that most respondents want the system removed. The most common
concerns relate to the system is in the wrong location(meaning it is not as reliable as camera detection, as
discussed above), that the warning signs are too bright at night, and that the system is not reliable.
Literature Survey 4 (Dr. D. J. Shah, Signal & Image Processing : An International
Journal (SIPIJ) Vol.4, No.3, June 2013)
TITLE: A brief overview on different animal detection methods
ABSTRACT
• The very first area that is detection of animals is applied in various fields of real life Applications Eg.
LIDAR, Micro-Doppler signals
• The second area that is tracking of animals is important for monitoring or observing the locomotive
behavior of animals and its surroundings thereby tracking helps in the behaviour and interaction of the
animal with environment.
• The third area that is identification of animals is very important in identifying the targeted animal and
its behavior.
INFERENCE
• Since animals are unpredictable, there are issues in designing an efficient detection algorithm.
• Lots of ground work and research has to be done in this area
 Comparison between Classifier Types
• AdaBoost preferably used to train the three features: Haar, LBP,HOG
• HOG-AdaBoost uses largest number of weak classifier in each stage, LBP uses around 18 features and
Haar, 15 to 56 features
• HOG-AdaBoost performs the best(49.1ms), faster than LBP-AdaBoost
by 8 ms
• Reason being calculating a single feature of Haar or LBP is complicated
than calculating an HOG feature
• Extracting LBP feature is less time-consuming than extracting a Haar
feature
 Comparison between Classifier Types
• LBP-AdaBoost and Haar-AdaBoost outperform HOG-AdaBoost,
especially for low FPPI values
• LBP-AdaBoost detector has a relatively low miss rate of around
30% compared to the 45% miss rate of the Haar-AdaBoost, when
the FPPI rate is less than 10−4
• As Haar-AdaBoost consumes much more time than
LBP-Adaboost and has a significant number of false
positives ,LBP-AdaBoost can be
considered the best solution, followed by
Haar-AdaBoost
• The LBP-AdaBoost detector wins the comparison in terms of true
positive results and false negative misjudgment
 Why CNN?
• The convolution layer is the main building block of a convolutional neural network.
• CNN can be thought of automatic feature extractors from the image.CNN effectively uses adjacent
pixel information to effectively downsample the image first by convolution and then uses a prediction
layer at the end.
• The convolution layer comprises of a set of independent filters. Each filter is independently convolved
with the image and we end up with feature maps, providing accuracy compared to the aforementioned
feature classifier algorithms.
• CNN provides feature detection and inbuilt deep learning model which reduces processing time
considerably with high accuracy.
• CNN has high computational efficiency that is it needs fewer operations to be able to learn.
Methodology
Data Acquisition

Resizing and
Annotation

Training of test images

Localization of images

Evaluation of trained
images
 Implementation
1. Data Acquisition:
a. Images(1700 pictures)
b. Extraction from videos(1435 frames):
2.a. Resizing of Images
b.Annotation of Images:
Cont….
Obtained XML File:
 3.a.About the model

Single Shot Detector(SSD):

• The first part (“feature extraction”) of the whole
network is a bunch of convolutional blocks from
some other network .
• The second part (“detection”) of the whole
network is a sequence of detection blocks, with
each block having its own output that contributes
to the final output of the network.
• Each of the detection blocks has 3 branches: box
generation, classification and correction of
localization.
• In the last layer, smart filtering of the predictions
(non-maxima suppresion) happens.
 b.About the Feature Extractors

i)MobileNet v1:
• This architecture uses depthwise separable convolutions
which significantly reduces the number of
parameters.
• The normal convolution is replaced by depthwise convolution
followed by pointwise convolution
• This results in the reduction of number of parameters
significantly and thereby reduces the total number of FLOP which
is favorable in mobile and embedded vision applications with less
compute power.
 About the Feature Extractors

ii)InceptionNet :
• The module basically acts as multiple convolution filters, that are
applied to the same input, with some pooling. The results are
then concatenated.
• This allows the model to take advantage of multi-level feature
extraction.
• Thus the major steps involved are:
– Factorizing Convolutions with Large Filter Size
– Factorization into smaller convolutions
– Spatial Factorization into Asymmetric Convolutions
4.a.Detection Results(100 images)
b.Detection Results(1700 images)
SSD-InceptionNet
Detection Results
SSD-MobileNet model
 5.Evaluation Parameters
a) Classification Loss
b) Localization Loss
c) Total Loss
d) Inference Time
e) Frames per second
f) Mean Average Precision
 a)Classification Loss
• Loss incurred in classifying whether the object belongs to the specified class.

SSD-MobileNet SSD-InceptionNet
 b)Localization Loss

• Loss Associated with detection prediction of bounding box with respect to true case.

SSD-Mobilenet SSD-InceptionNet
 c)Total Loss
• Sum of classification and localization losses

SSD-Mobilenet SSD-InceptionNet
 d)Inference Time
• Time taken to produce the loss functions(in Tensor board) and to output the process’
fps.
• SSD_MobileNet:number of images trained=1700Images Processed: 508 Count: 0
Process+Stitch_FPS: 991.09 Process_FPS: 53.11 Time Taken -> 11.60

• SSD_InceptionNet:number of images trained=1700Images Processed: 508 Count: 0

Process+Stitch_FPS: 1021.01 Process_FPS: 28.93 Time Taken -> 18.62

SSD+MobileNet SSD+InceptionNet

11.60 18.62
 e)Frames per second
Rate of Detection of the test data.

• SSD_MobileNet:number of images trained=1700Images Processed: 508 Count: 0

Process+Stitch_FPS: 991.09 Process_FPS: 53.11 Time Taken -> 11.60

• SSD InceptionNet:number of images trained=1700Images Processed: 508 Count: 0

Process+Stitch_FPS: 1021.01 Process_FPS: 28.93 Time Taken -> 18.62

SSD+MobileNet SSD+InceptionNet
53.11 28.93
 f)Mean Average Precision

• Average of precision values at ranks of relevant documents.

 Assumes user wants to find many relevant documents
 Biased towards top of the ranking
• Obtained as the mean of average precisions
• Higher MAP value indicates a better performance of the neural network.
SSD+MobileNet SSD+InceptionNet

0.9888 0.9885
 Results-Mean Average Precision

SSD-MobileNet SSD-InceptionNet
 Conclusions and Future work
 The model performs efficiently with satisfying performance metrics.
 The architecture implemented is suited for large-scale animal detection; suited for cattle detection.
 Performance could be improved with larger datasets and increased training steps.
 Model, after extended training time and for multi-class cattle detections, could be tested on ImageNet
database for improved classification and localisation.
 Data Augmentation could be performed to obtain unique datasets; prevent overfitting of dataset.
 References
[1] Mammeri, A., Zhou, D., & Boukerche, A. (2016) ‘Animal-Vehicle Collision Mitigation System for
Automated Vehicles’, IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 46, no. 9, pp.
1287–1299.
[2] Prof. Sachin Sharma and Dr. D. J. Shah (2013), ‘A Brief Overview of Different Animal Detection
Methods’, Signal & Image Processing: An International Journal (SIPIJ), vol.4, no.3, pp. 77-81.
[3] Zhou, D., Dillon, M., & Kwon, E. (2009), ‘Tracking-based Deer Vehicle Collision Detection using
Thermal Imaging’, IEEE International Conference on Robotics and Biomimetics (ROBIO), Guilin.
Guangxi, China, pp. 688-693.
[4] P. Michalopoulos(1991), ‘Vehicle detection video through image processing: the Autoscope system’,
IEEE Transactions on Vehicular Technology, vol. 40, No. 1, pp. 21-29.
[5] Dr. D. J. Shah, Signal & Image Processing (2013) ‘A brief overview on different animal detection
methods‘,An International Journal (SIPIJ) Vol.4, No.3, pp.40-47.
[6] Shaikh S, Jadhav M, Nehe N and Verma U. Automatic animal detection and warning system.
International Journal of Advance Foundation and Research in Computer 2015; 2:405-410.
[7] G. Howard, Andrew & Zhu, Menglong & Chen, Bo & Kalenichenko, Dmitry & Wang, Weijun &
Weyand, Tobias & Andreetto, Marco & Adam, Hartwig. (2017), ‘MobileNets: Efficient Convolutional
Neural Networks for Mobile Vision Applications’ vol. 8, no. 3, pp. 65-73.
[8] Guo, T., Dong, J., Li, H., & Gao, Y. (2017), ‘Simple Convolutional Neural Network on Image
Classification’, IEEE 2nd International Conference on Big Data Analysis (ICBDA), Beijing, pp. 721-
724.
[9] Liu, Weiyang & Wen, Yandong & Yu, Zhiding & Yang, Meng. (2016a), ‘Large-Margin Softmax Loss
for Convolutional Neural Networks, Proceedings of the 33rd International Conference on Machine
Learning, New York, USA, pp. 507-516.
[10] Mitra, V., Chia-Jiu Wang, & George Edwards (2003), ‘Neural network for LIDAR detection of
fish’, Proceedings of the International Joint Conference on Neural Networks, Portland, OR, pp. 1001-
1006.
[11] Szegedy, Christian & Liu, Wei & Jia, Yangqing & Sermanet, Pierre & Reed, Scott & Anguelov,
Dragomir & Erhan, Dumitru & Vanhoucke, Vincent & Rabinovich, Andrew. (2015), ‘Going Deeper with
Convolutions’, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA,
pp. 1-9.
[12] Szegedy, Christian & Vanhoucke, Vincent & Ioffe, Sergey &Shlens, Jon & Wojna, ZB. (2016),
‘Rethinking the Inception Architecture for Computer Vision’, IEEE Conference on Computer Vision and
Pattern Recognition (CVPR), Vegas, NV, pp. 4278-4284.
[13] Y. Le Cun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard and L. D. Jackel
(1990), ‘Handwritten Digit Recognition with a Back-Propagation Network’, Neural Information
Processing Systems Conference, AT&T Bell Laboratories, Holmdel, N. J., pp. 396-404.
[14] Mohammad (Ashkan) Sharafsaleh, P.E.,Marcel Huijser, PhD, Christopher Nowakowski, Mark C.
Greenwood, PhD, Larry Hayden, Jonathan Felder, and May Wang