A Surveillance System for Drone Localization and

Tracking Using Acoustic Arrays

Xianyu Chang∗ , Chaoqun Yang‡ , Junfeng Wu∗ , Xiufang Shi∗ and Zhiguo Shi‡
∗
Anti Unmanned-vehicle Group, State Key Lab. of Industrial Control Technology, Zhejiang University, China
‡
College of Information Science and Electronic Engineering, Zhejiang University, China
Email: xiany.chang@gmail.com, {chaoqunyang, jfwu}@zju.edu.cn, xfshi.zju@gmail.com, shizg@zju.edu.cn

Abstract—The wide proliferation of drones has posed great Drone surveillance using acoustic arrays [16–18], as a
threats to personal privacy and public security, which makes promising drone surveillance method, could circumvent the
it urgent to monitor the drones in the sensitive areas. Drone above limitations and be used to localize and track the
surveillance using acoustic arrays, as a promising method, could
localize and track the drones effectively. In this paper, we develop drones. In [17], a tetrahedral shape acoustic array with four
a systematic method for drone localization and tracking by microphone sensors is used to estimate the direction of arrival
using acoustic arrays. Specifically, we develop a time difference (DOA) of the drone. The azimuth and the elevation angles are
of arrival (TDOA) estimation algorithm based on Gauss priori obtained by multiple signal classification (MUSIC) algorithm.
probability density function to overcome the multipath effect and However, it has very low accuracy and fails to track the target.
the low signal-to-noise ratio (SNR), and design a localization
method by making full use of the TDOA estimation results, Besides, due to lack of ground truth, the performance has not
followed by tracking the drone by Kalman filter. Finally, field been evaluated. In [18, 19], the authors develop and build
experiments are carried out to verify the effectiveness and the acoustic aircraft detection (AAD) system. The system can
performance of the proposed methods. The results show that detect, track and classify small planes, helicopters and ultra-
they can achieve good performance in drone localization and light aircraft. However, the authors do not consider the effect
tracking.
Index Terms—drone surveillance, acoustic arrays, localization, of multipath and low signal-to-noise ratio (SNR) condition
tracking in a more complex environment. In [16], a acoustic array
consists of two perpendicular linear arrays of microphones
I. I NTRODUCTION along x-axis and y-axis, respectively, is applied to track the
flying drones. The estimation of flight parameters are based on
Recently, the proliferation of drones, i.e., small unmanned Doppler effect and the maximum likelihood method is chosen
aerial vehicles (UAV), has benefited us a lot in many appli- to estimate the DOA. However, they can only track the flying
cations, e.g., drone-assistant logistics, UAV aerial photograph, drones with the assumption that the drone is flying along a
etc. Nevertheless, in the wrong hands, drones can be dan- straight line. So it still can not give the three-dimensional
gerous. For example, from 2016 to 2017, some unauthorized locations of the drone.
drones flew over the airports leading to a serious threat to Motivated by above, in this paper, we build up a drone
airplanes taking off and landing [1, 2]. Besides, drones can surveillance system using acoustic arrays for drone local-
also be used to carry explosive, transport drugs, steal personal ization and tracking. We propose a series of algorithms to
privacy and attack people by criminals. Such security breach improve the accuracy of localization and tracking. Three-
incidences have met a sudden increase in recent years and dimensional trajectory of the flying drone can be calculated
posed great threats to public security and individual privacy. in real time. The contributions of this paper are summarized
Therefore, it is of great significance to localize and track the as follows:
drones in the sensitive areas.
Drone surveillance is urgent but challenging, due to the 1) According to actual acoustic signals, we develop a new
drone’s small size, low flying speed at low altitude. And drone time difference of arrival (TDOA) estimation algorithm
surveillance has received many researches in recent years [3– by using Gauss function based on GCC-PHAT [20] in
10]. Though many existing surveillance technologies, e.g., order to overcome the multipath effect and low SNR.
radar [11, 12], video-based surveillance [13, 14], radio fre- 2) To have good localization performance, we propose a
quency (RF) based surveillance [15], etc., are applicable, their localization scheme to estimate the location of the flying
performances are limited due to various reasons, such as small drone by making full use of the TDOA information.
radar cross section for radar systems, difficulty in detecting a 3) Field experiments are carried out to evaluate the perfor-
flying drone using video and lack of communication signals mance of the proposed methods and verify the effective-
in GPS navigation mode for RF-based surveillance. ness of the system.
The rest of this paper is organized as follows. The system
This work is partially supported by NSFC under Grant 61772467, Zhejiang deployment is illustrated in Section II. The drone localization
Provincial Natural Science Foundation of China under Grant LR16F010002,
973 Project under Grant 2015CB352503, the Fundamental Research Funds
and tracking methods are presented in Section III. Experiment
for the Central Universities (2017XZZX009-01) and Project funded by China evaluation is described in Section IV, followed by conclusion
Postdoctoral Science Foundation in Section V.
 II. S YSTEM DEPLOYMENT weaken the influence of low frequency wind noise. The
In this section, we will first introduce the drone surveillance acoustical-absorbent materials are wrapped up on the ground
system, then describe how the acoustic array is deployed. of every acoustic array in order to weaken the multipath effect.

A. System overview III. D RONE LOCALIZATION AND TRACKING METHODS

The system, which has been deployed in a building roof In this section, we will introduce the drone localization
in Zhejiang University, Hangzhou, China (see Fig. 1), mainly and tracking methods. In signal processing field, there are
consists of two tetrahedron-shape acoustic arrays and a pro- mainly three methods for acoustic source localization [21],
cessing center. The acoustic arrays are used to sample the i.e., direction of arrival (DOA) [22], received signal strength
surrounding acoustic signal periodically and the processing (RSS) and TDOA. The signal subspace decomposition meth-
center is used to control acoustic signal sampling, to access ods, i.e., MUSIC and incoherent signal-subspace method, are
the data and to run the localization and tracking algorithms. commonly used to get the DOA estimation, however, they are
This system was built in 2017. A large number of experiments sensitive to the noise and with high computational complexity;
were carried up on this system and rich data were collected. meanwhile, more microphone sensors are needed to ensure
For this system, we proposed a series of new algorithms which estimation accuracy. Due to passive localization, the RSS of
mainly include TDOA estimate algorithm based Gauss priori the drone’s acoustic signal can not be obtained beforehand
probability and a high precision localization method for drone and the accuracy is vulnerable to environmental interference
localization. The algorithms will be detailed in the following. , so it is not commonly used in acoustic signal localization.
TDOA is widely used for acoustic source localization. It is
usually calculated through the generalized cross correlation
(GCC) function, which is considered to have low computa-
tional complexity, high accuracy and good robustness. So we
consider to localize and track the drone based on TDOA by
using GCC.
However, it is important to improve the TDOA estimation
accuracy which will influence the localization and tracking
results significantly. Due to multipath effect and low SNR,
which are caused by acoustic signal reflection and low strength
of drone’s acoustic signals, the TDOA estimation will be a
tough task. So in this section, we first propose a algorithm to
improve the TDOA estimation accuracy based on GCC [20].
Fig. 1: The system deployment Then, we develop a localization method to get the locations of
the drone. Finally, we establish the dynamic system equations
B. Acoustic array deployment of the flying drone and apply Kalman filter to track it.
The acoustic array model is shown in Fig. 2, which mainly
consists of two completely same acoustic arrays and every A. TDOA estimation
acoustic array is composed of 4 microphone sensors. We We use m and n represent the Mic-m and Mic-n for
assign the label Mic-1, Mic-2, · · · , Mic-8 for every sensor. acoustic array-1 in Fig. 2, where m, n = 1, 2, 3, 4 and m 6= n.
The spacing between the microphone sensor and the bottom We use xm (t) and xn (t) to denote the acoustic signals
center of the tetrahedron-shape array is 1m in every single received by two microphone sensors in acoustic array-1 and
array. The distance between the two acoustic arrays is 14m. Gxm xn (f ) to denote the Fourier transform of their cross
correlation function. The GCC is used to calculate the TDOA:
Z∞
Rxm xn (τ, k) = Gxm xn (f )ϕmn (f )e−j2πf τ df, (1)
−∞

where Rxm xn (τ, k) is known as cross power spectral den-

sity function (CPDF) at time k, in which the time corre-
sponding to the peak value is the TDOA estimation result;
ϕmn (f ) is the frequency domain prefilter of the two signals;
in our system, the phase transform (PHAT) is used, then
Fig. 2: The acoustic arrays ϕmn (f )= |Gx 1x (f )| . However, due to multipath effect and
m n
low SNR, there exist large the estimation errors. The TDOA
The microphone sensors used in our system are type CHZ- results estimated by GCC-PHAT can not be used directly in
213 1/2-inch metrological microphones and every sensor is our system.
equipped with a pre-amplifiers of type YG-201. All the micro- According to the acoustic arrays in Fig. 2, we can calculate
phone sensors have been waterproof treated in order to work 6 independent TDOA estimation results between any pair of
well under rain and snow weathers. Meanwhile, windproof sensors in acoustic array-1. Fig. 3 shows the TDOA estimation
covers have been installed on every microphone sensors to results calculated by GCC-PHAT when a drone is flying
away from the acoustic array-1. Though acoustical-absorbent B. Drone localization
materials can partly overcome the multipath effect, the results
are still with large errors especially when the SNR is very low. We denote the locations of the microphone sensors as Sm ,
From this figure, we can see that the GCC-PHAT is invalid Sn for acoustic array-1 and the drone’s location as S0 , and
after 90s due to low SNR. Sm , Sn , S0 ∈ R3 . Then we can write the following equation
−3
according to the microphone sensor pair Mic-m and Mic-n:
x 10
5
t21 t32 dmn = kSm − S0 k − kSn − S0 k , (5)
4
t31 t43
3 t41 t42 where dmn = cτmn , dmn is the path difference when drone’s
2 acoustic signal arrives at microphone sensor Sm and Sn .
1 In our system, 12 independent TDOA results can be estimat-
TDOA (s)

0 ed from acoustic array-1&2. Therefore, 12 equations can be

−1 written as the same form of (5). In fact, we can’t get the exact
−2 TDOA estimation results, the noise are inevitably included in
−3 the TDOA estimation result τmn , so we can write:
−4
τmn =τ̃mn + εmn , (6)
−5
0 20 40 60 80 100 120 140 160
time (s) where εmn is the TDOA measurement noise, τ̃mn is
the exact TDOA estimation result. Then we substitute
Fig. 3: TDOA estimation results using GCC-PHAT (5) into (6) and combine all the 12 equations of the
Therefore, we propose a new algorithm to solve this prob-
lem based on the Gauss priori probability density function two acoustic arrays
 togetherT as a  optimizationT problem
(GPDF). This algorithm mainly makes use of the relevance of and define T = τ21 · · · τ87 , E= ε21 · · · ε87 , F =
1
 T
the TDOA estimation results for a flying drone between time c
kS2 − S0 k − kS1 − S0 k · · · kS8 − S0 k − kS7 − S0 k ,
k and time k −1. We suppose that the drone is not moving too Scov is the covariance matrix of the TDOA measurement
fast, then the TDOA will not change in a large range between noise. Then we minimize the following quadratic function:
two consecutive time. It means that the peak of CPDF at time T
−1
k will appear near the peak of CPDF at time k − 1. So we Q = (T − F ) Scov (T − F ) , (7)
construct a GPDF as following: A recursive solution can be obtained as [23]:
1 (τ −µ)2
−1 T −1
GPDF (τ, k) = √ e− 2σ2 , (2) −1
Ŝ k+1 = Ŝ k + FsT Scov Fs + λI Fs Scov (T − F ) , (8)
2πσ
P
Rxm xn (τi ,k−1)(τi −µ)2
where Fs is the partial derivative of F , λI is a regularization
where µ = τmn (k − 1), σ 2
=
i,τi ∈B
P , term. The convergence solution of Ŝ k will be the drone’s
Rxm xn (τi ,k−1)
i,τi ∈B
location estimation results.
τmn (k − 1) is the TDOA estimation result at time k − 1, However, this iterative method is sensitive to the initial
Rxm xn (τ, k − 1) is the CPDF at time k − 1, B is the block value. Therefore, we should give a good pre-estimate of the
restricted by the left and right zero points of Rxm xn (τmn (k − initial value for (8). In the following, we will illustrate how
1), k − 1) as shown in Fig. 4. Then the new CPDF at time k to estimate the initial location value of the drone.
can be given by: We square the (5) in both sides and then we transform the
R̃xm xn (τ, k) = GPDF (τ, k) Rxm xn (τ, k), (3) equation into a standard matrix form:
 
where Rxm xn (τ, k) is the CPDF at time k and calculated by  S0T T
− Sn SnT − d2mn , (9)

2(Sm − Sn ) 2dmn = Sm Sm
(1), R̃xm xn (τ, k) is the new CPDF at time k, and the TDOA dn0
estimation result at time k can be given as (see Fig. 4): where dn0 is the distance between drone and the Mic-n. We
τmn (k) = argmax R̃xm xn (τ, k) , τ ∈ [−d/c, d/c] , (4) choose n = 1 as the reference node and can write (9) for
m = 2, 3, 4 in acoustic array-1. Similarly, choose ñ = 5 as
where d is the distance between Mic-m and Mic-n, c is the the reference node and can write the similar equations for
speed of acoustic signal. m̃ = 6, 7, 8 in acoustic array-2. We denote dñ0 as the distance
0.4 between drone and the Mic-ñ, then we can write:
Rx x
(t ,k−1)
m n

0.3
B
Rx
m n
x
(t ,k)
AXLos = B, (10)
GPDF(t ,k)
0.2 −
amplitude

Rx x (t ,k)
m n

S2 − S1 d21 0
 
0.1

0
 S3 − S1 d31 0 
 S − S1 d41 0 
 
−0.1 A = 2 4 , (11)
TDOA  S6 − S5 0 d65 

−18 −16 −14 −12 −10 −8 −6 −4 −2 0 S − S
t (s) x 10
−4
7 5 0 d75 
S8 − S5 0 d85
Fig. 4: The GPDF
 −3
x 10
5
S2 S2T − S1 S1T − d221
 
t21 t
32

 S3 S3T − S1 S1T 2 4
− d31  t31 t43

 S ST − S1 S1T − d241 
  3 t41 t
42

B =  4 4T 2 . (12)
 S6 S6 − S5 S5T − d65  2

 S ST − S5 S5T − d275  1

TDOA (s)
7 7
S8 S8T − S5 S5T − d285 0

−1
The result can be given by least square solution: −2
 
S0T
−1 T
−3

XLos =  d10  = AT A A B. (13) −4

d50 −5
0 20 40 60 80 100 120 140 160
time (s)

The inial location value for (8) can be obtained from XLos ,
then we can calculate the final location of the flying drone. Fig. 5: The TDOA estimation results after filtering

C. Drone tracking reached more than 100m with the SNR lower than -5dB. The
estimation errors and CDF of the errors are shown in Fig. 7.
Kalman filter is widely used in tracking problems because
From these two figures, we can see that more than 95% of the
of good real-time and low computational complexity. So in
estimation errors are lower than 6m and 80% of the estimation
our system, we use Kalman filter to track the flying drone. We
errors are within 2m. The results shown in Fig. 6 and Fig. 7
use the first order system to approximate how the flying drone
also verify the effectiveness of the TDOA estimation algorithm
moves, then the dynamic system equations can be written as:
     in section III.A.
X(k) I ∆I X(k − 1)
= + wn (k − 1), (14)
Ẋ(k) 0 I Ẋ(k − 1) acoustic array−1
acoustic array−2
GPS
150
  trajectory of the drone
  X(k)
Y(k) = I 0 + vn (k), (15) 100
Ẋ(k)
Z (m)

 T 50
where X(k)= x̂ ŷ ẑ is the trajectory of the drone, I is a
unit matrix, ∆ is the sampling time with a constant value, 0

the covariance of process noise wn (k) and measurement

40
noise vn (k) matrices might change with each time step or
measurement, however here we assume they are constant. 20
−50
−40
Then the ultimate drone’s tracjectory can be estimated by the 0 −20
−30
−10
Kalman filter. X (m) 10
0
Y (m)

IV. E XPERIMENT E VALUATION

Fig. 6: The trajectory of the flying drone
In this section, we conduct field experiments to verify
20 1

the performance of all the proposed algorithms respectively. 18

error
0.9

The DJI Phantom 2 is used in our experiments for drone 16

14
0.8

0.7

localization and tracking. The TDOA estimation results of 12 0.6

error (m)

CDF

acoustic array-1 by applying the proposed algorithm in section 10

8
0.5

0.4

in III.A are shown in Fig. 5. From this figure, we can find 6 0.3

that the abnormity TDOA estimation results are filtered. A 4

2
0.2

0.1
CDF of the error

complete and single TDOA estimation path is obtained for 0

0 20 40 60 80 100 120 140 160
0
0 2 4 6 8 10
time (s) error (m)

every microphone sensor pair, however, we still can’t evaluate

the accuracy. But the accuracy will be verified in the following (a) Estimation errors (b) CDF of the estimation errors
parts by applying these TDOA estimation results to localize
the drone when compared with the GPS ground truth. Fig. 7: Estimation errors of the drone’s trajectory
We use the algorithm described in section III.B to calculate
the locations of the flying drone. The locations of the drone are
calculated though (8). Then Kalman fiter is used to estimate V. C ONCLUSION
the final trajectory of the flying drone based on (14) and (15). In this paper, we introduce a drone surveillance system
The GPS data and the final trajectory results are shown in Fig. using acoustic arrays. A new TDOA estimation algorithm
6. From this figure, we can find that the proposed methods give and a localization algorithm are proposed in order to localize
a good estimation of the drone’s location, and the results are and track the drone more accurately. Field experiments are
nearly the same with the GPS data. The acoustic arrays have conducted to verify the the effectiveness and the performance
a good performance to track the flying drone, which verifies of these methods. It proves that this system can localize and
the effectiveness of our system. And the longest distance has track the drone effectively.
 R EFERENCES Matwyschuk, Pierre Naz, Stéphane Schertzer, and Gwenael
Schmitt, “Optical and acoustical UAV detection,” in Proceed-
[1] State Times, “Drone delays 55 flights in china,” http://news.
ings of SPIE Conference on Security+ Defence, 2016, pp. 1–13.
statetimes.in/drone-delays-55-flights-china/.
[18] Alexander Sedunov, Alexander Sutin, Nikolay Sedunov, Hady
[2] Euan McKirdy an d Serenitie Wang, “Drone’s operator detained
Salloum, Alexander Yakubovskiy, and David Masters, “Passive
for flying near Chinese airplane,” http://edition.cnn.com/2017/
acoustic system for tracking low-flying aircraft,” IET Radar
01/17/asia/china-drone-passenger-plane-near-miss/.
Sonar and Navigation, vol. 10, no. 9, pp. 1561–1568, 2017.
[3] Guoru Ding, Qihui Wu, Linyuan Zhang, Yun Lin, Theodoros A.
[19] H Salloum, A Sedunov, N Sedunov, and A Sutin, “Acoustic
Tsiftsis, and Yu Dong Yao, “An amateur drone surveillance
system for low flying aircraft detection,” in Proceedings of
system based on the cognitive internet of things,” IEEE
IEEE International Symposium on Technologies for Homeland
Communications Magazine, vol. 56, no. 1, pp. 29–35, 2018.
Security, 2015, pp. 1–6.
[4] Mohammad Mahdi Azari, Hazem Sallouha, Alessandro Chi-
[20] C. Knapp and G. Carter, “The generalized correlation method
umento, Sreeraj Rajendran, Evgenii Vinogradov, and Sofie
for estimation of time delay,” IEEE Transactions on Acoustics
Pollin, “Key technologies and system trade-offs for detection
Speech and Signal Processing, vol. 24, no. 4, pp. 320–327,
and localization of amateur drones,” IEEE Communications
1976.
Magazine, vol. 56, no. 1, pp. 51–57, 2018.
[21] Xiufang Shi, Guoqiang Mao, Brian D. O. Anderson, Zaiyue
[5] Zhiguo Shi, Chengwei Zhou, Yujie Gu, Nathan A. Goodman,
Yang, and Jiming Chen, “Robust localization using range
and Fengzhong Qu, “Source estimation using coprime array: A
measurements with unknown and bounded errors,” IEEE
sparse reconstruction perspective,” IEEE Sensors Journal, vol.
Transactions on Wireless Communications, vol. 16, no. 6, pp.
17, no. 3, pp. 755–765, 2017.
4065–4078, 2017.
[6] Dmitrii Solomitckii, Margarita Gapeyenko, Vasilii Semkin,
[22] Chengwei Zhou, Yujie Gu, Yimin D. Zhang, Zhiguo Shi, Tao
Sergey Andreev, and Yevgeni Koucheryavy, “Technologies for
Jin, and Xidong Wu, “Compressive sensing based coprime array
efficient amateur drone detection in 5g millimeter-wave cellular
direction-of-arrival estimation,” IET Communications, vol. 11,
infrastructure,” IEEE Communications Magazine, vol. 56, no.
no. 11, pp. 1719–1724, 2017.
1, pp. 43–50, 2018.
[23] Don J. Torrieri, “Statistical theory of passive location systems,”
[7] Chengwei Zhou, Yujie Gu, Shibo He, and Zhiguo Shi, “A robust
IEEE Transactions on Aerospace and Electronic Systems, vol.
and efficient algorithm for coprime array adaptive beamform-
20, no. 2, pp. 183–198, 1984.
ing,” IEEE Transactions on Vehicular Technology, vol. 67, no.
2, pp. 1099–1112, 2018.
[8] Rick Sturdivant and Edwin Chong, “Systems engineering
baseline concept of a multispectral drone detection solution for
airports,” IEEE Access, vol. 5, pp. 7123–7138, 2017.
[9] Joël Busset, Florian Perrodin, Peter Wellig, Beat Ott, Kurt
Heutschi, Torben Rühl, and Thomas Nussbaumer, “Detection
and tracking of drones using advanced acoustic cameras,” in
Proceedings of SPIE Conference on Security+ Defence, 2015.
[10] Xiufang Shi, Chaoqun Yang, Weige Xie, Chao Liang, Zhiguo
Shi, and Jiming Chen, “Anti-drone system with multiple
surveillance technologies: Architecture, implementation, and
challenges,” IEEE Communications Magazine, vol. 56, no. 4,
pp. 68–74, 2018.
[11] Multerer Thomas, Ganis Alexander, Prechtel Ulrich, Miralles
Enric, Meusling Askold, Mietzner Jan, Vossiek Martin, Loghi
Mirko, and Ziegler Volker, “Low-cost jamming system against
small drones using a 3d mimo radar based tracking,” in
Proceedings of the 14th European Radar Conference, 2017, pp.
299–302.
[12] Dong Hun Shin, Dae Hwan Jung, Dong Chan Kim, Jong Wook
Ham, and Seong Ook Park, “A distributed fmcw radar system
based on fiber-optic links for small drone detection,” IEEE
Transactions on Instrumentation and Measurement, vol. 66, no.
2, pp. 340–347, 2017.
[13] Artem Rozantsev, Vincent Lepetit, and Pascal Fua, “Detecting
flying objects using a single moving camera,” IEEE Transac-
tions on Pattern Analysis and Machine Intelligence, vol. 39, no.
5, pp. 879–892, 2017.
[14] Hakki Erhan Sevil, Atilla Dogan, Kamesh Subbarao, and Brian
Huff, “Evaluation of extant computer vision techniques for
detecting intruder suas,” in Proceedings of International Con-
ference on Unmanned Aircraft Systems, 2017, pp. 929–938.
[15] Phuc Nguyen, Hoang Truong, Mahesh Ravindranathan, Anh
Nguyen, Richard Han, and Tam Vu, “Matthan: Drone presence
detection by identifying physical signatures in the drone’s rf
communication,” in Proceedings of International Conference
on Mobile Systems, Applications, and Services, 2017, pp. 211–
224.
[16] Jianfei Tong, Wei Xie, Yu-Hen Hu, Ming Bao, Xiaodong Li, and
Wei He, “Estimation of low-altitude moving target trajectory
using single acoustic array,” The Journal of the Acoustical
Society of America, vol. 139, no. 4, pp. 1848–1858, 2016.
[17] Frank Christnacher, Sébastien Hengy, Martin Laurenzis, Alexis