Intelligent Transportation Infrastructure, 2022, 1, 1–11

https://doi.org/10.1093/iti/liac009
Advance access publication date 30 September 2022
Review

Structural health monitoring of railway bridges using

innovative sensing technologies and machine learning
algorithms: a concise review

Downloaded from https://academic.oup.com/iti/article/doi/10.1093/iti/liac009/6732506 by guest on 04 November 2024

1,2 1,2, 1,2
You-Wu Wang , Yi-Qing Ni *, Su-Mei Wang
1 Departmentof Civil and Environmental Engineering, The Hong Kong Polytechnic University, Hung Hom, Kowloon, HKG, Hong Kong, China
2 National
Rail Transit Electrification and Automation Engineering Technology Research Center (Hong Kong Branch), The Hong Kong Polytechnic University, Hung
Hom, Kowloon, HKG, Hong Kong, China

*Corresponding author: Yi-Qing Ni, Email: ceyqni@polyu.edu.hk (Y.Q. Ni)

Abstract
Railway bridges are a vital element of railway infrastructures, and their safety can directly affect the regional economy and commuter
transportation. However, railway bridges are often subjected to severe loading and working conditions, caused by rising traffic levels
and heavier vehicles, and increases in train running speeds makes the bridges extremely susceptible to degradation and failure. One
of the promising tools for evaluating the overall safety and reliability of railway bridges is the bridge structural health monitoring
(SHM) system, which not only monitors the structural conditions of bridges and maintains the safety of train operations, but also
helps to expand the lifespan of bridges by enhancing their durability and reliability. While a multitude of review papers on SHM and
vibration-based structural damage detection methods have been published in the past two decades, there is a paucity of literature that
provides a review or overview on the SHM of railway bridges. Some of the review papers have become obsolete and do not ref lect the
state-of-the-art research. Therefore, the main goal of this article is to summarize state-of-the-art SHM techniques and methods that
have been widely used and popular in recent years. First, two state-of-the-art SHM sensing technologies (i.e. fiber optic sensing (FOS)
technology and computer vision-based (CV) technology) are reviewed, including the working principles of various sensors and their
practical applications for railway bridge monitoring. Second, two state-of-the-art machine learning algorithms (i.e. convolutional neural
networks (CNN) and transfer learning (TL)) and their applications for railway bridge structural condition assessment are exemplified.
Third, the principle of digital twin (DT) and its applications for railway bridge monitoring are presented. Finally, issues related to the
future direction and challenges of the monitoring technologies and condition assessment methods of railway bridges are highlighted.

Keywords: railway bridges, structural health monitoring, fiber optic sensing technology, computer vision-based technology, convolu-
tional neural networks, transfer learning, digital twin

Introduction played an important role in detecting defects on a structure’s

Railway infrastructure has expanded hugely in many countries surface and assessing structural conditions. However, visual
due to increasing demand for transport capacity for both goods inspection is labor-intensive, time-consuming, and unable to track
and passengers in recent decades. For example, China has structural conditions in real time. Furthermore, visual inspection
expanded the total length of its high-speed rail (HSR) to more relies heavily on expert knowledge, which significantly affects
than 40,000 kilometers in the past two decades, making it the accuracy and reliability of structural condition assessment
the longest HSR network in the world (China Daily, 2021). As due to assessor subjectivity (Vagnoli et al., 2018). To address
railway infrastructure is exposed to various external effects, this challenge, structural health monitoring (SHM) techniques
such as increasing traffic loads and harsh environments, the have been proposed and applied to maintaining bridge operation
performance of railway infrastructure gradually deteriorates. safety.
This can lead to catastrophic failure and risks the occurrence The main purpose of implementing SHM systems on railway
of dangerous accidents. Railway bridges are a vital element of rail bridges is to maintain operational safety, expand the lifespan
infrastructure, which requires a deep understanding of the daily of the structure, and reduce maintenance costs by enhancing
operation and safety issues because their lines are used by a large early warning and timely damage detection (Farrar and Worden,
number of passengers every day. In China, half of the total length 2007). To the authors’ knowledge, there has been a large body of
of high-speed railways was built on bridges. For example, more literature delineating the application of SHM systems for mon-
than 85 per cent of the Beijing–Shanghai High-Speed Railway is itoring large-scale bridges, mainly focusing on highway bridges;
composed of pre-stressed concrete simply supported box girder while only very few publications have focused on reviewing SHM
bridges (Qin and Gao, 2017). Traditionally, visual inspection has application for railway bridges.

Received: July 18, 2022. Revised: September 9, 2022. Accepted: September 16, 2022
© The Author(s) 2022. Published by Oxford University Press and Southwest Jiaotong University.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which
permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.
2 | Intelligent Transportation Infrastructure, 2022, Vol. 1, No. 1

Seo et al. (2016) provided an extensive review of the application displacement sensing systems can simultaneously track multiple
of SHM systems to highway bridges, where damage detection, points from a long distance (Feng and Feng, 2018).
structural capacity evaluation, and remaining service life estima- Among the data interpretation methods for SHM signal pro-
tion of bridge structures were summarized. With a focus on struc- cessing, the emerging machine learning methods, such as con-
tural health monitoring and fault detection in railway bridges, volutional neural networks (CNNs) and transfer learning (TL)
Vagnoli et al. (Vagnoli et al., 2018) reviewed the most commonly will be highlighted in this paper. With recent advances in non-
adopted bridge fault detection methods, i.e. model-based finite contact sensing technology, such as cameras and unmanned
element updating strategies and non-model-based fault detec- aerial vehicles in the SHM field, there has been a significant trend
tion methods, and produced a comparative study using a short- towards using machine learning methods such as CNNs to con-
span railway bridge to evaluate and compare the performance of duct local damage assessment and anomaly detection in larger-
the reviewed methods, and finally highlighted the opportunities scale civil structures (Sony et al., 2021). As the most popular class
and future challenges of the fault detection methods for railway of machine learning methods, CNNs have been widely applied in

Downloaded from https://academic.oup.com/iti/article/doi/10.1093/iti/liac009/6732506 by guest on 04 November 2024

bridges. Sony et al. (2019) presented a comprehensive review of image-based feature extraction since their breakthrough in the
the next-generation smart sensing technologies that were devel- 2012 ImageNet Challenge. A major assumption in many machine
oped in recent years for SHM, especially the recent application learning and data mining algorithms is that the training and
of smartphones, cameras, unmanned aerial vehicles (UAVs), and future data are in the same feature space and stick to the same
robotic sensors. Sun et al. (2020) conducted a comprehensive distribution. However, this assumption may not hold in many
review of the use of big data (BD) and artificial intelligence (AI) real-world applications. In such a case, transfer learning (TL) has
techniques in addressing data interpretation in bridge SHM, and emerged as a promising methodology to address this problem
discussed the upper limits of monitoring, challenges, and future (Zhuang et al., 2020).
trends. Shokravi et al. (Shokravi et al., 2020) reviewed conventional This review article is organized as follows. Section 2 describes
vehicle-based health monitoring methods used in the SHM of the types of FOS technology and CV technology, and their applica-
bridges, and assessed the performance of vehicle-assisted meth- tions on monitoring railway bridges. Section 3 outlines the princi-
ods. Rizzo and Enshaeian (2021) reviewed the challenges in bridge ples of CNN and TL, as well as their application in addressing data
health monitoring with regard to the detection of specific bridge interpretation problems in relation to bridge structural health
characteristics such as anomalous behavior, with the aim of monitoring. Section 4 introduces the application of digital twin
helping newcomers, practitioners, and researchers navigate the (DT) in the SHM of railway bridges. Section 5 discusses the future
many methodologies that had been developed for damage iden- challenges in SHM of railway bridges and provides conclusions.
tification using monitoring data from various sensors installed in
real structures.
Although most of the review papers focus on highway bridges, Sensing technologies in SHM
the monitoring techniques and data interpretation methods are Fiber optic sensing technology
general and can also be applied to railway bridges. As a railway Currently, fiber optic sensing (FOS) technology is moving rapidly
bridge network is not as f lexible as a road network, where the towards becoming an intelligent, miniaturized, and digital health
latter usually involves a number of alternative routes, a service monitoring system (Barrias et al., 2016). A significant number of
interruption due to a railway bridge closure is very serious. There- fiber optic sensors have been utilized for SHM due to their unique
fore, a robust and reliable SHM system is crucial to ensure the nor- and specific advantages, such as their miniature size, light weight,
mal operation of railway bridges and minimize such interruptions. resistance to corrosion, immunity to electromagnetic interfer-
In 2015, an unexpected tugboat collision at the cable-stayed Kap ence, and distributed sensing capabilities. The FOS technology
Shui Bridge (carrying both highway and railway traffic) caused adopted for railway bridge monitoring can be divided into three
the emergency closure of all traffic lanes (including two lines of categories: grating based sensors, interferometry based sensors,
rail tracks) leading to the Hong Kong International Airport. As and ref lectometry based sensors (Leung et al., 2015; Du et al.,
the closure lasted for approximately two hours, it lead to more 2020; Sasy Chan et al., 2021).
than 10,000 travelers being stranded at different locations (Guo et
al., 2017). This article focuses on reviews of advanced monitoring Grating based sensors
techniques and state-of-the-art data interpretation methods as Grating based sensors detect the effect of grating in the core of
applied to railway bridges. In terms of monitoring technologies, optical fibers. When light travels through an optical fiber, the
the application of fiber optic sensing (FOS) and computer vision- grating will serve as a narrow band filter to modulate a range of
based (CV) to railway bridges will be particularly highlighted. wavelengths, known as Bragg wavelengths. Variations of temper-
Being a form of wired system, a number of innovative sensing ature, strain, and vibration due to the surrounding environment
systems based on fiber optic technology have been explored can cause the Bragg wavelength to shift. There are three typical
and exploited in the civil engineering community in the recent grating based sensors: fiber Bragg grating (FBG) sensor; long period
two decades as a result of their inherent distinctive advantages: fiber sensor; and tilted fiber Bragg grating sensor (Du et al., 2020);
small size, light weight, resistance to corrosion, and immunity among them, the most commonly used in SHM applications are
to electromagnetic interference. Fiber optic sensing technology is the FBG sensors, whose principle of operation is shown in Fig. 1.
capable of measuring acceleration, strain, temperature, displace- In an FBG sensor, the ref lected wavelength known as the Bragg
ment, cracks, and more (Figueiredo and Brownjohn, 2022) in a wavelength (λB ) is given by:
multiplexing manner. Non-contact sensing is another promising
technology for SHM, which has made great progress and gained
λB = 2n (1)
wide application in the past decade. For example, cameras (pho-
togrammetry and videogrammetry) have shown their capacity to
rapidly assess the surface condition of bridge components, even where λB represents the central wavelength of FBG at strain or
in otherwise hard-to-access areas of the structure. Vision-based temperature free stage, n is the effective refractive index and 
 Y. W. Wang et al. | 3

on the principle of optical time-domain ref lectometry (OTDR), as

shown in Fig. 3. The measurement mechanism of the distributed
fiber optic sensors is to record the backscattering of optical pulses
utilizing an optical fiber, serving as a sensor over its length. If the
temperature, strain, and vibration change in a localized position
of the fiber, the backscattering of signal will be modulated and
varied in amplitude, frequency and phase, thus enabling changes
in the surrounding environment to be detected.
Figure 1. Principle of a fiber Bragg grating (FBG) sensor (Du et al., 2020) Currently, distributed fiber optic sensors have three major
categories: Raman (Failleau et al., 2018), Rayleigh (Wheeler et al.,
2018), and Brillouin scattering (Bao & Chen, 2011). Raman based
is the grating period of optical fiber. The deformation of an FBG sensors only detect temperature, while Rayleigh and Brillouin

Downloaded from https://academic.oup.com/iti/article/doi/10.1093/iti/liac009/6732506 by guest on 04 November 2024

sensor causes a shift in the Bragg wavelength (λB ) due to the effect based sensors can measure temperature, strain, and vibration.
on the variation of grating period and refractive index induced by The spatial resolution (z) of scattering based sensors can be
strain or temperature. The relative wavelength shift is given by calculated by:
τc
λB z = (5)
= kε ε + kT T (2) 2neff
λB
where τ is the pulse width of input pulse light, c is the velocity
where kε is the strain coefficient and kT is the temperature coef- of light, and neff is the effective ref lective index of the fiber. The
ficient of the FBG sensor. range of pulse spectral width of Rayleigh, Brillouin, and Raman
scatting is MHz, GHz, and THz, respectively (Uchida et al., 2015).
Interferometry based sensors
According to the different interferometry principles, the interfer- FOS technology for railway bridge monitoring
ometry based sensors can be classified into two categories: Fabry- Owing to its unique and specific advantages, FOS technology
Perot (FP) based sensors (Wu et al., 2012) and Mach-Zehnder (MZ) has shown great potential in the real time monitoring of civil
based sensors (Zhang et al., 2013). The FP based sensors follow infrastructure systems, including railway bridges. The first com-
the principle of FP interferometry, which uses the intensity of prehensive review on monitoring railway infrastructure using
interferometry signals illuminated between two parallel ref lect- FOS technology was conducted by Du et al. (2020). The state-of-
ing interfaces, as shown in Fig. 2(a). The interference pattern shifts the-art of fiber optic sensing/monitoring technologies, including
depending on the optical path change and the length of the FP the basic principles of various fiber optic sensors, novel sensing
cavity. For FP based sensors, the intensity of interference signal I and computational methodologies, and practical applications on
is specified by: railway infrastructure monitoring were reviewed in detail in their
  paper. Chan et al. (2006) deployed three arrays of FBG sensors
 4π Z
I = 2I1 I2 I1 I2 × cos +φ (3) on the hanger cables, rocker bearing, and truss girders of Hong
λ
Kong’s landmark Tsing Ma bridge (TMB) to investigate the fea-
sibility of using the devised FBG sensors for bridge SHM, by
where I1 and I2 are ref lection at the ref lective surfaces, φ is
monitoring the strain at different portions of TMB under both
the initial phase of the interference, λ is the optical wavelength,
highway and railway loads and comparing the FBG sensors’ per-
and Z represents the optical path difference. The sensitivity of
formance with conventional strain sensors. Scott et al. (2013)
FP sensors is defined as the deformation at the center of the
described the design, commissioning, and evaluation of a fiber
diaphragm under pressure:
optic strain sensory system for the SHM of a prestressed concrete
post-tensioned box girder railway bridge. Cracks in the boxes were
   4
3 1 − υ 2 2d successfully identified during the load tests. Yoon et al. (2016)
Yc = (4) applied a distributed fiber optic sensory system based on Brillouin
16Eh3
optical correlation domain analysis (BOCDA) to the rail and girder
where Yc is the sensitivity of the FP sensor, d is the diameter of of a railway bridge to measure the distribution of longitudinal
diaphragm, h is the thickness of the film of the diaphragm, υ is strain occurring at the rail and girder during train passage. Van
the Poisson ratio, and E represents the elastic modulus. The FP Der Koori et al. (2018) investigated the use of distributed fiber optic
based sensors can be utilized for the detection of pressure change sensing systems in an in-service steel railway bridge. Three load-
and temperature variation (e.g. environmental change). ing scenarios were imposed on the bridge to evaluate the sensing
The MZ-based sensors follow the principle of Mach-Zehnder capabilities of distributed static and dynamic strain sensing for
interferometry, which is the intensity of the phase shift variations bridge evaluation. Ye et al. (2020) used the distributed (Brillouin
illuminated between two beams, as shown in Fig. 2(b). The inter- optical time domain analysis, BOTDA) and discrete (fiber Bragg
ference pattern shifts according to the optical path change and grating, FBG) optic sensory systems to measure strain and tem-
depending on the length of paths. The MZ-based sensors can be perature to investigate pre-stress loss in the pre-stressed concrete
employed to detect temperature change, air pressure change, and beams of a newly constructed railway bridge. Subsequently, Ye
f luid change. et al. (2021) evaluated the in-service structural behavior of an
operational pre-stressed concrete railway bridge using monitored
Ref lectometry based sensors data from a dense network of fiber optic strain and temperature
Scattering based distributed sensors are suitable for continuous sensors. Pimentel et al. (2021) installed a fiber optic weigh-in-
and long-distance measurement, such as long-span bridges, tun- motion system in a railway bridge for the identification of trains’
nels, and so on. The distributed fiber optic sensors are based speed, geometry, and static axle loads.
4 | Intelligent Transportation Infrastructure, 2022, Vol. 1, No. 1

Figure 2. (a) Fabry-Perot sensor; (b) Mach-Zehnder sensor (Du et al., 2020)

Downloaded from https://academic.oup.com/iti/article/doi/10.1093/iti/liac009/6732506 by guest on 04 November 2024

Figure 3. Principle of optical time-domain ref lectometry (OTDR) (Du et al., 2020)

Computer vision technology applications in computer vision, such as surveillance, human–

Computer vision algorithms computer interaction, and traffic f low monitoring. Deep SORT is a
Computer vision (CV) technology aims to create digital systems typical tracking-by-detection algorithm for multi-object tracking
that can process and analyze visual data (images and videos) in in videos (Wojke et al., 2017). By integrating appearance features
the same way as humans do These systems can be used for object extracted from a CNN-based object detector with original SORT
classification, object identification, and object tracking. A vison- algorithms, it enables real-time tracking to be achieved.
based sensory system typically consists of a video (or camera), a
zoom lens, and a computer. It may also require lighting lamps for CV for railway bridge monitoring
measurement conducted in a dimly lit scene. Computer vision based techniques have been recognized as
(1) Object classification. As a crucial role in the field of com- an innovative means for the inspection and monitoring of
puter vision, objective classification aims to automatically clas- civil engineering systems, which offer promising non-contact
sify images into pre-defined classes. Before the explosion of deep solutions to civil infrastructure condition assessment (Spencer
learning methods, extensive research efforts were made to design et al., 2019). The goal of such techniques is to automatically
scale-invariant features, feature representations, and classifiers and robustly convert image and video data into actionable
for image classification (Sánchez and Perronnin, 2011). Over the information. In conjunction with acquisition through remote
last decade, deep CNN has substantially advanced the classifica- cameras and unmanned aerial vehicles (UAVs), vision based
tion process, performing much more robustly than other conven- techniques offer the advantages of non-contact monitoring
tional methods in the presence of large-scale variations. Different methods, potentially addressing some of the challenges in using
CNN models, such as Alexines, VGGNet, GoogleNet, ResNet, and contact-type sensors.
DenseNet, were successively proposed to improve the classifica- Digital image correlation (DIC) is a vision based technique
tion performance and offer less computational complexity. that is frequently employed to measure static displacement and
(2) Object identification. Objective identification is another strain and crack propagation and to further identify structural
important and challenging task in computer vision, which aims to def lection and damage emergence (Mousa et al., 2021). The basic
detect and locate the object instances of a certain class in digital idea of DIC is to compare captured images at varying levels of
images and videos (Feng et al., 2019). A technical challenge in the deformation of a target structure and analyze them using
object identification, which is also encountered in object classi- correlation based matching algorithms. It can be divided into
fication, is that it involves processing a large number of highly three categories: two-dimensional digital image correlation (2D
variable objects. However, object identification is more difficult DIC), three-dimensional digital image correlation (3D DIC), and
than image classification, as it must determine the accurate loca- digital volume correlation (DVC) (Pan, 2018; Mousa et al., 2021).
tion of the object of interest. In the past, major research efforts The DIC methods have been applied to the field monitoring of
have been devoted to detecting a single category of given objects displacement and the strain of bridge structures. McCormick and
by designing a set of appropriate features. A striking improvement Lord (2012) used the 2D DIC technique to measure the vertical
in object identification accuracy has recently been achieved by displacement of a bridge deck under different loading condi-
taking advantage of deep learning techniques. Different object tions. Yoneyama and Ueda (2012) proposed a method to correct
detection architectures, such as R-CNN, SPPnet, Fast R-CNN, Mask the effect of camera movement in order to realize the bridge
R-CNN, and YOLO, were proposed to improve feature quality and def lection measurement by using the 2D DIC technique. Ye et
recognition accuracy (Feng et al., 2019). al. (2013) demonstrated the robustness of a devised vision based
(3) Object tracking. Object tracking is the task of automat- system through field measurement of the vertical displacement
ically identifying objects in a video and interpreting them as of the suspension Tsing Ma Bridge at mid-span under opera-
set of trajectories with high accuracy. It has a wide range of tional conditions, from which an excellent capability for real-time
 Y. W. Wang et al. | 5

remote displacement measurement was confirmed. Kim et al. for classifying an image. Pooling layers reduce the dimensions of
(2013) developed a vision-based monitoring system that utilizes the resulting features using a down-sampling operation, thereby
an image processing technique to estimate the tensile forces of reducing computational costs and the probability of overfitting.
stayed cables in a cable-stayed bridge during traffic operation, A fully connected layer has the shape of a f lattened vector and
which made it possible to estimate the forces on multiple cables is used to connect the two-dimensional convolutional layer with
using a single system. Ribeiro et al. (2014) developed an advanced the one-dimensional Softmax layer. The Softmax layer extracts fea-
video based system for non-contact measurement of the dynamic tures from the fully connected layer, calculates the probabilities
displacement of railway bridges, which can perform measure- of each class and outputs the class with highest probability as the
ment with acquisition frame rates rating from 64 fps to 500 fps classification result.
and be integrated with an SHM system. The measured displace- Both ordinary neural networks (NNs) and CNNs are feed-
ment of a railway bridge deck induced by the passage of trains forward neural networks and are generally trained using back-
demonstrated the precision of the proposed video based system. propagation. The main difference between them is the different

Downloaded from https://academic.oup.com/iti/article/doi/10.1093/iti/liac009/6732506 by guest on 04 November 2024

Feng et al. (2015a) developed a vision based sensory system for layers used to classify images. The NN uses hidden layers, whereas
the remote measurement of structural dynamic displacement by CNN uses convolutional and pooling layers along with input
tracking bridge surface features from a remote distance and then and output layers. The major disadvantage of NN is that it
applying it to two railway bridges subjected to freight trainloads trends to over-fit. Unlike NN, CNN is not prone to overfitting
travelling at various speeds. The field measurements demon- due to a reduction in weights and the number of neurons caused
strated the accuracy of the proposed remote-sensing system. Feng by the convolutional and pooling layers. Since the late 1990s,
et al. (2015b) applied the 2D DIC technique to extract the real- tremendous improvements in CNN learning methodology and
time displacement from video images based on the up-sampled architecture have been achieved to make CNN scalable to large,
cross correlation using Fourier transform. Field tests carried out heterogeneous, complex, and multiclass problems. LeNet (LeCun
on railway and pedestrian bridges showed that the displacements et al., 1998) was the first CNN architecture developed to classify
measured by a single camera were in agreement with those low-resolution images such as handwritten digit recognition.
measured by high-performance laser displacement sensors. Pan Later, AlexNet (Krizhevsky et al., 2017), VGGNet (Simonyan and
et al. (2016) proposed an off-axis 2D DIC technique by employ- Zisserman, 2014), GoogleNet (Szegedy et al., 2015), ResNet (He et
ing an inverse compositional Gauss-Newton algorithm for real- al., 2016) and others were developed to progress the deep learning.
time vertical displacement measurement of a railway bridge. It The development of newer CNN architecture evidenced a trend
is demonstrated that the proposed technique can practically and towards using more and more layers. However, application of
accurately measure the real-time displacement of railway bridges these architectures for structural damage detection requires
subjected to trains passing by. Tian et al. (2021) proposed a novel a large amount of damage-state data. Moreover, the issue of
approach for full-field bridge def lection measurement based on overfitting may arise, and the performance of high-performance
an off-axis DIC video def lectometer. Considering the bridge is CNNs may not generalize the required results for civil engineering
usually slender and narrow, the new approach can determine applications.
the scale factors of all points of interest with a spatial straight- Xu et al. (2019) proposed a modified fusion CNN architecture
line fitting scheme. Field tests on a high-speed railway bridge for crack identification in the steel box girder of bridges con-
demonstrated the technique’s effectiveness and practicality for taining complicated distracting backgrounds and handwriting.
SHM applications. Narazaki et al. (2022) proposed an approach The recognition errors of the fusion CNN during both training
for vision based autonomous UAV navigation planning for rapid and validation processes were smaller than those of the regular
post-earthquake inspections of reinforced concrete railway bridge CNN. Zhang et al. (2019) proposed a simple 1D CNN for the
viaducts. The results demonstrated that the proposed approach detection of tiny local structural stiffness and mass changes.
can reach accuracy at the centimeter level and has great potential Three independent databases, based on a T-shaped steel beam,
for post-earthquake structural inspections. a short steel girder bridge, and a long steel girder bridge were
used to validate the proposed CNN. Ni et al. (2020) proposed
a 1D CNN based technique in combination with autoencoder
Machine learning algorithms in SHM data compression for anomaly detection in long-span suspension
Convolutional neural networks (CNNs) bridges. The acceleration data measured from the SHM system
Non-contact monitoring techniques and computer vision have of a long-span bridge were employed to validate the proposed
opened up a new era for the next generation of autonomous SHM method and the results showed the method with high accuracy.
and inspection of large-scale structures (Feng and Feng, 2018; Liu and Zhang (2020) developed a CNN based condition rating
Spencer et al., 2019). These sensors result in images and videos, modeling approach for predicting the condition rating of bridge
requiring deep learning (DL) methods to train predictive mod- components in future inspection. A case study using National
els for damage classification, location, and prediction once new Bridge Inventory (NBI) data of Maryland and Delaware highway
measurements of vision signals are ready. Convolutional neural bridges demonstrated that the proposed method is a promising
networks (CNNs) are the most popular variant of DL networks. tool. Lee et al. (2021) used supervised and semi-supervised deep
The underlying architecture of CNN is comprised of three layers: learning techniques to investigate the structural integrity of a rail-
(i) convolutional layers (feature extraction), (ii) pooling layers way bridge based on initial measurements of acceleration data.
(dimensionality reduction), and (iii) fully-connect layer (Sony et For this, the conventional supervised CNN modes with different
al., 2021). Convolutional layers take an input image and convolve network architectures were compared with each other, while a
it with a filter or kernel, where the size of the kernel matrix is semi-supervised OC-CNN was considered to detect novelty in
much smaller than the size of the input matrix. Convolutions the measurement data of a simply supported railway bridge.
generate invariant local features. At the lower level, filters can be Sony et al. (2021) conducted a systematic review of CNN based
used to detect edges in image, whereas at the higher level, they techniques in the context of infrastructure monitoring and main-
can detect more complex shapes and objects that are important tenance. The f lowchart of the state-of-the-art CNN based SHM
6 | Intelligent Transportation Infrastructure, 2022, Vol. 1, No. 1

Definition of task
A task T consists of two components: A label space Y and an
objective predictive function f (·), that is,

T = Y , f (·) .

The f (·) can be used to predict the corresponding label, f (x),

given a new instance x. From a probabilistic viewpoint, f (x) can be
written as
 
P y|x .

Downloaded from https://academic.oup.com/iti/article/doi/10.1093/iti/liac009/6732506 by guest on 04 November 2024

We simply consider the case where there are one source
domain Ds and one target domain DT . The source domain data is
denoted as
   
Ds = xS1 , yS1 , . . . , xSn , ySn ,

where xSi ∈ XS is the data instance and ySi ∈ YS is the correspond-

ing class label; while the target-domain data is denoted as

   
Figure 4. A schematic of state-of-the-art CNN-based SHM practice (Sony DT = xT1 , yT1 , . . . , xTn , yTn ,
et al., 2021)
where the input xTi is in XT and yTi ∈ YT is the corresponding
output. In most cases, 0 ≤ nT  ns .

operations is summarized in Fig. 4. Hajializadeh (2022) attempted Definition of transfer learning

to employ 2D CNN algorithms for vibration based damage detec- Given a source domain Ds and a learning task Ts , transfer learning
tion using train-borne acceleration signals, where the well-known aims to help improve the learning of the target predictive function
pre-trained GoogleNet architecture was used as the basis for a fT (·) in DT using the knowledge in Ds and Ts , where Ds  = DT , or
CNN algorithm, and the hyperparameters in the algorithm were Ts  = TT .
fine-tuned by using Bayesian optimization. Ghiasi et al. (2022) In the above definition, a domain is a pair D = {X , P(X)}. Thus,
proposed a practical deep learning method for the classification of the condition Ds = DT implies that either XS = XT or PS (X)  =
various categories of damage, such as corrosion, for the safety and PT (X). Likewise, a task is a pair T = {Y , P(Y|X)}. Thus, the condition
regulatory management of in-service steel railway bridges using a Ts = TT implies that either YS = YT or PS (YS |XS ) = PT (YT |XT ). When
1D CNN developed by combining a finite element model and field the target and source domains are the same, i.e. Ds = DT , and
testing data. their learning tasks are the same, i.e. Ts = TT , the learning prob-
lem becomes a traditional machine learning problem. When the
domains are different, then either (i) the feature spaces between
Transfer learning
the domains are different, i.e. XS = XT , or (ii) the feature spaces
Although traditional machine learning methods have achieved
between the domains are the same but the marginal probability
significant success in many science and engineering areas that
distributions of domain data are different, i.e.
involve regression, classification, and clustering (Yang and Wu,
2006; Wu et al., 2008), there are still some limitations for certain
P (XS )  = P (XT ) ,
real-world scenarios. Many machine learning methods work well
only under a common assumption that the training and test data
where XSi ∈ XS and XTi ∈ XT .
have the same distribution and feature space. Once the distri-
Based on the definition of transfer learning, the relationship
bution changes, most statistical models need to be rebuilt using
between traditional machine learning and various transfer learn-
newly collected training data. In many practical applications, col-
ing settings is summarized in Table 1. According to different
lecting further training data to rebuild models is highly expensive,
situations between the source and target domains and tasks,
time-consuming, or even unrealistic. In such cases, knowledge
we categorize transfer learning into inductive transfer learning,
transfer or transfer learning between task domains would be
transductive transfer learning, and unsupervised transfer learn-
desirable. Transfer learning, which focuses on transferring the
ing.
knowledge across domains, can surmount the above difficulty
Approaches to transfer learning in the above three different
(Zhuang et al., 2020). Before introducing transfer learning, let us
settings can be summarized into four cases based on ‘What to
give the definition of domain and task, which are fundamental to
transfer’, i.e. instance-transfer, feature-representation-transfer,
transfer learning (Pan and Yang, 2009).
parameter-transfer, and relational-knowledge-transfer (Pan and
Yang, 2009). The instance-transfer approach assumes partial
Definition of domain data in the source domain can be reused for learning in the
A domain D contains two components: a feature space X and a target domain by reweighting. The idea behind the feature-
marginal distribution P(X), where X = {x1 , x2 , . . . , xn } ∈ X . In other representation-transfer approach is to learn a good feature
words, D = {X , P(X)}. representation for the target domain and improve the task
 Y. W. Wang et al. | 7

Table 1. Relationship between traditional machine learning and transfer learning.

Learning settings Source and target domains Source and target Ttsks

Traditional machine learning Same Same

Trans- Inductive transfer learning Same Different but related
fer Unsupervised transfer learning Different but related Different but related
learn- Transductive transfer learning Different but related Same
ing

performance. The parameter-transfer approach assumes the devices, and external excitations (Dang et al., 2021). The main
source tasks and target tasks share partial parameters or structural components play an essential role in assuring the
prior distributions of the hyperparameters of the models. The integrity and safety of the structure, which are designed according

Downloaded from https://academic.oup.com/iti/article/doi/10.1093/iti/liac009/6732506 by guest on 04 November 2024

relational-knowledge-transfer approach assumes the relationship to various specified specifications. For railway bridges, the main
among the data in the source and target domains is similar. components could be a bridge tower, girder, deck, main cables, and
Recently, transfer learning techniques have been applied suc- so on, whose structural performance during operation should
cessfully in many real-world applications, including the SHM meet predefined ultimate limit states (ULS) and serviceability
of bridges. Xin et al. (2020) proposed a novel acoustic emission limit states (SLS):
(AE) monitoring CNN by combining wavelet analysis and transfer
F (α, β, γ , t, . . . ) ≤ D, (6)
learning to enable early detection of defects in the stay cables of a
cable-stayed bridge, which helps to assess the service status and where α, β, γ are time-variant properties of the main structural
evaluate the remaining fatigue behaviors of cables in the bridge. components, such as acceleration, displacement, strain, deforma-
Zhang et al. (2020) proposed a single-stage detector, termed You tion, corrosion, and cracks; t denotes time; D is the threshold
Only Look Once (YOLOv3), for detecting multiple concrete bridge value specified by various specifications. It is worth noting that
damage, in which a field inspection image dataset labeled with the time-variant parameters are random and have large vari-
four types of concrete damage, i.e. crack, spalling, pop-out, and ances. With the DT method, the states of a structure are continu-
exposed rebar, was used for training and testing. To enhance the ously updated, thus reducing uncertainty in the parameters and
detection accuracy, the original YOLOv3 is improved by embed- increasing the reliability of a structural performance evaluation.
ding a novel transfer learning algorithm with pretrained weights The auxiliary components do not directly inf luence the safety
from a geometrically similar dataset. Lee et al. (2021) applied of the structure, but they provide significant protection for the
the transfer learning method to investigate the efficient damage structure and improve users’ comfort.
detection model based on initial measurement data of a railway The excitations acting on structures, including permanent
bridge within the minimum number of sensors, where only the loads, live loads, environmental changes, and accidental events
label of the normal data was known as a prior. Aliyari et al. are also random and time-varying. These could be modelled
(2021) investigated the performance of state-of-the-art convolu- by stochastic processes in design practice and a corresponding
tional neural networks (CNNs) through transfer learning for crack structural behavior assessment. The measurement devices
detection in UAV based bridge inspection, in which the selection of contain various types of sensors, such as accelerometers, strain
an optimized amount of transfer learning was specifically studied gauges, GPS, and so on, which provide actual structural responses
as a challenging task. and environment variables.

Virtual structure
Digital twin The virtual part is expected to accurately replicate the behavior
Intelligent and digital management of railway bridges is an essen- of the physical structure in real-world scenarios, predicting events
tial part of future transportation infrastructure systems. With and bahavior in the future. Computed results of the digital model
the development of the Internet of Things (IoT), deep learning can be expressed as follows (Kennedy and O’Hagan, 2001):
algorithms, and cloud framework, a new data-driven paradigm,
termed Digital Twin (DT), has emerged and received much recent z(x) = y(x) + ε (7)
attention (Dang et al., 2021). The concept of DT includes both
the physical structure in physical space and a digital model where z(x) denotes the observational outputs; y(x) is the model
(virtual structure) in cyber space. The most important element discrepancy-corrected outputs;ε is the additional error, account-
is to establish a connection between the physical object and the ing for uncertainty in observations. The model output y(x) can be
digital object in an accurate and real-time manner. The digital obtained by ensemble methods, including the exact mathematical
replica of a physical model could have a number of sub-models: model, the fine finite element model (FEM), and the promising
mathematical models, machine learning models and numerical machine learning (ML) model (Fig. 6). By combining these models,
models, which work jointly to mimic the real physical object and it is able to create a digital mirror of the actual infrastructure, thus
predict future structural behavior (Dang et al., 2021). The DT allowing the prediction of the structural capacity, performing a
framework allows engineers to discover defects before disasters structural health evaluation and proposing maintenance strate-
happen, improve the control system, and be better prepared to gies.
prevent sudden damage. Dang et al. (2021) addressed a cloud based digital twin (DT)
framework for the real-time monitoring and proactive mainte-
Physical structure nance of bridges. The proposed method facilitates two-way map-
The physical structure includes four subcategories (Fig. 5): the ping between a physical structure and a virtual structure, paving
main structural elements, auxiliary elements, measurement the way towards a real-time intelligent monitoring system. The
8 | Intelligent Transportation Infrastructure, 2022, Vol. 1, No. 1

Downloaded from https://academic.oup.com/iti/article/doi/10.1093/iti/liac009/6732506 by guest on 04 November 2024

Figure 5. Four subcategories of physical structure (Dang et al., 2021)

Figure 6. Digital mirror involving multiple computation models (Dang et al., 2021)

proposed DT framework was applied to a real railway bridge for Jiang et al. (2021b) presented a DT framework to predict the non-
verification. With field measurement data from the bridge and deterministic fatigue life of steel bridges. The uncertainties of
simulated data by FEM form the database, a 1D CNN based DT material microstructures were incorporated in modified Fine and
model is formulated for local pattern recognition and damage Bhat model, and the random macroscopic factors were considered
detection. Zhao et al. (2020) designed a deep learning regression via a Bayesian model. The implementation of the probability-
network of long short-term memory (LSTM) for modeling the based multiscale models enables the fatigue life of steel bridges
nonlinear mapping between the structural input–output (temper- under various sources of uncertainties to be accurately predicted.
ature, strain, and dynamic displacement) and structural global– Lin et al. (2021) proposed a DT based collapse fragility assess-
local response (strain), and applied it to assess an in-service ment approach for a long-span cable-stayed bridge under strong
bridge. Jiang et al. (2021a) conducted a comprehensive and in- earthquakes. Three FE models of the bridge were established to
depth review of the academic publications related to DT in civil demonstrate the necessity of the DT based assessment. Rageh
engineering from the perspective of concepts, and a research clus- et al. (2022) introduced guidelines for developing a precise dig-
ter of applications. The digital clusters were divided into design, ital twin of physical structures by using measured operational
construction, operation, and maintenance from the perspective of and ambient structural responses. The developed framework was
DT services. Dan et al. (2021) proposed a DT system for group of applied to an in-service, double-track railway bridge for accurate
bridges in the regional transportation infrastructure network. By response prediction.
establishing the digital model using the traffic loads measured by
weigh-in-motion (WIM) and muti-source heterogeneous machine
vision, the working condition perception and safety warnings for
Challenges and conclusions
all bridges in the group were achieved, forming significant sup- Monitoring the long-term performance of railway bridges is cru-
port for developing an intelligent transportation infrastructure. cial for the real-time assessment of potential risks, improving
 Y. W. Wang et al. | 9

the structure’s safety and reliability through early-stage damage data obtained all the time. One way is to adopt a hybrid scheme
detection. This paper presents a concise review of the recent to augment the SHM database with synthetic data obtained
development in emerging sensing technologies (i.e. FOS tech- from reliable FEMs. Another way to increase data efficiency is
nology and CV technology), machine learning algorithms (i.e. to engineer a hierarchical strategy. First, some basic features are
CNN and TL) and DT, and their applications to the SHM of rail- extracted from data to detect the existence of damage and only
way bridges. The goal is to help broaden the applications of the basic features are stored if no damage occurs. Second, for
these emerging technologies and methods not only in scientific cases of damage, more advanced feature extractions are executed
research but also in engineering practice. Based on the reviews to establish DL models to perform complex tasks such as damage
given in this article, the following summaries and future chal- location and damage severity.
lenges are highlighted.
(i) Although FOS technology offers distinct advantages over
other technologies, challenges and difficulties still exist for Acknowledgments

Downloaded from https://academic.oup.com/iti/article/doi/10.1093/iti/liac009/6732506 by guest on 04 November 2024

its application to railway bridges. One major challenge is its
The work described in this paper was supported by a grant from
installation in the field. Difficulties encountered in the field
the Research Grants Council of the Hong Kong Special Adminis-
deployment of fiber optic sensors include low strain sensitivity
trative Region (SAR), China (Grant No. R-5020-18), a grant from
jacked fibers, broken bare fibers, and loss of light intensity due
the National Natural Science Foundation of China (Grant No.
to bending. Therefore, the use of special jacketed fibers for strain
U1934209), and a grant from the Hong Kong, Macao and Taiwan
measurement and their installation will lead to extremely high
Science and Technology Innovation Cooperation Key Project of
expenses for long-range FOS systems, especially for long-span
Sichuan Province, China (Grant No. 2020YFH0178). The authors
railway bridges. Another major challenge is the effective compen-
would also like to appreciate funding support by the Innovation
sation of non-uniform, time-varying distributed environmental
and Technology Commission of the Hong Kong SAR Government
temperature. The cross-sensitivity between thermal strain and
(Grant No. K-BBY1).
mechanical strain can affect the accuracy of FOS systems.
(ii) Computer vision for bridge inspection and monitoring is a Conf lict of Interest statement. One of the authors, Dr Yi-Qing Ni,
natural step forward and will progressively replace manual visual is a Co-Editor-in-Chief of Intelligent Transportation Infrastruc-
inspection, while offering new advantages and opportunities. The ture. He was blinded from reviewing or making decisions on
use of image data is also a double-edged sword. Although each the manuscript. The authors declare that they have no known
image contains rich textural, spatial, and contextual informa- competing financial interests or personal relationships that could
tion, the process of extracting actionable information from these have appeared to inf luence the work reported in this paper.
images to realize automated vision-based inspection and moni-
toring is challenging. These challenges broadly lie in converting
the features and signals extracted by vision based methods into
actionable data that can aid decision-making at a higher level.
References
(iii) The CNN techniques offer new advantages and opportu- Aliyari, M., Droguett, E. L. and Ayele, Y. Z. (2021) ‘UAV-based bridge
nities in remote and autonomous SHM of civil infrastructure. inspection via transfer learning’, Sustainability, 13: 11359.
However, the existing CNN techniques present several challenges Bao, X., and Chen, L. (2011) ‘Recent progress in Brillouin scattering
that must be addressed. Due to the large scale and life-span of based fiber sensors’, Sensors, 11: 4152–87.
bridge structures, monitoring data collected from limited sparse Barrias, A., Casas, J. R. and Villalba, S. (2016) ‘A review of distributed
measurements may correspond to a variety of potential damage optical fiber sensors for civil engineering applications’, Sensors,
states, causing data imbalance in SHM and limiting the applicabil- 16: 748.
ity of CNN in structural condition assessment. The utilization of Chan, T. H., Yu, L., Tam, H. Y., Ni, Y. Q. et al. (2006) ‘Fiber Bragg
mobile or handheld devices is a future trend, and thus strategies grating sensors for structural health monitoring of Tsing Ma
which are computationally efficient and help to reduce the cost Bridge: Background and experimental observation’, Engineering
of CNN algorithms are highly desirable. Structures, 28: 648–59.
(iv) CNN enhanced by TL would provide attractive solutions China Daily, High-Speed Rail Network Expands Past 40,000 Km
to railway bridge SHM. Statistically similar datasets of identical (2021): https://global.chinadaily.com.cn/a/202112/31/WS61ce70
structures can be leveraged to replace the requirement of aug- cca310cdd39bc7e855.html.
menting a training dataset, especially when some of the actual Dan, D., Ying, Y. and Ge, L. (2021) ‘Digital twin system of bridges group
measurement data are difficult to obtain. With the aid of TL, a based on machine vision fusion monitoring of bridge traffic load’,
CNN model trained in one domain can be transferred for use in IEEE Transactions on Intelligent Transportation Systems. https://doi.
other domains where training data are lacking. Deep TL uses a org/10.1109/TITS.2021.3130025.
well-established NN benchmark for training and feature extrac- Dang, H. V., Tatipamula, M. and Nguyen, H. X. (2021) ‘Cloud-based
tion and refines the fully-connected classification layers for dam- digital twinning for structural health monitoring using deep
age diagnosis. learning’, IEEE Transactions on Industrial Informatics, 18: 3820–30.
(v) When implementing DT in the SHM of railway bridges, the Du, C., Dutta, S., Kurup, P., Yu, T. et al. (2020) ‘A review of railway
fidelity of the DL models is a key issue. Multi-physics modeling infrastructure monitoring using fiber optic sensors’, Sensors and
and continuous model updating through the whole lifecycle Actuators a: Physical, 303: 111728.
are required for the purpose of simulating the physical twin’s Failleau, G., Beaumont, O., Razouk, R., Delepine-Lesoille, S. et al. (2018)
behavior as precisely as possible. In addition, real-time DL models ‘a metrological comparison of Raman-distributed temperature
are required to realize relatively low latency between the digital sensors’, Measurement, 116: 18–24.
twin and physical twin. Another challenge is the scarcity of Farrar, C. R., and Worden, K. (2007) ‘an introduction to structural
relevant data. Most of the data are related to the normal state health monitoring’, Philosophical Transactions of the Royal Society a:
of the structures, so it is not necessary to transmit and store full Mathematical, Physical and Engineering Sciences, 365: 303–15.
10 | Intelligent Transportation Infrastructure, 2022, Vol. 1, No. 1

Feng, D., and Feng, M. Q. (2018) ‘Computer vision for SHM of civil Engineers-Civil Engineering, Vo. 165, pp. 185–90. Institution of Civil
infrastructure: From dynamic response measurement to damage Engineers.
detection–A review’, Engineering Structures, 156: 105–17. Mousa, M. A., Yussof, M. M., Udi, U. J., Nazri, F. M. et al. (2021) ‘Applica-
Feng, M. Q., Fukuda, Y., Feng, D. and Mizuta, M. (2015a) ‘Nontar- tion of digital image correlation in structural health monitoring
get vision sensor for remote measurement of bridge dynamic of bridge infrastructures: A review’, Infrastructures, 6: 176.
response’, Journal of Bridge Engineering, 20: 04015023. Narazaki, Y., Hoskere, V., Chowdhary, G. and Spencer, B. F., Jr.
Feng, D., Feng, M. Q., Ozer, E. and Fukuda, Y. (2015b) ‘A vision-based (2022) ‘Vision-based navigation planning for autonomous post-
sensor for noncontact structural displacement measurement’, earthquake inspection of reinforced concrete railway viaducts
Sensors, 15: 16557–75. using unmanned aerial vehicles’, Automation in Construction, 137:
Feng, X., Jiang, Y., Yang, X., Du, M. et al. (2019) ‘Computer vision 104214.
algorithms and hardware implementations: A survey’, Integration, Ni, F., Zhang, J. and Noori, M. N. (2020) ‘Deep learning for data
69: 309–20. anomaly detection and data compression of a long-span suspen-

Downloaded from https://academic.oup.com/iti/article/doi/10.1093/iti/liac009/6732506 by guest on 04 November 2024

Figueiredo, E., and Brownjohn, J. (2022) ‘Three decades of statisti- sion bridge’, Computer-Aided Civil and Infrastructure Engineering, 35:
cal pattern recognition paradigm for SHM of bridges’, Structural 685–700.
Health Monitoring. https://doi.org/10.1177/14759217221075241. Pan, B. (2018) ‘Digital image correlation for surface deformation mea-
Ghiasi, A., Moghaddam, M. K., Ng, C. T., Sheikh, A. H. et al. (2022) surement: Historical developments, recent advances and future
‘Damage classification of in-service steel railway bridges using a goals’, Measurement Science and Technology, 29: 082001.
novel vibration-based convolutional neural network’, Engineering Pan, S. J., and Yang, Q. (2009) ‘A survey on transfer learning’, IEEE
Structures, 264: 114474. Transactions on Knowledge and Data Engineering, 22: 1345–59.
Guo, Y. L., Ni, Y. Q. and Chen, S. K. (2017) ‘Optimal sensor placement Pan, B., Tian, L. and Song, X. (2016) ‘Real-time, non-contact and
for damage detection of bridges subject to ship collision’, Struc- targetless measurement of vertical deflection of bridges using
tural Control and Health Monitoring, 24: e1963. off-axis digital image correlation’, Ndt & E International, 79: 73–80.
Hajializadeh, D. (2022) ‘Deep-learning-based drive-by damage detec- Pimentel, R., Ribeiro, D., Matos, L., Mosleh, A. et al. (2021) ‘Bridge
tion system for railway bridges’, Infrastructures, 7: 84. Weigh-in-Motion system for the identification of train loads
He, K., Zhang, X., Ren, S., and Sun, J. (2016). ‘Deep residual learning for using fiber-optic technology’, Structure, 30: 1056–70.
image recognition’, Proceedings of the IEEE Conference on Computer Qin, S., and Gao, Z. (2017) ‘Developments and prospects of long-span
Vision and Pattern Recognition, pp. 770–8. IEEE: Las Vegas, USA. high-speed railway bridge technologies in China’, Engineering, 3:
https://doi.org/10.1109/CVPR.2016.90. 787–94.
Jiang, F., Ma, L., Broyd, T. and Chen, K. (2021a) ‘Digital twin and its Rageh, A., Azam, S. E., Alomari, Q., Linzell, D. et al. (2022) ‘Model
implementations in the civil engineering sector’, Automation in updating and parameter identification for developing digital
Construction, 130: 103838. twins for riveted steel railway bridges’. In: Haldar A. and Al-
Jiang, F., Ding, Y., Song, Y., Geng, F. et al. (2021b) ‘Digital twin-driven Hussein A. (eds.) Recent Developments in Structural Health Monitoring
framework for fatigue life prediction of steel bridges using a prob- and Assessment–Opportunities and Challenges: Bridges, Buildings and
abilistic multiscale model: Application to segmental orthotropic Other Infrastructures, pp. 285–318. USA: World Scientific Publish-
steel deck specimen’, Engineering Structures, 241: 112461. ing.
Kennedy, M. C., and O’Hagan, A. (2001) ‘Bayesian calibration of Ribeiro, D., Calçada, R., Ferreira, J. and Martins, T. (2014) ‘Non-contact
computer models’, Journal of the Royal Statistical Society: Series B measurement of the dynamic displacement of railway bridges
(Statistical Methodology), 63: 425–64. using an advanced video-based system’, Engineering Structures, 75:
Kim, S. W., Jeon, B. G., Kim, N. S. and Park, J. C. (2013) ‘Vision-based 164–80.
monitoring system for evaluating cable tensile forces on a cable- Rizzo, P., and Enshaeian, A. (2021) ‘Challenges in bridge health mon-
stayed bridge’, Structural Health Monitoring, 12: 440–56. itoring: A review’, Sensors, 21: 4336.
Krizhevsky, A., Sutskever, I. and Hinton, G. E. (2017) ‘Imagenet clas- Sánchez, J. and Perronnin, F. (2011) ‘High-dimensional signature
sification with deep convolutional neural networks’, Communica- compression for large-scale image classification’, Proceedings of
tions of the ACM, 60: 84–90. the IEEE Conference on Computer Vision and Pattern Recognition, pp.
LeCun, Y., Bottou, L., Bengio, Y. and Haffner, P. (1998) ‘Gradient-based 1665–72. Colorado Springs, USA: IEEE. https://doi.org/10.1109/
learning applied to document recognition’, Proceedings of the IEEE, CVPR.2011.5995504.
86: 2278–324. Sasy Chan, Y. W., Wang, H. P. and Xiang, P. (2021) ‘Optical fiber sensors
Lee, J. S., Kim, H. M., Kim, S. I. and Lee, H. M. (2021) ‘Evaluation of for monitoring railway infrastructures: A review towards smart
structural integrity of railway bridge using acceleration data and concept’, Symmetry, 13: 2251.
semi-supervised learning approach’, Engineering Structures, 239: Scott, R. H., Banerji, P., Chikermane, S., Srinivasan, S. et al. (2013)
112330. ‘Commissioning and evaluation of a fiber-optic sensor system for
Leung, C. K., Wan, K. T., Inaudi, D., Bao, X. et al. (2015) ‘Optical fiber bridge monitoring’, IEEE Sensors Journal, 13: 2555–62.
sensors for civil engineering applications’, Materials and Structures, Seo, J., Hu, J. W. and Lee, J. (2016) ‘Summary review of structural
48: 871–906. health monitoring applications for highway bridges’, Journal of
Lin, K., Xu, Y. L., Lu, X., Guan, Z. et al. (2021) ‘Digital twin- Performance of Constructed Facilities, 30: 04015072.
based collapse fragility assessment of a long-span cable-stayed Shokravi, H., Shokravi, H., Bakhary, N., Heidarrezaei, M. et al. (2020)
bridge under strong earthquakes’, Automation in Construction, 123: ‘Vehicle-assisted techniques for health monitoring of bridges’,
103547. Sensors, 20: 3460.
Liu, H., and Zhang, Y. (2020) ‘Bridge condition rating data modeling Simonyan, K., and Zisserman, A. (2014) ‘Very deep convolutional net-
using deep learning algorithm’, Structure and Infrastructure Engi- works for large-scale image recognition’, arXiv:1409.1556. https://
neering, 16: 1447–60. arxiv.org/abs/1409.1556.
McCormick, N. and Lord, J. (2012). ‘Digital image correlation for Sony, S., Laventure, S. and Sadhu, A. (2019) ‘A literature review
structural measurements’, Proceedings of the Institution of Civil of next-generation smart sensing technology in structural
 Y. W. Wang et al. | 11

health monitoring’, Structural Control and Health Monitoring, 26: engineering application using deep transfer learning and wavelet
e2321. analysis’, Advances in Bridge Engineering, 1: 1–16.
Sony, S., Dunphy, K., Sadhu, A. and Capretz, M. (2021) ‘A systematic Xu, Y., Bao, Y., Chen, J., Zuo, W. et al. (2019) ‘Surface fatigue crack
review of convolutional neural network-based structural condi- identification in steel box girder of bridges by a deep fusion
tion assessment techniques’, Engineering Structures, 226: 111347. convolutional neural network based on consumer-grade camera
Spencer, B. F., Jr., Hoskere, V. and Narazaki, Y. (2019) ‘Advances images’, Structural Health Monitoring, 18: 653–74.
in computer vision-based civil infrastructure inspection and Yang, Q., and Wu, X. (2006) ‘10 challenging problems in data mining
monitoring’, Engineering, 5: 199–222. research’, International Journal of Information Technology & Decision
Sun, L., Shang, Z., Xia, Y., Bhowmick, S. et al. (2020) ‘Review of bridge Making, 5: 597–604.
structural health monitoring aided by big data and artificial Ye, X. W., Ni, Y. Q., Wai, T. T., Wong, K. Y. et al. (2013) ‘A vision-based
intelligence: From condition assessment to damage detection’, system for dynamic displacement measurement of long-span
Journal of Structural Engineering, 146: 04020073. bridges: algorithm and verification’, Smart Structures and Systems,

Downloaded from https://academic.oup.com/iti/article/doi/10.1093/iti/liac009/6732506 by guest on 04 November 2024

Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, 12: 363–79.
D., Vanhoucke, V., and Rabinovich, A. (2015) ‘Going deeper with Ye, C., Butler, L. J., Elshafie, M. Z. and Middleton, C. R. (2020) ‘Evaluat-
convolutions’, Proceedings of the IEEE Conference on Computer Vision ing prestress losses in a prestressed concrete girder railway bridge
and Pattern Recognition, pp. 1–9. Boston, USA: IEEE. https://doi. using distributed and discrete fibre optic sensors’, Construction
org/10.1109/CVPR.2015.7298594. and Building Materials, 247: 118518.
Tian, L., Zhao, J., Pan, B. and Wang, Z. (2021) ‘Full-field bridge deflec- Ye, C., Butler, L., Huseynov, F. and Middleton, C. (2021) ‘Evaluating
tion monitoring with off-axis digital image correlation’, Sensors, in-service structural behaviour of an operational railway bridge
21: 5058. using fibre optic sensing and structural model updating’, Engi-
Uchida, S., Levenberg, E. and Klar, A. (2015) ‘On-specimen strain neering Structures, 247: 113116.
measurement with fiber optic distributed sensing’, Measurement, Yoneyama, S., and Ueda, H. (2012) ‘Bridge deflection measure-
60: 104–13. ment using digital image correlation with camera movement
Vagnoli, M., Remenyte-Prescott, R. and Andrews, J. (2018) ‘Railway correction’, Materials Transactions, 53: 285–90.
bridge structural health monitoring and fault detection: State- Yoon, H. J., Song, K. Y., Choi, C., Na, H. S. et al. (2016) ‘Real-time
of-the-art methods and future challenges’, Structural Health Mon- distributed strain monitoring of a railway bridge during train
itoring, 17: 971–1007. passage by using a distributed optical fiber sensor based on
Van Der Kooi, K., Hoult, N. A. and Le, H. (2018) ‘Monitoring an in- Brillouin optical correlation domain analysis’, Journal of Sensors,
service railway bridge with a distributed fiber optic strain sensing 2016: 9137531.
system’, Journal of Bridge Engineering, 23: 05018007. Zhang, Y., Wang, K., Wang, H., Yu, C., Xu, C., and Zhang, Y. (2013)
Wheeler, L. N., Pannese, E., Hoult, N. A., Take, W. A. et al. (2018) ‘Mea- ‘Mach-Zehnder interferometer as a temperature sensor based on
surement of distributed dynamic rail strains using a Rayleigh the nested fiber ring resonator’, 2013 Seventh International Con-
backscatter based fiber optic sensor: lab and field evaluation’, ference on Sensing Technology, pp. 226–9. Wellington, New Zealand
Transportation Geotechnics, 14: 70–80. (ICST): IEEE. https://doi.org/10.1109/ICSensT.2013.6727647.
Wojke, N., Bewley, A., and Paulus, D. (2017) ‘Simple online and real- Zhang, Y., Miyamori, Y., Mikami, S. and Saito, T. (2019) ‘Vibration-
time tracking with a deep association metric’, Proceedings of the based structural state identification by a 1-dimensional convo-
IEEE international conference on image processing, pp. 3645–9. Beijing, lutional neural network’, Computer-Aided Civil and Infrastructure
China: IEEE. https://doi.org/10.1109/ICIP.2017.8296962. Engineering, 34: 822–39.
Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J. et al. (2008) ‘Top 10 Zhang, C., Chang, C. C. and Jamshidi, M. (2020) ‘Concrete bridge sur-
algorithms in data mining’, Knowledge and Information Systems, 14: face damage detection using a single-stage detector’, Computer-
1–37. Aided Civil and Infrastructure Engineering, 35: 389–409.
Wu, N., Zou, X., Tian, Y., Fitek, J. et al. (2012) ‘An ultra-fast fiber optic Zhao, H., Ding, Y., Li, A., Sheng, W. et al. (2020) ‘Digital modeling on the
pressure sensor for blast event measurements’, Measurement Sci- nonlinear mapping between multi-source monitoring data of in-
ence and Technology, 23: 055102. service bridges’, Structural Control and Health Monitoring, 27: e2618.
Xin, H., Cheng, L., Diender, R. and Veljkovic, M. (2020) ‘Fracture Zhuang, F., Qi, Z., Duan, K., Xi, D. et al. (2020) ‘A comprehensive survey
acoustic emission signals identification of stay cables in bridge on transfer learning’, Proceedings of the IEEE, 109: 43–76.