Sheng Xiang, Yi Qin, Jun Luo, Huayan Pu, Baoping Tang,

Multicellular LSTM-based deep learning model for aero-engine remaining useful life
prediction,
Reliability Engineering & System Safety,
Volume 216,
2021,
107927,
ISSN 0951-8320,
https://doi.org/10.1016/j.ress.2021.107927.
(https://www.sciencedirect.com/science/article/pii/S0951832021004439)
Abstract: The prediction of aero-engine remaining useful life (RUL) is helpful for
its operation and maintenance. Aiming at the challenge that most neural networks
(NNs), including long short-term memory (LSTM), cannot process the input data in
different update modes based on its importance degree, a novel variant of LSTM
named multicellular LSTM (MCLSTM) is constructed. The level division unit is
proposed to determine the importance degree of input data, and then multiple
cellular units are designed to update the cell states according to the data level.
Thus, MCLSTM can well mine different levels of degradation trends. Based on MCLSTM
and a deep NN (DNN), a deep learning model for RUL prediction is set up, where
MCLSTM and a branch of the DNN is used to extract health indicators (HIs) of aero-
engine from raw data, and the other part of the DNN is applied to generate the HIs
from human-made features and predict the RUL based on the concatenated HIs. The
proposed RUL prediction model is successfully applied to predict the RULs of aero-
engines via the Commercial Modular Aero Propulsion System Simulation datasets, and
the comparative results show that it has a better comprehensive prediction
performance than the commonly-used machine learning methods.
Keywords: RUL prediction; Degradation trend; Multi-resource data; Health feature;
Data level

Yucheng Liu, Wenyang Duan, Limin Huang, Shiliang Duan, Xuewen Ma,
The input vector space optimization for LSTM deep learning model in real-time
prediction of ship motions,
Ocean Engineering,
Volume 213,
2020,
107681,
ISSN 0029-8018,
https://doi.org/10.1016/j.oceaneng.2020.107681.
(https://www.sciencedirect.com/science/article/pii/S0029801820306739)
Abstract: Vessel motions due to the ocean waves contribute to maritime operational
safety and efficiency. Real-time prediction of deterministic ship motions in the
coming future seconds is essential in decision-making when performing motions
sensitive activities. The Long-Short Term Memory (LSTM) deep learning model
provides a potential way for nonlinear ship motions prediction due to its
capability in nonlinearity processing. Determination of a reasonable dimension of
the input vector is critical in training the LSTM model. Conventionally, the
optimal dimension for the input vector is selected by traversing an empirical
preset range. Hence, it suffers both high computational cost and poor adaptation in
determining the optimal input vector dimension. In the present work, an input
vector space optimization method is proposed based on the dependence hidden in ship
motion records of a sequence. Taking different correlation expressions into
consideration, both the Impulse Response Function (IRF) based and Auto-correlation
Function (ACF) based techniques are investigated for input vector space
optimization. Numerical simulations are carried out for vilification and comparison
purpose. The ACF technique is better in representing the auto-correlation hidden in
the stochastic ship motions. And the ACF-based LSTM model performs better in both
training efficiency and prediction accuracy.
Keywords: Ship motions real-time prediction; LSTM deep learning model; Input vector
space optimization; Impulse response function; Auto-correlation function
Iftikhar Rasheed, Fei Hu, Lin Zhang,
Deep reinforcement learning approach for autonomous vehicle systems for maintaining
security and safety using LSTM-GAN,
Vehicular Communications,
Volume 26,
2020,
100266,
ISSN 2214-2096,
https://doi.org/10.1016/j.vehcom.2020.100266.
(https://www.sciencedirect.com/science/article/pii/S2214209620300371)
Abstract: The success of autonomous vehicles (AVhs) depends upon the effectiveness
of sensors being used and the accuracy of communication links and technologies
being employed. But these sensors and communication links have great security and
safety concerns as they can be attacked by an adversary to take the control of an
autonomous vehicle by influencing their data. Especially during the state
estimation process for monitoring of autonomous vehicles' dynamics system, these
concerns require immediate and effective solution. In this paper we present a new
adversarial deep reinforcement learning algorithm (NDRL) that can be used to
maximize the robustness of autonomous vehicle dynamics in the presence of these
attacks. In this approach the adversary tries to insert defective data to the
autonomous vehicle's sensor readings so that it can disrupt the safe and optimal
distance between the autonomous vehicles traveling on the road. The attacker tries
to make sure that there is no more safe and optimal distance between the autonomous
vehicles, thus it may lead to the road accidents. Further attacker can also add
fake data in such a way that it leads to reduced traffic flow on the road. On the
other hand, autonomous vehicle will try to defend itself from these types of
attacks by maintaining the safe and optimal distance i.e. by minimizing the
deviation so that adversary does not succeed in its mission. This attacker-
autonomous vehicle action reaction can be studied through the game theory
formulation with incorporating the deep learning tools. Each autonomous vehicle
will use Long-Short-Term-Memory (LSTM)-Generative Adversarial Network (GAN) models
to find out the anticipated distance variation resulting from its actions and input
this to the new deep reinforcement learning algorithm (NDRL) which attempts to
reduce the variation in distance. Whereas attacker also chooses deep reinforcement
learning algorithm (NDRL) and wants to maximize the distance variation between the
autonomous vehicles.
Keywords: Autonomous vehicles; Deep reinforcement learning; Generative adversarial
network (GAN); Long-short-term-memory (LSTM); 5G; ITS-security

Ahmed A. Ewees, Mohammed A.A. Al-qaness, Laith Abualigah, Mohamed Abd Elaziz,
HBO-LSTM: Optimized long short term memory with heap-based optimizer for wind power
forecasting,
Energy Conversion and Management,
Volume 268,
2022,
116022,
ISSN 0196-8904,
https://doi.org/10.1016/j.enconman.2022.116022.
(https://www.sciencedirect.com/science/article/pii/S0196890422008147)
Abstract: The forecasting and estimation of wind power is a challenging problem in
renewable energy generation due to the high volatility of wind power resources,
inevitable intermittency, and complex fluctuation. In recent years, deep learning
techniques, especially recurrent neural networks (RNN), showed prominent
performance in time-series forecasting and prediction applications. One of the main
efficient RNNs is the long short term memory (LSTM), which we adopted in this study
to forecast the wind power from different wind turbines. We adopted the advances of
the metaheuristic optimization algorithms to train the LSTM and to boost its
performance by optimizing its parameters. The Heap-based optimizer (HBO) is a new
human-behavior-based metaheuristic algorithm that was inspired by corporate rank
hierarchy, and it was employed to solve complex optimization and engineering
problems. In this study, HBO is used to train the LSTM, and it showed significant
enhancement on the LSTM prediction performance. We used four datasets from the
well-known wind turbines in France, La Haute Borne wind turbines, to evaluate the
developed HBO-LSTM. We also considered several optimized LSTM models using several
optimization algorithms for comparisons, as well as several existing models. The
comparison outcome confirmed the capability of HBO to boost the prediction
performance of the basic LSTM model.
Keywords: Forecasting; Deep learning; Wind power; Heap-based optimizer; Long short
term memory

Xing-Ke Ma, Hong-Quan Huang, Bo-Rui Huang, Zhi-Wen Shen, Qing-Tai Wang, Yu-Yu Xiao,
Cheng-Lin Zhong, Hao Xin, Peng Sun, Kai-Ming Jiang, Lin Tang, Wei-Cheng Ding, Wei
Zhou, Jian-Bin Zhou,
X-ray spectra correction based on deep learning CNN-LSTM model,
Measurement,
Volume 199,
2022,
111510,
ISSN 0263-2241,
https://doi.org/10.1016/j.measurement.2022.111510.
(https://www.sciencedirect.com/science/article/pii/S0263224122007321)
Abstract: The two most important indicators in the measurement process for the X-
ray spectra are energy resolution and counting rate. However, in the actual
detection process, when the interval time between adjacent pulses is less than the
pulse shaping time, the pulses pile up. If the pile-up pulse is not separated and
recognized, then it greatly affects the X-ray spectrum’s accuracy. A method of X-
ray spectrum correction is proposed on the basis of a deep learning model, which
realizes the separation of the pile-up pulse by recognizing its parameters, and
then realizes the correction of the X-ray spectrum. Standard sources 55Fe and 238Pu
are used as the measurement objects, and the spectra correction method is used to
recognize the pile-up pulses. Measurement results show that the method can
effectively recognize the pile-up pulses, improve the spectrum’s counting rate, and
obtain more accurate X-ray spectra.
Keywords: X-ray spectra; Pile-up pulse; Deep learning; Pulse recognition; Radiation
measurement

Baiying Lei, Yuwen Zhang, Dongdong Liu, Yanwu Xu, Guanghui Yue, Jiuwen Cao, Huoyou
Hu, Shuangzhi Yu, Peng Yang, Tianfu Wang, Yali Qiu, Xiaohua Xiao, Shuqiang Wang,
Longitudinal study of early mild cognitive impairment via similarity-constrained
group learning and self-attention based SBi-LSTM,
Knowledge-Based Systems,
Volume 254,
2022,
109466,
ISSN 0950-7051,
https://doi.org/10.1016/j.knosys.2022.109466.
(https://www.sciencedirect.com/science/article/pii/S0950705122007353)
Abstract: Alzheimer’s disease (AD) is an incurable neurodegenerative disease. Mild
cognitive impairment (MCI) is often considered a critical time window for
predicting early conversion to Alzheimer’s disease (AD), with approximately 80% of
amnestic MCI patients developing AD within 6 years. MCI can be further categorized
into two stages (i.e., early MCI (EMCI) and late MCI (LMCI)). To identify EMCI
effectively and understand how it changes brain function, the brain functional
connectivity network (BFCN) has been widely used. However, the conventional methods
mainly focused on detection from a single time-point data, which could not discover
the changes during the disease progression without using multi-time points data.
Therefore, in this work, we carry out a longitudinal study based on multi-time
points data to detect EMCI and validate them on two public datasets. Specifically,
we first construct a similarity-constrained group network (SGN) from the resting
state functional magnetic resonance imaging (rs-fMRI) data at different time-
points, and then use a stacked bidirectional long short term memory (SBi-LSTM)
network to extract features for longitudinal analysis. Also, we use a self-
attention mechanism to leverage high-level features to further improve the
detection accuracy. Evaluated on the public Alzheimer’s Disease Neuroimaging
Initiative Phase II and III (ADNI-2 and ADNI-3) databases, the proposed method
outperforms several state-of-the-art methods.
Keywords: Mild cognitive impairment; Similarity-constrained group learning; SBi-
LSTM; Self-attention; Longitudinal study

Huy Vu Ngoc, J.R.R. Mayer, Elie Bitar-Nehme,

Deep learning LSTM for predicting thermally induced geometric errors using rotary
axes’ powers as input parameters,
CIRP Journal of Manufacturing Science and Technology,
Volume 37,
2022,
Pages 70-80,
ISSN 1755-5817,
https://doi.org/10.1016/j.cirpj.2021.12.009.
(https://www.sciencedirect.com/science/article/pii/S1755581722000037)
Abstract: Temperature changes within a machine tool, which are partly due to
internal heat sources related to the power consumption at the machine drives,
result in thermally induced geometric errors of the machine. Predicting the
thermally affected machine geometry facilitates the prediction of volumetric errors
throughout the 5-axis machine volume. Measuring power instead of temperature is
easier to implement in practice but causes drift issues. Stacked Long Short Term
Memory (LSTM) with two hidden LSTM layers is applied to determine the relationship
between powers and the thermal variation of geometric error parameters by
remembering previous states and learning complex non-linear relationships through
optimized generalized parameters of the predicting model. Optimizers Adam, RMSprop,
and SGD based on exponential moving average of first order and second order moments
of gradients are combined with Stacked LSTM for faster learning convergence to an
optimal solution. Validation trials with various B- and C-axis motion sequences
demonstrate a capability of predicting multi time steps ahead over a 40 hours
period without re-measuring the geometric error parameters of the model. Stacked
LSTM with the Adam optimizer has the best capability by predicting up to 93% of the
main geometric error compared to the other optimizers.
Keywords: Geometric errors; Deep learning LSTM; Machine tool; Thermal errors

Arsalan Mahmoodzadeh, Hamid Reza Nejati, Mokhtar Mohammadi, Hawkar Hashim Ibrahim,
Shima Rashidi, Tarik Ahmed Rashid,
Forecasting tunnel boring machine penetration rate using LSTM deep neural network
optimized by grey wolf optimization algorithm,
Expert Systems with Applications,
Volume 209,
2022,
118303,
ISSN 0957-4174,
https://doi.org/10.1016/j.eswa.2022.118303.
(https://www.sciencedirect.com/science/article/pii/S0957417422014361)
Abstract: Achieving an accurate and reliable estimation of tunnel boring machine
(TBM) performance can diminish the hazards related to extreme capital costs and
planning tunnel construction. Here, a hybrid long short-term memory (LSTM) model
enhanced by grey wolf optimization (GWO) is developed for predicting TBM-
penetration rate (TBM-PR). 1125 datasets were considered including six input
parameters. To vanish overfitting, the dropout technique was used. The effect of
input time series length on the model performance was studied. The TBM-PR results
of the LSTM-GWO model were compared to some other machine learning (ML) models such
as LSTM. The results were evaluated using root mean square error (RMSE), mean
absolute percentage error (MAPE), and correlation coefficient (R2). Finally, the
LSTM-GWO model produced the most accurate results (test: R2 = 0.9795; RMSE = 0.004;
MAPE = 0.009 %). The mutual information test revealed that input parameters of rock
fracture class and uniaxial compressive strength have the most and least impact on
the TBM-PR, respectively.
Keywords: Machine learning; Long short-term memory; Grey wolf optimization;
Metaheuristic optimization, Tunnel boring machine penetration rate

Jialan Liu, Chi Ma, Hongquan Gui, Shilong Wang,

Transfer learning-based thermal error prediction and control with deep residual
LSTM network,
Knowledge-Based Systems,
Volume 237,
2022,
107704,
ISSN 0950-7051,
https://doi.org/10.1016/j.knosys.2021.107704.
(https://www.sciencedirect.com/science/article/pii/S0950705121009564)
Abstract: The thermal error is a dominant factor that seriously hinders the high-
accuracy machining of complex parts. The weak robustness and low predictive
accuracy have always been barriers to the wide use of data-based models. To improve
the robustness, the transfer learning-based error control method is proposed in
this study. The error mechanism modeling is conducted to demonstrate the memory
behavior of thermal errors, and the applicability of a long short-term memory
network (LSTMN) for the error prediction is proven. Then an improved least mean
square (ILMS) is proposed to filter the high-frequency noises and remove singular
values. A pre-activated residual block is designed, and is embedded into the deep
residual LSTMN (DRLSTMN). The differential spotted hyenas optimization algorithm
(DSHOA) is proposed based on the chaos initialization strategy, differential
mutation operator, and nonlinear control factor to optimize the hyper-parameters of
DRLSTMN. Then the ILMS-DSHOA-DRLSTMN error prediction model is proposed for machine
tool #1. The transfer learning model is established for machine tool #2 based on
ILMS-DSHOA-DRLSTMN to enhance the robustness. The predictive abilities of the
transfer learning models of ILMS-DSHOA-DRLSTMN, ILMS-DRLSTMN, ILMS-DSHOA-LSTMN,
ILMS-back propagation network (ILMS-BP), ILMS-multiple linear regression analysis
(ILMS-MLRA), ILMS-least squared support vector machine (ILMS-LSSVM), ILMS-CNNs-LSTM
(ILMS-CL), and ILMS-deep calibration (ILMS-DC) are 98.37%, 97.95%, 97.60%, 94.51%,
95.41%, 96.02%, 96.43%, and 96.06%, respectively. Finally, the actual machining
experiments were performed. When the thermal error is controlled with the transfer
learning model, the fluctuation ranges for the geometric errors for D1 and D2 are
[−4μm, 4μm] and [−3μm, 3μm], respectively.
Keywords: Machine tool; Thermal error; Error compensation; Temperature rise;
Spindle system

First Ali Agga, Second Ahmed Abbou, Yassine El Houm, Moussa Labbadi,
Short-Term Load Forecasting Based on CNN and LSTM Deep Neural Networks,
IFAC-PapersOnLine,
Volume 55, Issue 12,
2022,
Pages 777-781,
ISSN 2405-8963,
https://doi.org/10.1016/j.ifacol.2022.07.407.
(https://www.sciencedirect.com/science/article/pii/S2405896322008084)
Abstract: In the coming years, the world will witness a global transition towards
the adoption of photovoltaic technology for large-scale plants to produce
electricity at a grid scale, and more householders will also be encouraged to
produce their electricity. However, the reliance of the photovoltaic plants on
erratic weather conditions requires the development of solutions that could help in
preventing any electricity blackout or overproduction. Hence, comes the role of
forecasting models that help in overcoming that issue. In this work, two deep
learning models are developed and tested (LSTM, CNN). Both architectures will go
under several different configurations to witness the impact of changing the number
of hidden layers on the accuracy of the forecasts. The findings reveal that the
models behave differently when the number of layers changed over the different
configurations. In addition, two-time windows were considered (1-Day, 2-Days) for
even deeper insight..
Keywords: Deep Learning; CNN; LSTM; Load Forecast

Sujan Ghimire, Ravinesh C. Deo, David Casillas-Pérez, Sancho Salcedo-Sanz, Ekta

Sharma, Mumtaz Ali,
Deep learning CNN-LSTM-MLP hybrid fusion model for feature optimizations and daily
solar radiation prediction,
Measurement,
Volume 202,
2022,
111759,
ISSN 0263-2241,
https://doi.org/10.1016/j.measurement.2022.111759.
(https://www.sciencedirect.com/science/article/pii/S0263224122009629)
Abstract: Global solar radiation (GSR) prediction plays an essential role in
planning, controlling and monitoring solar power systems. However, its stochastic
behaviour is a significant challenge in achieving satisfactory prediction results.
This study aims to design an innovative hybrid prediction model that integrates a
feature selection mechanism using a Slime-Mould algorithm, a Convolutional-Neural-
Network (CNN), a Long–Short-Term-Memory Neural Network (LSTM) and a final CNN with
Multilayer-Perceptron output (SCLC algorithm hereafter). The proposed model was
applied to six solar farms in Queensland (Australia) at daily temporal horizons in
six different time steps. The comprehensive benchmarking of the obtained results
with those from two Deep-Learning (CNN-LSTM, Deep-Neural-Network) and three
Machine-Learning (Artificial-Neural-Network, Random-Forest, Self-Adaptive
Differential-Evolutionary Extreme-Learning-Machines) models highlighted a higher
performance of the proposed prediction model in all the six selected solar farms.
From the results obtained, this work establishes that the designed SCLC algorithm
could have a practical utility for applications in renewable and sustainable energy
resource management.
Keywords: Global solar prediction; Deep Learning networks; Convolutional Neural
Networks; Slime Mould Algorithm; Renewable energy; Global climate models

Dounia El Bourakadi, Ali Yahyaouy, Jaouad Boumhidi,

Intelligent energy management for micro-grid based on deep learning LSTM prediction
model and fuzzy decision-making,
Sustainable Computing: Informatics and Systems,
Volume 35,
2022,
100709,
ISSN 2210-5379,
https://doi.org/10.1016/j.suscom.2022.100709.
(https://www.sciencedirect.com/science/article/pii/S2210537922000488)
Abstract: In this paper, we present a new intelligent system based on multi-agent
system for energy management in micro-grid with grid-connected mode and mainly
based on wind energy. The main contribution of this work is to present a model for
wind power prediction for the next hour using the wind speed values of previous
hours. It is a model based on a deep recurrent neural network with Long Short Term
Memory which takes as input a number of previous values of the wind speed and as
output the wind power produced for the next hour. The proposed model is compared to
a Multi-Layer Perceptron with the same inputs and output. Both models aim to find a
hidden correlation between historical weather data (previous wind speeds) and the
next wind power produced. Using as a case study the dataset from a real site
located in Tetouan city in Morocco, experimental results show that Long Short Term
Memory based prediction model yields good results and successfully achieve high
accuracy compared to Multi-Layer Perceptron based model. To establish a robust
energy management strategy, a fuzzy logic controller is used as decision-making
tool to manage the different energy flows in the micro-grid and to control the
amount of electricity delivered or taken from the distribution network in order to
reduce the costs and maximize the profits.
Keywords: Micro-grid; Multi-agent system; Wind power prediction; Deep recurrent
neural network; Long-short term memory; Decision-making; Fuzzy logic control

Jun Zhang, Pengli Wu, Xia Xu, Ming Han, Bin Pan,
PCS-LSTM: A hybrid deep learning model for multi-stations joint temperature
prediction based on periodicity and closeness,
Neurocomputing,
Volume 501,
2022,
Pages 151-161,
ISSN 0925-2312,
https://doi.org/10.1016/j.neucom.2022.06.015.
(https://www.sciencedirect.com/science/article/pii/S0925231222007342)
Abstract: Temperature is one of the most important meteorological elements, which
affects the daily lives of people all over the world. Owing to the rapid
development of meteorological facilities, the number of meteorological observation
stations on earth is gradually increasing, which brings challenges to the spatial
association between stations. Many researchers focus on how to predict temperature
more accurately utilizing these associations. However, the existing deep learning
methods of temperature prediction have difficulty in capturing the interactions
between neighboring stations in the spatial dimension. In addition, in the time
dimension, the temperature in nature exhibits not only nearby variations but also
periodic characteristics, which further increases the difficulty of temperature
prediction. To solve the aforementioned two problems, we propose the periodicity
and closeness social long short-term memory (PCS-LSTM) model, which includes PS-
LSTM and CS-LSTM modules. Specifically, to model the relationships between multiple
meteorological observation stations, we utilized the social pooling in the PS-LSTM
and CS-LSTM modules to establish spatial associations. To further refine the
temperature variation, we combine PS-LSTM and CS-LSTM to model the periodicity and
closeness of the time series. Compared with the LSTM basic model, the experiments
show that the MAE of our model prediction results is reduced by 0.109°C in the next
24 h compared.
Keywords: Temperature prediction; LSTM; Periodicity; Closeness

Rahim Barzegar, Mohammad Taghi Aalami, Jan Adamowski,

Coupling a hybrid CNN-LSTM deep learning model with a Boundary Corrected Maximal
Overlap Discrete Wavelet Transform for multiscale Lake water level forecasting,
Journal of Hydrology,
Volume 598,
2021,
126196,
ISSN 0022-1694,
https://doi.org/10.1016/j.jhydrol.2021.126196.
(https://www.sciencedirect.com/science/article/pii/S0022169421002432)
Abstract: Developing accurate lake water level (WL) forecasting models is important
for flood control, shoreline maintenance and sustainable water resources planning
and management. In this study, improved accuracy of forecasts (up to three months)
of Lake Michigan and Lake Ontario WLs was achieved by coupling Boundary Corrected
(BC) Maximal Overlap Discrete Wavelet Transform (MODWT) data preprocessing, with a
hybrid Convolutional Neural Network (CNN) Long-Short Term Memory (LSTM) deep
learning (DL) model. Hybrid DL-based model performance was compared to that of BC-
MODWT machine learning (ML) [e.g., Random Forest (RF) and Support Vector Regression
(SVR)] models. For each lake, all models were calibrated using 70% of the monthly
WL (in meters) data series (January 1918 to February 1988), with the remaining 30%
(March 1988 to December 2018) serving for validation. In both standalone and
wavelet-machine learning models, a hybrid correlation-based feature selection
(CFS)-particle swarm optimization (PSO) search method served to select input
variables among candidate WL lags of up to twelve months, whereas for the CNN-LSTM
DL models, input variable selection was carried out automatically by the CNN
structure. For the MODWT-based ML and DL models, input time series were decomposed
using a BC-MODWT approach. Scaling coefficients were developed through several
mother wavelet approaches (i.e., Haar, Daubechies, Symlets, Fejer-Korovkin and
Coiflets) with different filter lengths (up to twelve) and decomposition levels (up
to seven). Model performance was evaluated using several visual and statistical
metrics, including the correlation coefficient, r; the root mean standard error,
RMSE; and Willmot’s Index, WI. The CNN-LSTM DL model outperformed the standalone
SVR and RF models. For all time horizons, coupled MODWT-based CNN-LSTM models
outperformed standalone and hybrid models in WL forecasting. Not all wavelet
family/filter length/decomposition combinations improved standalone models;
however, the proposed BC-MODWT-CNN-LSTM model implementing the Haar mother wavelet
(for Lake Michigan — one-month ahead: r = 0.994, RMSE = 0.04 m, WI = 0.996; two-
months ahead: r = 0.979, RMSE = 0.07 m, WI = 0.989; three-months ahead: r = 0.957,
RMSE = 0.102 m, WI = 0.976; for Lake Ontario — one-month ahead: r = 0.956,
RMSE = 0.082 m, WI = 0.978; two-months ahead: r = 0.864, RMSE = 0.141, WI = 0.912;
three-months ahead: r = 0.755, RMSE = 0.182 m, WI = 0.841) outperformed standalone
ML and BC-MODWT-ML-based models. Accordingly, the BC-MODWT-CNN-LSTM model can be
viewed as a potentially useful approach to increase the accuracy of lake WL
forecasts.
Keywords: Forecasting; Wavelet transform; Deep learning; Machine learning; Water
level; Great Lakes

Reuben Varghese Joseph, Anshuman Mohanty, Soumyae Tyagi, Shruti Mishra, Sandeep
Kumar Satapathy, Sachi Nandan Mohanty,
A hybrid deep learning framework with CNN and Bi-directional LSTM for store item
demand forecasting,
Computers and Electrical Engineering,
Volume 103,
2022,
108358,
ISSN 0045-7906,
https://doi.org/10.1016/j.compeleceng.2022.108358.
(https://www.sciencedirect.com/science/article/pii/S0045790622005754)
Abstract: In the era of ever-changing market landscape, enterprises tend to make
quick and informed decisions to survive and prosper in the competition. Decision
makers within an organization must be supplied with data in a way that could be
easily analyzed and comprehended to build strategies in order to achieve business
goals. Accurate demand forecasting of products is one of such decisions which is
crucial for retail operators to have a clear picture on the future demand of their
products and services. With a certainty in estimation, retailers might keep a check
on how many items to allocate, order and restock thus boosting their gross sales
and profits. Machine Learning approaches are widely used for demand forecasting of
different items. In this work, we have used the Store Item Demand Forecasting
Challenge dataset from Kaggle to implement our proposed framework. The main novelty
of this study was to build a coupled CNN-BiLSTM framework with Lazy Adam optimizer
to make an accurate forecast of product demand of store items. Various State-of-art
machine learning techniques like SGD (Stochastic Gradient Descent), Linear
Regression, K-Nearest Neighbour, Bagging, Random Forest, SVR, XgBoost (extreme
gradient boosting) and CNN-LSTM. for demand forecasting has been implemented and
the results were compared with the proposed model. On evaluation with metrics
including Mean Absolute Percentage Error (MAPE), R-Squared (R2) value and Mean
Absolute Error (MAE), it was observed that the proposed framework having more
accurecy as compare to the traditional approaches.
Keywords: Convolutional neural network; Bidirectional long short-term memory;
Product demand forecasting; Lazy Adam optimizer; Inventory prediction; Supply
chain; R-squared score; Mean absolute error; Mean absolute percentage error;
Machine learning; Time series analysis

Yaoran Chen, Yan Wang, Zhikun Dong, Jie Su, Zhaolong Han, Dai Zhou, Yongsheng Zhao,
Yan Bao,
2-D regional short-term wind speed forecast based on CNN-LSTM deep learning model,
Energy Conversion and Management,
Volume 244,
2021,
114451,
ISSN 0196-8904,
https://doi.org/10.1016/j.enconman.2021.114451.
(https://www.sciencedirect.com/science/article/pii/S0196890421006270)
Abstract: Short-term wind speed forecast is of great importance to wind farm
regulation and its early warning. Previous studies mainly focused on the prediction
at a single location but few extended the task to 2-D wind plane. In this study, a
novel deep learning model was proposed for a 2-D regional wind speed forecast,
using the combination of the auto-encoder of convolutional neural network (CNN) and
the long short-term memory unit (LSTM). The 12-hidden-layer deep CNN was adopted to
encode the high dimensional 2-D input into the embedding vector and inversely, to
decode such latent representation after it was predicted by the LSTM module based
on historical data. The model performance was compared with parallel models under
different criteria, including MAE, RMSE and R2, all showing stable and considerable
enhancements. For instance, the overall MAE value dropped to 0.35 m/s for the
current model, which is 32.7%, 28.8% and 18.9% away from the prediction results
using the persistence, basic ANN and LSTM model. Moreover, comprehensive
discussions were provided from both temporal and spatial views of analysis,
revealing that the current model can not only offer an accurate wind speed forecast
along timeline (R2 equals to 0.981), but also give a distinct estimation of the
spatial wind speed distribution in 2-D wind farm.
Keywords: Regional wind speed prediction; CNN; LSTM; Temporal series fitness;
Spatial distribution

Shaojiang Dong, Jiafeng Xiao, Xiaolin Hu, Nengwei Fang, Lanhui Liu, Jinbao Yao,
Deep transfer learning based on Bi-LSTM and attention for remaining useful life
prediction of rolling bearing,
Reliability Engineering & System Safety,
Volume 230,
2023,
108914,
ISSN 0951-8320,
https://doi.org/10.1016/j.ress.2022.108914.
(https://www.sciencedirect.com/science/article/pii/S0951832022005294)
Abstract: Many transfer learning methods focus on training models between domains
with large differences. However, the data feature distribution varies greatly in
different bearing degradation processes, which affects the prediction accuracy of
Remaining useful life (RUL). To solve this problem, a novel method for RUL
prediction with more refined transfer is proposed, which added failure behavior
judgment. Firstly, a failure behavior judgment method is proposed by using the
convolutional autoencoder (CAE) and Pearson correlation coefficient to determine
whether the bearing fails gradually or suddenly. Then, a multi-channel transfer
network is proposed for extracting multi-scale features of bearing degradation.
Each channel uses convolutional network and bidirectional long short-term memory
(Bi-LSTM) to extract global and temporal information. To reduce the difference in
feature distribution between the training and test bearing data, a domain adaptive
structure is added after feature fusion in each channel to enable the model to
learn domain invariant features. By applying this method to experimental data and
comparing it with other methods, the superiority and effectiveness of the proposed
method are verified.
Keywords: Rolling bearings; Remaining useful life prediction; Feature distribution;
Domain adaptive

Sumudu Senanayake, Biswajeet Pradhan, Abdullah Alamri, Hyuck-Jin Park,

A new application of deep neural network (LSTM) and RUSLE models in soil erosion
prediction,
Science of The Total Environment,
Volume 845,
2022,
157220,
ISSN 0048-9697,
https://doi.org/10.1016/j.scitotenv.2022.157220.
(https://www.sciencedirect.com/science/article/pii/S0048969722043182)
Abstract: Rainfall variation causes frequent unexpected disasters all over the
world. Increasing rainfall intensity significantly escalates soil erosion and soil
erosion related hazards. Forecasting accurate rainfall helps early detection of
soil erosion vulnerability and can minimise the damages by taking appropriate
measures caused by severe storms, droughts and floods. This study aims to predict
soil erosion probability using the deep learning approach: long short-term memory
neural network model (LSTM) and revised universal soil loss equation (RUSLE) model.
Daily rainfall data were gathered from five agro-meteorological stations in the
Central Highlands of Sri Lanka from 1990 to 2021 and fed into the LSTM model
simulation. The LSTM model was forecasted with the time-series monthly rainfall
data for a long lead time period, rainfall values for next 36 months in each
station. Geo-informatics tools were used to create the rainfall erosivity map layer
for the year 2024. The RUSLE model prediction indicates the average annual soil
erosion over the Highlands will be 11.92 t/ha/yr. Soil erosion susceptibility map
suggests around 30 % of the land area will be categorised as moderate to very-high
soil erosion susceptible classes. The resulted map layer was validated using past
soil erosion map layers developed for 2000, 2010 and 2019. The soil erosion
susceptibility map indicates an accuracy of 0.93 with the area under the receiver
operator characteristic curve (AUC-ROC), showing a satisfactory prediction
performance. These findings will be helpful in policy-level decision making and
researchers can further tested different deep learning models with the RUSLE model
to enhance the prediction capability of soil erosion probability.
Keywords: Deep learning-LSTM model; Soil erosion susceptibility; Rainfall
erosivity; RUSLE; GIS

Aatish Anshuman, T.I. Eldho,

Entity aware sequence to sequence learning using LSTMs for estimation of
groundwater contamination release history and transport parameters,
Journal of Hydrology,
Volume 608,
2022,
127662,
ISSN 0022-1694,
https://doi.org/10.1016/j.jhydrol.2022.127662.
(https://www.sciencedirect.com/science/article/pii/S0022169422002372)
Abstract: Groundwater contaminant sources identification and parameter estimation
using the simulation–optimization (S/O) approach require numerous runs of the
computationally expensive simulation model through the optimization algorithm. The
computational cost can be effectively reduced by using a surrogate model which can
accurately approximate the simulation model. With the advent of deep learning, Long
Short-Term Memory (LSTM) networks, which are suitable to learn sequential data, are
being increasingly applied to regression problems involving time dependencies.
However, for the simultaneous contaminant source identification and parameter
estimation problem, the surrogate model requires to establish a relationship
between release histories at the source locations to concentration measurements at
observation points subject to given values of aquifer parameters. In this study, a
novel deep neural network framework Entity Aware Sequence to sequence learning
using Long Short-Term Memory (EAS-LSTM) is proposed as a surrogate model which
takes both sequential i.e., release histories at the source locations at different
stress periods and static variables i.e., transport parameters as inputs to predict
breakthrough curves (BTCs) at observation points. The proposed surrogate model is
applied to a heterogeneous field scale aquifer with 4 zones. The Mean Squared Error
(MSE) using EAS-LSTM is 0.0033 ppm2 which is significantly better than that of
Kriging and Support Vector Regression (SVR) which are 0.048 ppm2 and 1.302 ppm2
respectively. Comparison of EAS-LSTM model performance with Kriging and SVR based
models demonstrates its higher accuracy. Further, the optimization algorithms for
inverse modelling, the performances of Multiverse Optimizer (MVO), Grey wolf
optimization (GWO) and Particle swarm optimization (PSO) are investigated. It is
observed that the combination of EAS-LSTM and MVO provides better results in
comparison to other surrogate simulation optimization (SSO) models.
Keywords: Contaminant source identification; Parameter estimation; Groundwater
contamination; Surrogate model; Long Short-Term Memory (LSTM); Multiverse
Optimization (MVO)

Motiur Rahman, Dilshad Islam, Rokeya Jahan Mukti, Indrajit Saha,

A deep learning approach based on convolutional LSTM for detecting diabetes,
Computational Biology and Chemistry,
Volume 88,
2020,
107329,
ISSN 1476-9271,
https://doi.org/10.1016/j.compbiolchem.2020.107329.
(https://www.sciencedirect.com/science/article/pii/S1476927120304692)
Abstract: Diabetes is a chronic disease that occurs when the pancreas does not
generate sufficient insulin or the body cannot effectively utilize the produced
insulin. If it remains unidentified and untreated, then it could be very deadliest.
One can lead a healthy life with proper treatment if the presence of diabetes can
be detected at an early stage. When the conventional process of detecting diabetes
is tedious, there is a need of an automated system for identifying diabetes from
the clinical and physical data. In this study, we developed a novel diabetes
classifying model based on Convolutional Long Short-term Memory (Conv-LSTM) that
was not applied yet in this regard. We applied another three popular models such as
Convolutional Neural Network (CNN), Traditional LSTM (T-LSTM), and CNN-LSTM and
compared the performance with our developed model over the Pima Indians Diabetes
Database (PIDD). Significant features were extracted from the dataset using Boruta
algorithm that returned glucose, BMI, insulin, blood pressure, and age as important
features for classifying diabetes patients more accurately. We performed
hyperparameter optimization using Grid Search algorithm in order to find the
optimal parameters for the applied models. Initial experiment by splitting the
dataset into separate training and testing sets, the Conv-LSTM-based model
classified the diabetes patients with the highest accuracy of 91.38 %. In later,
using cross-validation technique the Conv-LSTM model achieved the highest accuracy
of 97.26 % and outperformed the other three models along with the state-of-the-art
models.
Keywords: Diabetes detection; Feature selection; Parameter optimization; LSTM;
Conv-LSTM; CNN

Jayanthi E, Vallikannu R,
Enhancing the performance of asymmetric architectures and workload characterization
using LSTM learning algorithm,
Advances in Engineering Software,
Volume 173,
2022,
103266,
ISSN 0965-9978,
https://doi.org/10.1016/j.advengsoft.2022.103266.
(https://www.sciencedirect.com/science/article/pii/S0965997822001673)
Abstract: Asymmetric multiple core architectures are becoming increasingly popular
because of their QoS, energy efficiency, and high performance. Because of the
heterogeneous piping layout and execution procedure, the overall system performance
is highly influenced by workload characterization becomes a difficult issue.
Recently, it has become vital to create a performance evaluator for faster response
FPGA structures. With the embedded computerized visual hardware resources (e.g.
GPUs, FPGAs, and multi-core CPUs) and its associated software manages this
heterogeneous system design is tough for the researchers. A thorough analysis of
the performance of run-time and various embedded kernels' energy efficiencies to
assist developers in determining which embedded substrate is best for their
project. To solve these challenges, resource-based prediction models are developed
for FPGA-SoC relying on the characterization of intellectual workload, and its
performance is thoroughly examined. Xilinx FPGA frameworks are used to run a
variety of real-time applications that perform workloads depending on projected
processing cores. When compared to existing Ml classifiers, this LSTM predictor
achieves a mean of 96% of the experimental findings and also improves energy
efficiency.
Keywords: Performance Enhancement; Asymmetric architectures; LSTM; Workload
Characterization

Xiangling Li, Kang Xiao, Xianbing Li, Chunye Yu, Dongyan Fan, Zhixue Sun,
A well rate prediction method based on LSTM algorithm considering manual
operations,
Journal of Petroleum Science and Engineering,
Volume 210,
2022,
110047,
ISSN 0920-4105,
https://doi.org/10.1016/j.petrol.2021.110047.
(https://www.sciencedirect.com/science/article/pii/S0920410521016582)
Abstract: Manual operations such as changing the size of chokes as well as opening
and closing of the well have a great impact on oil and gas production from the
well. This scenario is not considered in most deep learning methods for predicting
productivity. Therefore, a deep learning method based on a long short-term memory
(LSTM) neural network model was established to predict well performance considering
the manual operations. The input dataset was composed of data related to choke
size, daily opening time series, and production; the first 90% of the dataset was
used as the training set and the remaining 10% was used as the test set. The deep
learning model was constructed using a LSTM module, regularization process, and
dropout network. The formulated LSTM model was proficient compared with a model
that did not consider the manual operation process, and showed better prediction
accuracy. Through multiple experiments, the production-related time step was
optimized at three, indicating that prediction for the subsequent step was most
relevant to the initial three step inputs. Overall, the operation of opening and
closing of wells, changing the size of chokes, and variations in daily production
time can be considered in our LSTM deep learning model, which provides more
reasonable results.
Keywords: Manual operations; Time series; LSTM; Deep learning method; Rate
prediction

Ramdas Vankdothu, Mohd Abdul Hameed, Husnah Fatima,

A Brain Tumor Identification and Classification Using Deep Learning based on CNN-
LSTM Method,
Computers and Electrical Engineering,
Volume 101,
2022,
107960,
ISSN 0045-7906,
https://doi.org/10.1016/j.compeleceng.2022.107960.
(https://www.sciencedirect.com/science/article/pii/S0045790622002361)
Abstract: Brain tumors are one of the most often diagnosed malignant tumors in
persons of all ages. Recognizing its grade is challenging for radiologists in
health monitoring and automated determination; however, IoT can help. It is
critical to detect and classify contaminated tumor locations using Magnetic
Resonance Imaging (MRI) images. Numerous tumors exist, including glioma tumor,
meningioma tumor, pituitary tumor, and no tumor (benign). Detecting the type of
tumor and preventing it is one of the most challenging aspects of brain tumor
categorization. Numerous deep learning-based approaches for categorizing brain
tumors have been published in the literature. A CNN (Convolutional Neural Network),
the most advanced method in deep learning, was used to detect a tumor using brain
MRI images. However, there are still issues with the training procedure, which is
lengthy. The main goal of this project is to develop an IoT computational system
based on deep learning for detecting brain tumors in MRI images. This paper
suggests combining A CNN(Convolutional Neural Network) with an STM(Long Short Term
Memory), LSTMs can supplement the ability of CNN to extract features. When used for
image classification, the layered LSTM-CNN design outperforms standard CNN
classification. Experiments are undertaken to forecast the proposed model's
performance using the Kaggle data set, which contains 3264 MRI scans. The dataset
is separated into two sections: 2870 photos of training sets and 394 images of
testing sets. The experimental findings demonstrate that the proposed model
outperforms earlier CNN and RNN models in terms of accuracy.
Keywords: A Brain Tumor; Convolutional Neural Network(CNN); Classification; Deep
Learning; Magnetic Resonance Imaging; LSTM

Yan Hu, Jian Ni, Liu Wen,

A hybrid deep learning approach by integrating LSTM-ANN networks with GARCH model
for copper price volatility prediction,
Physica A: Statistical Mechanics and its Applications,
Volume 557,
2020,
124907,
ISSN 0378-4371,
https://doi.org/10.1016/j.physa.2020.124907.
(https://www.sciencedirect.com/science/article/pii/S0378437120304696)
Abstract: Forecasting the copper price volatility is an important yet challenging
task. Given the nonlinear and time-varying characteristics of numerous factors
affecting the copper price, we propose a novel hybrid method to forecast copper
price volatility. Two important techniques are synthesized in this method. One is
the classic GARCH model which encodes useful statistical information about the
time-varying copper price volatility in a compact form via the GARCH forecasts. The
other is the powerful deep neural network which combines the GARCH forecasts with
both domestic and international market factors to search for better nonlinear
features; it also combines the long short-term memory (LSTM) network with
traditional artificial neural network (ANN) to generate better volatility
forecasts. Our method synthesizes the merits of these two techniques and is
especially suitable for the task of copper price volatility prediction. The
empirical results show that the GARCH forecasts can serve as informative features
to significantly increase the predictive power of the neural network model, and the
integration of the LSTM and ANN networks is an effective approach to construct
useful deep neural network structures to boost the prediction performance. Further,
we conducted a series of sensitivity analyses of the neural network architecture to
optimize the prediction results. The results suggest that the choice between LSTM
and BLSTM networks for the hybrid model should consider the forecast horizon, while
the ANN configurations should be fine-tuned depending on the choice of the measure
of prediction errors.
Keywords: Copper price volatility; Deep learning; GARCH; LSTM-ANN

S. Sowmya, Deepa Jose,

Contemplate on ECG signals and classification of arrhythmia signals using CNN-LSTM
deep learning model,
Measurement: Sensors,
Volume 24,
2022,
100558,
ISSN 2665-9174,
https://doi.org/10.1016/j.measen.2022.100558.
(https://www.sciencedirect.com/science/article/pii/S2665917422001921)
Abstract: An electrocardiogram (ECG) is a schematic illustration of heart signals
that is being used to measure the electric signals of the heart and to detect any
abnormalities. Due to non-invasive qualities, the Electrocardiogram (ECG) has
become a commonly employed auxiliary diagnostic index for heart problems in pre-
screening and give information of heart diseases. Many methods are used to find out
the abnormalities in heartbeat. In this paper survey is done to find out what are
the methods used to classify the ECG recordings to predict cardiovascular diseases
which affects middle aged as well as older people causing severe illness leading to
death. One of the major abnormality was due to arrhythmia disease. Hence many deep
learning methods were used to find early prediction of arrhythmia to save lives of
people. From the survey it is found that many ECG classifications done using
existing database such as MIT-BIH arrhythmia. Most of the methods work on
classifications, feature extractions to find abnormalities in ECG signals and found
to have higher accuracies of more than 94%.In this paper, the study is based on
abnormalities in ECG signals due to arrhythmia and its identification using a
network architecture based on LSTM and CNN deep learning methods. The simulation
result shows the CNN- LSTM algorithm has higher accuracy compared to CNN.
Keywords: Arrhythmia detection; Machine learning; Convolutional Neural Networks
(CNN); Long Short-Term Memory (LSTM); Massachusetts Institute of Technology-Beth
Israel Hospital (MIT-BIH)

Sachi Nandan Mohanty, E. Laxmi Lydia, Mohamed Elhoseny, Majid M. Gethami Al Otaibi,
K. Shankar,
Deep learning with LSTM based distributed data mining model for energy efficient
wireless sensor networks,
Physical Communication,
Volume 40,
2020,
101097,
ISSN 1874-4907,
https://doi.org/10.1016/j.phycom.2020.101097.
(https://www.sciencedirect.com/science/article/pii/S1874490720301737)
Abstract: Wireless sensor network (WSN) comprises a collection of sensor nodes
employed to monitor and record the status of the physical environment and organize
the gathered data at a central location. This paper presents a deep learning based
distributed data mining (DDM) model to achieve energy efficiency and optimal load
balancing at the fusion center of WSN. The presented DMM model includes a recurrent
neural network (RNN) based long short-term memory (LSTM) called RNN-LSTM, which
divides the network into various layers and place them into the sensor nodes. The
proposed model reduces the overhead at the fusion center along with a reduction in
the number of data transmission. The presented RNN-LSTM model is tested under a
wide set of experimentation with varying number of hidden layer nodes and signaling
intervals. At the same time, the amount of energy needed to transmit data by RNN-
LSTM model is considerably lower than energy needed to transmit actual data. The
simulation results indicated that the RNN-LSTM reduces the signaling overhead,
average delay and maximizes the overall throughput compared to other methods. It is
noted that under the signaling interval of 240 ms, it can be shown that the RNN-
LSTM achieves a minimum average delay of 190 ms whereas the OSPF and DNN models
shows average delay of 230 ms and 230 ms respectively.
Keywords: Deep learning; Energy efficiency; Recurrent neural network; WSN

Chenfei Ma, Chuang Lin, Oluwarotimi Williams Samuel, Lisheng Xu, Guanglin Li,
Continuous estimation of upper limb joint angle from sEMG signals based on SCA-LSTM
deep learning approach,
Biomedical Signal Processing and Control,
Volume 61,
2020,
102024,
ISSN 1746-8094,
https://doi.org/10.1016/j.bspc.2020.102024.
(https://www.sciencedirect.com/science/article/pii/S1746809420301804)
Abstract: Robotic arm control has drawn a lot of attention along with the
development of industrialization. The methods based on myoelectric pattern
recognition have been proposed with multiple degrees of freedom for years. While
these methods can support the actuation of several classes of discrete movements
sequentially, they do not allow simultaneous control of multiple movements in a
continuous manner like natural arms. In this study, we proposed a short connected
autoencoder long short-term memory (SCA-LSTM) based simultaneous and proportional
(SP) scheme that estimates continuous arm movements using kinematic information
extracted from surface electromyogram (sEMG) recordings. The sEMG signals
corresponding to seven classes of shoulder-elbow joint angle movements acquired
from eleven participants were preprocessed using max root mean square envelope.
Afterwards, the proposed SCA-LSTM model and two commonly applied models, namely,
multilayer perceptrons (MLPs) and convolutional neural network (CNN), were trained
and tested using the preprocessed data for continuous estimation of arm movements.
Our experimental results showed that the proposed SCA-LSTM model could achieve a
significantly higher estimation accuracy of approximately 95.7% that is
consistently stable across the subjects in comparison to the CNN (86.8%) and MLP
(83.4%) models. These results suggest that the proposed SCA-LSTM would be a
promising model for continuous estimation of upper limb movements from sEMG signals
for prosthetic control.
Keywords: Robotic arm control; Surface electromyogram; Simultaneous and
proportional control; Joint angle estimation; Deep learning

Ghufran Isam Drewil, Riyadh Jabbar Al-Bahadili,

Air pollution prediction using LSTM deep learning and metaheuristics algorithms,
Measurement: Sensors,
Volume 24,
2022,
100546,
ISSN 2665-9174,
https://doi.org/10.1016/j.measen.2022.100546.
(https://www.sciencedirect.com/science/article/pii/S2665917422001805)
Abstract: Air pollution is a leading cause of health concerns and climate change,
one of humanity's most dangerous problems. This problem has been exacerbated by an
overabundance of automobiles, industrial output pollution, transportation fuel
consumption, and energy generation. As a result, air pollution forecasting has
become vital. As a result of the large amount and variety of data acquired by air
pollution monitoring stations, air pollution forecasting has become a popular
topic, particularly when applying deep learning models of long short-term memory
(LSTM). The ability of these models to learn long-term dependencies in air
pollution data sets them apart. However, LSTM models using many other statistical
and machine learning approaches may not offer adequate prediction results due to
noisy data and improper hyperparameter settings. As a result, to define the
pollution levels for a group of contaminants, an ideal representation of the LSTM
is required. To address the problem of identifying the best hyperparameters for the
LSTM model, In this paper, we propose a model based on the Genetic Algorithm (GA)
algorithm as well as the long short-term memory (LSTM) deep learning algorithm. The
model aims to find the best hyperparameters for LSTM and the pollution level for
the next day using four types of pollutants PM10, PM2.5, CO, and NOX. The proposed
model modified by optimization algorithms shows more accurate results with less
experience and more speed than machine learning models and LSTM models.
Keywords: Deep learning; Long short-term memory (LSTM); Genetic algorithm (GA);
Time series data; Air pollution

Yang Wang, Pandi Vijayakumar, Brij B. Gupta, Wadee Alhalabi, Audithan Sivaraman,
An improved entity recognition approach to cyber-social knowledge provision of
intellectual property using a CRF-LSTM model,
Pattern Recognition Letters,
Volume 163,
2022,
Pages 145-151,
ISSN 0167-8655,
https://doi.org/10.1016/j.patrec.2022.10.001.
(https://www.sciencedirect.com/science/article/pii/S016786552200294X)
Abstract: With the development of cutting-edge IT technologies, e.g. Big Data,
Knowledge Engineering, etc., traditional Intellectual Property (IP) services have
depicted high redundancy and low efficiency during management of such large-scale
of data. Recent advancement of Artificial Intelligence (AI) and Deep Learning (DL)
models has been accelerating relevant research activities being investigated on
Knowledge Graph (KG) schemes and applications in different domains, such as medical
services, social media, etc. However, when IP services and their cyber-social
provision are taken into account, relevant approaches suffer from unbalanced labels
against training results, and inappropriate evaluation metrics not well reflecting
the impact of the unbalance. In this paper, a deep learning model combining
Conditional Random Field and Bidirectional LSTM has been proposed, in order to
achieve named entity recognition with unbalanced labels. An adaptive metric, G-
Score was introduced to compare the fitting ability of models by evaluating the gap
between Precision and Recall. According to the results, the proposed model can
effectively recognize the potential named entities with outperformance over other
relevant models.
Keywords: Intellectual property; Named-entity recognition; LSTM; CRF; Knowledge
graph

Rinku Supakar, Parthasarathi Satvaya, Prasun Chakrabarti,

A deep learning based model using RNN-LSTM for the Detection of Schizophrenia from
EEG data,
Computers in Biology and Medicine,
Volume 151, Part A,
2022,
106225,
ISSN 0010-4825,
https://doi.org/10.1016/j.compbiomed.2022.106225.
(https://www.sciencedirect.com/science/article/pii/S0010482522009337)
Abstract: Normal life can be ensured for schizophrenic patients if diagnosed early.
Electroencephalogram (EEG) carries information about the brain network connectivity
which can be used to detect brain anomalies that are indicative of schizophrenia.
Since deep learning is capable of automatically extracting the significant features
and make classifications, the authors proposed a deep learning based model using
RNN-LSTM to analyze the EEG signal data to diagnose schizophrenia. The proposed
model used three dense layers on top of a 100 dimensional LSTM. EEG signal data of
45 schizophrenic patients and 39 healthy subjects were used in the study.
Dimensionality reduction algorithm was used to obtain an optimal feature set and
the classifier was run with both sets of data. An accuracy of 98% and 93.67% were
obtained with the complete feature set and the reduced feature set respectively.
The robustness of the model was evaluated using model performance measure and
combined performance measure. Outcomes were compared with the outcome obtained with
traditional machine learning classifiers such as Random Forest, SVM, FURIA, and
AdaBoost, and the proposed model was found to perform better with the complete
dataset. When compared with the result of the researchers who worked with the same
set of data using either CNN or RNN, the proposed model's accuracy was either
better or comparable to theirs.
Keywords: Schizophrenia; Electroencephalogram; Deep learning; RNN-LSTM; CNN

Peipei Wang, Xinqi Zheng, Gang Ai, Dongya Liu, Bangren Zhu,
Time series prediction for the epidemic trends of COVID-19 using the improved LSTM
deep learning method: Case studies in Russia, Peru and Iran,
Chaos, Solitons & Fractals,
Volume 140,
2020,
110214,
ISSN 0960-0779,
https://doi.org/10.1016/j.chaos.2020.110214.
(https://www.sciencedirect.com/science/article/pii/S096007792030610X)
Abstract: The COVID-19 outbreak in late December 2019 is still spreading rapidly in
many countries and regions around the world. It is thus urgent to predict the
development and spread of the epidemic. In this paper, we have developed a
forecasting model of COVID-19 by using a deep learning method with rolling update
mechanism based on the epidemical data provided by Johns Hopkins University. First,
as traditional epidemical models use the accumulative confirmed cases for training,
it can only predict a rising trend of the epidemic and cannot predict when the
epidemic will decline or end, an improved model is built based on long short-term
memory (LSTM) with daily confirmed cases training set. Second, considering the
existing forecasting model based on LSTM can only predict the epidemic trend within
the next 30 days accurately, the rolling update mechanism is embedded with LSTM for
long-term projections. Third, by introducing Diffusion Index (DI), the
effectiveness of preventive measures like social isolation and lockdown on the
spread of COVID-19 is analyzed in our novel research. The trends of the epidemic in
150 days ahead are modeled for Russia, Peru and Iran, three countries on different
continents. Under our estimation, the current epidemic in Peru is predicted to
continue until November 2020. The number of positive cases per day in Iran is
expected to fall below 1000 by mid-November, with a gradual downward trend expected
after several smaller peaks from July to September, while there will still be more
than 2000 increase by early December in Russia. Moreover, our study highlights the
importance of preventive measures which have been taken by the government, which
shows that the strict controlment can significantly reduce the spread of COVID-19.
Keywords: Covid-19; LSTM; Rolling update mechanism; Modeling; Forecasting

Sanjay Kumar, Abhishek Mallik, B.S. Panda,

Influence maximization in social networks using transfer learning via graph-based
LSTM,
Expert Systems with Applications,
Volume 212,
2023,
118770,
ISSN 0957-4174,
https://doi.org/10.1016/j.eswa.2022.118770.
(https://www.sciencedirect.com/science/article/pii/S0957417422017882)
Abstract: Social networks have emerged as efficient platforms to connect people
worldwide and facilitate the rapid spread of information. Identifying influential
nodes in social networks to accelerate the spread of particular information across
the network formulates the study of influence maximization (IM). In this paper,
inspired by deep learning techniques, we propose a novel approach to solve the
influence maximization problem as a classical regression task using transfer
learning via graph-based long short-term memory (GLSTM). We start by calculating
three popular node centrality methods as feature vectors for the nodes in the
network and every node’s individual influence under susceptible–infected–recovered
(SIR) information diffusion model, which forms the labels of the nodes in the
network. The generated feature vectors and their corresponding labels for all the
nodes are then fed into a graph-based long short-term memory (GLSTM). The proposed
architecture is trained on a vast and complex network to generalize the model
parameters better. The trained model is then used to predict the probable influence
of every node in the target network. The proposed model is compared with some of
the well-known and recently proposed algorithms of influence maximization on
several real-life networks using the popular SIR model of information diffusion.
The intensive experiments suggest that the proposed model outperforms these well-
known and recently proposed influence maximization algorithms.
Keywords: Graph-based LSTM; Influence maximization (IM); Node centrality;
Information diffusion; Social networks; Transfer learning

Kshitij Kakade, Ishan Jain, Aswini Kumar Mishra,

Value-at-Risk forecasting: A hybrid ensemble learning GARCH-LSTM based approach,
Resources Policy,
Volume 78,
2022,
102903,
ISSN 0301-4207,
https://doi.org/10.1016/j.resourpol.2022.102903.
(https://www.sciencedirect.com/science/article/pii/S0301420722003476)
Abstract: This study proposes a new hybrid model that combines LSTM and BiLSTM
neural networks with GARCH type model forecasts using an ensemble approach to
forecast volatility for one-day ahead 95% and 99% Value-at-Risk (VaR) estimates
using the Parametric (PAR) and Filtered Historical Simulation (FHS) method. The
forecasting abilities of the standard GARCH (GARCH), exponential GARCH (eGARCH),
and threshold GARCH (tGARCH) models are combined with the LSTM networks to capture
different characteristics of the underlying volatility. We evaluate the model using
log returns on Crude Oil during two periods of extreme volatility: the 2007-09
Financial Crisis and the Covid Recession of 2020–21. The performance of hybrid
models is compared against several traditional VaR methods like the Historical
Simulation, Bootstrap, Age weighted method, and the volatility-based VaR models
using the GARCH, LSTM, and BiLSTM model forecasts. The unconditional and
conditional coverage tests and a combination of regulator and firm loss functions
are used to evaluate the quality of VaR forecasts. We find a significant
improvement in the quality and accuracy of the VaR forecasts of the hybrid models
over all the other models across all loss functions and coverage tests. The FHS-
BiLSTM-HYBRID, a proposed FHS-based hybrid model, combining the BiLSTM model with
three GARCH-type models, is the best performing, with the lowest values for both
loss functions. The traditional and GARCH-type models do not efficiently model
volatility during the crisis periods resulting in poor VaR forecasts. The FHS
consistently performs as the best method for generating VaR compared to all other
approaches.
Keywords: Value-at-Risk; BiLSTM; LSTM; GARCH; Ensemble; Crude oil

Abdul Quadir Md, Sanjit Kapoor, Chris Junni A.V., Arun Kumar Sivaraman, Kong Fah
Tee, Sabireen H., Janakiraman N.,
Novel optimization approach for stock price forecasting using multi-layered
sequential LSTM,
Applied Soft Computing,
2022,
109830,
ISSN 1568-4946,
https://doi.org/10.1016/j.asoc.2022.109830.
(https://www.sciencedirect.com/science/article/pii/S1568494622008791)
Abstract: Stock markets can often be one of the most volatile places to invest.
Statistical analysis of past stock performance and external factors play a major
role in the decision to buy or sell stocks. These factors are all used to maximize
profits. Stock price index forecasting has been a subject of great research for
many years, and several machine learning and deep learning algorithms have been
proposed to simplify this complex task, but little success has been found so far.
In order to forecast stocks accurately, it is crucial to understand the context-
specific dependence of stock prices on their past values. The use of Long Short
Term Memory (LSTM), which is capable of understanding long-term data dependencies,
can help overcome this obstacle. In this context, this paper proposes a novel
optimization approach for stock price prediction that is based on a Multi-Layer
Sequential Long Short Term Memory (MLS LSTM) model which makes use of the adam
optimizer. Additionally, the MLS LSTM algorithm uses normalized time series data
divided into time steps to determine the relationship between past values and
future values in order to make accurate predictions. Furthermore, it eliminates the
vanishing gradient problem associated with simple recurrent neural networks. The
stock price index is forecasted by taking into account past performance information
along with past trends and patterns. The results illustrate that a 95.9% prediction
accuracy is achieved on the training data set and a 98.1% accuracy on the testing
data set with the MLS LSTM algorithm, which dramatically exceeds the performance of
other machine learning and deep learning algorithms. The mean absolute percentage
error was observed to be 1.79% on the training set and 2.18% on the testing set,
respectively. Moreover, the proposed model is able to estimate the stock price with
a normalized root mean squared error of 0.019, thus giving an accurate forecast and
making it a feasible real-world solution.
Keywords: Recurrent neural network; Forecasting; Stock market; Adam optimizer; Long
short-term memory

Ali Agga, Ahmed Abbou, Moussa Labbadi, Yassine El Houm, Imane Hammou Ou Ali,
CNN-LSTM: An efficient hybrid deep learning architecture for predicting short-term
photovoltaic power production,
Electric Power Systems Research,
Volume 208,
2022,
107908,
ISSN 0378-7796,
https://doi.org/10.1016/j.epsr.2022.107908.
(https://www.sciencedirect.com/science/article/pii/S0378779622001389)
Abstract: Climate change is pushing an increasing number of nations to use green
energy resources, particularly solar power as an applicable substitute to
traditional power sources. However, photovoltaic power generation is highly
weather-dependent, relying mostly on solar irradiation that is highly unstable, and
unpredictable which makes power generation challenging. Accurate photovoltaic power
predictions can substantially improve the operation of solar power systems. This is
vital for supplying prime electricity to customers and ensuring the resilience of
power plants’ operation. This research is motivated by the recent adoption and
advances in DL models and their successful use in the sector of energy. The
suggested model merges two deep learning architectures, the long short-term memory
(LSTM) and convolutional neural network (CNN). Using a real-world dataset from
Rabat, Morocco, as a case study to illustrate the effectiveness of the suggested
topology. According to error metrics, MAE, MAPE, and RMSE, the suggested
architecture CNN-LSTM performance exceeds that of standard machine learning and
single DL models in terms of prediction, precision, and stability.
Keywords: Solar energy; Short-term forecasting; Long short-term memory;
Convolutional Neural Network; CNN-LSTM; Photovoltaic Power; Forecasts; Hybrid
Models

Jicheng Li, Peng Tan, S. Joe Qin,

LSTM and Statistical Learning for Dynamic Inferential Modeling with Applications to
a 660MW Boiler,
IFAC-PapersOnLine,
Volume 55, Issue 7,
2022,
Pages 600-605,
ISSN 2405-8963,
https://doi.org/10.1016/j.ifacol.2022.07.509.
(https://www.sciencedirect.com/science/article/pii/S2405896322009156)
Abstract: Statistical learning methods have been widely studied and practiced in
the past for inferential modeling. In recent years, deep learning methods have been
implemented for inferential sensor modeling. As a popular deep learning model, the
long short-term memory (LSTM) network is capable of handling data nonlinearity and
dynamics and is therefore applied for dynamic inferential modeling. In this paper,
we analyze and compare LSTM with other statistical learning methods for the dynamic
NOx emission prediction of a 660 MW industrial boiler. Support vector regression
(SVR), partial least squares (PLS), and Least absolute shrinkage and selection
operator (Lasso) with embedded dynamics are compared with LSTM for dynamic
inferential modeling. The experimental results indicate that SVR, PLS, and Lasso
outperform LSTM. By disabling the LSTM gates to realize a simple memory structure,
the LSTM performance is signifcantly improved. The main goal of the paper is to
demonstrate that a deep neural network that is effective in other domains requires
close scrutiny and detailed study to show its superiority in process applications.
Keywords: Dynamic inferential modeling; LSTM; Statistical learning; Dynamic
modeling

Farah Shahid, Aneela Zameer, Muhammad Muneeb,

Predictions for COVID-19 with deep learning models of LSTM, GRU and Bi-LSTM,
Chaos, Solitons & Fractals,
Volume 140,
2020,
110212,
ISSN 0960-0779,
https://doi.org/10.1016/j.chaos.2020.110212.
(https://www.sciencedirect.com/science/article/pii/S0960077920306081)
Abstract: COVID-19, responsible of infecting billions of people and economy across
the globe, requires detailed study of the trend it follows to develop adequate
short-term prediction models for forecasting the number of future cases. In this
perspective, it is possible to develop strategic planning in the public health
system to avoid deaths as well as managing patients. In this paper, proposed
forecast models comprising autoregressive integrated moving average (ARIMA),
support vector regression (SVR), long shot term memory (LSTM), bidirectional long
short term memory (Bi-LSTM) are assessed for time series prediction of confirmed
cases, deaths and recoveries in ten major countries affected due to COVID-19. The
performance of models is measured by mean absolute error, root mean square error
and r2_score indices. In the majority of cases, Bi-LSTM model outperforms in terms
of endorsed indices. Models ranking from good performance to the lowest in entire
scenarios is Bi-LSTM, LSTM, GRU, SVR and ARIMA. Bi-LSTM generates lowest MAE and
RMSE values of 0.0070 and 0.0077, respectively, for deaths in China. The best
r2_score value is 0.9997 for recovered cases in China. On the basis of demonstrated
robustness and enhanced prediction accuracy, Bi-LSTM can be exploited for pandemic
prediction for better planning and management.
Keywords: Deep learning models; Bi-LSTM; GRU; Corona virus; COVID-19; epidemic
prediction

Ahmed Al Hamoud, Amber Hoenig, Kaushik Roy,

Sentence subjectivity analysis of a political and ideological debate dataset using
LSTM and BiLSTM with attention and GRU models,
Journal of King Saud University - Computer and Information Sciences,
Volume 34, Issue 10, Part A,
2022,
Pages 7974-7987,
ISSN 1319-1578,
https://doi.org/10.1016/j.jksuci.2022.07.014.
(https://www.sciencedirect.com/science/article/pii/S1319157822002415)
Abstract: Subjectivity analysis is one of the key tasks in the field of natural
language processing. Used to annotate data as subjective or objective, subjectivity
analysis can be implemented on its own or as a precursor to other NLP applications
such as sentiment analysis, emotion analysis, consumer review analysis, political
opinion analysis, document summarization, and question answering systems. The main
objective of this article is to test and compare six deep learning methods for
subjectivity classification, including Long Short-Term Memory Networks (LSTM),
Gated Recurrent Units (GRU), bidirectional GRU, bidirectional LSTM, LSTM with
attention, and bidirectional LSTM with attention. We introduced a combination
method for subjectivity annotation using lexicon-based and syntactic pattern-based
methods. We evaluated the performance of GloVe versus one-hot encoding. We also
reformatted, preprocessed, and annotated a political and ideological debate dataset
for use in subjectivity analysis. Our research compares favorably with the
performance of existing research on subjectivity analysis, achieving very high
accuracy and evaluation metrics. LSTM with attention performed the best out of all
the methods we tested with an accuracy of 97.39%.
Keywords: Subjectivity analysis; Deep learning; Long Short-Term Memory (LSTM);
Gated Recurrent Unit (GRU); LSTM with attention; Attention network

Yuchen Liao, Rong Lin, Ruiyang Zhang, Gang Wu,

Attention-based LSTM (AttLSTM) neural network for Seismic Response Modeling of
Bridges,
Computers & Structures,
Volume 275,
2023,
106915,
ISSN 0045-7949,
https://doi.org/10.1016/j.compstruc.2022.106915.
(https://www.sciencedirect.com/science/article/pii/S0045794922001754)
Abstract: Accurate prediction of bridge responses plays an essential role in health
monitoring and safety assessment of bridges subjected to dynamic loads such as
earthquakes. To this end, this paper leverages the recent advances in deep learning
and proposes an innovative attention-based recurrent neural network for
metamodeling of bridge structures under seismic hazards. The key concept is to
establish an attention-based long short-term memory neural network (AttLSTM) to
learn the dynamics from limited training data and make predictions of bridge
responses against unseen earthquakes. In particular, an attention mechanism is
proposed to enhance the selection of more informative components among sequential
data for better learning from limited data. The performance of the proposed AttLSTM
neural network was validated through both numerical and real-world data of a girder
bridge and a cable-stayed bridge to systematically evaluate the prediction
performance of the proposed method. In addition, the classical LSTM neural network
was selected as the baseline model to show the favorable performance of the
proposed attention mechanism. Results indicate that the proposed method with
attention mechanism outperforms the compared state-of-the-art LSTM in terms of both
accuracy and reliability.
Keywords: Bridge engineering; Deep learning; Long short term memory neural network;
Attention mechanism; Seismic response modelling

Soufiane Belagoune, Noureddine Bali, Azzeddine Bakdi, Bousaadia Baadji, Karim Atif,
Deep learning through LSTM classification and regression for transmission line
fault detection, diagnosis and location in large-scale multi-machine power systems,
Measurement,
Volume 177,
2021,
109330,
ISSN 0263-2241,
https://doi.org/10.1016/j.measurement.2021.109330.
(https://www.sciencedirect.com/science/article/pii/S0263224121003286)
Abstract: Fault detection, diagnosis, identification and location are crucial to
improve the sensitivity and reliability of system protection. This maintains power
systems continuous proper operation; however, it is challenging in large-scale
multi-machine power systems. This paper introduces three novel Deep Learning (DL)
classification and regression models based on Deep Recurrent Neural Networks (DRNN)
for Fault Region Identification (FRI), Fault Type Classification (FTC), and Fault
Location Prediction (FLP). These novel models explore full transient data from pre-
and post-fault cycles to make reliable decisions; whereas current and voltage
signals are measured through Phasor Measurement Units (PMUs) at different terminals
and used as input features to the DRNN models. Sequential Deep Learning (SDL) is
employed herein through Long Short-Term Memory (LSTM) to model spatiotemporal
sequences of high-dimensional multivariate features to achieve accurate
classification and prediction results. The proposed algorithms were tested in a
Two-Area Four-Machine Power System. Training and testing data are collected during
transmission lines faults of different types introduced at various locations in
different regions. The presented algorithms achieved superior detection,
classification and location performance with high accuracy and robustness compared
to contemporary techniques.
Keywords: Multi-machine power system; Power transmission lines; Short-circuit
fault; Long short-term memory; Fault detection and isolation; Sequential deep
learning

Do-Eun Choe, Hyoung-Chul Kim, Moo-Hyun Kim,

Sequence-based modeling of deep learning with LSTM and GRU networks for structural
damage detection of floating offshore wind turbine blades,
Renewable Energy,
Volume 174,
2021,
Pages 218-235,
ISSN 0960-1481,
https://doi.org/10.1016/j.renene.2021.04.025.
(https://www.sciencedirect.com/science/article/pii/S0960148121005371)
Abstract: This paper proposes and tests a sequence-based modeling of deep learning
(DL) for structural damage detection of floating offshore wind turbine (FOWT)
blades using Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) neural
networks. The complete framework was developed with four different designs of deep
networks using unidirectional or bidirectional layers of LSTM and GRU networks.
These neural networks, specifically developed to learn long-term and short-term
dependencies within sequential information such as time-series data, are
successfully trained with the sensor signals of damaged FOWT. The sensor data were
simulated due to the limited availability of field data from damaged FOWTs using
multiple computational methods previously validated with experimental tests. The
simulations accounted for the damage scenarios with various intensities, locations,
and damage shapes, totaling 1320 damage scenarios. Both the presence of damage and
its location were detected up to an accuracy of 94.8% using the best performing
model of the selected network when tested for independent signals. The K-fold
cross-validation accuracy of the selected network is estimated to be 91.7%. The
presence of damage itself was detected with an accuracy of 99.9% based on the
cross-validation regardless of the damage location. Structural damage detection
using deep learning is not restricted by the assumptions of the systems or the
environmental conditions as the networks learn the system directly from the data.
The framework can be applied to various types of civil and offshore structures.
Furthermore, the sequence-based modeling enables engineers to harness the vast
amounts of digital information to improve the safety of structures.
Keywords: Offshore wind energy; Machine learning; Deep learning; Long-short-term
memory; Gated recurrent unit

Mainak Sarkar, Arnaud De Bruyn,

LSTM Response Models for Direct Marketing Analytics: Replacing Feature Engineering
with Deep Learning,
Journal of Interactive Marketing,
Volume 53,
2021,
Pages 80-95,
ISSN 1094-9968,
https://doi.org/10.1016/j.intmar.2020.07.002.
(https://www.sciencedirect.com/science/article/pii/S1094996820301080)
Abstract: In predictive modeling, firms often deal with high-dimensional data that
span multiple channels, websites, demographics, purchase types, and product
categories. Traditional customer response models rely heavily on feature
engineering, and their performance depends on the analyst's domain knowledge and
expertise to craft relevant predictors. As the complexity of data increases,
however, traditional models grow exponentially complicated. In this paper, we
demonstrate that long-short term memory (LSTM) neural networks, which rely
exclusively on raw data as input, can predict customer behaviors with great
accuracy. In our first application, a model outperforms standard benchmarks. In a
second, more realistic application, an LSTM model competes against 271 hand-crafted
models that use a wide variety of features and modeling approaches. It beats 269 of
them, most by a wide margin. LSTM neural networks are excellent candidates for
modeling customer behavior using panel data in complex environments (e.g., direct
marketing, brand choices, clickstream data, churn prediction).
Keywords: Long-short term memory neural network (LSTM); Recurrent neural network
(RNN); Feature engineering; Response model; Panel data; Direct marketing

Jie Liu, Lei Xu, Nengcheng Chen,

A spatiotemporal deep learning model ST-LSTM-SA for hourly rainfall forecasting
using radar echo images,
Journal of Hydrology,
Volume 609,
2022,
127748,
ISSN 0022-1694,
https://doi.org/10.1016/j.jhydrol.2022.127748.
(https://www.sciencedirect.com/science/article/pii/S0022169422003237)
Abstract: Accurate and timely short-term forecasting services of precipitation
variable are significant for people's lives and property security. The data-driven
approaches demonstrate promising performance in the extrapolation of precipitation.
In this paper, we proposed a spatiotemporal prediction model, namely the Spatial
Temporal Long Short-Term Memory based on the self-attentive mechanism (ST-LSTM-SA).
This model enables better aggregation of sequence features inspired by proposed
improvements. The encapsulated 3D convolution is developed to fully exploit the
short-term spatiotemporal information, and the channel correlation is modeled by
self-attention mechanism to further improve representations in the long-term
interaction, the effectiveness of which is validated in the ablation study.
Comprehensive experiments have been conducted on the radar echo sequence, we
successfully predicted future radar reflectivity images for next 3 h with data for
previous three hours as inputs in Wuhan, China. Three machine learning methods:
multiple linear regression (MLR), support vector regression (SVR) and artificial
neural networks (ANN) and deep learning model—ConvLSTM have been chosen as
comparative groups to corroborate the nowcasting availability of this model for
radar echo extrapolation. Moreover, the supplementary experiment on minute dataset
also illustrated the superiority of ST-LSTM-SA. The studies analyzed the
forecasting performance in terms of image quality and rainfall error. The
experimental results demonstrated the better versatility and performance of ST-
LSTM-SA. These conclusions and attempts may provide efficient guidance for
precipitation nowcasting in urban areas.
Keywords: Precipitation nowcasting; Spatiotemporal LSTM; Self-Attention; Radar echo
image

Aditi Sakalle, Pradeep Tomar, Harshit Bhardwaj, Divya Acharya, Arpit Bhardwaj,
A LSTM based deep learning network for recognizing emotions using wireless
brainwave driven system,
Expert Systems with Applications,
Volume 173,
2021,
114516,
ISSN 0957-4174,
https://doi.org/10.1016/j.eswa.2020.114516.
(https://www.sciencedirect.com/science/article/pii/S095741742031160X)
Abstract: Positive and Negative emotions are experienced by the majority of
individuals in their day-to-day life. It is important to control access of negative
emotions because it may lead to several chronic health issues like depression and
anxiety. The purpose of this research work is to develop a portable brainwave
driven system for recognizing positive, negative, and neutral emotions. This
research considers the classification of four negative class of emotions using
genres sadness, disgust, angry, and surprise along with the classification of three
basic class of emotions i.e., positive, negative, and neutral. This paper
introduces a long short term memory deep learning (LSTM) network to recognize
emotions using EEG signals. The primary goal of this approach is to assess the
classification performance of the LSTM model. The secondary goal is to assess the
human behavior of different age groups and gender. We have compared the performance
of Multilayer Perceptron (MLP), K-nearest neighbors (KNN), Support Vector Machine
(SVM), LIB-Support Vector Machine (LIB-SVM), and LSTM based deep learning model for
classification. The analysis shows that, for four class of emotions LSTM based deep
learning model provides classification accuracy as 83.12%, 86.94%, 91.67%, and
94.12% for 50–50, 60–40, 70–30, and 10-fold cross-validations. For three class of
emotions LSTM based deep learning model provides classification accuracy as 81.33%,
85.41%, 89.44%, and 92.66% for 50–50, 60–40, 70–30, and 10-fold cross-validation.
The generalizability and reliability of this approach are evaluated by applying our
approach to publicly available EEG datasets DEAP and SEED. In compliance with the
self-reported feelings, brain signals of 18–25 years of age group provided the
highest emotional identification. The results show that among genders, females are
more emotionally active as compared to males. These results affirmed the potential
use of our method for recognizing positive, negative, and neutral emotions.
Keywords: Emotion recognition; Deep learning; EEG; Empirical mode decomposition;
LSTM

Lihua Lin, Min Li, Li Ma, Aliasghar Baziar, Ziad M. Ali,

Hybrid RNN-LSTM deep learning model applied to a fuzzy based wind turbine data
uncertainty quantization method,
Ad Hoc Networks,
Volume 123,
2021,
102658,
ISSN 1570-8705,
https://doi.org/10.1016/j.adhoc.2021.102658.
(https://www.sciencedirect.com/science/article/pii/S1570870521001736)
Abstract: In addition to the plenty of advantages that the penetration of wind
turbine (WT) brings to the smart networks, uncertainty problems can be considered
as an unavoidable phenomenon that requires to be addressed. The results of high
uncertainty are able to lead to the instability of management plans and schedules
that is able to lead to the serious issues for operators and users. The following
case study tries to propose a novel reinforcement learning based hybrid-based
quantification technique to capture the prediction fault into the output power of
WT. The offered approach has been applied the hybrid recurrent neural network (RNN)
and long-short term memory (LSTM) layout with the aim of learning the utmost
efficient Spatio-temporal properties of WT's output power. Because of the wide
complication of the information, a novel optimization approach according to the
modified sine cos algorithm has been suggested to aid in further steady layout
training. The possibility and efficacy of the model are evaluated via the test
analysis of two datasets in wind lands of Australian.
Keywords: Optimization; Reinforcement Deep Learning; Hybrid LSTM and RNN;
Uncertainty; Wind Turbine

Ailing Tan, Yunxin Wang, Yong Zhao, Bolin Wang, Xiaohang Li, Alan X. Wang,
Near infrared spectroscopy quantification based on Bi-LSTM and transfer learning
for new scenarios,
Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy,
Volume 283,
2022,
121759,
ISSN 1386-1425,
https://doi.org/10.1016/j.saa.2022.121759.
(https://www.sciencedirect.com/science/article/pii/S1386142522009088)
Abstract: This study proposed a deep transfer learning methodology based on an
improved Bi-directional Long Short-Term Memory (Bi-LSTM) network for the first time
to address the near infrared spectroscopy (NIR) model transfer issue between
samples. We tested its effectiveness on two datasets of manure and polyglutamic
acid (γ-PGA) solution, respectively. First, the optimal primary Bi-LSTM networks
for cattle manure and the first batch of γ-PGA were developed by ablation
experiments and both proved to outperform one-dimensional convolutional neural
network (1D-CNN), Partial Least Square (PLS) and Extreme Learning Machine (ELM)
models. Then, two types of transfer learning approaches were carried out to
determine model transferability to non-homologous samples. For poultry manure and
the second batch of γ-PGA, the obtained predicting results verified that the second
approach of fine-tuning Bi-LSTM layers and re-training FC layers transcended the
first approach of fixing Bi-LSTM layers and only re-training FC layers by reducing
the RMSEPtarget of 23.4275% and 50.7343%, respectively. Finally, comparisons with
fine-tuning 1D-CNN and other traditional model transfer methods further identified
the superiority of the proposed methodology with exceeding accuracy and smaller
variation, which decreased RMSEPtarget of poultry manure and the second batch of γ-
PGA of 7.2832% and 48.1256%, 67.1117% and 80.6924% when compared to that acquired
by fine-tuning 1D-CNN, Tradaboost-ELM and CCA-PLS which were the best of five
traditional methods, respectively. The study demonstrates the potential of the
Fine-tuning-Bi-LSTM enabled NIR technology to be used as a simple, cost effective
and reliable detection tool for a wide range of applications under various new
scenarios.
Keywords: Near infrared spectroscopy; Bi-directional Long Short-Term Memory;
Transfer learning; Fine-tuning; Manure; γ-PGA

Xinyu Dong, Rachel Wong, Weimin Lyu, Kayley Abell-Hart, Jianyuan Deng, Yinan Liu,
Janos G. Hajagos, Richard N. Rosenthal, Chao Chen, Fusheng Wang,
An integrated LSTM-HeteroRGNN model for interpretable opioid overdose risk
prediction,
Artificial Intelligence in Medicine,
2022,
102439,
ISSN 0933-3657,
https://doi.org/10.1016/j.artmed.2022.102439.
(https://www.sciencedirect.com/science/article/pii/S0933365722001919)
Abstract: Opioid overdose (OD) has become a leading cause of accidental death in
the United States, and overdose deaths reached a record high during the COVID-19
pandemic. Combating the opioid crisis requires targeting high-need populations by
identifying individuals at risk of OD. While deep learning emerges as a powerful
method for building predictive models using large scale electronic health records
(EHR), it is challenged by the complex intrinsic relationships among EHR data.
Further, its utility is limited by the lack of clinically meaningful
explainability, which is necessary for making informed clinical or policy decisions
using such models. In this paper, we present LIGHTED, an integrated deep learning
model combining long short term memory (LSTM) and graph neural networks (GNN) to
predict patients' OD risk. The LIGHTED model can incorporate the temporal effects
of disease progression and the knowledge learned from interactions among clinical
features. We evaluated the model using Cerner's HealthFacts database with over 5
million patients. Our experiments demonstrated that the model outperforms
traditional machine learning methods and other deep learning models. We also
proposed a novel interpretability method by exploiting embeddings provided by GNNs
and clustering patients and EHR features respectively, then conducted qualitative
feature cluster analysis for clinical interpretations. Our study shows that LIGHTED
can take advantage of longitudinal EHR data and the intrinsic graph structure of
EHRs among patients to provide effective and interpretable OD risk predictions that
may potentially improve clinical decision support.
Keywords: Opioid overdose; Opioid poisoning; Deep learning; Clinical decision
support; Electronic health records; Long short-term memory; Graph neural network

Ricardo Navares, José L. Aznarte,

Predicting air quality with deep learning LSTM: Towards comprehensive models,
Ecological Informatics,
Volume 55,
2020,
101019,
ISSN 1574-9541,
https://doi.org/10.1016/j.ecoinf.2019.101019.
(https://www.sciencedirect.com/science/article/pii/S1574954119303309)
Abstract: In this paper we approach the problem of predicting air quality in the
region of Madrid using long short term memory recurrent artificial neural networks.
Air quality, in this study, is represented by the concentrations of a series of air
pollutants which are proved as risky for human health such as CO, NO2, O3, PM10,
SO2 and airborne pollen concentrations of two genus (Plantago and Poaceae). These
concentrations are sampled in a set of locations in the city of Madrid. Instead of
training an array of models, one per location and pollutant, several comprehensive
deep network configurations are compared to identify those which are able to better
extract relevant information out of the set of time series in order to predict one
day-ahead air quality. The results, supported by statistical evidence, indicate
that a single comprehensive model might be a better option than multiple individual
models. Such comprehensive models represent a successful tool which can provide
useful forecasts that can be thus applied, for example, in managerial environments
by clinical institutions to optimize resources in expectation of an increment of
the number of patients due to the exposure to low air quality levels.
Keywords: Air quality; Forecast; Neural networks; Deep learning

Archana Nigam, Sanjay Srivastava,

Hybrid deep learning models for traffic stream variables prediction during
rainfall,
Multimodal Transportation,
Volume 2, Issue 1,
2023,
100052,
ISSN 2772-5863,
https://doi.org/10.1016/j.multra.2022.100052.
(https://www.sciencedirect.com/science/article/pii/S2772586322000521)
Abstract: Adverse weather conditions like fog, rainfall, and snowfall affect the
driver’s visibility, mobility of vehicle, and road capacity. Accurate prediction of
the macroscopic traffic stream variables such as speed and flow is essential for
traffic operation and management in an Intelligent Transportation System (ITS). The
accurate prediction of these variables is challenging because of the traffic
stream’s non-linear and complex characteristics. Deep learning models are proven to
be more accurate for predicting traffic stream variables than shallow learning
models because it extracts hidden abstract representation using layerwise
architecture. The impact of weather conditions on traffic is dependent on various
hidden features. The rainfall effect on traffic is not directly proportional to the
distance between the weather station and the road because of terrain feature
constraints. The prolonged rainfall weakens the drainage system, affects soil
absorption capability, which causes waterlogging. Therefore, to capture the spatial
and prolonged impact of weather conditions, we proposed a soft spatial and temporal
threshold mechanism. To fill out the missing weather data spatial interpolation
techniques are used. The traffic condition on a target road depends on the
surrounding area’s traffic and weather conditions and relies on its own traffic
characteristics. We designed the hybrid deep learning models, CNN-LSTM and LSTM-
LSTM. The former model in the hybrid model extracts the spatiotemporal features and
the latter model uses these features as memory. The latter model predicts the
traffic stream variables depending upon the passed features and temporal input. We
perform multiple experiments to validate the deep learning model’s performance. The
experiments show that a deep learning model trained with traffic and rainfall data
gives better prediction accuracy than the model trained without rainfall data. The
performance of the LSTM-LSTM model is better than other models in extracting long-
term dependency between the traffic and weather data.
Keywords: Traffic speed prediction; Traffic flow; Rainfall; CNN; LSTM; SUMO;
Waterlogging

Hui Liu, Xiwei Mi, Yanfei Li,

Smart multi-step deep learning model for wind speed forecasting based on
variational mode decomposition, singular spectrum analysis, LSTM network and ELM,
Energy Conversion and Management,
Volume 159,
2018,
Pages 54-64,
ISSN 0196-8904,
https://doi.org/10.1016/j.enconman.2018.01.010.
(https://www.sciencedirect.com/science/article/pii/S0196890418300104)
Abstract: Accurate and robust wind speed forecasting is essential for the planning,
scheduling and maintenance of wind power. In this study, a novel wind speed
multistep prediction model is proposed by combing the VMD (Variational Mode
Decomposition), SSA (Singular Spectrum Analysis), LSTM (Long Short Term Memory)
network and ELM (Extreme Learning Machine), in which, the VMD is employed to
decompose the original wind speed data into a series of sub-layers; the SSA is
adopted to further extract the trend information of all the sub-layers; the LSTM
network is used to complete the forecasting for the low-frequency sub-layers
obtained by the VMD-SSA; and the ELM is used to complete the forecasting for the
high-frequency sub-layers obtained by the VMD-SSA. To investigate the multistep
prediction performance of the proposed models, eight models are included in the
comparisons. The four experiments results show that: (a) among all the involved
models, the proposed model has the best multistep prediction performance; (b)
compared to the other involved models, the proposed model is more effective and
robust in extracting the trend information.
Keywords: Wind speed forecasting; Variational mode decomposition; Singular spectrum
analysis; Deep learning; Long short term memory network; Extreme learning machine

Fatemeh Rezaie Adaryani, S. Jamshid Mousavi, Fatemeh Jafari,

Short-term rainfall forecasting using machine learning-based approaches of PSO-SVR,
LSTM and CNN,
Journal of Hydrology,
Volume 614, Part A,
2022,
128463,
ISSN 0022-1694,
https://doi.org/10.1016/j.jhydrol.2022.128463.
(https://www.sciencedirect.com/science/article/pii/S0022169422010332)
Abstract: Short-term rainfall forecasting plays an important role in hydrologic
modeling and water resource management problems such as flood warning and real time
control of urban drainage systems. This paper compares the performances of three
machine and deep learning-based rainfall forecasting approaches including a hybrid
optimized-by-PSO support vector regression (PSO-SVR), long-short term memory
(LSTM), and convolutional neural network (CNN). The approaches are used to develop
both 5-minute and 15–minute ahead forecast models of rainfall depth based on
datasets of Niavaran station, Tehran, Iran. Results of applying the models to all
data points indicated that PSO-SVR and LSTM approaches performed almost the same
and better than CNN. Subsequently, rainfall events were divided into four classes
depending on their severity and duration using K-nearest neighbor method, and a
separate forecast model was built for each of the classes. Classification of the
events improved the forecast models accuracy where PSO-SVR and LSTM were the best
approaches for the 15-minute and 5-minute ahead rainfall forecast models,
respectively. Investigating the impact of more predictors on the forecast quality,
adding differences of rainfall depths to model predictors improved the accuracy of
PSO-SVR approach for the 5-minute ahead forecast model up to 13%. Furthermore,
depending on the rainfall event, additional input variables considering rainfall
depth fluctuations over shorter time periods than the forecast lead time increased
the performances of the PSO-SVR and LSTM approaches between 3–15% and 2–10%,
respectively.
Keywords: Rainfall forecasting; Machine learning; Deep learning; Particle swarm
optimization

Han Ma, Shunlin Liang,

Development of the GLASS 250-m leaf area index product (version 6) from MODIS data
using the bidirectional LSTM deep learning model,
Remote Sensing of Environment,
Volume 273,
2022,
112985,
ISSN 0034-4257,
https://doi.org/10.1016/j.rse.2022.112985.
(https://www.sciencedirect.com/science/article/pii/S0034425722000992)
Abstract: Leaf area index (LAI) is a terrestrial essential climate variable that is
required in a variety of ecosystem and climate models. The Global LAnd Surface
Satellite (GLASS) LAI product has been widely used, but its current version (V5)
from Moderate Resolution Imaging Spectroradiometer (MODIS) data has several
limitations, such as frequent temporal fluctuation, large data gaps, high
dependence on the quality of surface reflectance, and low computational efficiency.
To address these issues, this paper presents a deep learning model to generate a
new version of the LAI product (V6) at 250-m resolution from MODIS data from 2000
onward. Unlike most existing algorithms that estimate one LAI value at one time for
each pixel, this model estimates LAI for 2 years simultaneously. Three widely used
LAI products (MODIS C6, GLASS V5, and PROBA-V V1) are used to generate global
representative time-series LAI training samples using K-means clustering analysis
and least difference criteria. We explore four machine learning models, the general
regression neural network (GRNN), long short-term memory (LSTM), gated recurrent
unit (GRU), and Bidirectional LSTM (Bi-LSTM), and identify Bi-LSTM as the best
model for product generation. This new product is directly validated using 79 high-
resolution LAI reference maps from three in situ observation networks. The results
show that GLASS V6 LAI achieves higher accuracy, with a root mean square (RMSE) of
0.92 at 250 m and 0.86 at 500 m, while the RMSE is 0.98 for PROBA-V at 300 m, 1.08
for GLASS V5, and 0.95 for MODIS C6 both at 500 m. Spatial and temporal consistency
analyses also demonstrate that the GLASS V6 LAI product is more spatiotemporally
continuous and has higher quality in terms of presenting more realistic temporal
LAI dynamics when the surface reflectance is absent for a long period owing to
persistent cloud/aerosol contaminations. The results indicate that the new Bi-LSTM
deep learning model runs significantly faster than the GLASS V5 algorithm, avoids
the reconstruction of surface reflectance data, and is resistant to the noises
(cloud and snow contamination) or missing values contained in surface reflectance
than other methods, as the Bi-LSTM can effectively extract information across the
entire time series of surface reflectance rather than a single time point. To our
knowledge, this is the first global time-series LAI product at the 250-m spatial
resolution that is freely available to the public (www.geodata.cn and
www.glass.umd.edu).
Keywords: Leaf area index; GLASS; MODIS; PROBA-V; Deep learning; LSTM

Weifang Mao, Huiming Zhu, Hao Wu, Yijie Lu, Haidong Wang,
Forecasting and trading credit default swap indices using a deep learning model
integrating Merton and LSTMs,
Expert Systems with Applications,
Volume 213, Part B,
2023,
119012,
ISSN 0957-4174,
https://doi.org/10.1016/j.eswa.2022.119012.
(https://www.sciencedirect.com/science/article/pii/S0957417422020309)
Abstract: Using macroeconomic and financial conditions to forecast credit default
swap (CDS) spreads is a challenging task. In this paper, we propose the Merton-LSTM
model, a modified LSTM model formed by integrating with the Merton determinants
model, to forecast the CDS indices. We provide the rigorous math behind the Merton-
LSTM model, which demonstrates that by leveraging the nonlinear learning ability of
LSTM with increased model capacity, the Merton-LSTM model is expected to learn the
inherent association between the Merton determinants and CDS spreads. Further, the
Merton-LSTM model is compared with the machine learning models LSTM, gated
recurrent unit (GRU), multilayer perceptron network (MLP), support vector machine
(SVM) and a typical stochastic series model in forecasting the two most liquid
five-year CDS indices, North America High Yield index (CDX.NA.HY) and North America
Investment Grade index (CDX.NA.IG) through the root mean squared error (RMSE) and
the Diebold-Mariano test. The comparison results show that the RMSEs of the Merton-
LSTM model are the lowest (6.2570–27.2000 for CDX.NA.HY and 1.3168–6.4772 for
CDX.NA.IG) compared to other competitive models. The superiority of the Merton-LSTM
model in forecasting performance is highlighted in long-term prediction even with a
forecasting horizon extended to 28 days. Simulated trading with different
thresholds and horizons is conducted in this study. We find that the Merton-LSTM
trading strategy yields the highest annualized Sharpe ratios and lowest maximum
losses at most thresholds and horizons, highlighting the economic significance of
the proposed model.
Keywords: Forecasting; Trading; Credit default swap; LSTM; Deep learning

Md. Kowsher, Anik Tahabilder, Md. Zahidul Islam Sanjid, Nusrat Jahan Prottasha, Md.
Shihab Uddin, Md Arman Hossain, Md. Abdul Kader Jilani,
LSTM-ANN & BiLSTM-ANN: Hybrid deep learning models for enhanced classification
accuracy,
Procedia Computer Science,
Volume 193,
2021,
Pages 131-140,
ISSN 1877-0509,
https://doi.org/10.1016/j.procs.2021.10.013.
(https://www.sciencedirect.com/science/article/pii/S1877050921020548)
Abstract: Machine learning is getting more and more advanced with the progression
of state-of-the-art technologies. Since existing algorithms do not provide a
palatable learning performance most often, it is necessary to carry on the trail of
upgrading the current algorithms incessantly. The hybridization of two or more
algorithms can potentially increase the performance of the blueprinted model.
Although LSTM and BiLSTM are two excellent far and widely used algorithms in
natural language processing, there still could be room for improvement in terms of
accuracy via the hybridization method. Thus, the advantages of both RNN and ANN
algorithms can be obtained simultaneously. This paper has illustrated the deep
integration of BiLSTM-ANN (Fully Connected Neural Network) and LSTM-ANN and
manifested how these integration methods are performing better than single BiLSTM,
LSTM and ANN models. Undertaking Bangla content classification is challenging
because of its equivocalness, intricacy, diversity, and shortage of relevant data,
therefore, we have executed the whole integrated models on the Bangla content
classification dataset from newspaper articles. The proposed hybrid BiLSTM-ANN
model beats all the implemented models with the most noteworthy accuracy score of
93% for both validation & testing. Moreover, we have analyzed and compared the
performance of the models based on the most relevant parameters.
Keywords: BiLSTM-ANN; LSTM-ANN; Supervised machine learning; Hybrid ML model;
Fusion of ML model; NLP

Haibo Huang, Xiaorong Huang, Weiping Ding, Siwen Zhang, Jian Pang,
Optimization of electric vehicle sound package based on LSTM with an adaptive
learning rate forest and multiple-level multiple-object method,
Mechanical Systems and Signal Processing,
Volume 187,
2023,
109932,
ISSN 0888-3270,
https://doi.org/10.1016/j.ymssp.2022.109932.
(https://www.sciencedirect.com/science/article/pii/S0888327022010007)
Abstract: The sound absorption and sound insulation performance of an acoustic
package (AP) system directly affect the noise, vibration and harshness performance
of a vehicle. Numerous studies have studied the optimization of vehicle sound
package, however, there are two deficiencies in the current research of sound
package: (1) The noise transmission path of acoustic package is complex and
hierarchical. Most of the related works focus on the data-driven part while
ignoring the knowledge attributes behind the acoustic package design problem, which
limits the further improvement of prediction and optimization of acoustic package
performance. (2) In using intelligent neural networks-based methods such as long
short-term memory (LSTM), reducing the learning rate during training gradually
narrows the search interval of a solution, and adjusting the learning rate in a
small range may tend to trap local optima. In this study, a knowledge- and data-
driven approach is proposed for the development of acoustic package systems. A
multiple-level multiple-object method is proposed as the knowledge model, and a
multilayer structure of the acoustic package system that contains the system,
subsystem and component layers is developed. In addition, an improved long short-
term memory model based on an adaptive learning rate forest, which can increase and
decrease the learning rate adaptively, is proposed as the data-driven model. The
knowledge- and data-driven method is applied to optimize the sound absorption and
insulation of the acoustic package system. In the experimental validation, the
effectiveness and robustness of the proposed method outperformed the traditional
direct mapping method and the conventional long short-term memory method.
Keywords: Noise, vibration and harshness; Acoustic package; Knowledge- and data-
driven; Long short-term memory; Multiple-level multiple-object

Shuyuan Zhang, Tong Qiu,

Semi-supervised LSTM ladder autoencoder for chemical process fault diagnosis and
localization,
Chemical Engineering Science,
Volume 251,
2022,
117467,
ISSN 0009-2509,
https://doi.org/10.1016/j.ces.2022.117467.
(https://www.sciencedirect.com/science/article/pii/S0009250922000513)
Abstract: Deep learning is attracting widespread attention in the field of chemical
process fault diagnosis recently. However, most deep learning methods are based on
supervised learning and heavily rely on labeled data, with massive unlabeled data
underutilized. Moreover, these supervised deep learning methods are uninterpretable
and cannot facilitate fault localization, which is necessary for supervising the
process back to normal. In this study, long short-term memory (LSTM) is used to
extract temporal features and ladder autoencoder (LAE) is adopted for semi-
supervised learning. Combining LSTM and LAE, LSTM-LAE is innovatively proposed to
effectively utilize unlabeled data, with fault diagnosis performance largely
improved. Moreover, LSTM-LAE achieves the interpretability to extract fault-
relevant process variables with its elaborately designed internal features. When
applied on a continuous stirred tank heater and the benchmark Tennessee Eastman
process, LSTM-LAE exhibited a state-of-the-art fault diagnosis performance and
localized faults to their relevant variables correctly.
Keywords: Fault diagnosis and localization; Semi-supervised learning; Ladder
autoencoder; Long short-term memory; Tennessee Eastman process

Asiye Kaymaz Ozcanli, Mustafa Baysal,

Islanding detection in microgrid using deep learning based on 1D CNN and CNN-LSTM
networks,
Sustainable Energy, Grids and Networks,
Volume 32,
2022,
100839,
ISSN 2352-4677,
https://doi.org/10.1016/j.segan.2022.100839.
(https://www.sciencedirect.com/science/article/pii/S2352467722001230)
Abstract: Islanding detection is a critical task due to safety hazards and
technical issues for the operation of microgrids. Deep learning (DL) has been
applied for islanding detection and achieved good results due to the ability of
automatic feature learning in recent years. Long short term memory (LSTM) and two
dimensional (2D) convolutional neural networks (CNN) based DL techniques are
implemented and demonstrated well performance on islanding detection. However, one
dimensional (1D) CNN is more suitable for real-time implementations since it has
relatively low complexity and cost-effective than 2D CNN. In this paper, for the
first time, the 1D CNN and the combination of 1D CNN-LSTM are proposed for
islanding detection to better exploit the global information of islanding data
samples using the strengths of both networks. The proposed methods utilize only
voltage and current harmonic measurements as input at the point of common coupling
(PCC). About 4000 cases under the modified CERTS microgrid model are simulated to
evaluate the performance of the proposed architectures. The simulation results and
the presented analysis show that the proposed networks have achieved the maximum
accuracy of 100% on the task of islanding detection; especially the proposed CNN-
LSTM model outperforms the other approaches. Furthermore, the robustness of the
proposed methods is demonstrated with unseen samples under low none detection zone
and the expansion of microgrid topology.
Keywords: CNN; Deep learning; Islanding detection; LSTM; Microgrid; THD

Yiyin Tang, Yalin Wang, Chenliang Liu, Xiaofeng Yuan, Kai Wang, Chunhua Yang,
Semi-supervised LSTM with historical feature fusion attention for temporal sequence
dynamic modeling in industrial processes,
Engineering Applications of Artificial Intelligence,
Volume 117, Part A,
2023,
105547,
ISSN 0952-1976,
https://doi.org/10.1016/j.engappai.2022.105547.
(https://www.sciencedirect.com/science/article/pii/S0952197622005371)
Abstract: In modern industrial processes, the data-driven soft sensor technology
has been widely used for the prediction of key quality variables. Due to the
important of dynamics and nonlinearity in industrial process data, deep learning
models like long short-term memory (LSTM) network are well suited for temporal
sequence dynamic modeling due to their excellent long-term memory function and
feature extraction capability. Furthermore, industrial processes generate a large
amount of process data with irregular sampling frequencies. However, traditional
LSTM cannot fully utilize the process data with irregular sampling frequency and
the guidance value of historical data samples for feature learning. To address
these issues, a novel semi-supervised LSTM with history feature fusion attention
(HFFA-SSLSTM) model is proposed in this paper. First, the semi-supervised learning
strategy is implemented in LSTM to fully utilize the unlabeled data and mine the
temporal sequence features of labeled samples and unlabeled samples with irregular
sampling frequencies. Then, a novel historical feature fusion attention (HFFA)
mechanism is developed, which utilizes historical hidden features to learn
attention scores for obtaining weighted historical information-related features.
Finally, the extracted features are combined to form the soft sensor model to
perform time series prediction tasks for key quality variables in industrial
processes. The experimental results on the actual industrial hydrocracking data set
demonstrate the effectiveness of the proposed HFFA-SSLSTM model and its possibility
of applicating in real industrial processes.
Keywords: Deep learning; Semi-supervised long short-term memory (SSLSTM);
Historical feature fusion attention (HFFA); Temporal sequence dynamic modeling;
Industrial processes

Akhter Mohiuddin Rather,

LSTM-based Deep Learning Model for Stock Prediction and Predictive Optimization
Model,
EURO Journal on Decision Processes,
Volume 9,
2021,
100001,
ISSN 2193-9438,
https://doi.org/10.1016/j.ejdp.2021.100001.
(https://www.sciencedirect.com/science/article/pii/S2193943821001175)
Abstract: A new method of predicting time-series-based stock prices and a new model
of an investment portfolio based on predictions obtained is proposed here. For this
purpose, a new regression scheme is implemented on a long-short-term-memory-based
deep neural network. The predictions once obtained are used to construct an
investment portfolio or more specifically a predicted portfolio. A large set of
experiments have been carried on stock data of NIFTY-50 obtained from the National
stock exchange of India. The results confirm that the proposed model outperforms
various standard predictive models as well as various standard portfolio
optimization models.
Keywords: Artificial Neural Networks; Deep Learning; LSTM; Time Series; Portfolio
Model
Ke Wang, Changxi Ma, Yihuan Qiao, Xijin Lu, Weining Hao, Sheng Dong,
A hybrid deep learning model with 1DCNN-LSTM-Attention networks for short-term
traffic flow prediction,
Physica A: Statistical Mechanics and its Applications,
Volume 583,
2021,
126293,
ISSN 0378-4371,
https://doi.org/10.1016/j.physa.2021.126293.
(https://www.sciencedirect.com/science/article/pii/S0378437121005665)
Abstract: With the rapid development of social economy, the traffic volume of urban
roads has raised significantly, which has led to increasingly serious urban traffic
congestion problems, and has caused much inconvenience to people’s travel. By
focusing on the complexity and long-term dependence of traffic flow sequences on
urban road, this paper considered the traffic flow data and weather conditions of
the road section comprehensively, and proposed a short-term traffic flow prediction
model based on the attention mechanism and the 1DCNN-LSTM network. The model
combined the time expansion of the CNN and the advantages of the long-term memory
of the LSTM. First, the model employs 1DCNN network to extract the spatial features
in the road traffic flow data. Second, the output spatial features are considered
as the input of LSTM neural network to extract the time features in road traffic
flow data, and the long-term dependence characteristics of LSTM neural network are
adopted to improve the prediction accuracy of traffic flow. Next, the spatio-
temporal characteristics of road traffic flow were regarded as the input of the
regression prediction layer, and the prediction results corresponding to the
current input were calculated. Finally, the attention mechanism was introduced on
the LSTM side to give enough attention to the key information, so that the model
can focus on learning more important data features, and further improve the
prediction performance. The experimental results showed that the prediction effect
of the 1DCNN-LSTM-Attention model under the weather factor was better than that
without considering the weather factor. At the same time, compared with traditional
neural network models, the prediction effect of the proposed model revealed faster
convergence speed and higher prediction accuracy. It can be found that for short-
term traffic flow prediction on urban roads, the 1DCNN-LSTM network structure
considering the attention mechanism provides superior features.
Keywords: Traffic flow prediction; Deep learning; One-dimensional convolutional
neural network; Long short-term memory network; Attentional mechanism

Heshan Wang, Yiping Zhang, Jing Liang, Lili Liu,

DAFA-BiLSTM: Deep Autoregression Feature Augmented Bidirectional LSTM network for
time series prediction,
Neural Networks,
Volume 157,
2023,
Pages 240-256,
ISSN 0893-6080,
https://doi.org/10.1016/j.neunet.2022.10.009.
(https://www.sciencedirect.com/science/article/pii/S0893608022003938)
Abstract: Time series forecasting models that use the past information of exogenous
or endogenous sequences to forecast future series play an important role in the
real world because most real-world time series datasets are rich in time-dependent
information. Most conventional prediction models for time series datasets are time-
consuming and fraught with complex limitations because they usually fail to
adequately exploit the latent spatial dependence between pairs of variables. As a
successful variant of recurrent neural networks, the long short-term memory network
(LSTM) has been demonstrated to have stronger nonlinear dynamics to store
sequential data than traditional machine learning models. Nevertheless, the common
shallow LSTM architecture has limited capacity to fully extract the transient
characteristics of long interval sequential datasets. In this study, a novel deep
autoregression feature augmented bidirectional LSTM network (DAFA-BiLSTM) is
proposed as a new deep BiLSTM architecture for time series prediction. Initially,
the input vectors are fed into a vector autoregression (VA) transformation module
to represent the time-delayed linear and nonlinear properties of the input signals
in an unsupervised way. Then, the learned nonlinear combination vectors of VA are
progressively fed into different layers of BiLSTM and the output of the previous
BiLSTM module is also concatenated with the time-delayed linear vectors of the VA
as an augmented feature to form new additional input signals for the next adjacent
BiLSTM layer. Extensive real-world time series applications are addressed to
demonstrate the superiority and robustness of the proposed DAFA-BiLSTM. Comparative
experimental results and statistical analysis show that the proposed DAFA-BiLSTM
has good adaptive performance as well as robustness even in noisy environment.
Keywords: Time series prediction; Long short-term memory; Deep recurrent neural
network; Feature augmented; Vector autoregression transformation

Wanqing Zhang, Zi Lin, Xiaolei Liu,

Short-term offshore wind power forecasting - A hybrid model based on Discrete
Wavelet Transform (DWT), Seasonal Autoregressive Integrated Moving Average
(SARIMA), and deep-learning-based Long Short-Term Memory (LSTM),
Renewable Energy,
Volume 185,
2022,
Pages 611-628,
ISSN 0960-1481,
https://doi.org/10.1016/j.renene.2021.12.100.
(https://www.sciencedirect.com/science/article/pii/S0960148121018279)
Abstract: Short-term time series wind power predictions are extremely essential for
accurate and efficient offshore wind energy evaluation and, in turn, benefit large
wind farm operation and maintenance (O&M). However, it is still a challenging task
due to the intermittent nature of offshore wind, which significantly increases
difficulties in wind power forecasting. In this paper, a novel hybrid model, using
unique strengths of Discrete Wavelet Transform (DWT), Seasonal Autoregressive
Integrated Moving Average (SARIMA), and Deep-learning-based Long Short-Term Memory
(LSTM), was proposed to handle different components in the power time series of an
offshore wind turbine in Scotland, where neither the approximation nor the detail
was considered as purely nonlinear or linear. Besides, an integrated pre-processing
method, incorporating Isolation Forest (IF), resampling, and interpolation was
applied for the raw Supervisory Control and Data Acquisition (SCADA) datasets. The
proposed DWT-SARIMA-LSTM model provided the highest accuracy among all the observed
tests, indicating it could efficiently capture complex times series patterns from
offshore wind power.
Keywords: Short-term wind power forecasting; Offshore wind turbine; Wavelet
transform; Seasonal auto-regression integrated moving average (SARIMA); Deep
learning

Noopur Singh, Ravindra Nath, Dev Bukhsh Singh,

Splice-site identification for exon prediction using bidirectional LSTM-RNN
approach,
Biochemistry and Biophysics Reports,
Volume 30,
2022,
101285,
ISSN 2405-5808,
https://doi.org/10.1016/j.bbrep.2022.101285.
(https://www.sciencedirect.com/science/article/pii/S2405580822000851)
Abstract: Machine learning methods played a major role in improving the accuracy of
predictions and classification of DNA (Deoxyribonucleic Acid) and protein
sequences. In eukaryotes, Splice-site identification and prediction is though not a
straightforward job because of numerous false positives. To solve this problem,
here, in this paper, we represent a bidirectional Long Short Term Memory (LSTM)
Recurrent Neural Network (RNN) based deep learning model that has been developed to
identify and predict the splice-sites for the prediction of exons from eukaryotic
DNA sequences. During the splicing mechanism of the primary mRNA transcript, the
introns, the non-coding region of the gene are spliced out and the exons, the
coding region of the gene are joined. This bidirectional LSTM-RNN model uses the
intron features that start with splice site donor (GT) and end with splice site
acceptor (AG) in order of its length constraints. The model has been improved by
increasing the number of epochs while training. This designed model achieved a
maximum accuracy of 95.5%. This model is compatible with huge sequential data such
as the complete genome.
Keywords: Splice-site; Intron; Exon; Machine learning; Deep learning; Bidirectional
LSTM-RNN

Asiye Kaymaz Özcanlı, Mustafa Baysal,

A novel Multi-LSTM based deep learning method for islanding detection in the
microgrid,
Electric Power Systems Research,
Volume 202,
2022,
107574,
ISSN 0378-7796,
https://doi.org/10.1016/j.epsr.2021.107574.
(https://www.sciencedirect.com/science/article/pii/S0378779621005551)
Abstract: Microgrid (MG) is a key part of the future energy system that can operate
in either grid-connected or island mode by enabling the growing integration of
renewable energy sources such as photovoltaic energy, wind energy and hydroelectric
power. One of the most substantial phenomena in microgrids is unintentional
islanding which can cause significant problems such as power quality, voltage
stability and safety hazards. This paper introduces a new passive islanding
detection method (IDM) for synchronous and inverter interfaced MGs. The multi-long
short-term memory (LSTM) architecture which is one of the most recent and popular
techniques of deep learning is first proposed by utilizing voltage and current
harmonic distortion measured at the point of common coupling (PCC) of MG. For the
first time, the distorted main grid is taken into account with various operating
conditions. Numerical simulations are performed in MATLAB/Simulink and comparative
analysis of the proposed method with intelligent IDMs is realized to verify its
overall superiorities. The proposed method has achieved remarkable performance like
average accuracy of 99.3% and minimum loss of 0.06. The multi-LSTM model is able to
detect islanding events with accuracy of 97.93% for small than ± 0.5% power
mismatch within 50 ms detection time.
Keywords: Islanding detection; Deep learning; Harmonic distortion; LSTM; Microgrid

Xinxin Zhou, Jingru Feng, Yang Li,

Non-intrusive load decomposition based on CNN–LSTM hybrid deep learning model,
Energy Reports,
Volume 7,
2021,
Pages 5762-5771,
ISSN 2352-4847,
https://doi.org/10.1016/j.egyr.2021.09.001.
(https://www.sciencedirect.com/science/article/pii/S2352484721008064)
Abstract: With the rapid development of science and technology, the problem of
energy load monitoring and decomposition of electrical equipment has been receiving
widespread attention from academia and industry. For the purpose of improving the
performance of non-intrusive load decomposition, a non-intrusive load decomposition
method based on a hybrid deep learning model is proposed. In this method, first of
all, the data set is normalized and preprocessed. Secondly, a hybrid deep learning
model integrating convolutional neural network (CNN) with long short-term memory
network (LSTM) is constructed to fully excavate the spatial and temporal
characteristics of load data. Finally, different evaluation indicators are used to
analyze the mixture. The model is fully evaluated, and contrasted with the
traditional single deep learning model. Experimental results on the open dataset
UK-DALE show that the proposed algorithm improves the performance of the whole
network system. In this paper, the proposed decomposition method is compared with
the existing traditional deep learning load decomposition method. At the same time,
compared with the obtained methods: spectral decomposition, EMS, LSTM–RNN, and
other algorithms, the accuracy of load decomposition is significantly improved, and
the test accuracy reaches 98%.
Keywords: Non-intrusive load decomposition; Convolutional neural network; Long
short-term memory network; Hybrid deep learning

Chunlei Ji, Chu Zhang, Lei Hua, Huixin Ma, Muhammad Shahzad Nazir, Tian Peng,
A multi-scale evolutionary deep learning model based on CEEMDAN, improved whale
optimization algorithm, regularized extreme learning machine and LSTM for AQI
prediction,
Environmental Research,
Volume 215, Part 1,
2022,
114228,
ISSN 0013-9351,
https://doi.org/10.1016/j.envres.2022.114228.
(https://www.sciencedirect.com/science/article/pii/S0013935122015559)
Abstract: With the rapid development of economy, air pollution occurs frequently,
which has a huge negative impact on human health and urban ecosystem. Air quality
index (AQI) can directly reflect the degree of air pollution. Accurate AQI trend
prediction can provide reliable information for the prevention and control of air
pollution, but traditional forecasting methods have limited performance. To this
end, a dual-scale ensemble learning framework is proposed for the complex AQI time
series prediction. First, complete ensemble empirical mode decomposition adaptive
noise (CEEMDAN) and sample entropy (SE) are used to decompose and reconstruct AQI
series to reduce the difficulty of direct modeling. Then, according to the
characteristics of high and low frequencies, the high-frequency components are
predicted by the long short-term memory neural network (LSTM), and the low-
frequency items are predicted by the regularized extreme learning machine (RELM).
At the same time, the improved whale optimization algorithm (WOA) is used to
optimize the hyper-parameters of RELM and LSTM models. Finally, the hybrid
prediction model proposed in this paper predicts the AQI of four cities in China.
This work effectively improves the prediction accuracy of AQI, which is of great
significance to the sustainable development of the cities.
Keywords: Regularized extreme learning machine; Long short-term memory; CEEMDAN;
Whale optimization algorithm

Yongxiang Lei, Hamid Reza Karimi, Xiaofang Chen,

A novel self-supervised deep LSTM network for industrial temperature prediction in
aluminum processes application,
Neurocomputing,
Volume 502,
2022,
Pages 177-185,
ISSN 0925-2312,
https://doi.org/10.1016/j.neucom.2022.06.080.
(https://www.sciencedirect.com/science/article/pii/S0925231222008116)
Abstract: This article studies the influence of pot temperature or electrolyte
temperature in the aluminum reduction production. Specifically, these indexes
reflect the distribution of the physical and energy field of the reduction cell,
the current efficiency, and the lifespan of the aluminum reduction cell. Therefore,
the pot temperature detection and identification are two critical and significant
issues in the whole production process of aluminum electrolysis. However, due to
the low measurement accuracy and high maintenance costs with the thermocouple
sensor in the practical production process, the real-time measurement of pot
temperature index is still a major challenge, which motivate us to develop a self-
supervised soft sensor method based on deep long-short term memory (LSTM). Under
the constraint of the limited samples, the proposed method achieves a competitive
performance. First, the input variables are selected according to the expert
experience. Then, a deep self-supervised model is built. Finally, the proposed
self-supervised LSTM model is applied to real-time detection in an industrial
electrolysis production case. The performance in the experiment outperforms other
existing methods in terms of both accuracy and robustness aspects.
Keywords: PT prediction; LSTM; Aluminum electrolysis; Recurrent neural network;
Self-supervised learning (SSL)

S. Nath, B. Chetia, S. Kalita,

Ionospheric TEC prediction using hybrid method based on ensemble empirical mode
decomposition (EEMD) and long short-term memory (LSTM) deep learning model over
India,
Advances in Space Research,
2022,
,
ISSN 0273-1177,
https://doi.org/10.1016/j.asr.2022.10.067.
(https://www.sciencedirect.com/science/article/pii/S0273117722010195)
Abstract: Total electron data (TEC) from GPS nowadays can be used as a tool for
understanding the space weather phenomena. The development of prediction model for
TEC is quiet crucial and challenging due to the dynamic behavior of the ionosphere,
since it depends on different factors such as seasonal, diurnal and spatial
variations, solar geomagnetic conditions etc. In this paper, an attempt is made for
predicting the GPS derived TEC values for different GNSS stations over India using
a hybrid method based on Ensemble empirical mode decomposition (EEMD) and Long
Short-Term Memory (LSTM) deep learning method. The daily TEC time series data from
the IISc Bangalore (Latitude 13.021, Longitude 77.570), Lucknow (Latitude 26.912,
Longitude 80.956) and Hyderabad (Latitude 17.417, Longitude 78.551) stations over
India during the period 2008 to 2015 of solar cycle 23 and 24 is used for analysis.
The assessment of model performance for testing predicted output compared with LSTM
and EMD-LSTM models, and their comparison results show that the hybrid EEMD-LSTM
model presents better than the other models.
Keywords: Ionosphere; Total Electron Content; EEMD; LSTM; GPS

Xingyu Tang, Peijie Zheng, Xueyong Li, Hongyan Wu, Dong-Qing Wei, Yuewu Liu, Guohua
Huang,
Deep6mAPred: A CNN and Bi-LSTM-based deep learning method for predicting DNA N6-
methyladenosine sites across plant species,
Methods,
Volume 204,
2022,
Pages 142-150,
ISSN 1046-2023,
https://doi.org/10.1016/j.ymeth.2022.04.011.
(https://www.sciencedirect.com/science/article/pii/S1046202322001001)
Abstract: DNA N6-methyladenine (6mA) is a key DNA modification, which plays
versatile roles in the cellular processes, including regulation of gene expression,
DNA repair, and DNA replication. DNA 6mA is closely associated with many diseases
in the mammals and with growth as well as development of plants. Precisely
detecting DNA 6mA sites is of great importance to exploration of 6mA functions.
Although many computational methods have been presented for DNA 6mA prediction,
there is still a wide gap in the practical application. We presented a convolution
neural network (CNN) and bi-directional long-short term memory (Bi-LSTM)-based deep
learning method (Deep6mAPred) for predicting DNA 6mA sites across plant species.
The Deep6mAPred stacked the CNNs and the Bi-LSTMs in a paralleling manner instead
of a series-connection manner. The Deep6mAPred also employed the attention
mechanism for improving the representations of sequences. The Deep6mAPred reached
an accuracy of 0.9556 over the independent rice dataset, far outperforming the
state-of-the-art methods. The tests across plant species showed that the
Deep6mAPred is of a remarkable advantage over the state of the art methods. We
developed a user-friendly web application for DNA 6mA prediction, which is freely
available at http://106.13.196.152:7001/ for all the scientific researchers. The
Deep6mAPred would enrich tools to predict DNA 6mA sites and speed up the
exploration of DNA modification.
Keywords: 6mA; DNA modification; Convolution neural network; Long-short term
memory; Feed-forward attention; Deep learning

Bhargava K Reddy, Dursun Delen,

Predicting hospital readmission for lupus patients: An RNN-LSTM-based deep-learning
methodology,
Computers in Biology and Medicine,
Volume 101,
2018,
Pages 199-209,
ISSN 0010-4825,
https://doi.org/10.1016/j.compbiomed.2018.08.029.
(https://www.sciencedirect.com/science/article/pii/S0010482518302567)
Abstract: Hospital readmission is one of the critical metrics used for measuring
the performance of hospitals. The HITECH Act imposes penalties when patients are
readmitted to hospitals if they are diagnosed with one of the six conditions
mentioned in the Act. However, patients diagnosed with lupus are the sixth highest
in terms of rehospitalization. The heterogeneity in the disease and patient
characteristics makes it very hard to predict rehospitalization. This research
utilizes deep learning methods to predict rehospitalization within 30 days by
extracting the temporal relationships in the longitudinal EHR clinical data.
Prediction results from deep learning methods such as LSTM are evaluated and
compared with traditional classification methods such as penalized logistic
regression and artificial neural networks. The simple recurrent neural network
method and its variant, gated recurrent unit network, are also developed and
validated to compare their performance against the proposed LSTM model. The results
indicated that the deep learning method RNN-LSTM has a significantly better
performance (with an AUC of .70) compared to traditional classification methods
such as ANN (with an AUC of 0.66) and penalized logistic regression (with an AUC of
0.63). The rationale for the better performance of the deep learning method may be
due to its ability to leverage the temporal relationships of the disease state in
patients over time and to capture the progression of the disease—relevant clinical
information from patients' prior visits is carried forward in the memory, which may
have enabled the higher predictability for the deep learning methods.
Keywords: Readmission; Lupus; Machine learning; Predictive analytics; Deep
learning; LSTM

Md. Ferdous Ahammed, A. Alim Molla, Rafiul Kadir, Mohammad Ismat Kadir,
Deep bidirectional LSTM for the signal detection of universal filtered multicarrier
systems,
Machine Learning with Applications,
Volume 10,
2022,
100425,
ISSN 2666-8270,
https://doi.org/10.1016/j.mlwa.2022.100425.
(https://www.sciencedirect.com/science/article/pii/S2666827022001001)
Abstract: Universal filtered multicarrier (UFMC) has emerged as a potential
waveform contender of orthogonal frequency division multiplexing (OFDM) for the
fifth generation (5G) and beyond wireless systems. In this paper, we propose a
bidirectional long short-term memory (Bi-LSTM)-based detector for the UFMC system.
The proposed detector directly detects the transmitted symbols using the deep
learning (DL)-based training data. The system is first trained with the aid of
training data and pilot symbols. The training tunes the DL-based network
parameters. During the testing phase, the signal is detected using the trained
network. The performance of the proposed scheme is compared with that of the DL-
aided OFDM system, and with the signal detection strategies using the conventional
channel estimation techniques. Our simulations show that the proposed Bi-LSTM-based
DL can flexibly and effectively detect UFMC signals.
Keywords: Universal filtered multicarrier; Deep learning; Long short-term memory
(LSTM); Bidirectional LSTM; Rayleigh fading channel

Admin Husic, Nabil Al-Aamery, James F. Fox,

Simulating hydrologic pathway contributions in fluvial and karst settings: An
evaluation of conceptual, physically-based, and deep learning modeling approaches,
Journal of Hydrology X,
Volume 17,
2022,
100134,
ISSN 2589-9155,
https://doi.org/10.1016/j.hydroa.2022.100134.
(https://www.sciencedirect.com/science/article/pii/S2589915522000165)
Abstract: Hydrologic models are robust tools for estimating key parameters in the
management of water resources, including water inputs, storage, and pathway fluxes.
The selection of process-based versus data-driven modeling structure is an
important consideration, particularly as advancements in machine learning yield
potential for improved model performance but at the cost of lacking physical
analogues. Despite recent advancement, there exists an absence of cross-model
comparison of the tradeoffs between process-based and data-driven model types in
settings with varying hydrologic controls. In this study, we use physically-based
(SWAT), conceptually-based (LUMP), and deep-learning (LSTM) models to simulate
hydrologic pathway contributions for a fluvial watershed and a karst basin over a
twenty-year period. We find that, while all models are satisfactory, the LSTM model
outperformed both the SWAT and LUMP models in simulating total discharge and that
the improved performance was more evident in the groundwater-dominated karst system
than the surface-dominated fluvial stream. Further, the LSTM model was able to
achieve this improved performance with only 10–25% of the observed time-series as
training data. Regarding pathways, the LSTM model coupled with a recursive digital
filter was able to successfully match the magnitude of process-based estimates of
quick, intermediate, and slow flow contributions for both basins (ρ ranging from
0.58 to 0.71). However, the process-based models exhibited more realistic time-
fractal scaling of hydrologic flow pathways compared to the LSTM model which,
depending on project objectives, presents a potential drawback to the use of
machine learning models for some hydrologic applications. This study demonstrates
the utility and potential extraction of physical-analogues of LSTM modeling, which
will be useful as deep learning approaches to hydrologic modeling become more
prominent and modelers look for ways to infer physical information from data-driven
predictions.
Keywords: Hydrologic modeling; Machine learning; LSTM; Rainfall-runoff; Process-
based; Data-driven

Ruoyu Yang, Shubhendu Kumar Singh, Mostafa Tavakkoli, Nikta Amiri, Yongchao Yang,
M. Amin Karami, Rahul Rai,
CNN-LSTM deep learning architecture for computer vision-based modal frequency
detection,
Mechanical Systems and Signal Processing,
Volume 144,
2020,
106885,
ISSN 0888-3270,
https://doi.org/10.1016/j.ymssp.2020.106885.
(https://www.sciencedirect.com/science/article/pii/S0888327020302715)
Abstract: The conventional modal analysis involves physically-attached wired or
wireless sensors for vibration measurement of structures. However, this method has
certain disadvantages, owing to the sensor’s weight and its low spatial resolution,
which limits the analysis precision or the high cost of optical vibration sensors.
Besides, the sensor installation and calibration in itself is a time consuming and
labor-intensive process. Non-contact computer vision-based vibration measurement
techniques can address the shortcomings mentioned above. In this paper, we
introduce CNN-LSTM (Convolutional Neural Network, Long Short-Term Memory) deep
learning based approach that can serve as a backbone for computer vision-based
vibration measurement techniques. The key idea is to use each pixel of an image
taken from an off the shelf camera, encapsulating the Spatio-temporal information,
like a sensor to capture the modal frequencies of a vibrating structure. Non-
contact “pixel-sensor” does not alter the system’s dynamics and is relatively low-
cost, agile, and provides measurements with very high spatial resolution. Our
computer vision-based deep learning model takes the video of a vibrating structure
as input and outputs the fundamental modal frequencies. We demonstrate, using
reliable empirical results, that “pixel-sensor” is more efficient, autonomous, and
accurate. Robustness of the deep learning model has been put to the test by using
specimens of a variety of materials, and varying dimensions and results have shown
high levels of sensing accuracy.
Keywords: CNN (convolutional neural network); LSTM (long short-term memory);
Computer vision; Modal analysis

K.N. Ravikumar, Akhilesh Yadav, Hemantha Kumar, K.V. Gangadharan, A.V.

Narasimhadhan,
Gearbox fault diagnosis based on Multi-Scale deep residual learning and stacked
LSTM model,
Measurement,
Volume 186,
2021,
110099,
ISSN 0263-2241,
https://doi.org/10.1016/j.measurement.2021.110099.
(https://www.sciencedirect.com/science/article/pii/S0263224121010216)
Abstract: Fault diagnosis methods based on signal analysis techniques are widely
used to diagnose faults in gear and bearing. This paper introduces a fault
diagnosis model that includes a multi-scale deep residual learning with a stacked
long short-term memory (MDRL-SLSTM) to address sequence data in a gearbox health
prediction task in an internal combustion (IC) engine. In the MDRL-SLSTM network,
CNN and residual learning is firstly utilized for local feature extraction and
dimension reduction. The experiment is carried out on the gearbox of an IC engine
setup, two datasets are used; one is from bearing and the other from 2nd driving
gear of gearbox. To reduce the number of parameters, down-sampling is carried out
on input data before giving to the architecture. The model achieved better
diagnostic performance with vibration data of gearbox. Classification accuracy of
94.08% and 94.33% are attained on bearing datasets and 2nd driving gear of gearbox
respectively.
Keywords: Ball bearing; Gear; Deep learning techniques; IC engine; Gearbox; LSTM

Yakubu Imrana, Yanping Xiang, Liaqat Ali, Zaharawu Abdul-Rauf,

A bidirectional LSTM deep learning approach for intrusion detection,
Expert Systems with Applications,
Volume 185,
2021,
115524,
ISSN 0957-4174,
https://doi.org/10.1016/j.eswa.2021.115524.
(https://www.sciencedirect.com/science/article/pii/S0957417421009337)
Abstract: The rise in computer networks and internet attacks has become alarming
for most service providers. It has triggered the need for the development and
implementation of intrusion detection systems (IDSs) to help prevent and or
mitigate the challenges posed by network intruders. Over the years, intrusion
detection systems have played and continue to play a very significant role in
spotting network attacks and anomalies. Numerous researchers around the globe have
proposed many IDSs to combat the threat of network invaders. However, most of the
previously proposed IDSs have high rates of raising false alarms. Additionally,
most existing models suffer the difficulty of detecting the different attack types,
especially User-to-Root (U2R) and Remote-to-Local (R2L) attacks. These two types of
attacks often appear to have lower detection accuracy for the existing models.
Hence, in this paper, we propose a bidirectional Long-Short-Term-Memory (BiDLSTM)
based intrusion detection system to handle the challenges mentioned above. To train
and measure our model’s performance, we use the NSL-KDD dataset, a benchmark
dataset for most IDSs. Experimental results show and validate the effectiveness of
the BiDLSTM approach. It outperforms conventional LSTM and other state-of-the-art
models in terms of accuracy, precision, recall, and F-score values. It also has a
much more reduced false alarm rate than the existing models. Furthermore, the
BiDLSTM model achieves a higher detection accuracy for U2R and R2L attacks than the
conventional LSTM.
Keywords: Machine learning; Deep learning; Recurrent neural networks; Bidirectional
LSTM; Intrusion detection

Liqun Yang, You Zhai, Zhoujun Li,

Deep learning for online AC False Data Injection Attack detection in smart grids:
An approach using LSTM-Autoencoder,
Journal of Network and Computer Applications,
Volume 193,
2021,
103178,
ISSN 1084-8045,
https://doi.org/10.1016/j.jnca.2021.103178.
(https://www.sciencedirect.com/science/article/pii/S1084804521001880)
Abstract: The Power system is a crucial Cyber-Physical system and is prone to the
False Data Injection Attack (FDIA). The existing FDIA detection mechanism focuses
on DC state estimation. In this paper, we propose a phased AC FDIA targeting at
generation rescheduling and load shedding. After injecting the false data into the
measurements, the estimated states will be deviated from those in normal
conditions. The proposed mechanism extracts the spatial and spectral features of
the modes decomposed from the estimated states using variational mode decomposition
(VMD). Then LSTM-Autoencoder is trained by learning the temporal correlations
between the multi-dimensional feature vectors. The reconstruction error deviation
vectors of the feature vectors are calculated and updated by LSTM-Autoencoder.
Based on these error deviation vectors, the Logistic Regression (LR) classifier is
trained to determine whether the error deviation vector is abnormal. We evaluate
the performance of the proposed mechanism with comprehensive simulations on IEEE 14
and 118-bus systems. The results indicate that the mechanism can achieve a
satisfactory attack detection accuracy.
Keywords: False data injection attack; AC state Estimation; Variational mode
decomposition; LSTM-Autoencoder; Deviation vector; Logistic regression

Won Hee Chung, Yeong Hyeon Gu, Seong Joon Yoo,

District heater load forecasting based on machine learning and parallel CNN-LSTM
attention,
Energy,
Volume 246,
2022,
123350,
ISSN 0360-5442,
https://doi.org/10.1016/j.energy.2022.123350.
(https://www.sciencedirect.com/science/article/pii/S0360544222002535)
Abstract: Accurate heat load forecast is important to operate combined heat and
power (CHP) efficiently. This paper proposes a parallel convolutional neural
network (CNN) - long short-term memory (LSTM) attention (PCLA) model that extracts
spatiotemporal characteristics and then intensively learns importance. PCLA learns
by derived spatial and temporal features parallelly from CNNs and LSTMs. The
novelty of this paper lies in the following three aspects: 1) a PCLA model for heat
load forecasting is proposed; 2) it is demonstrated that the performance is
superior to 12 models including the serial coupled model; 3) the model using CNNs
and LSTMs is better than the one using principal component analysis. The dataset
includes district heater related variables, heat load-derived variables, weather
forecasts and time factors that affect heat loads. The forecasting accuracy of the
PCLA is reflected by the lowest values of the mean absolute and mean squared errors
of 0.571 and 0.662, respectively, and the highest R-squared value of 0.942. The
performance of the PCLA is therefore better than the previously proposed heat load
and demand forecasting models and is expected to be useful for CHP plant
management.
Keywords: Heat load forecasting; Convolutional neural network; Long short-term
memory; Attention mechanism; Bayesian optimization

K. Venkatachalam, Pavel Trojovský, Dragan Pamucar, Nebojsa Bacanin, Vladimir Simic,

DWFH: An improved data-driven Deep Weather forecasting Hybrid Model using
Transductive Long Short Term Memory (T-LSTM),
Expert Systems with Applications,
2022,
119270,
ISSN 0957-4174,
https://doi.org/10.1016/j.eswa.2022.119270.
(https://www.sciencedirect.com/science/article/pii/S0957417422022886)
Abstract: Forecasting climate and the development of the environment have been
essential in recent days since there has been a drastic change in nature. Weather
forecasting plays a significant role in decision-making in traffic management,
tourism planning, crop cultivation in agriculture, and warning the people nearby
the seaside about the climate situation. It is used to reduce accidents and
congestion, mainly based on climate conditions such as rainfall, air condition, and
other environmental factors. Accurate weather prediction models are required by
meteorological scientists. The previous studies have shown complexity in terms of
model building, and computation, and based on theory-driven and rely on time and
space. This drawback can be easily solved using the machine learning technique with
the time series data. This paper proposes the state-of-art deep learning model Long
Short-Term Memory (LSTM) and the Transductive Long Short-Term Memory (T-LSTM)
model. The model is evaluated using the evaluation metrics root mean squared error,
loss, and mean absolute error. The experiments are carried out on HHWD and Jena
Climate datasets. The dataset comprises 14 weather forecasting features including
humidity, temperature, etc. The T-LSTM method performs better than other
methodologies, producing 98.2% accuracy in forecasting the weather. This proposed
hybrid T-LSTM method provides a robust solution for the hydrological variables.
Keywords: Forecasting; rainfall; LSTM; Transductive T-LSTM; deep learning

Zahra Khademi, Farideh Ebrahimi, Hussain Montazery Kordy,

A transfer learning-based CNN and LSTM hybrid deep learning model to classify motor
imagery EEG signals,
Computers in Biology and Medicine,
Volume 143,
2022,
105288,
ISSN 0010-4825,
https://doi.org/10.1016/j.compbiomed.2022.105288.
(https://www.sciencedirect.com/science/article/pii/S0010482522000804)
Abstract: In the Motor Imagery (MI)-based Brain Computer Interface (BCI), users'
intention is converted into a control signal through processing a specific pattern
in brain signals reflecting motor characteristics. There are such restrictions as
the limited size of the existing datasets and low signal to noise ratio in the
classification of MI Electroencephalogram (EEG) signals. Machine learning (ML)
methods, particularly Deep Learning (DL), have overcome these limitations
relatively. In this study, three hybrid models were proposed to classify the EEG
signal in the MI-based BCI. The proposed hybrid models consist of the convolutional
neural networks (CNN) and the Long-Short Term Memory (LSTM). In the first model,
the CNN with different number of convolutional-pooling blocks (from shallow to deep
CNN) was examined; a two-block CNN model not affected by the vanishing gradient
descent and yet able to extract desirable features employed; the second and third
models contained pre-trained CNNs conducing to the exploration of more complex
features. The transfer learning strategy and data augmentation methods were applied
to overcome the limited size of the datasets by transferring learning from one
model to another. This was achieved by employing two powerful pre-trained
convolutional neural networks namely ResNet-50 and Inception-v3. The continuous
wavelet transform (CWT) was used to generate images for the CNN. The performance of
the proposed models was evaluated on the BCI Competition IV dataset 2a. The mean
accuracy vlaues of 86%, 90%, and 92%, and mean Kappa values of 81%, 86%, and 88%
were obtained for the hybrid neural network with the customized CNN, the hybrid
neural network with ResNet-50 and the hybrid neural network with Inception-v3,
respectively. Despite the promising performance of the three proposed models, the
hybrid neural network with Inception-v3 outperformed the two other models. The best
obtained result in the present study improved the previous best result in the
literature by 7% in terms of classification accuracy. From the findings, it can be
concluded that transfer learning based on a pre-trained CNN in combination with
LSTM is a novel method in MI-based BCI. The study also has implications for the
discrimination of motor imagery tasks in each EEG recording channel and in
different brain regions which can reduce computational time in future works by only
selecting the most effective channels.
Keywords: BCI; MI; Hybrid neural network; Convolutional neural network; LSTM;
Transfer learning

Zengyi Lyu, Xiaowei Jia, Yao Yang, Keqi Hu, Feifei Zhang, Gaofeng Wang,
A comprehensive investigation of LSTM-CNN deep learning model for fast detection of
combustion instability,
Fuel,
Volume 303,
2021,
121300,
ISSN 0016-2361,
https://doi.org/10.1016/j.fuel.2021.121300.
(https://www.sciencedirect.com/science/article/pii/S0016236121011790)
Abstract: In this paper, we propose a deep learning model to detect combustion
instability using high-speed flame image sequences. The detection model combines
Convolutional Neural Network (CNN) and Long Short-Term Memory network (LSTM) to
learn both spatial features and temporal correlations from high-speed images, and
then outputs combustion instability detection results. We also visualize the
extracted spatial features and their temporal evolution to interpret the detection
process of model. In addition, we discuss the effect of different complexity of CNN
layers and different amounts of training data on model performance. The proposed
method achieves superior performance under various combustion conditions in swirl
chamber with high accuracy and a short processing time about 1.23 ms per frame.
Hence, we show that the proposed deep learning model is a promising detection tool
for combustion instability under various combustion conditions.
Keywords: Premixed swirling flame; Combustion instability; Deep learning;
Convolutional neural network; LSTM

Lin Lin, Xin Guan, Benran Hu, Jun Li, Ning Wang, Di Sun,
Deep reinforcement learning and LSTM for optimal renewable energy accommodation in
5G internet of energy with bad data tolerant,
Computer Communications,
Volume 156,
2020,
Pages 46-53,
ISSN 0140-3664,
https://doi.org/10.1016/j.comcom.2020.03.024.
(https://www.sciencedirect.com/science/article/pii/S0140366419321103)
Abstract: With the high penetration of large scale distributed renewable energy
generations, there is a serious curtailment of wind and solar energy in 5G internet
of energy. A reasonable assessment of large scale renewable energy grid-connected
capacities under random scenarios is critical to promote the efficient utilization
of renewable energy and improve the stability of power systems. To assure the
authenticity of the data collected by the terminals and describe data
characteristics precisely are crucial problems in assessing the accommodation
capability of renewable energy. To solve these problems, in this paper, we propose
an L-DRL algorithm based on deep reinforcement learning (DRL) to maximize renewable
energy accommodation in 5G internet of energy. LSTM as a bad data tolerant
mechanism provides real state value for the solution of accommodation strategy,
which ensures the accurate assessment of renewable energy accommodation capacity.
DDPG is used to obtain optimal renewable energy accommodation strategies in
different scenarios. In the numerical results, based on real meteorological data,
we validate the performance of the proposed algorithm. Results show considering the
energy storage system and demand response mechanism can improve the capacity of
renewable energy accommodation in 5G internet of energy.
Keywords: 5G internet of energy; Renewable energy accommodation; Deep reinforcement
learning; Demand response; LSTM

Zahra Fazlipour, Elaheh Mashhour, Mahmood Joorabian,

A deep model for short-term load forecasting applying a stacked autoencoder based
on LSTM supported by a multi-stage attention mechanism,
Applied Energy,
Volume 327,
2022,
120063,
ISSN 0306-2619,
https://doi.org/10.1016/j.apenergy.2022.120063.
(https://www.sciencedirect.com/science/article/pii/S0306261922013204)
Abstract: This paper presents an innovative univariate Deep LSTM-based Stacked
Autoencoder (DLSTM-SAE) model for short-term load forecasting, equipped with a
Multi-Stage Attention Mechanism (MSAM), including an input AM and several temporal
AM in the pre-training phase. The input AM is used to capture the high-impact load
sequence time steps of univariate input data. It should be noted that the model's
performance is improved by increasing the network depth; however, finding the
optimal network parameters is a challenging task due to the random assignment of
the initial weights of the network. An unsupervised greedy layer-wise pre-training
structure equipped with the MSAM is expanded to solve setting the random initial
weight problem of the DLSTM-SAE model. The multi-stage temporal AM in the pre-
training structure leads the DLSTM-SAE to properly learn the time dependencies
related to remarkably long sequence input data and capture the temporal merit
features lied in the LSTM memory. The performance of the proposed model is
evaluated through various comparative tests with current prevalent models using
actual energy market data New England ISO using three criteria indexes. The results
show the superiority of the proposed model and its robustness in offline and online
load forecasting.
Keywords: Deep learning; Attention mechanism; Short-term load forecasting; LSTM;
Stacked autoencoder

Yong'an Zhang, Binbin Yan, Memon Aasma,

A novel deep learning framework: Prediction and analysis of financial time series
using CEEMD and LSTM,
Expert Systems with Applications,
Volume 159,
2020,
113609,
ISSN 0957-4174,
https://doi.org/10.1016/j.eswa.2020.113609.
(https://www.sciencedirect.com/science/article/pii/S0957417420304334)
Abstract: Deep learning is well-known for extracting high-level abstract features
from a large amount of raw data without relying on prior knowledge, which is
potentially attractive in forecasting financial time series. Long short-term memory
(LSTM) networks are deemed as state-of-the-art techniques in sequence learning,
which are less commonly applied to financial time series predictions, yet
inherently suitable for this domain. We propose a novel methodology of deep
learning prediction, and based on this, construct a deep learning hybrid prediction
model for stock markets—CEEMD-PCA-LSTM. In this model, complementary ensemble
empirical mode decomposition (CEEMD), as a sequence smoothing and decomposition
module, can decompose the fluctuations or trends of different scales of time series
step by step, generating a series of intrinsic mode functions (IMFs) with different
characteristic scales. Then, with retaining the most of information on raw data,
PCA reduces dimension of the decomposed IMFs component, eliminating the redundant
information and improving prediction response speed. After that, high-level
abstract features are separately fed into LSTM networks to predict closing price of
the next trading day for each component. Finally, synthesizing the predicted values
of individual components is utilized to obtain a final predicted value. The
empirical results of six representative stock indices from three types of markets
indicate that our proposed model outperforms benchmark models in terms of
predictive accuracy, i.e., lower test error and higher directional symmetry.
Leveraging key research findings, we perform trading simulations to validate that
the proposed model outperforms benchmark models in both absolute profitability
performance and risk-adjusted profitability performance. Furthermore, model
robustness test unveils the more stable robustness compared to benchmark models.
Keywords: Deep learning; Long short-term memory; Complementary ensemble empirical
mode decomposition; Financial time series; Stock market forecasting; Principal
component analysis

Dinesh Jackson Samuel R., Fenil E, Gunasekaran Manogaran, Vivekananda G.N,

Thanjaivadivel T, Jeeva S, Ahilan A,
Real time violence detection framework for football stadium comprising of big data
analysis and deep learning through bidirectional LSTM,
Computer Networks,
Volume 151,
2019,
Pages 191-200,
ISSN 1389-1286,
https://doi.org/10.1016/j.comnet.2019.01.028.
(https://www.sciencedirect.com/science/article/pii/S1389128618308521)
Abstract: Football is the most popular sport in the world with four billion fans
all over the world. Reportedly, the violence incidence rates are high during or
after the matches. The violent or destructive behavior carried out by a person or
player, who watches or plays the game in the stadium is known as football
hooliganism. To prevent or control the violence, a real time violence detection
system is exclusively needed to monitor the behavior of the crowd and players to
take necessary action before the violence is about to happen. Even it is necessary
for the system to find whether the attack is non-intentional or intentional in the
game. In this paper, a real time violence detection system is proposed which
processes the huge input streaming data and recognize the violence with human
intelligence simulation. The input to the system is the enormous amount of real
time video streams from different sources which is processed in Spark framework. In
the Spark framework, the frames are separated and the features of individual frames
are extracted by using HOG (Histogram of Oriented Gradients) function. Then the
frames are labeled based on features as violence model, human part model and
negative model, which are used to train the Bidirectional Long Short-Term Memory
(BDLSTM) network for recognition of violence scenes. The bidirectional LSTM can
access the information both in forward and reverse direction. Thus the output is
generated in context to both past and future information. The network is trained
with the violent interaction dataset (VID), containing 2314 videos with 1077 fight
ones and 1237 no-fight ones. Moreover to make the model robust to violence
detection, we have created a dataset with 410 video clips having non-violence
scenes and 409 video clips having violence scenes, acquired from the football
stadium. The performance of this model is validated and it proves the sturdiness of
the system with an accuracy of 94.5 percentage in recognizing the violent action.
Keywords: Violence detection; Video surveillance; Big data analysis; Histogram of
Oriented Gradients; Bidirectional Long Short-Term Memory

K.E. ArunKumar, Dinesh V. Kalaga, Ch. Mohan Sai Kumar, Masahiro Kawaji, Timothy M.
Brenza,
Comparative analysis of Gated Recurrent Units (GRU), long Short-Term memory (LSTM)
cells, autoregressive Integrated moving average (ARIMA), seasonal autoregressive
Integrated moving average (SARIMA) for forecasting COVID-19 trends,
Alexandria Engineering Journal,
Volume 61, Issue 10,
2022,
Pages 7585-7603,
ISSN 1110-0168,
https://doi.org/10.1016/j.aej.2022.01.011.
(https://www.sciencedirect.com/science/article/pii/S1110016822000138)
Abstract: Several machine learning and deep learning models were reported in the
literature to forecast COVID-19 but there is no comprehensive report on the
comparison between statistical models and deep learning models. The present work
reports a comparative time-series analysis of deep learning techniques (Recurrent
Neural Networks with GRU and LSTM cells) and statistical techniques (ARIMA and
SARIMA) to forecast the country-wise cumulative confirmed, recovered, and deaths.
The Gated Recurrent Units (GRU), Long Short-Term Memory (LSTM) cells based on
Recurrent Neural Networks (RNN), ARIMA and SARIMA models were trained, tested, and
optimized to forecast the trends of the COVID-19. We deployed python to optimize
the parameters of ARIMA which include (p, d, q) representing autoregressive and
moving average terms and parameters of SARIMA model include additional seasonal
terms which are denoted by (P, D, Q). Similarly, for LSTM and GRU based RNN models’
parameters (number of layers, hidden size, learning rate and number of epochs) are
optimized by deploying PyTorch machine learning framework. The best model was
chosen based on the lowest Mean Square Error (MSE) and Root Mean Squared Error
(RMSE) values. For most of the time-series data of the countries, deep learning-
based models LSTM and GRU outperformed statistical ARIMA and SARIMA models, with an
RMSE values that are 40 folds less than that of the ARIMA models. But for some
countries statistical (ARIMA, SARIMA) models outperformed deep learning models.
Further, we emphasize the importance of various factors such as age, preventive
measures and healthcare facilities etc. that play vital role on the rapid spread of
COVID-19 pandemic.
Keywords: COVID-19 pandemic; Gated Recurrent Units (GRUs); Long Short-Term Memory
(LSTM) cells; Recurrent Neural Networks (RNNs); Auto Regressive Integrated Moving
Average (ARIMA); Seasonal Auto Regressive Integrated Moving Average (SARIMA)

R. Kiran, Pradeep Kumar, Bharat Bhasker,

Oslcfit (organic simultaneous LSTM and CNN Fit): A novel deep learning based
solution for sentiment polarity classification of reviews,
Expert Systems with Applications,
Volume 157,
2020,
113488,
ISSN 0957-4174,
https://doi.org/10.1016/j.eswa.2020.113488.
(https://www.sciencedirect.com/science/article/pii/S0957417420303122)
Abstract: Review sentiment influences purchase decisions and indicates user
satisfaction. Inferring the sentiment from reviews is an essential task in Natural
Language Processing and has managerial implications for improving customer
satisfaction and item quality. Traditional approaches to polarity classification
use bag-of-words techniques and lexicons combined with machine learning. These
approaches suffer from an inability to capture semantics and context. We propose a
Deep Learning solution called OSLCFit (Organic Simultaneous LSTM and CNN Fit). In
our architecture, we include all the components of a CNN until but not including
the final fully connected layer and do the same in case of a bi-directional LSTM.
The final fully connected layer in our architecture consists of fixed length
features from the CNN, and features for both variable length and temporal
dependencies from the bi-directional LSTM. The solution fine-tunes Language Model
embeddings for the specific task of polarity classification using transfer
learning, enabling the capture of semantics and context. The key contribution of
this paper is the combination of features from both a CNN and a bi-directional LSTM
into a single architecture with a single optimizer. This combination forms an
organic combination and uses embeddings fine-tuned to the reviews for the specific
purpose of sentiment polarity classification. The solution is benchmarked on six
different datasets such as SMS Spam, YouTube Spam, Large Movie Review Corpus,
Stanford Sentiment Treebank, Amazon Cellphone & Accessories and Yelp, where it
beats existing benchmarks and scales to large datasets. The source code is
available for the purposes of reproducible research on
GitHub.11https://github.com/efpm04013/finalexp34
Keywords: Bi-directional LSTM; CNN; Sentiment Classification; Organic Simultaneous
LSTM and CNN Fit; Review Polarity; NLP
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

Sangjin Park, Jae-Suk Yang,

Interpretable deep learning LSTM model for intelligent economic decision-making,
Knowledge-Based Systems,
Volume 248,
2022,
108907,
ISSN 0950-7051,
https://doi.org/10.1016/j.knosys.2022.108907.
(https://www.sciencedirect.com/science/article/pii/S0950705122004348)
Abstract: For sustainable economic growth, information about economic activities
and prospects is critical to decision-makers such as governments, central banks,
and financial markets. However, accurate predictions have been challenging due to
the complexity and uncertainty of financial and economic systems amid repeated
changes in economic environments. This study provides two approaches for better
economic prediction and decision-making. We present a deep learning model based on
the long short-term memory (LSTM) network architecture to predict economic growth
rates and crises by capturing sequential dependencies within the economic cycle. In
addition, we provide an interpretable machine learning model that derives economic
patterns of growth and crisis through efficient use of the eXplainable AI (XAI)
framework. For major G20 countries from 1990 to 2019, our LSTM model outperformed
other traditional predictive models, especially in emerging countries. Moreover, in
our model, private debt in developed economies and government debt in emerging
economies emerged as major factors that limit future economic growth. Regarding the
economic impact of COVID-19, we found that sharply reduced interest rates and
expansion of government debt increased the probability of a crisis in some emerging
economies in the future.
Keywords: LSTM; Deep learning; Interpretable machine learning; Economic prediction

Zeng Chen, Huan Xu, Peng Jiang, Shanen Yu, Guang Lin, Igor Bychkov, Alexey Hmelnov,
Gennady Ruzhnikov, Ning Zhu, Zhen Liu,
A transfer Learning-Based LSTM strategy for imputing Large-Scale consecutive
missing data and its application in a water quality prediction system,
Journal of Hydrology,
Volume 602,
2021,
126573,
ISSN 0022-1694,
https://doi.org/10.1016/j.jhydrol.2021.126573.
(https://www.sciencedirect.com/science/article/pii/S002216942100620X)
Abstract: In recent years, water quality monitoring has been crucial to improve
water resource protection and management. Under the relevant laws and regulations,
environmental protection department agencies monitor lakes, streams, rivers, and
other types of water bodies to assess water quality conditions. The valid and high-
quality data generated from these monitoring activities help water resource
managers understand the existing pollution situations, energy consumption problems
and pollution control needs. However, there are inevitably many problems with water
quality data in the real world due to human mistakes or system failures. One of the
most frequently occurring issues is missing data. Although most existing studies
have explored classic statistical methods or emerging machine/deep learning methods
to fill gaps in data, these methods are not suitable for large-scale consecutive
missing data problems. To address this issue, this paper proposes a novel algorithm
called TrAdaBoost-LSTM, which integrates state-of-the-art deep learning theory
through long short-term memory (LSTM) and instance-based transfer learning through
TrAdaBoost. This model inherits the full advantages of the LSTM model and transfer
learning technique, namely the powerful ability to capture the long-term
dependencies among time series and the flexibility of leveraging the related
knowledge from complete datasets to fill in large-scale consecutive missing data. A
case study involving Dissolved Oxygen concentrations obtained from water quality
monitoring stations is conducted to validate the effectiveness and superiority of
the proposed method. The results show that the proposed TrAdaBoost-LSTM model not
only improves the imputation accuracy by 15%~25% compared with that of alternative
models based on the obtained performance indicators, but also provides potential
ideas for similar future research.
Keywords: Water quality; Transfer learning; LSTM; TrAdaBoost; Large-scale
consecutive missing data

Mingyang Li, Zequn Wang,

LSTM-augmented deep networks for time-variant reliability assessment of dynamic
systems,
Reliability Engineering & System Safety,
Volume 217,
2022,
108014,
ISSN 0951-8320,
https://doi.org/10.1016/j.ress.2021.108014.
(https://www.sciencedirect.com/science/article/pii/S0951832021005238)
Abstract: This paper presents a long short-term memory (LSTM)-augmented deep
learning framework for time-dependent reliability analysis of dynamic systems. To
capture the behavior of dynamic systems under time-dependent uncertainties,
multiple LSTMs are trained to generate local surrogate models of dynamic systems in
the time-independent system input space. With these local surrogate models, the
time-dependent responses of dynamic systems at specific input configurations can be
predicted as an augmented dataset accordingly. Then feedforward neural networks
(FNN) can be trained as global surrogate models of dynamic systems based on the
augmented data. To further enhance the performance of the global surrogate models,
the Gaussian process regression technique is utilized to optimize the architecture
of the FNNs by minimizing a validation loss. With the global surrogates, the time-
dependent system reliability can be directly approximated by the Monte Carlo
simulation (MCS). Three case studies are used to demonstrate the effectiveness of
the proposed approach.
Keywords: LSTM; dynamic systems; time-variant reliability; deep learning; Gaussian
Process

Chao Peng, Yifan Tao, Zhipeng Chen, Yong Zhang, Xiaoyan Sun,
Multi-source transfer learning guided ensemble LSTM for building multi-load
forecasting,
Expert Systems with Applications,
Volume 202,
2022,
117194,
ISSN 0957-4174,
https://doi.org/10.1016/j.eswa.2022.117194.
(https://www.sciencedirect.com/science/article/pii/S0957417422005814)
Abstract: Generally, it is difficult to establish an accurate building load
forecasting model by using insufficient energy data. Although the transfer of
knowledge from similar buildings can effectively solve this problem, there is still
a lack of effective methods for both the selection of source domain buildings and
the use of transfer knowledge when many candidate buildings are available. In view
of this, this paper proposes a multi-source transfer learning guided ensemble LSTM
method for building multi-load forecasting (MTE-LSTM). Firstly, a two-stage source-
domain building matching method based on dominance comparison is developed to find
multiple source-domain buildings similar to the target building. Next, an LSTM
modeling strategy combining transfer learning and fine-tune technology is proposed,
which uses multiple source-domain data to generate multiple basic load forecasting
models for the target building. Following that, a model ensemble strategy based on
similarity degree is given to weight the output results of basic forecasting
models. Applications in many real buildings shows that the proposed building multi-
energy load forecasting method can obtain high-precision load forecasting results
when the target building data is relatively few.
Keywords: Building load forecasting; Transfer learning; Multi-source; LSTM

Iman Ranjbar, Vahab Toufigh,

Deep long short-term memory (LSTM) networks for ultrasonic-based distributed damage
assessment in concrete,
Cement and Concrete Research,
Volume 162,
2022,
107003,
ISSN 0008-8846,
https://doi.org/10.1016/j.cemconres.2022.107003.
(https://www.sciencedirect.com/science/article/pii/S0008884622002952)
Abstract: This paper presented a comprehensive study on developing a deep learning
approach for ultrasonic-based distributed damage assessment in concrete. In
particular, two architectures of long short-term memory (LSTM) networks were
proposed: (1) a classification model to evaluate the concrete’s damage stage; (2) a
regression model to predict the concrete’s absorbed energy ratio. Two input
configurations were considered and compared for both architectures: (1) the input
was a single signal; (2) the inputs were four signals from four sides of the
specimen. A comprehensive experimental study was designed and conducted on ground
granulated blast furnace slag-based geopolymer concrete, providing a total number
of 1920 ultrasonic signals from different damage stages. Unsupervised k-means
clustering based on dynamic time warping (DTW) was implemented to cluster the
ultrasonic response signals from the experimental study into five defined damage
stages. The proposed LSTM architectures were successfully trained and validated
using the experimental dataset. Moreover, the performance of the LSTM models was
evaluated in noisy environments. The proposed LSTM models in this study used the
time series of response signals for damage assessment. Therefore, the damage-
sensitive features were automatically extracted by the LSTM layers. For comparison,
a set of linear and nonlinear ultrasonic features were manually extracted from the
response signals as damage-sensitive features, and their sensitivity to damage was
investigated. Artificial neural networks were implemented to combine the extracted
features and perform the same tasks defined for LSTM models. Comparing the two
approaches showed that using the time series of ultrasonic response signals as the
input of LSTM models outperforms the idea of using the manually extracted features.
This study showed that the presented method is efficient, reliable, and promising
for nondestructive evaluation of damage in concrete.
Keywords: Concrete damage assessment; Ultrasonic; Deep learning; Long short-term
memory; LSTM; k-means clustering; Dynamic time warping

Jingxu Lin, Sheng-hua Zhong, Ahmed Fares,

Deep hierarchical LSTM networks with attention for video summarization,
Computers & Electrical Engineering,
Volume 97,
2022,
107618,
ISSN 0045-7906,
https://doi.org/10.1016/j.compeleceng.2021.107618.
(https://www.sciencedirect.com/science/article/pii/S0045790621005504)
Abstract: This paper studies the video summarization task by formulating it as a
sequential decision-making process, in which the input is a sequence of video
frames and the output is a subset of the original frames. Long Short-Term Memory
(LSTM) is a commonly used framework of prior video summarization methods due to
great temporal dependencies modeling ability. However, the frame sequence in the
video summarization task is relatively long, and LSTM can only handle short video
clips of up to 80 frames in length. This paper proposes a novel deep summarization
framework named Deep Hierarchical LSTM Networks with Attention for Video
Summarization (DHAVS) that consists of delicate feature extraction, temporal
dependencies modeling, and video summary generation. Specifically, we employ 3D CNN
instead of 2D CNN to extract spatial–temporal features and design an attention-
based hierarchical LSTM module to capture the temporal dependencies among video
frames. Additionally, we treat video summarization as an imbalanced class
distribution problem and design a cost-sensitive loss function. Experimental
results show that the proposed method has 0.7% ∼ 21.3% and 4.5% ∼ 12.2% improved to
the conventional methods on SumMe and TVSum datasets.
Keywords: Video summarization; Attention mechanism; LSTM; Cost-sensitive learning

Jiateng Yin, Chenhe Ning, Tao Tang,

Data-driven models for train control dynamics in high-speed railways: LAG-LSTM for
train trajectory prediction,
Information Sciences,
Volume 600,
2022,
Pages 377-400,
ISSN 0020-0255,
https://doi.org/10.1016/j.ins.2022.04.004.
(https://www.sciencedirect.com/science/article/pii/S0020025522003280)
Abstract: The construction of an accurate train control model (TCM) is crucial to
the design of automatic train operation and real-time traffic management systems in
high-speed railways. Traditional physical-driven models usually fail to reflect the
“true” dynamics of high-speed trains (HSTs) because of the strong nonlinearity and
uncertainty due to air resistance, frequently switching working conditions, and
variations in external influencing factors such as weather and temperature.
Although some data-driven deep learning models have recently been proposed for
environmental adaptation, they are all “black-box” models, which cannot explain how
the input of the models affects the HST output. To overcome these issues, this
study constructs a novel long short-term memory with lagged information (LAG-LSTM)
model by combining the physical-driven HST model and an “interpretable” deep
learning model. Specifically, our LAG-LSTM model contains three modules: a time-
delay variable module to model the transform delay of control variables, state
variable enhancement module to extract the key features among high-dimensional
input data, and Pre-LSTM module to predict the future train trajectory with given
control variables. We collected field data from the Beijing-Shanghai high-speed
railway and developed a data filter method and a normalization procedure to
overcome the positioning errors of HSTs and construct a standard data set. Finally,
we tested the effectiveness of our LAG-LSTM by comparing it with six deep learning
structures, including fully connected neural network, recursive neural network,
standard LSTM, and LSTM with convolutional layers. The results show that LAG-LSTM
can accurately predict the trajectories of HSTs and outperforms other deep learning
models. Regarding prediction accuracy, LAG-LSTM improved the performance of the
traditional LSTM by 13.5% to 23.3%.
Keywords: High-speed train; Data-driven; Train control model; Time delay; LAG-LSTM;
Trajectory prediction

Zhen Fang, Xu Ma, Huifeng Pan, Guangbing Yang, Gonzalo R. Arce,

Movement forecasting of financial time series based on adaptive LSTM-BN network,
Expert Systems with Applications,
2022,
119207,
ISSN 0957-4174,
https://doi.org/10.1016/j.eswa.2022.119207.
(https://www.sciencedirect.com/science/article/pii/S0957417422022254)
Abstract: Long-short term memory (LSTM) network is one of the state-of-the-art
models to forecast the movement of financial time series (FTS). However, existing
LSTM networks do not perform well in the long-term forecasting FTS with sharp
change points, which significantly influences the accumulated returns. This paper
proposes a novel long-term forecasting method of FTS movement based on a modified
adaptive LSTM model. The adaptive network mainly consists of two LSTM layers
followed by a pair of batch normalization (BN) layers, a dropout layer and a binary
classifier. In order to capture the important profit points, we propose to use an
adaptive cross-entropy loss function that enhances the prediction capacity on the
sharp changes and deemphasizes the slight oscillations. Then, we perform the
forecasting on multiple independent networks and vote on their output data to
obtain stable forecasting result. Considering the temporal correlation of FTS, an
inherited training strategy is introduced to accelerate the retraining procedure
when performing the long-term forecasting task. The proposed methods are assessed
and verified by the numerical experiments on the stock index datasets, including
“Standard’s & Poor’s 500 Index”, “China Securities Index 300” and “Shanghai Stock
Exchange 180”. A substantial improvement of forecasting performance is proved.
Moreover, the proposed hybrid forecasting framework can be generalized to different
FTS datasets and deep learning models.
Keywords: Finance; LSTM; Deep learning

Ümit Atila, Furkan Sabaz,

Turkish lip-reading using Bi-LSTM and deep learning models,
Engineering Science and Technology, an International Journal,
Volume 35,
2022,
101206,
ISSN 2215-0986,
https://doi.org/10.1016/j.jestch.2022.101206.
(https://www.sciencedirect.com/science/article/pii/S221509862200115X)
Abstract: In recent years, lip-reading has been one of the studies whose importance
has increased considerably, especially with the spread of deep learning
applications. In this topic, researchers try to detect what a person says from
video frames without sound. When the previous studies are analysed, it is seen that
automatic lip-reading systems have been developed for various languages such as
Chinese, Korean, English and German. However, these studies reveal that the
development of the system is difficult because lip-reading from video frame images
without audio data depends on many parameters such as light, shooting distance, and
the gender of the person. Lip-reading systems were first developed using classical
machine learning methods. However, especially in recent years, with the popularity
of deep learning applications, this subject has started to be studied more than
before and studies reveal that in general, deep learning-based lip-reading gives
more successful results. Even though there are studies in this field in different
languages, there is no current study and dataset in Turkish. Therefore, this study
aims to investigate the performances of the state-of-the art deep learning models
on Turkish lip-reading. To this aim, two new datasets, one with 111 words and other
with 113 sentences were created using image processing techniques. The model used
in this study to perform lip-reading extracts features from video frames using CNN
based models and performs classification using Bidirectional Long Short-Term Memory
(Bi-LSTM). Results of experiments reveal that, ResNet-18 and Bi-LSTM pair gives the
best results in both word and sentence datasets with accuracy values 84.5% and
88.55%, respectively. It is also observed that, better performances are obtained in
sentence recognition than word recognition in almost every model implemented.
Keywords: Lip-reading; Bi-lstm; Deep learning; Dataset; Turkish

Jingxian Liu, Zulin Wang, Mai Xu,

DeepMTT: A deep learning maneuvering target-tracking algorithm based on
bidirectional LSTM network,
Information Fusion,
Volume 53,
2020,
Pages 289-304,
ISSN 1566-2535,
https://doi.org/10.1016/j.inffus.2019.06.012.
(https://www.sciencedirect.com/science/article/pii/S1566253518306122)
Abstract: In the field of radar data processing, traditional maneuvering target-
tracking algorithms assume that target movements can be modeled by pre-defined
multiple mathematical models. However, the changeable and uncertain maneuvering
movements cannot be timely and precisely modeled because it is difficult to obtain
sufficient information to pre-define multiple models before tracking. To solve this
problem, we propose a deep learning maneuvering target-tracking (DeepMTT) algorithm
based on a DeepMTT network, which can quickly track maneuvering targets once it has
been well trained by abundant off-line trajectory data from existent maneuvering
targets. To this end, we first build a LArge-Scale Trajectory (LAST) database to
offer abundant off-line trajectory data for network training. Second, the DeepMTT
algorithm is developed to track the maneuvering targets using a DeepMTT network,
which consists of three bidirectional long short-term memory layers, a filtering
layer, a maxout layer and a linear output layer. The simulation results verify that
our DeepMTT algorithm outperforms other state-of-the-art maneuvering target-
tracking algorithms.
Keywords: Maneuvering target-tracking; Bidirectional long short-term memory
network; Multiple models; Trajectory database

Nurul Absar, Nazim Uddin, Mayeen Uddin Khandaker, Habib Ullah,

The efficacy of deep learning based LSTM model in forecasting the outbreak of
contagious diseases,
Infectious Disease Modelling,
Volume 7, Issue 1,
2022,
Pages 170-183,
ISSN 2468-0427,
https://doi.org/10.1016/j.idm.2021.12.005.
(https://www.sciencedirect.com/science/article/pii/S2468042721000865)
Abstract: The coronavirus disease that outbreak in 2019 has caused various health
issues. According to the WHO, the first positive case was detected in Bangladesh on
7th March 2020, but while writing this paper in June 2021, the total confirmed,
recovered, and death cases were 826922, 766266 and 13118, respectively. Due to the
emergence of COVID-19 in Bangladesh, the country is facing a major public health
crisis. Unfortunately, the country does not have a comprehensive health policy to
address this issue. This makes it hard to predict how the pandemic will affect the
population. Machine learning techniques can help us detect the disease's spread. To
predict the trend, parameters, risks, and to take preventive measure in Bangladesh;
this work utilized the Recurrent Neural Networks based Deep Learning methodologies
like LongShort-Term Memory. Here, we aim to predict the epidemic's progression for
a period of more than a year under various scenarios in Bangladesh. We extracted
the data for daily confirmed, recovered, and death cases from March 2020 to August
2021. The obtained Root Mean Square Error (RMSE) values of confirmed, recovered,
and death cases indicates that our result is more accurate than other contemporary
techniques. This study indicates that the LSTM model could be used effectively in
predicting contagious diseases. The obtained results could help in explaining the
seriousness of the situation, also mayhelp the authorities to take precautionary
steps to control the situation.
Keywords: COVID-19; LSTM; Recurrent neural networks (RNN); Prediction; RMSE

Wei Li, Denis Mike Becker,

Day-ahead electricity price prediction applying hybrid models of LSTM-based deep
learning methods and feature selection algorithms under consideration of market
coupling,
Energy,
Volume 237,
2021,
121543,
ISSN 0360-5442,
https://doi.org/10.1016/j.energy.2021.121543.
(https://www.sciencedirect.com/science/article/pii/S0360544221017916)
Abstract: The availability of accurate day-ahead electricity price forecasts is
pivotal for electricity market participants. In the context of trade liberalisation
and market harmonisation in the European markets, accurate price forecasting
becomes difficult for electricity market participants to obtain because electricity
forecasting requires the consideration of features from ever-growing coupling
markets. This study provides a method of exploring the influence of market coupling
on electricity price prediction. We apply state-of-the-art long short-term memory
(LSTM) deep neural networks combined with feature selection algorithms for
electricity price prediction under the consideration of market coupling. LSTM
models have a good performance in handling nonlinear and complex problems and
processing time series data. In our empirical study of the Nordic market, the
proposed models obtain considerably accurate results. The results show that feature
selection is essential to achieving accurate prediction, and features from
integrated markets have an impact on prediction. The feature importance analysis
implies that the German market has a salient role in the price generation of Nord
Pool.
Keywords: Deep learning; Electricity price forecasting (EPF); Electricity market
coupling; Feature selection; Long short-term memory (LSTM); The Nord Pool system
price

Haowen Hu, Xin Xia, Yuanlin Luo, Chu Zhang, Muhammad Shahzad Nazir, Tian Peng,
Development and application of an evolutionary deep learning framework of LSTM
based on improved grasshopper optimization algorithm for short-term load
forecasting,
Journal of Building Engineering,
Volume 57,
2022,
104975,
ISSN 2352-7102,
https://doi.org/10.1016/j.jobe.2022.104975.
(https://www.sciencedirect.com/science/article/pii/S2352710222009858)
Abstract: Accurate short-term load forecasting (STLF) plays an important role in
the daily operation of a smart grid. In order to forecast short-term load more
effectively, this article proposes an integrated evolutionary deep learning
approach based on complete ensemble empirical mode decomposition with adaptive
noise (CEEMDAN), improved grasshopper optimization algorithm (IGOA), and long
short-term memory (LSTM) network. First of all, CEEMDAN is used to decompose the
original data into a certain number of periodic intrinsic mode functions (IMFs) and
a residual. Secondly, the nonlinear strategy is used to improve the attenuation
coefficient of GOA, and the golden sine operator is introduced to update the
individual position of GOA. Then the improved GOA is used to optimize the
parameters of the LSTM model, which are the number of hidden neurons and learning
rate. The optimized LSTM is applied to the decomposed modal components. Finally,
the prediction results of each modal component are aggregated to get the real STLF
results. Through comparative experiments, the effectiveness of the CEEMDAN method,
the IGOA method, and the combined model is verified, respectively. The experimental
results show that the integrated evolutionary deep learning method proposed in this
article is an effective tool for STLF.
Keywords: Short-term load forecasting; Complete ensemble empirical mode
decomposition with adaptive noise; Improved grasshopper optimization algorithm;
Long short-term memory

Rogério Luís de C. Costa,

Convolutional-LSTM networks and generalization in forecasting of household
photovoltaic generation,
Engineering Applications of Artificial Intelligence,
Volume 116,
2022,
105458,
ISSN 0952-1976,
https://doi.org/10.1016/j.engappai.2022.105458.
(https://www.sciencedirect.com/science/article/pii/S0952197622004481)
Abstract: Solar panels can generate energy to meet almost all of the energy needs
of a house. Batteries store energy generated during daylight hours for future use.
Also, it may be possible to sell extra electricity back to distribution companies.
However, the efficiency of photovoltaic systems varies according to several
factors, such as the solar exposition at ground levels, atmospheric temperature,
and relative humidity, and predicting the energy generated by such a system is not
easy. This work is on the use of deep learning to predict the generation of
photovoltaic energy by residential systems. We use real-world data to evaluate the
performance of LSTM, Convolutional, and hybrid Convolutional-LSTM networks in
predicting photovoltaic power generation at different forecasting horizons. We also
assess the generalizability of the solutions, evaluating the use of models trained
with data aggregated by geographic areas to predict the energy generation by
individual systems. We compare the performance of deep networks with Prophet in
terms of MAE, RMSE, and NRMSE, and in most cases, Convolutional and Convolutional-
LSTM networks achieve the best results. Using models trained with region-based data
to predict the power generation of individual systems is confirmed to be a
promising approach.
Keywords: Time series forecasting; Photovoltaic power generation; Deep learning;
LSTM; Convolutional neural networks