Deep Learning Application for Communication

Engineering
Subject Code: ECE7419

By

Dr. RAM SEWAK SINGH

Associate Professor

Electronics and Communication Engineering Department

School of Electrical Engineering and Computing
Adama Science and Technology University,
Ethiopia, Po. Box: 1888 1
ASTU
 2
Chapter 3: : Deep Learning

Although the neural network is applicable to both classification and regression, it is

seldom used for regression. This is not because it yields poor performance, but
because most of regression problems can be solved using simpler models.

In the application of the neural network to classification, the output layer is usually
formulated differently depending on how many groups the data should be divided
into. The selection of the number of nodes and suitable activation functions for the
classification of two groups is different when using more groups.

ASTU
Neural Network and Classification:
Binary Classification

Multiclass Classification

Binary Classification
 Binary classification neural network, which classifies the input data into one of the
two groups.

 For binary classification, a single output node is sufficient for the neural network.
This is because the input data can be classified by the output value, which is either
greater than or less than the threshold.

 For example, if the sigmoid function is employed as the activation function of the
output node, the data can be classified by whether the output is greater than 0.5 or
not.
 4

ASTU
Multiclass Classification
5

ASTU
 6

The activation functions that we have discussed so far, including the sigmoid
function, account only for the weighted sum of inputs. They do not consider the
output from the other output nodes.

However, the softmax function accounts not only for the weighted sum of the
inputs, but also for the inputs to the other output nodes.

For example, when the weighted sum of the inputs for the three output nodes are 2,
1, and 0.1, respectively, ASTU
 7

The output from the ith output node of the softmax function is calculated as follows:

ASTU
 8

ASTU
 9

ASTU
The neural network model contains a single hidden layer, as shown in Figure 4-13.
As each image is set on a matrix, we set 25 input nodes. 10

In addition, as we have five digits to classify, the network contains five output
nodes. The softmax function is used as the activation function of the output node.

The hidden layer has 50 nodes and the sigmoid function is used as the activation
function.

ASTU
Input: Muticlassification
11

ASTU
 Testing
12

ASTU
 13

Deep Learning is a Machine Learning technique that employs the deep neural
network. As you know, the deep neural network is the multi-layer neural network that
contains two or more hidden layers.

ASTU
 14
The innovation of Deep Learning is a result of many small technical improvements.
This section briefly introduces why the deep neural network yielded poor
performance and how Deep Learning overcame this problem.
The backpropagation algorithm experiences the following three primary difficulties in
the training process of the deep neural network:

Vanishing Gradient

The vanishing gradient in the training process with the back-propagation algorithm
occurs when the output error is more likely to fail to reach the farther nodes. The
back-propagation algorithm trains the neural network as it propagates the output error
backward to the hidden layers.

However, as the error hardly reaches theASTU first hidden layer, the weight cannot be
adjusted. Therefore, the hidden layers that are close to the input layer are not properly
trained.
 15

ASTU
 Overfitting
16
The reason that the deep neural network is especially vulnerable to overfitting is that
the model becomes more complicated as it includes more hidden layers, and hence
more weight.

The most representative solution is the dropout, which trains only some of the
randomly selected nodes rather than the entire network.

It is very effective, while its implementation is not very complex. Figure 5-4 explains
the concept of the dropout. Some nodes are randomly selected at a certain percentage
and their outputs are set to be zero to deactivate the nodes.

The dropout effectively prevents overfitting as it continuously alters the nodes and
weights in the training process. The adequate percentages of the dropout are
approximately 50% and 25% for hidden and ASTU
input layers, respectively.
 17

Another prevailing method used to prevent overfitting is adding regularization

terms, which provide the magnitude of the weights, to the cost function. This method
works as it simplifies the neural network’ architecture as much as possible, and
hence reduces the possible onset of overfitting.
ASTU
Computational Load
18
The last challenge is the time required to complete the training. The number of
weights increases geometrically with the number of hidden layers, thus requiring
more training data. This ultimately requires more calculations to be made.

The more computations the neural network performs, the longer the training takes.
This problem is a serious concern in the practical development of the neural
network.

ASTU
Consider the deep neural network with the three hidden layers, as shown in Figure 5-
6. Each hidden layer contains 20 nodes. The network has 25 input nodes for 19 the
matrix input and five output nodes for the five classes. The output nodes employ the
softmax activation function.

ASTU
 20

function y = ReLU(x)
y = max(0, x);
end

ASTU
 21

ASTU
 22

This section presents the code that implements the dropout. We use the sigmoid
activation function for the hidden nodes. This code is mainly used to see how the
dropout is coded, as the training data may be too simple for us to perceive the actual
improvement of overfitting.

ASTU
 23

ASTU
 Automatic Digital Modulation Detection by Neural Network
The automatic digital modulation detection is a new technology which applied 24 at the
communication receiver to automatically recognize the modulation type of a received signal.

ASTU
Feature Extraction:
25
Two types of feature sets are extracted and combined. These sets of features are
spectral and statistical features.
Spectral Attributes (Features)
 Power spectral density of a signal.

 The standard deviation of the absolute value-centered nonlinear component of the instantaneous phase.

 The standard deviation of direct (non-absolute) value-centered nonlinear component of the instantaneous phase.

 The standard deviation of the absolute value of the normalized instantaneous amplitude of the signal.

 Mean value of the received signal strength on the basis of open distribution called cumulates.

 The variance of the simulated signal.

 Kurtosis of the normalized centered instantaneous amplitude.

 Kurtosis of the normalized centered instantaneous frequency.

Statistical Attributes

1st ,2nd, 3rd and 4th cumulants at zero lag

ASTU
 26

Thank you for your attention!!

ASTU