Communication Systems

Unit-1
1. Anjali Gupta (01101182023)
2. Annie Smith (01301182023)
3. Anushka Shanker (01701182023)
4. Areeba Khanam (02101182023)
5. Deepanjali Kumari (03001182023)
 INTRODUCTION TO PROBABILITY THEORY
Till now we have studied signals whose valve at any instant t can be determined by its analytical or
graphical description. These are called deterministic signals, implying complete certainty about their
values at any moment t. Such signals which can be specified with certainty cannot convey
information. If a message to be received is specified (i.e., if it is known beforehand), then It
contains m uncertainty and conveys no new information to the receiver. The higher the
uncertainty about a signal, the higher is it's in formation content.

HENCE SIGNALS THAT CONVEY INFORMATION MUST BE UNPREDICTABLE. In addition to

information bearing signals, noise signals that perturb information signals in a system are also
unpredictable. These unpredictable message signals and noise waveforms are examples of
RANDOM PROCESSES that play key roles in communication systems and their analysis.

Random phenomenon arise either because of our partial ignorance of the generating mechanism
(as in message or noise signals) or because the laws governing the phenomenon may be
fundamentally random (as in quantum mechanics).

We shall begin with a review of the basic concepts of the theory of probability, which forms the
basis for describing RANDOM PROCESSES.
 CONCEPT OF PROBABILITY
To begin the discussion of probability, we must define some basic elements and important terms,

(i) Experiment :- This term is used in probability theory to describe a process whose outcome cannot be fully
predicted because the conditions under which it is performed cannot be predetermined with sufficient accuracy
and completeness. Tossing a coin, rolling a die, and drawing a card from a deck are some examples of such
experiments.
(ii) Outcomes :- An experiment may have several separately identifiable outcomes. For example, rolling a die has six
possible identifiable outcomes (1,2,3,4,5 and 6).
(iii) Event :- An event is a subset of outcomes that share some common characteristics. An event occurs if the
outcome of the experiment belongs to the specific subsets of outcomes defining the event. In the experiment of
rolling a die, for example, the event "odd number on a throw" can result from any one of three outcomes (viz. 1, 3
and 5). Hence this event is a set consisting of three outcomes (1, 3 and S).
(iv)Sample space :- A sample space in probability is the set of all possible outcomes of a random experiment. It is
usually denoted by the symbol ( S ). For example:
- In a coin toss, the sample space is S = {Heads, Tails}
- In rolling a six-sided die, the sample space is S = { 1, 2, 3, 4, 5, 6 }.
- For drawing a card from a standard deck, the sample space consists of 52 cards.
(v) COMPLIMENT :- In probability, the complement of an event refers to all outcomes in the sample space that are
not part of that event. If an event A occurs, its complement, denoted A' or Ā includes all outcomes where A does
not occur. The probability of the complement can be calculated using the formula :
P(A') = 1 - P(A)
where P(A)is the probability of event A . This relationship is useful for finding probabilities of events indirectly.

(vi) UNION :- The union of two events refers to the occurrence of at least one of those events. If you have two
events A and B , the union is denoted as A ∪ B. The probability of the union of two events can be calculated using
the formula:
P(A∪B)=P(A)+P(B)−P(A∩B)
Here , P(A∩B)is the probability that both events occur simultaneously. This formula accounts for the fact that if both
events happen, their probabilities are counted twice, so we subtract the overlap.

(vii)INTERSECTION :- In probability, the intersection of two events refers to the occurrence of both events
simultaneously. If you have two events A and B, the intersection is denoted as A∩B. The probability of the
intersection of two events can be expressed as: P(A∩B)
For independent events, the probability can be calculated using:
P(A∩B)=P(A)×P(B)
For non-independent events, the formula may involve conditional probability:
P(A∩B)=P(A)×P(B∣A)
where P(B∣A) is the probability of B occurring given that A has occurred. The intersection essentially captures the
scenario where both events are true at the same time.
 CONDITIONAL PROBABILITY
Conditional probability is the probability of an event occurring given that another event has already occurred. It is
denoted as P(A | B) , which reads as "the probability of A given B. The formula for conditional probability
is P(A∣B)= P(A∩B) / P (B)
where P(A∩B) is the probability that both events A and B occur, and P(B) is the probability of event B. This concept
is essential for understanding how the probability of an event changes in the context of other known information.

Bayes' theorem;-It provides a way to calculate conditional probability, allowing us to update our beliefs based on
new evidence. It relates the conditional probability of an event A given B to the conditional probability of B given
A , along with their prior probabilities.
Bayes' Theorem Formula is expressed as:- P(A∣B)= P(B∣A)⋅P(A) / P(B)

Components Explained
- P(A | B) : The *posterior probability*—the probability of event A occurring given that event B has occurred.
- P(B | A): The *likelihood*—the probability of event B occurring given that event A is true.
- P(A): The *prior probability*—the initial probability of event A occurring before considering even B.
- P(B): The *marginal probability*—the total probability of event B occurring, calculated as:
P(B)=P(B∣A)⋅P(A)+P(B∣A′)⋅P(A′)
where A' is the complement of A.
 CONDITIONAL PROPERTY OF INDEPENDENT EVENTS
When two events are independent, those conditional probability is the same as the probability of the event
individually i.e., P (A | B) is the same as P(A) as there is no effect of event B on the probability of event A. For
independent events, A and B, the conditional probability of A and B with respect to each other is given as follows:
P(B|A) = P(B)
P(A|B) = P(A)

Properties of Conditional Probability

Property 1: Let’s consider an event A in any sample space S of an experiment.
P(S|A) = P(A|A) = 1
Property 2: For any two events A and B of a sample space S, and an event X such that P(X) ≠ 0,
P((A ∪ B)|X) = P(A|X) + P(B|X) – P((A ∩ B)|X)
Property 3: The order of set or events is important in conditional probability, i.e.,
P(A|B) ≠ P(B|A)
Property 4: The complement formula for probability only holds conditional probability if it is given in the context of
the first argument in conditional probability i.e.,
P(A’|B)=1-P(A|B)
P(A|B’) ≠ 1-P(A|B)
Property 5: For any two or three independent events, the intersection of events can be calculated using the
following formula: For the intersection of two events A and B,
P(A ⋂ B) = P(A) P(B)
For the intersection of three events A, B, and C,
P (A ⋂ B ⋂ C) = P(A) P(B) ©

RANDOM VARIABLES
Random Variable Probability is a mathematical concept that assigns numerical values to outcomes of a sample
space. They can describe the outcomes of objective randomness (like tossing a coin) or subjective
randomness(results of a cricket game).
There are two types of Random Variables- (1) Discrete
(2)Continuous.
A random variable is considered a discrete random variable when it takes specific, or distinct values within an
interval. Conversely, if it takes a continuous range of values, then it is classified as a continuous random variable.
 DEFINITION OF RANDOM VARIABLES
Random variable in statistics is a variable whose possible values are numerical outcomes of a random phenomenon.
It is a function that assigns a real number to each outcome in the sample space of a random experiment. We define
a random variable as a function that maps from the sample space of an experiment to the real numbers.
Mathematically, Random Variable is expressed as ,
X: S →R
where, X is Random Variable (It is usually denoted using capital letter)
S is Sample Space
R is Set of Real Numbers.
Random variables are generally represented by capital letters like X and Y.
This is explained by the example below:
If two unbiased coins are tossed then find the random variable associated with that event.
Solution:
Suppose Two (unbiased) coins are tossed
X = number of heads. [X is a random variable or function]
Here, the sample space S = {HH, HT, TH, TT}
 CUMULATIVE DISTRIBUTIVE FUNCTION
The Cumulative Distribution Function (CDF), of a real-valued random variable X, evaluated at x, is the probability
function that X will take a value less than or equal to x. It is used to describe the probability distribution of random
variables in a table.
In other words, CDF finds the cumulative probability for the given value. To determine the probability of a
random variable, it is used and also to compare the probability between values under certain conditions. For
discrete distribution functions, CDF gives the probability values till what we specify and for continuous distribution
functions, it gives the area under the probability density function up to the given value specified.

Cumulative Distribution Function Formula

The CDF defined for a discrete random variable and is given as Fx(x) = P(X ≤ x)
Where X is the probability that takes a value less than or equal to x and that lies in the semi-closed interval (a,b],
where a < b. Therefore the probability within the interval is written as
P(a < X ≤ b) = Fx(b) – Fx(a)
The CDF defined for a continuous random variable is given as;
 Here, X is expressed in terms of integration of its probability density function fx. In case, if the distribution of the
random variable X has the discrete component at value b,
P(X = b) = Fx(b) – limx→b- Fx(x)

PROPERTIES
The cumulative distribution function Fx(x) of a random variable has the following important properties:
Every CDF Fx is non decreasing and right continuous
limx→-∞Fx(x) = 0 and limx→+∞Fx(x) = 1
For all real numbers a and b with continuous random variable X, then the function fx is equal to the derivative of Fx,
such that

If X is a completely discrete random variable, then it takes the values x1, x2, x3,… with probability pi = p(xi), and
the CDF of X will be discontinuous at the points xi:
FX(x) = P(X ≤ x) =
This function is defined for all real values, sometimes it is defined implicitly rather than defining it explicitly.
The CDF is an integral concept of PDF ( Probability Distribution Function )

Consider a simple example for CDF which is given by rolling a fair six-sided die, where X is the random variable
We know that the probability of getting an outcome by rolling a six-sided die is given as:
Probability of getting 1 = P(X≤ 1 ) = 1 / 6

Probability of getting 2 = P(X≤ 2 ) = 2 / 6

Probability of getting 3 = P(X≤ 3 ) = 3 / 6

Probability of getting 4 = P(X≤ 4 ) = 4 / 6

Probability of getting 5 = P(X≤ 5 ) = 5 / 6

Probability of getting 6 = P(X≤ 6 ) = 6 / 6 = 1

From this, it is noted that the probability value always lies between 0 and 1 and it is non-decreasing and right
continuous in nature.
 PROBABILITY MASS FUNCTION
The Probability Mass Function (PMF) is also called a probability function or frequency function which characterizes
the distribution of a discrete random variable. Let X be a discrete random variable of a function, then the probability
mass function of a random variable X is given by
Px (x) = P( X=x ),
For all x belongs to the range of X
It is noted that the probability function should fall on the condition :
Px (x) ≥ 0 and ∑xϵRange(x) Px (x) = 1
Here the Range(X) is a countable set and it can be written as { x1, x2, x3, ….}. This means that the random variable X
takes the value x1, x2, x3, …

The Probability Mass function is defined on all the values of R, where it takes all the arguments of any real
number. It doesn’t belong to the value of X when the argument value equals to zero and when the argument
belongs to x, the value of PMF should be positive.

The probability mass function is usually the primary component of defining a discrete probability distribution, but it
differs from the probability density function (PDF) where it produces distinct outcomes
PROPERTIES :-
There are three important properties of the probability mass function. With the help of these, the cumulative
distribution function of a discrete random variable can be determined. The probability mass function properties
are given as follows: P(X = x) = f(x) > 0.
This implies that for every element x associated with a sample space, all probabilities must be positive.
∑xϵSf(x)=1
The sum of all probabilities associated with x values of a discrete random variable will be equal to 1.
P(X ∈ T) = ∑xϵTf(x)
The probability associated with an event T can be determined by adding all the probabilities of the x values in T. This
property is used to find the CDF of the discrete random variable.
Probability Mass Function of Poisson Distribution
Poisson distribution is another type of probability distribution. It models the probability that a given number of
events will occur within an interval of time independently and at a constant mean rate. The probability mass
function of Poisson distribution with parameter
λ > 0 is as follows:
P(X = x) = λxeλ / x!
PROBABILITY MASS FUNCTION OF BINOMIAL DISTRIBUTION
Binomial distribution is a discrete distribution that models the number of successes in n Bernoulli trials. These
trials are experiments that can have only two outcomes, i.e, success (with probability p) and failure (with
probability 1 - p). The probability mass function of a binomial distribution is given as follows:
P(X = x) = (nx)px(1−p)n−x

PROBABILITY DENSITY FUNCTION

The Probability Density Function(PDF) defines the probability function representing the density of a continuous
random variable lying between a specific range of values. In other words, the probability density function
produces the likelihood of values of the continuous random variable. Sometimes it is also called a probability
distribution function or just a probability function. However, this function is stated in many other sources as the
function over a broad set of values. Often it is referred to as cumulative distribution function or sometimes as
probability mass function(PMF). However, the actual truth is PDF (probability density function ) is defined for
continuous random variables, whereas PMF (probability mass function) is defined for discrete random variables.
FORMULA :- In the case of a continuous random variable, the probability taken by X on some given value x is
always 0. In this case, if we find P(X = x), it does not work. Instead of this, we must calculate the probability of X
lying in an interval (a, b). Now, we have to figure it for P(a< X< b), and we can calculate this using the formula of
PDF. The Probability density function formula is given as,
 This is because, when X is continuous, we can ignore the endpoints of intervals while finding probabilities of
continuous random variables. That means, for any constants a and b,
P(a ≤ X ≤ b) = P(a < X ≤ b) = P(a ≤ X < b) = P(a < X < b).

Probability Density Function Graph

The probability density function is defined as an integral of the density of the variable density over a given range. It
is denoted by f (x). This function is positive or non-negative at any point of the graph, and the integral, more
specifically the definite integral of PDF over the entire space is always equal to one. The graph of PDFs typically
resembles a bell curve, with the probability of the outcomes below the curve. The below figure depicts the graph of
a probability density function for a continuous random variable x with function f(x).
PROPERTIES
Let x be the continuous random variable with density function f(x), and the probability
density function should satisfy the following conditions:

For a continuous random variable that takes some value between certain limits, say a and b,
the PDF is calculated by finding the area under its curve and the X-axis within the lower limit
(a) and upper limit (b).
Thus, the PDF is given by

The probability density function is non-negative for all the possible values, i.e. f(x)≥ 0, for all x.
The area between the density curve and horizontal X-axis is equal to 1, i.e.

Due to the property of continuous random variables, the density function curve is continued for
all over the given range. Also, this defines itself over a range of continuous values or the domain
of the variable.
 Bayes Rule for Continuous Random Variables
If X and Y are both continuous random variables with joint pdf fX,Y(x, y), we know that
fX,Y(x, y)= fY |X(y|x)
fX(x) = fX|Y (x|y) fY(y)

Thus we can turn a conditional pdf in y, fY |X(y|x) into one for X using
fX|Y (x|y) = fY |X(y|x) fX(x) fY(y)

For a fixed observation of Y = y,

• fX(x) is a function of x (a pdf)
• fY|X(y|x) is a particular function of x determined by y (although not a pdf),
• fY(y) is a number
Using the fact that
∞
fY (y) = −∞ ʃ fY|X(y|u) fX(u)du,

we will often find it useful to rewrite the denominator above to get,

fY|X(y|x) fX(x)
fX|Y (x|y) = ------------------------------
∞
ʃ f
−∞ Y|X(y|u) fX(u) du
 Bayes Rule for Mixed Random Variables
A joint distribution fX,Z(x,z) of continuous variable X∼fX, and discrete variable Z∼pZ,
is defined as any non-negative function of x and z that satisfies

∫fX,Z(x,z)dx = pZ(z)

z∑fX,Z(x,z) = fX(x)
For a given distribution fX,Z, the conditional distributions are defined:
fX,Z(x,z)
pZ∣X(z) ≡ ----------------
fX(x)
and
fX,Z(x,z)
fX∣Z(x) ≡ -------------------
pZ(z).
Note that both expressions satisfy the proper unity condition when you apply
the sum or integral from earlier.

The mixed form of Bayes theorem can be obtained simply by rearranging

the above formulas for the conditional distribution. Rearranging the second
equation for
fX,Z(x,z) and substituting the result into the first equation, you get,

fX∣Z(x).pZ(z)
pZ∣X(z) = ---------------------
fX(x)
 Sum of 2 Independent Random Variables

1. Cumulative Distribution Function of Sum

Let X and Y be two independent random variables. Denote their CDF by FX(x) and
FY(y). Let,
Z=X+Y
and denote the CDF of Z by FZ(z). Then,

FZ(z) = E[FX(z-Y)] or FZ(z) = E[FY(z-X)]

Example:
Let X be a uniform random variable with support RX=[0,1] and probability density function

Let Y be a uniform random variable with support RY=[0,1] and probability density function

The CDF of X is,

The CDF of Z = X + Y is,

There are 4 cases to consider:

1. If z <= 0, then
2. If 0 < z <= 1, then

3. If 1 < z <= 2, then

4. If z > 2, then

By combining these four possible cases, we obtain

 Probability Mass Function of Sum

When the two summands are discrete random variables, the probability mass function (PMF)
of their sum can be derived as follows.
Let X and Y be two independent discrete random variables. Denote their respective PMFs by
pX(x) and pY(y) and their supports by RX and RY . Let

Z=X+Y
and denote the PMF of Z by pZ(z) Then,

Or
Proof:
The two summations above are called convolutions (of two PMFs).
Example:
Let X be a discrete random variable with support RX= {0,1} and probability mass function

Let Y be a discrete random variable with support RY= {0,1} and probability mass function

Define, Z = X + Y
Support, RZ = {0,1,2}

The pmf of Z, evaluated at z=0 is

Evaluated at z=1, it is

Evaluated at z=2, it is

Therefore, the pmf of Z is

 Probability Density Function of Sum
When the two summands are continuous variables, the probability density function (pdf)
of their sum can be derived as follows.
Let X and Y be two independent continuous random variables and denote their respective
pdfs by fX(x) and fY(y). Let
Z=X+Y
and denote the pdf of Z by fZ(z). Then,
Or

Define, Z = X + Y
Support, RZ = [0, ∞)

Proof:
The two integrals above are called convolutions (of two pdfs).
Example:
Let X be an exponential random variable with support RX = [0, ∞) and pdf

Let Y be another exponential random variable, independent of X, with support RY = [0, ∞) and
pdf

Define, Z = X + Y
Support, RZ = [0, ∞)
When z ∈ RZ, the pdf of Z is
Therefore, the pdf of Z is,
 Statistical Averages of Random Variables
Mean:
The mean, expected value, or expectation of a random variable X is written as E(X) or µX. The
expectation is defined differently for continuous and discrete random variables.

Let X be a continuous random variable with p.d.f. fX(x). The expected value of X is
E(X) = −∞ ʃ∞ x.fX(x) dx

Let X be a discrete random variable with probability function fX(x). The expected value of X is

E(X) = x ∑ x.fX(x) = x∑ x.P(X = x)

Properties:

1. Let g and h be functions, and let a and b be constants. For any random variable X (discrete
or continuous),
E[ag(X) + bh(X)] = aE[g(X)] + bE[h(X)]
In particular,
E[aX + b] = aE[X] + b

2. Let X and Y be ANY random variables (discrete, continuous, independent, or non-

independent). Then
E(X + Y ) = E(X) + E(Y )

More generally, for ANY random variables X1, . . . , Xn,

E(X1 + . . . + Xn) = E(X1) + . . . + E(Xn).
3. Let X and Y be independent random variables, and g, h be independent functions. Then,

E[XY] = E[X].E[Y]

E[g(X)h(Y)] = E[g(X)].E[h(Y)]
Variance:

The variance is the mean squared deviation of a random variable from its own mean.
• If X has high variance, we can observe values of X a long way from the mean.
• If X has low variance, the values of X tend to be clustered tightly around the mean value.

Let X be any random variable. The variance of X is

Var(X) = E[(X - µX)2] = E[X2] – (E[X])2

Example:
Let X be a continuous random variable with p.d.f.

Find E(X) and Var(X).

For Var(X), we use
Var(X) = E[X2] – (E[X])2
Now,

Thus,
Properties:
1. Let g be a function, and let a and b be constants. For any random variable X (discrete or
continuous),
Var{ag(X) + b} = a2. Var{g(X)}
In particular,
Var(aX + b) = a2Var(X)

2. Let X and Y be independent random variables. Then

Var(X + Y) = Var(X) + Var(Y )

3. If X and Y are NOT independent, then

Var(X + Y ) = Var(X) + Var(Y ) + 2cov(X, Y )
Moments:
The moment measures how spread out or concentrated the number in a dataset is around the
central value, such as the mean etc.

Four Keys of Moments in Statistics:

1. First Moment – Mean
2. Second Moment – Variance
3. Third Moment - Skewness
4. Fourth Moment – Kurtosis

Out of these four, first two are already discussed above.

3. Skewness:
Skewness is the third moment, which measures the deviation of the given distribution of a
random variable from a symmetric distribution. In simple terms, skewness means the lack of
straightness or symmetry.

Positive Skewness: In positive skewness, the extreme data values are greater, which, in turn,
increases the mean value of the dataset.
In positive skewness: Mode < Median < Mean.
Negative Skewness: In negative skewness, the extreme data values are smaller,
which, in turn, decreases the mean value of the dataset.
In negative skewness: Mean < Median < Mode.

Note: When there is no skewness in the dataset, it is referred to as Symmetrical

Distribution.
In Symmetrical Distribution: Mean = Median = Mode.
4. Kurtosis

Kurtosis is the fourth moment, which measures the presence of outliers in the distribution. It
gives the graph as either heavily-tailed or lightly-tailed due to the presence of outliers. In
simple terms, kurtosis measures the peakedness or flatness of a distribution.

• If the graph has a shorter tail and a flat top, then Kurtosis is said to be high.
• If the graph has a higher peak and lower tail, then the kurtosis is said to be low.
There are three types of Kurtosis:

1. Mesokurtic: This is the same as Normal distribution, i.e., a type of distribution in which
the extreme ends of the graph are similar.
2. Leptokurtic: This distribution indicates that a more significant percentage of data is
present near the tail, which implies the longer tail. Leptokurtic has a greater value of
kurtosis than Mesokurtic.
3. Platykurtic: This distribution indicates that there is less data in the tail portion, which
implies a shorter tail. Platykurtic has a lesser value of kurtosis than Mesokurtic.
 COVARIANCE
Covariance: In communication systems, covariance is used to describe how two signals change together. For
example, the covariance between noise signals in different channels can indicate interference.

Covariance measures the degree to which two random variables (or processes) vary together. If one signal
increases when another increases, their covariance is positive, and if one decreases when the other increases,
their covariance is negative.

Covariance equation : OR

• X,Y : Two random variables (e.g., signals).

• Xi,Yi ​: Individual data points or signal values.
• Xˉ,Yˉ: Mean (average) of the variables X and Y.
• n : Number of data points or signal observations.
.
Covariance in the context of random processes in
communication.
In noise analysis, covariance helps in describing how the noise at different time instances or between different
receivers behaves.

In antenna diversity systems, covariance is used to understand how signals from different antennas (or
different paths in a multipath environment) are related. This can improve signal-to-noise ratio (SNR) in systems
like MIMO.
 Correlation
Correlation is a normalized version of covariance that measures the strength of the linear relationship between two
random variables or signals. In communication systems, correlation is often used for signal detection and to assess the
relationship between received signals at different points (antennas or times).

Correlation equation:

Where:
• Cov(X,Y) is the covariance between the two signals X and Y.
• σX​ and σY​ are the standard deviations of the signals X and Y, respectively.
 Role of correlation in signal processing and
detection
correlation helps measure how similar two signals are over time, which is useful for:

•Auto-correlation: Used in systems to detect repetitive patterns in signals or to synchronize a receiver with the
incoming signal.
.
•Cross-correlation: Applied to measure the similarity between two different signals, which is critical in detection
and decoding processes in communication systems.
 Gaussian Distribution
The Gaussian Distribution is fundamental in communication systems, as noise is often
modeled as Gaussian. It’s also crucial in analyzing signal distortion and for the
performance of modulation schemes like Quadrature Amplitude Modulation (QAM) under
noise.

Equation :

Where x= variable
µ = Mean
𝜎 2 = variance

We are yet to study µ and 𝜎 2. For the time being and for simplicity , we assume that µ = 0 and 𝜎 2= 1 . This is the case of
well known standard gaussian or normal probability density . It has zero means and unit variance .
 Uniform Distribution
•In communication systems, if all values of a signal or noise within a range are equally probable, it follows a uniform
distribution
•A uniform distribution assigns equal probability to all values in a given range.
.
A type of distribution where all outcomes are equally likely. It can be either continuous or discrete.
1.Discrete Uniform Distribution: A finite number of discrete outcomes are equally likely.
2.Continuous Uniform Distribution: All values within a specified range are equally likely.

•Continuous Uniform Distribution: The probability density function (PDF) is constant across the interval [a,b][a,
b][a,b].

Equation:

Where
•x: Random variable representing the signal level or noise amplitude.
•a: Lower bound of the interval (e.g., minimum signal amplitude).
•b: Upper bound of the interval (e.g., maximum signal amplitude).
•f(x): Probability density function (PDF) indicating how likely each value of x is.
Discrete Uniform Distribution: The probability mass function (PMF) is constant across discrete values.

Equation:

Where:
• X is the random variable.
• x1,x2,…,xn are the possible values of X.
• n is the total number of distinct outcomes.
 Rayleigh Distribution
This distribution is used to model data in which a non-negative random variable depends on the magnitude of two
independent normal variables. It is often used in signal processing.

•Probability Density Function (PDF):

•Equation:

Normalized Rayleigh distribution

Where

• x: Signal amplitude in a multipath fading environment (e.g., received signal strength).

• σ: Scale parameter (related to the spread of the distribution).
• f(x): Probability density function (PDF) representing the likelihood of a signal having amplitude x.
 Poisson Distribution
It is a probability model which can be used when the outcome of an experiment is a random variable taking on positive
integer values and where the only information available is a measurement of its average value.

•If we let X = The number of events in a given interval,

Then, if the mean number of events per interval is λ

The probability of observing x events in a given interval is given by

•Equation:
Where λ is the average rate of packet arrivals or errors.

where:
•X= is a discrete random variable representing the number of events,
•Λ= is the average rate (mean) of event occurrences per interval,
•k= is the number of events,
•e= is the base of the natural logarithm, and
•k! = is the factorial of k.
 Multivariate Gaussian Distribution

The extension of the Gaussian distribution to multiple variables, where each variable can be correlated with the others.

The Multivariate Gaussian Distribution is used when multiple signals are jointly Gaussian, such as in MIMO (Multiple
Input Multiple Output) systems, where several antennas are used to transmit/receive signals.

•Equation:

Where:
• x is the vector of variables,
• μ is the mean vector,
• Σ is the covariance matrix,
• ∣Σ∣ is the determinant of the covariance matrix.
 Binomial Distribution
Models the number of successes in nnn independent Bernoulli trials (yes/no outcomes), where the probability of
success is p.
The Binomial Distribution is used to model the number of successes in transmission, for example, the number of
correct bits received out of nnn bits sent over a noisy channel.

Equation:

• P(X=k): The probability of having exactly k successes (correct bits, for example) in n trials.
• n: The number of trials (e.g., bits transmitted).
• k: The number of successful outcomes (e.g., correctly received bits).
• p: Probability of success in each trial (e.g., probability of correct bit reception).
Random Variable
Random Variable - A random variable is a numerical outcome of a random experiment or process. It assigns a
number to each possible outcome of an experiment, allowing us to analyze the probabilities of different results in a
structured way.
A random variable is a function that maps outcomes from a sample space S to the real number line R:
X:S→R
This means that for every outcome ω∈S , the random variable X(ω) assigns a real number.
Ex- Coin toss
Random Process / stochastic Process
Random Process- A Random Process is a collection of Random Variable generally indexed by time.
OR

Random Process- Random variable as a function of time is called Random Process.

Denoted as - X(x,t) or X(s,t) or X(t) where X is the process, x belongs to sample space, and t is the time index.
Random Process is used to study statistical parameters of random signals like noise in communication system
Example:
Consider a simple random process X(t) representing the temperature measured at different times throughout
the day. At each time t, X(t) is a random variable. If we measure the temperature at noon for a week, the
outcome will be different each day, but all those values form a random process.
Discrete and continuous time Process

1. Discrete-Time Processes: These are random processes where time takes discrete
values (e.g., t=1,2,3,…).
Example: Think of the daily stock price measured at the end of each day. Let Xn be the
daily stock price where n is the day number. So, X1 ,X2 and so on represent prices on
different days.
1. Continuous-Time Processes: These are random processes where time is continuous
(e.g., t=0.1,0.5,1.0,…).
Example:A good example is temperature readings, which change continuously over
time.
X(t) might represent the temperature at any time t.
Statistical Properties Of Random
Process
Stationary Random Process

Stationarity-A process is stationary if its statistical properties do not change with time. Stationarity
comes in two main types:
Strict-Sense Stationary (SSS)- A process X(t) is strict-sense stationary if the joint distribution of
[X(t1),X(t2),…,X(tn)] is the same as the joint distribution of[X(t1+τ),X(t2+τ),…,X(tn+τ)] for any shift τ
This means that if we shift the process in time, its statistical behavior stays the same.
Wide-Sense Stationary (WSS)-
A process is wide-sense stationary if:
1. The mean is constant for all t.
2. The autocorrelation function only depends on the time difference: Rx(t1,t2)=Rx(t1−t2)
Autocorrelation Function of A WSS process and Its
Properties
For a wide-sense stationary process, the autocorrelation function Rx(τ) (which only depends on τ=t1−t2)
has the following properties:
1. Symmetry: Rx(τ)=Rx(−τ)

1. Maximum at Zero: Rx(0) is the largest value of the autocorrelation function.

1. Decreases with ∣τ∣ : The autocorrelation function typically decreases as the time lag τ increases.
Numericals
CROSS POWER SPECTRALDENSITY
Definition:

The cross power spectral density or cross power spectrum 𝑆𝑋𝑌(𝜔) of two

continuous random process {X(t)} and {Y(t)} is defined as the Fourier

Transform of 𝑅𝑋𝑌(𝜏) (i.e) 𝑆𝑋𝑌(𝜔) = F[𝑅𝑋𝑌(𝜏)]

Hence the inverse Fourier transform of 𝑆𝑋𝑌(𝜔) is 𝑅𝑋𝑌(𝜏)

PROPERTIES OF CROSS POWER SPECTRALDENSITY

If 𝑆𝑋𝑌(𝜔) is the cross power spectral density of two random processes {

X(t)} and {Y(t)} , then the following properties hold:

(𝑖)𝑆𝑋𝑌(𝜔) = 𝑆𝑌𝑋(−𝜔)
(𝑖𝑖)𝑆𝑋𝑌(−𝜔) = 𝑆∗𝑋𝑌(𝜔)

(𝑖𝑖𝑖)𝐼𝑓 {𝑋(𝑡)} 𝑎𝑛𝑑 {𝑌(𝑡)}𝑎𝑟𝑒 𝑜𝑟𝑡ℎ𝑜𝑔𝑜𝑛𝑎𝑙 , 𝑡ℎ𝑒𝑛 𝑆𝑋𝑌(𝜔) = 0

Proof:
 Replacing 𝜔 by −𝜔, 𝑤𝑒
𝑔𝑒𝑡

𝑃𝑢𝑡 𝑢 = −𝜏 ⇒ 𝜏 = −𝑢, 𝑑𝜏 = −𝑢

When 𝜏 = −∞, 𝑢 = ∞𝑎𝑛𝑑 𝑤ℎ𝑒𝑛 𝜏 = ∞, 𝑢 = −∞

𝑅𝑒𝑝𝑙𝑎𝑐𝑒 𝑢 𝑏𝑦
𝜏

(∵ 𝑅𝑌𝑋(−𝜏) = 𝑅𝑋𝑌(𝜏))

∴ 𝑆𝑋𝑌(𝜔) = 𝑆𝑌𝑋(−𝜔)
 (∵ [𝑅𝑋𝑌(𝜏)]∗ = 𝑅𝑌𝑋(−𝜏) = 𝑅𝑋𝑌(𝜏))

(𝑖𝑖𝑖)𝐼𝑓 {𝑋(𝑡)} 𝑎𝑛𝑑 {𝑌(𝑡)}𝑎𝑟𝑒 𝑜𝑟𝑡ℎ𝑜𝑔𝑜𝑛𝑎𝑙 , 𝑡ℎ𝑒𝑛 𝑅𝑌𝑋(𝜏) = 𝑅𝑋𝑌(𝜏) = 0

Now
Gaussian Theorem
Gaussian Random Variable
A continuous random variable with pdf of the
form

p , −∞ < x < .

∞,the
where µ is 0.3

mean and σ2 is the .

variance
– .
– –
Notation
• N(µ,σ2) denotes a Gaussian distribution with mean µ and
variance
σ2
• X ∼N(µ,σ2) ⇒X is a Gaussian RV with mean µ and variance
σ2
• X ∼N(0, 1) is termed a standard Gaussian RV

Affine Transformations Preserve Gaussianity

Theorem
Remarks
If X is Gaussian, then aX + b is Gaussian for a, b ∈R.
• If X ∼N(µ,σ2), then aX + b ∼N(aµ + b, a2σ2). • If
X∼
N(µ,σ2), then X−µ ∼N(0, 1).

CDF andCCDF ofStandardGaussian

 Φ(x) = P [N(0, 1) ≤ x] = dt
• Complementary cumulative distribution
function
Q dt

Properties of
Q(x)
• Φ(x) + Q(x) =
1

• Q(−∞) = 1

• X ∼N(µ,σ2)
P
Jointly Gaussian Random Variables
Definition (Jointly Gaussian RVs)
Random variables X1, X2,..., Xn are jointly Gaussian if any non-trivial
linear combination is a Gaussian random variable. a1X1 + ··· + anXn is
Gaussian for all (a1,..., an) ∈Rn \ 0
Example (Not Jointly Gaussian)

X ∼N(0, 1)

Y
Y ∼N(0, 1) and X + Y is not Gaussian.

Gaussian Random Vector

Definition (Gaussian Random Vector)
A random vector X = (X1,..., Xn)T whose components are
jointly Gaussian.
Notation

X ∼N(m,C) where m = E[X], C = E h(X − m)(X − m)T i

Definition (Joint Gaussian Density)

If C is invertible, the joint density is given by p
 Uncorrelated Random Variables
Definition

X1 and X2 are uncorrelated if cov(X1, X2) = 0

Remarks
For uncorrelated random variables X1,..., Xn, var(X1 + ···
+ Xn) =

var(X1) + ··· + var(Xn).

If X1 and X2 are independent, cov(X1, X2) = 0.

Correlation coefficient is defined as cov(X1, X2) ρ(X1,
Complex Gaussian Random Variable
Definition (Complex Random Variable)
A complex random variable Z = X + jY is a pair of real random
variables
X and Y.
Remarks
• The pdf of a complex RV is the joint pdf of its real and

imaginary parts.
• E[Z] = E[X] + jE[Y]
• var[Z] = E[|Z|2] − |E[Z]|2 = var[X] + var[Y]

Definition (Complex Gaussian RV)

If X and Y are jointly Gaussian, Z = X + jY is a complex Gaussian
Complex Random Vectors
Definition (Complex Random Vector)
A complex random vector is defined as Z = X + jY where X
and Y are
real random vectors having dimension n × 1. • There
are four matrices associated with X and Y
 h i
CX = E (X − E[X])(X −
E[X])T
E (Y− E[Y])(Y
= −
CY
E[Y])T i h
E(X− E[X])(Y
= −
CXY
E[Y])T i h
E (Y− E[Y])(X
• The Cpdf = Z −is the joint pdf of its real and imaginary
YX of
parts E[X])T i h
i.e. the pdf of

Z˜
Markov Chain
 A Markov process is a stochastic process that satisfies the Markov property
(sometimes characterized as
"memorylessness") it is a process for which predictions can be made regarding future
outcomes based solely on its
present state This means that, conditional on the present state of the system, its
future and past states

are independent.
 A Markov chain is a type of Markov process that has
either a discrete state space or a discrete index set
(often
representing time, a Markov chain as a Markov
process in either discrete or continuous time with a
countable state space but it is also common to define
a Markov chain as
having discrete time in either countable or continuous
state space (thus regardless of the state space).
 The probability that n+1th steps will be x depends only
on the nth steps not the complete sequence of steps
Types of Markov Chain :

discrete-time Markov chains : This implies the index set T( state

of the process at time t ) is a countable set or changes occur at
specific the term “Markov chain” is used for DTMC.
states.
Generally,
continuous-time Markov chains: the index set T( state of the
process at time t ) is a continuum, which means changes are
continuous in CTMC.
Properties of Markov Chain :

1. A Markov chain is said to be Irreducible if we can go

from one state to another in a single or more than one
step.
2. A state in a Markov chain is said to be Periodic if
returning to it requires a multiple of some integer larger
than 1, the greatest
common divisor of all the possible return path lengths will
be the period of that state.

3. A state in a Markov chain is said to be Transient if there is a

non- zero probability that the chain will never return to the
same state, otherwise, it is Recurrent.

4. A state in a Markov chain is called Absorbing if there is no

possible way to leave that state. Absorbing states do not
have any outgoing transitions from it.
Application of Markov Chain :
1. MCMC(Markov Chain Monte Carlo), which gives a solution to the
problems that come from the normalization factor, is based on Markov
Chain.

2. Markov Chains are used in information theory, search engines, speech

recognition etc.

3. Markov chain has huge possibilities, future and importance in the field of
Data Science and the interested readers are requested to learn this stuff
properly for being a competent person in the field of Data Science.
 Representation Of Markov
Chain
1. +——+——+

• | A | E |— Each element denotes the probability

weight of the edge
• 2. +——+——+——+ connecting the two corresponding
vertices
• | A | 0.6 | 0.4 | — 0.4 is the probability for
state A to go to state E and 0.6 is the probability
• 3. +——+——+——+ to remain at the same state
• | E | 0.7 | 0.3 | — 0.7 is the probability for
state E to go to state A and 0.3 is the probability
The Poisson process
The Poisson process can be used to model the number of
occurrences of events, such as patient arrivals at the ER,
during a certain period of time, such as 24 hours, assuming
that one knows the average occurrence of those events over
some period of
time. For example, an average of 10 patients walk into the
ER per hour.

The sub-structure of the Poisson

process
The Poisson process has the following properties:

1. It is made up of a sequence of random variables X1,

X2, X3,
…Xk such that each variable represents the number of
occurrences of some event, such as patients walking into
an ER, during some interval of time.
2. It is a stochastic process. Each time you run the Poisson
process, it will produce a different sequence of random
outcomes as per

some probability distribution which we will soon see.

3. It is a discrete process. The Poisson process’s
outcomes are the number of occurrences of some event
4. It has independent increments. What this is means is that the

number of events that the process predicts will occur in any given

interval, is independent of the number in any other disjoint

interval. For e.g. the number of people walking into the ER from

time zero (start of the observation) up through 10am, is

independent of the number walking in from 3:33pm to 8:26pm, or from

11:00pm to 11:05pm and so on.

5. The Poisson process’s constituent variables X1, X2, X3,…Xk all
have identical distribution.

6. The Poisson process’s constituent variables X1, X2, X3,…Xk all have a
Poisson distribution, which is given by

the Probability Mass Function:

 the probability of occurrence of k events in unit time, given that the average
occurrence rate is λ events per unit time.

The following 4 plots show the shape of the PMF for different values
of λ:

P(k) for λ = 0.1, 1.0, 5 and 20 events per unit

the sum of probabilities for all possible values of k is
always 1.0
Probability of k arrivals in one hour given that an average of 5 patients walk into the ER per hour
Modeling inter-arrival times
Even though the number of occurrence of events is modeled using
a
discrete Poisson distribution, the interval of time between
consecutive
events can be modeled using the Exponential
distribution,which is a continuous distribution.
Let X1, X2, X3,…etc. be random variables such that:
X1 = the interval of time between the start of the process and
the 1st event, i.e. 1st arrival,
X2 = the inter-arrival time between the first and the second arrival,
X3 = the inter-arrival time between the second and the third
arrival, and so forth.
The distribution of random variable Xk which represents the
Inter-arrival times are exponentially
distributed
The Probability Density Function (PDF) of the random variable
Xk is as follows:

PDF of inter-arrival times in a Poisson

process
And the Cumulative Distribution Function
(CDF) is:

CDF of interval-arrival times in a Poisson

process
probability values from the continuous
uniform distribution Uniform(0,1).
The table of patient inter-arrival times in hours at the ER for the
first 10 patients. We have generated this date using the above
formula,
with λ set to 5 patients per hour.
λ = 5 arrivals per
hour
Modeling the arrival times in a Poisson
process
To generate inter-arrival times, it is easy to generate the
patient arrival times.

From the table of 10 sample inter-arrival times shown

above, we can deduce the following:
Arrival time of first patient = x1 =

0.431257556 Arrival time of second

patient =

x1 + inter-arrival time between first and second patient =

x1 + x2 = 0.431257556 + 0.264141966 = 0.6954

Arrival time of third patient =

x1 + x2 + x3 = 0.431257556 + 0.264141966 + 0.190045932
= 0.885445

… and so on
Keeping in mind that X1, X2, X3,…Xk are the inter-arrival
patient arrival times at the ER,

T1 = X1
T2 = X1 + X2
T3 = X1 + X2 + X3
Tk = X1 + X2 + X3 + … + Xk
since T1, T2, T3…Tk are defined as linear combinations of random
variables X1, X2, X3,…Xk, the variables T1, T2, T3,…Tk are also
random variables.

Since T1, T2, T3…Tk are each the sum of exponentially distributed
random variables X1, X2, X3,…Xk, the random variables T1, T2,

T3,…,Tk follow the Gamma distribution.

Spectral factorization
theorem
Ergodicit
y:
Importance of Ergodicity :
 Statistical Stability: In ergodic systems, time
averages and ensemble averages converge, which
means that long-term
behavior can be reliably predicted from short-term
observations. This is essential for analyzing communication
channels where we need to understand how signals
behave over time.
 Resource Allocation: Ergodicity allows for efficient
resource
allocation in communication networks. When a system is
ergodic,
it can be assumed that users will experience similar
performance over time, enabling better planning and
 ergodic capacity considers the average throughput of a channel
over time, which is vital for system design.
 Channel Modeling: In wireless communication, channels often exhibit
variations due to factors like fading and interference. Understanding ergodic
properties helps in developing accurate models that can predict channel
behavior, leading to improved modulation and coding schemes.
 Data Compression and Transmission: Ergodicity supports techniques like
source coding and data compression, where the statistical properties of the data
can be leveraged. Knowing that certain patterns will repeat over time helps in
designing more efficient encoding schemes.
 Adaptability: Systems that exhibit ergodic behavior can adapt to changing
conditions. For example, adaptive modulation and
 coding techniques rely on the ergodic nature of the channel to
optimize performance dynamically.
 Error Analysis: In the context of error-correcting codes,
ergodicity allows for a better understanding of how errors will be
distributed over time, which is essential for designing robust
communication systems that can recover from errors effectively.