Scribd
Upload
38 views

University

This document contains student Nguyen Dinh Hoang's answers to two questions on Information Theory Assignment 3. Question 1 examines the mutual information between different sides of coins and dice. The student calculates that the mutual information is log2(2) for a coin and log2(6) for a standard 6-sided die between the top and bottom sides, and log2(3)-1 between the top side and front face of a die. Question 2 considers the entropy H(X) of the number of coin flips X needed to get the first head. The student calculates H(X) to be 2 bits.

Uploaded by

Sudo27
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
38 views

University

This document contains student Nguyen Dinh Hoang's answers to two questions on Information Theory Assignment 3. Question 1 examines the mutual information between different sides of coins and dice. The student calculates that the mutual information is log2(2) for a coin and log2(6) for a standard 6-sided die between the top and bottom sides, and log2(3)-1 between the top side and front face of a die. Question 2 considers the entropy H(X) of the number of coin flips X needed to get the first head. The student calculates H(X) to be 2 bits.

Uploaded by

Sudo27
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

Student Name: Nguyen Dinh Hoang

Student ID: 20021361

Information Theory (INT2044E)


Assignment 3

Question 1 Problem 4
Mutual information of heads and tails.

(a) Consider a fair coin flip. What is the mutual information between the top side and the
bottom side of the coin?

(b) A 6-sided fair dice is rolled. What is the mutual information between the top side and the
bottom side?

(c) What is the mutual information between the top side and the front face (the face most
facing you)?

(a) We have the chain rule: H(B) − H(B|T ) = I(T ; B) = log2 (2)

(b) Same with coin flip: H(B) − H(B|T ) = I(T ; B) = log2 (2)

(c) We use X represent for Top and Y for Front:

H(X) = log2 (6)


If the top is given, so the front will have four equally likely value:

H(X|Y = 1) = log2 (4)

and H(X|Y ) is the average of the H(X|Y = y) values; since they are all equally likely

H(X|Y ) = log2 (4) = 2

Lastly,
I(X; Y ) = H(X) − H(X|Y ) = log2 (6) − 2 = log2 (3) − 1

Question 2 Problem 7
Coin flip. A fair coin is flipped until the first head occurs. Let X denote the number of flips
required.
(a) Find the entropy H(X) in bits. The following expressions may be useful:
∞ ∞
X 1 X r
rn = , nrn =
n=0
1 − r n=0 (1 − r)2

1
If X = n that means Tail occurs n − 1 times and the last flip is Head.
 n−1
1 1
P (X = n) = ×
2 2
And then,
∞  n  n
X 1 1
H(X) = − log2 ( )
n=1
2 2
∞ ∞
−n
n2−n
X X
= (2 log2 (2n ) =
n=1 n=1
∞  n 1
X 1 2
= n = = 2.
n=1
2 (1 − 12 )2

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy