DC(CA-1)

SECTION-A

Q1) What is data compression?

Ans) Data compression is the process of encoding information using fewer bits than the original representation.
This technique is used to reduce the size of data files, making them easier to store and transmit. There are two
main types of data compression:

1. Lossless Compression: This method reduces file size without losing any information. It works by
identifying and eliminating statistical redundancy. Common algorithms include Huffman coding, Lempel-

Ziv-Welch (LZW), and DEFLATE12.

2. Lossy Compression: This method reduces file size by removing less important information, which can
result in some loss of quality. It is often used for multimedia files like images, audio, and video. Examples

include JPEG for images, MP3 for audio, and MPEG for video12.

Q2) Define Compression ratio?

Ans) The compression ratio is a measure of how much a data compression algorithm can reduce the size of a
file. It is defined as the ratio between the original size of the data and the compressed size. Mathematically, it can
be expressed as:
Compression Ratio=Compressed Size/Original Size
For example, if a file originally takes up 100 MB and is compressed to 25 MB, the compression ratio would be:
Compression Ratio=25 MB/100 MB=4
This means the compressed file is four times smaller than the original file. A higher compression ratio indicates
more effective compression.

Q3) What do you mean by Distortion?

Ans) In the context of data compression, distortion refers to the loss of information that occurs when data is
compressed using lossy compression techniques. This loss results in the decompressed data being an
approximation of the original data rather than an exact replica.

Q4) What do you mean by fidelity and quality?

Ans) Fidelity and quality are terms often used to describe the accuracy and excellence of reproduced data,
especially in the context of media and data compression. Here’s what they mean:

Fidelity

Fidelity refers to the degree to which a copy or reproduction accurately represents the original. In data
compression, high fidelity means that the compressed and then decompressed data closely matches the original
data. Fidelity is crucial in applications where precise replication of the original data is necessary, such as in
medical imaging or scientific data.

Quality

Quality refers to the overall excellence or superiority of the reproduced data. In the context of media, quality
often involves subjective assessments of how good the reproduced image, sound, or video appears or sounds to
human senses. Quality can be influenced by factors such as resolution, color accuracy, and sound clarity.

Q5) What is self information?

Ans) Self-information, also known as surprisal, is a concept from information theory that quantifies the
amount of information gained from observing a particular event. It measures how surprising or unexpected an
event is. The more unlikely an event, the higher its self-information. Mathematically, self-information ( I(x) ) for
an event ( x ) with probability ( P(x) ) is defined as:
I(x)=−log2(P(x))

Key Points:

High Probability Events: If an event is very likely (high probability), its self-information is low because
it is not surprising.
Low Probability Events: If an event is unlikely (low probability), its self-information is high because it is
surprising.

Example:

Consider a fair six-sided die:

The probability of rolling a 1 is ( P(1) = \frac{1}{6} ).

The self-information of rolling a 1 is:
I(1)=−log2(1/6)≈2.585 bits

Q6) Full form of NYT is?

Ans) In the context of data compression, NYT stands for Normalized Yule Walker Transform. This is a
technique used in signal processing and data compression to analyze and model time series data. It helps in
identifying patterns and reducing the amount of data needed to represent the original signal.

Q7) Difference between reconstruction data and original data is called?

Ans)The difference between the reconstructed data and the original data is called distortion.

Q8) The first phase of development of compression algorithms referred to as ?

Ans) The first phase of the development of compression algorithms is referred to as modeling. In this phase, the
goal is to analyze the data to identify and describe any redundancy present. This involves creating a model that

captures the statistical properties and patterns within the data1

Q9) The difference between the data and model is often referred to as?
Ans) The difference between the data and the model is often referred to as the residual or error.

Q10) The average self information associated with random experiment is called?
Ans) The average self-information associated with a random experiment is called entropy. In information
theory, entropy quantifies the expected amount of information produced by a stochastic source of data. It
measures the uncertainty or unpredictability of the random variable’s outcomes.
 SECTION-B

Q1) What do you mean by data Compression? Compare lossless and lossy data compression
techniques in detail.
Ans) Data compression is the process of encoding information using fewer bits than the original representation.
The primary goal is to reduce the size of data files, making them easier to store and transmit. Compression can
be achieved through various algorithms that identify and eliminate redundancy in the data.

Types of Data Compression

There are two main types of data compression: lossless and lossy.

Lossless Data Compression

Lossless compression reduces file size without losing any information. When the data is decompressed, it is
restored to its original form. This type of compression is essential for applications where data integrity is crucial,
such as text files, executable files, and certain types of images.
Common Algorithms:

Huffman Coding: Uses variable-length codes to represent symbols based on their frequencies.
Lempel-Ziv-Welch (LZW): Builds a dictionary of sequences encountered in the data.
DEFLATE: Combines LZ77 and Huffman coding, used in formats like ZIP and PNG.

Example:

Original: AAAAAABBBCCDAA

Compressed: 6A3B2C1D2A

Lossy Data Compression

Lossy compression reduces file size by removing less important information, which can result in some loss of
quality. This type of compression is often used for multimedia files like images, audio, and video, where a
perfect reproduction is not necessary, and some loss of quality is acceptable.
Common Algorithms:

JPEG: Used for images, it reduces file size by discarding less noticeable details.
MP3: Used for audio, it removes frequencies that are less audible to the human ear.
 MPEG: Used for video, it reduces file size by removing redundant frames and compressing the remaining
data.

Example:

Original Image: High resolution, large file size.

Compressed Image: Lower resolution, smaller file size, some loss of detail.

Comparison of Lossy and Lossless Compression

Table

Feature Lossless Compression Lossy Compression

Data Integrity Preserves original data Some data is lost
Use Cases Text files, executables, certain images Images, audio, video
Compression Ratio Generally lower Generally higher
Quality No loss of quality Some loss of quality
Common Algorithms Huffman, LZW, DEFLATE JPEG, MP3, MPEG

Q2) Define the prefix code and UDC code and also determine whether the given code is UDC or not.
1) {0,01,11,111}
2) {0,01,110,111}

Ans) Prefix Code

A prefix code is a type of code system where no code word is a prefix of any other code word. This property
ensures that each encoded message can be uniquely decoded without ambiguity. Prefix codes are also known as
prefix-free codes or instantaneous codes. They are commonly used in data compression algorithms like Huffman

coding12.

Uniquely Decodable Code (UDC)

A Uniquely Decodable Code (UDC) is a code in which every encoded message can be decoded in exactly one
way. All prefix codes are uniquely decodable, but not all uniquely decodable codes are prefix codes. UDCs

ensure that there is a one-to-one mapping between the encoded and decoded messages2.
Determining if the Given Codes are UDC

Let’s analyze the given codes to determine if they are uniquely decodable:

1. Code Set 1: {0, 01, 11, 111}

This set is not a prefix code because “0” is a prefix of “01”.
To check if it is a UDC, we need to ensure that no sequence of code words can be interpreted in
more than one way. However, since “0” is a prefix of “01”, the sequence “01” could be interpreted
as “0” followed by “1” or as “01”. Therefore, this set is not uniquely decodable.
2. Code Set 2: {0, 01, 110, 111}
This set is not a prefix code because “0” is a prefix of “01”.
To check if it is a UDC, we need to ensure that no sequence of code words can be interpreted in
more than one way. Since “0” is a prefix of “01”, the sequence “01” could be interpreted as “0”
followed by “1” or as “01”. Therefore, this set is not uniquely decodable.

Summary

Code Set 1: {0, 01, 11, 111} is neither a prefix code nor a uniquely decodable code.
Code Set 2: {0, 01, 110, 111} is neither a prefix code nor a uniquely decodable code.

Q3) Explain physical model and probability models with example in detail.

Ans)
Physical Model in Data Compression
A physical model in data compression refers to the tangible representation of the data and its structure. This
model is used to understand how data is stored, organized, and accessed at the physical level. It involves the
actual implementation details of the compression algorithm, including how data is encoded, stored, and retrieved.

Example:

Consider the JPEG image compression algorithm:

Discrete Cosine Transform (DCT): This is a physical model that transforms the image data from the
spatial domain to the frequency domain. It helps in identifying the parts of the image that can be
compressed more effectively.
 Quantization: This step reduces the precision of the transformed coefficients, which is a physical
representation of the data reduction process.
Entropy Coding: Techniques like Huffman coding or arithmetic coding are used to further compress the
quantized coefficients by representing frequently occurring patterns with shorter codes.

Probability Model in Data Compression

A probability model in data compression involves using statistical methods to predict the likelihood of different
data patterns. This model helps in designing efficient coding schemes by assigning shorter codes to more
probable events and longer codes to less probable events.

Example:

Consider the Huffman Coding algorithm:

Frequency Analysis: The algorithm starts by analyzing the frequency of each symbol in the data. This
frequency analysis forms the basis of the probability model.
Tree Construction: A binary tree is constructed where each leaf node represents a symbol, and the path
from the root to the leaf represents the code for that symbol. Symbols with higher probabilities are placed
closer to the root, resulting in shorter codes.
Encoding: The data is then encoded using the generated codes, which minimizes the overall length of the
encoded data based on the probability model.

Comparison

Aspect Physical Model in Data Compression Probability Model in Data Compression

Nature Tangible representation of data structure Statistical representation of data patterns
Purpose Understand and implement data storage Predict data patterns for efficient coding
Example JPEG (DCT, Quantization, Entropy Huffman Coding (Frequency Analysis, Tree
Coding) Construction)
Fields of Use Image, audio, and video compression Text and general data compression

Summary

Physical Models: Focus on the actual implementation and storage of compressed data, such as the steps in
JPEG compression.
Probability Models: Use statistical methods to predict data patterns and design efficient coding schemes,
such as in Huffman coding.
Q4) Consider the following sequence: 12123333123333123312. Find the entropy for given sequence.
Ans) To find the entropy of the sequence 12123333123333123312 we need to follow these steps:

Step 1: Count the Frequencies of Each Symbol

Let's tally the occurrences of each unique digit in the sequence:

1: appears 4 times
2: appears 4 times
3: appears 6 times

Step 2: Calculate Total Length of the Sequence

The total length N of the sequence is:

N = 20

Step 3: Calculate the Probability of Each Symbol

Now we calculate the probability P(x) of each symbol x:

- For digit 1:
P(1) = 4/20
= 0.2

- For digit 2:
P(2) = 4/20
= 0.2

- For digit 3:
P(3) = 6/20
= 0.3
Step 4: Apply the Entropy Formula

The entropy H of the sequence can be calculated using the formula:

Now we calculate the entropy contribution from each symbol:

1. For digit 1:
H1 = -P(1) log2(P(1)) = -0.2 log2(0.2)

Using log2(0.2) ≈ -2.32193:

H1 = -0.2 * (-2.32193) ≈ 0.4644

2. For digit 2 :
H2 = -P(2) log2(P(2)) = -0.2 log2(0.2) ≈ 0.4644

3. For digit 3 :
H3 = -P(3) log2(P(3)) = -0.3 log2(0.3)

Using log2(0.3) ≈ -1.73697 :

H3 = -0.3 ≈ (-1.73697) ≈ 0.5211

Step 5: Sum the Contributions

Now we sum these contributions to find the total entropy:

H = H1 + H2 + H3 ≈ 0.4644 + 0.4644 + 0.5211 ≈1.4499

Final Result

The entropy of the sequence 12123333123333123312 is approximately:

1.45