||Shri Hari||

BCSC0011: THEORY OF
AUTOMATA & FORMAL
LANGUAGES

Prof. Charul Bhatnagar

charul@gla.ac.in
WA: 7055510537
 What are Regular Languages?
• Many simple
▪ Languages languages are non
accepted by regular:
Finite Automata
{anbn | n = 0, 1, 2, …}
are regular
languages {w | w is a palindrome}
▪ Finite Automata
accept only ▪ and there is no finite
Regular automata that
Languages accepts them.
Context-free languages are a larger class of
languages that encompass all regular languages and
many others, including the two above.
2
Push Down Automata

3
Module II (Part 1)
 Context Free Grammar and Context Free Languages:
Introduction, Derivation Trees
 Ambiguity in Grammar, Ambiguous to Unambiguous CFG
 Simplification of CFGs
 Normal Forms for CFGs - CNF and GNF
 Pumping lemma for CFLs
 Equivalence of PDA and CFG

4
Formal Definition
Context Free
Grammar All productions in P are
of the form

G = (V, Σ, P, S)
Finite Set
A 
of
Variables /
Non Variable
Terminals
Start
Finite Set of variable String of
terminal variables and
symbols terminals
Set of
Productions
rules
 Language
generated by
Example of Context-Free Grammar this
grammar?
S  aSb | 
P {S  aSb, S  }
productions
G V ,  , S, P 

V {S }  {a, b} start variable

variables terminals
6
Context Free Language
▪ Given a context-free grammar G = (V, , P, S), the
language generated or derived from G is the set:

L(G ) {w : S  w, w   *}

▪ A language L is context-free if there is a context-

free grammar G = (V, , P, S), such that L is
generated from G

7
What languages are generated by the
CFG which have the following
productions?

 S  aSa | bSb | λ

Palindromes of even length

 S  aSa | bSb | a | b

Palindromes of odd length

 S  aS | bS | λ

All strings over  = {a, b} i.e. (a + b)*

8
What languages are generated by the
CFG which have the following
productions?

 S  aS | bS | a

All strings that end with a

 S  SS | bS | a

All strings that end with a

 S  aSa | bSb | aXb | bXa X  aX | bX |
λ

All strings which are not palindromes

9
 Find CFGs which generate the
following languages
 The set of odd-length strings in {a, b}* with
middle symbol a.
S  XSX | a Xa|b
 The set of even-length strings in {a, b}* with
the two middle symbols equal.
S  XSX | aa | bb Xa|b
 The set of odd-length strings in {a, b}* whose
first, middle, and last symbols are all the same.
S  aAa | bBb A  XAX | a B  XBX | b Xa
10
|b
Find CFGs which generate the
following languages
 {anbn+1 | n > 1}

S  aSb | aabbb

 {ai bj ck | i = j + k}

S  aSc | aXb | λ X  aXb | λ

 {ai bj ck | j = i + k}

S  XY X  aXb | λ Y  bYc | λ

11
 Derivation Trees

▪ Given a derivable string, we may represent the

derivation by a derivation / parse tree.
□ At the root node is the start symbol S.
□ At each node, the children are the symbols on
the right hand side of the production rule
that was used.

▪ Two derivations are equivalent if they have the

same parse tree. 12
Example of a S
Derivation Tree AB
Þ aAbB
▪ Let the grammar G be Þ aaAbbB
Þ aabbB
□ S → AB Þ aabbbB
Þ aabbbbB
□ A → aAb | λ
Þ aabbbb.
□ B → bB | λ
▪ Let the string be w = S AB AbB AbbB Abb
aabbbb. aAbbb aaAbbbb 
aabbbb.
▪ Derive the string w.
13
 Now draw the
Parse Tree of S AB aAbB  aaAbbB
Þ aabbB  aabbbB
aabbbb Þ aabbbbB  aabbbb.
S
S AB AbB AbbB Abb
aAbbb aaAbbbb 
A B aabbbb.

a A b b B
This is the parse tree of
a A b b B many different, but
equivalent, derivations
λ of aabbbb.
λ
14
Consider G whose productions

are

S  aabbaa
S  aAS | a, A  SbA | SS |ba.
Show that

Construct a derivation tree

whose yield is aabbaa.

S  aAS  aSbAS  aabAS  aabbaS  aabbaa

15
Leftmost and Rightmost
Derivations
▪ A leftmost derivation of a string is a
derivation in which each production rule is
applied to the leftmost non terminal in the
string.

▪ A rightmost derivation of a string is a

derivation in which each production rule is
applied to the rightmost non terminal in
the string.
16
Ambiguous Grammars

 If there exists w  L(G) such that w has

two or more different
 parse trees or
 leftmost (or rightmost) derivations ,
 then G is ambiguous.

17
 Let us consider this grammar : E  E+E | id
 We can create 2 parse tree from this grammar to obtain the
string id+id+id

+
+

+ +

18
Consider the grammar G with production rules −

S → S+S | S*S |S| a

Find two Left Most Derivations of the string ”a+a*a”

Derivation 1 : S → S+S → a +S → a+ S*S → a+a*S → a+a*a

Derivation 2 : S → S*S → S+S*S → a+ S*S → a+a*S → a+a*a

19
If G is the grammar S  SbS | a, show that G is
ambiguous.

20
▪ Ambiguity can be There are some inherently
removed by rewriting ambiguous languages like
the grammar such that L = {anbncm} ∪ {anbmcm}
there is only one
derivation or parse
tree possible for a
string of the language
which the grammar
E.g.
represents.
 Ambiguous Grammar
S  SbS | a
 Can be rewritten as
S  abS | a
21
Simplification
of CFGs

22
 CFG simplification & restriction useful
theoretically & pragmatically.
□ Simpler grammars are easier to understand.
□ Simpler grammars can lead to faster parsing.
□ Restricted forms useful for some parsing
algorithms.
□ Restricted forms can give you more knowledge
about derivations.
23
 Context Free Grammar can be
simplified / reduced by:
 production elimination
 Unit production elimination
 Useless production elimination
24
 Elimination of Null Productions
 If λ  L, we can remove all productions of the
type
Aλ
 If λ  L,
 we obtain a new G by adding to V the new
variable S0, making S0 the start variable, and
adding to P the productions : S0  S | λ
 let G′ = (V, T, S, P ) be a context-free grammar
for 25
Step 1: Construction of S  aS | AB
the set W of all Aλ|a
nullable variables
Bλ|b
 W1={A ∈ V | A → λ is a
D  b | aAbB
production in P}
 Wi+1= Wi ∪{B ∈ V |  a
production B → α with W1={A, B}
α ∈ W i* } W2={A, B, S}
 At some point Wi+1= Wi W3={A, B, S}
= W2 26
 S  aS | AB
Step 2: Construction of
P′ Aλ|a Bλ
|b
 All prodns whose RHS
doesn’t have any D  b | aAbB
Aa
nullable variables are Bb
included in P′
Db
 If A  X1 X2 … Xk is in P
S  aS | AB
A  1include
then  2  k where
Xi if X i  W S  aS | a | AB | A | B
 i 
 or X i if X i  W D  aAbB
Pr ovided A  1 2  k  D  aAbB | abB | aAb | 27
 S  aS | AB
Are we done?
Aλ|a Bλ
 Does λ  L? |b
 S  W, so λ  L D  b | aAbB
Aa
 So, 1 more variable &
Bb
2 more prodns
Db
S0  S | 
S  aS | AB
S  aS | a | AB | A | B
Total 14
D  aAbB
prodns
D  aAbB | abB | aAb | 28
Eliminate
The CFG will have
Null total 40 prodns!!
Productions Ca
S  ABCBCDA  A  CD | C | D

A  CD  B  Cb | b

B  Cb  D  bD | b

Ca|λ  How many S

productions?
D
W=bD | λD,
{C,
 32 S prodns 29
A}
 Elimination of Unit Productions
A  B where A, B  V
Step 1: Construction of S  A | bb
the set of variables
AB|b W(S) ={S, A, B}
derivable from A
BS|a W(A) ={A, B, S}
 W1 (A)={A}
W(B) ={B, S, A}
 Wi+1 (A)= Wi (A) ∪ {C ∈ W1(S) ={S}
V| W2(S) ={S, A}
there is a production
B → C & B  Wi(A) } W3(S) ={S, A, B} =
W4(S)
 At some point 30
Wi+1(A)= Wi(A) W(S) ={S, A, B}
 S  A | bb W(S) ={S, A, B}
Step 2: All non unit AB|b W(A) ={S, A, B}
prodns are included as BS|a W(B) ={S, A, B}
such
S  bb
Ab Hence prodn s
Step 3: Add prodn A  
whenever Ba
in G′ are:
S  a | b | bb
 B  W(A) and
A  a | b | bb
 B   is in P Sb|a
and B  a | b | bb
A  bb | a
   V
B  bb | b
31
 Eliminate Unit Productions
Step 2: All non unit prodns are included as
S → 0A | 1B | C such
S → 0A | 1B
A → 0S | 00
A → 0S | 00
B→1|A
B→1
C → 01 C → 01
Step 1: Construction of the
set of derivable variables Step 3: Add prodn to handle unit
productions Total 9
W(S) = {S, C}
W(A) = {A} S → 01 production
W(B) = {B, A} s
W(C) = {C} B → 0S | 00
32
Useless
Productions
▪A non-terminal X is Derive some strings
useless if:
□ X does not generate S  aSb | c | Ac
any string of terminals. A  aA
▫ It cannot derive a
terminal string B  bB | b
OR
□ X does not occur in any Any production
sentential form involving a useless
▫ It cannot be reached from symbol is a useless
start symbol 33 production.
Reduced Grammar / Elimination
of Useless Productions
▪ There should not be any Null
Productions
▪ There should not be any Unit
Productions

Then we find Reduced Grammar in Two Phases

1. Find G′, from the CFG, G, such that each
variable derives some terminal string.
2. Find G″, from the CFG, G′, such that each symbol 34
Reduced Grammar

Phase 1 −
Derivation of an
equivalent
grammar, G′, from
the CFG, G, such
that each variable
derives some 35
Construction of V′:
 We define Vi′ ⊆ V by recursion
 V1′={A ∈ V | there exists a production A → α
where α ∈ Σ*}
 Vi+1′=Vi′ ∪ {B ∈ V| there exists some production B
→β
with β ∈ (Σ ∪ Vi′)*}
 At some point Vk′=Vk+1′. Then we get V′=Vk′
 Construction of P′ 36
Let G contain following productions
S →AB A →a B →b B →C E→c

Phase 1: Find G′ (from G) such that every variable in

G′ derives some terminal string.
V1′ = {A, B, E} A →a, B →b, E → c
V2′ = {A, B, E, S} S →AB
V3′ = {A, B, E, S}
= V 2′
Here we get V′={S,A,B,E}.
So P′= S →AB, A →a, B →b and E → c
37
Phase 2 − Derivation of an equivalent grammar, G″, from the
CFG, G′, such that each symbol appears in a
sentential form.

Construction of V″:
 We define Vi″ ⊆ V by recursion
 V1″={S}
 Vi+1″=Vi″ ∪ {X ∈ V′| there exists some production A →α
with A ∈ Vi″ & α contains the symbol X}
 At some point Vk″=Vk+1″. Then we get V″=Vk″
38
 Construction of P″
 We had G as S →AB, A →a, B →b, B →C, and E
→c
Then we got G′ as S →AB, A →a, B →b and E
→ c 2: Find G′′ such that every variable in G′
Phase
appears in a sentential form.
V1″ = {S}
V2″ = {S, A, B} S →AB
V3″ = {S, A, B} A →a, B →b

= V 2″
Here we get V″={S,A,B}.
So P″= S →AB, A →a, B →b
39
Find reduced grammars equivalent to the following
1) S →aAa, A →Sb | bCC | DaA, C →abb | DD, E →ac, D
→aDa
2) S →AB | CA, B →BC| AB, A →a, C →aB | b
What would
Answers: happen if you
reverse the order:
1) S →aAa, A →Sb|bCC, C →abb i.e., test
2) S →CA, A →a, C → b reachability
before
generating?
40
1. Give context-free grammars that generate the following
languages

• { ai bj ck | i, j, k ≥ 0 and i = j or i = k }
G = (V, Σ, P, S) with set of variables V = {S, B, X, Y, Z},
where S is the start variable; set of terminals Σ = {a, b,
c}; and rules in P are
S  XY | Z
X  aXb | 
Y  cY | 
Z  aZc | B
B  bB | 
41
 S  XY | Z
2. Now find the Reduced Grammar
X  aXb | 
Y  cY | 
Z  aZc | B
B  bB | 

42
The language A of strings of properly balanced
left and right brackets: every left bracket can
be paired with a unique subsequent right
bracket, and every right bracket can be paired
with a unique preceding left bracket. Moreover,
the string between any such pair has the same
property.
For example,
G = (V, [ ] [set
Σ, R, S) with [ [ of
] [variables
]][]]∈ A.{S}, where S is
V=
the start variable; set of terminals Σ = {[, ]}; and rules
S → ε | SS | [S]

43
Normal Forms
₪ Chomsky Normal Form (CNF)

₪ Griebach Normal Form (GNF)

44
 Advantages of CNF

 Parsers can use binary trees

 Ambiguity is removed

 Exact length of derivations is known

 For a string of length n, the derivations are 2n – 1.

 Some proofs become easier if grammar is in

CNF 45
Chomsky Normal
Form
▪ A CFG is in CNF if all
the prodns are in any
one of the following
forms
□ A→a
□ A → BC
□S→ε
where A, B, and C are
non-terminals and a is 46
 Conversion to CNF
Step 1 : Find the reduced grammar G′ by
eliminating ε , Unit & useless
productions
L(G′) = L(G) – {ε}
Step 2 :If ε  L, then
o create a new start symbol S0 and new
productions as follows:
o S0→ ε and S0 → α for all prodn S → α
. 47
Step 3 − Eliminate rules with non solitary
terminals
 For each A → X1 ... a ... Xn where
terminal symbol a is not the only symbol
on the RHS add new rules
 Na → a,
 A → X1 ... Na ... Xn,
 If several terminal symbols occur on the
RHS, simultaneously replace each of
them by its associated non terminal
symbol. This doesn't change the 48
Step 4 − Eliminate RHS with more than 2
non terminals
 Replace each rule A → X1 X2 ... Xn with
more than 2 non terminals X1,...,Xn by
rules
 A → X1 A1,
 A1 → X2 A2,
 ... ,
 An-2 → Xn-1 Xn,
 where Ai are new non terminal symbols. 49
Convert the following CFG into CNF
S → aSa | bSb | aa | bb | a | b
Total no.
Sa|b
₪ No null or unit prodns or of
Na  a
εL prodns:
S  N a S Na 10
₪ Add prodns already in S  Na S1
CNF
S1  S N a
₪ Convert S → aSa
Nb  b
₪ Convert S → bSb
S  N b S Nb
₪ Convert S → aa
S  Nb S2
₪ Convert S → bb S2  S N b
50 S  Na Na
SN N
Convert the following S0  λ S  abX
CFG into CNF Xa|b S  NaN bX
Ya|b|c S  NaS3
S → aXbX | abX | aXb | ab
Na  a S3  NbX
|λ
Nb  b
X → aY | bY | a | b S0  abX
S  aXbX
Y → aY | bY | a | b | c S  NaXNbX S0  NaS3
S  NaS1 S → aXb
S0  λ | aXbX | abX | aXb |
S1  XS2 S → NaXNb
ab
S2  NbX S  NaS4
S → aXbX | abX | aXb | ab
S4  XNb
X → aY | bY | a | b S0  aXbX
Y → aY | bY | a | b | c S51
0  NaS1 S0  aXb
S0  NaS4
X → aY | bY S0  λ | NaS1 | NaS3 | NaS4
Y → aY | bY S  NaS1 | NaS3 | NaS4
X  N a Y | N bY
X  a | b | N a Y | N bY

Y  N a Y | N bY Y  a | b | c | N a Y | N bY

Na  a

Nb  b Total 22
S1  XS2 production
S2  NbX s
52
S3  NbX
 Greibach Normal
Form
▪ A context-free grammar is S → aB | bA
in Greibach Normal Form
if the right-hand side of A → a | aS | bAA
each rule has one terminal B → b | bS | aBB
followed by zero or more
non-terminals:

AaX
where
a 
X  V* 53
 Advantages:
● Every derivation of a string s contains |s| rule
applications. So strings can be quickly parsed.
● Greibach normal form grammars can easily be
converted to pushdown automata with no - transitions.
This is useful because such PDAs are guaranteed to
Shalt.
→ aB | bA S  aB # of derivation
A → a | aS | bAA  aaBB steps = |aababb|
B → b | bS | aBB  aabSB =6
 aabaBB
 aababB
Derive the string
 aababb
aababb

54
 Conversion to Greibach Normal Form using
substitution
S → aBc S → aBC S → Bc S → bC
B→b B→b B→b Bb
C→c C→c
S  AB S  aAB | bBB |
A  aA | bB | b bB
Bb A  aA | bB | b S  AB
Bb A  BS | b
S → aABSB | aA B  SA | a
S → aabSb | aa
B→b
A→a
55
To convert a CFG in to a GNF:

1. the grammar has to be in

Chomsky Normal Form

2. there should be no Left-

Recursive Rules

56
But first we need to understand the concept of
Left – Recursive Rules
This is a left-recursive rule: Ak → Akα | β
The alternative (β) allows a derivation to “break out of”
the recursion.
Every left-recursive rule must have an alternative (β)
that allows breaking out of the recursion.
The rule Ak → Akα | β generates the language: βαn, n≥0
To see this, look at a few derivations:
Language generated by Ak → Akα | β ?

57
 Eliminate left-recursion
• We want to eliminate the left-recursion in
this rule: Ak → Akα | β
• And we know the rule produces βαn
• But the language can also be generated
using these rules:
Ak →β|βZ
Z → α Z| α
• With those two types of rules we have
eliminated the left recursion.

58
 Multiple left-recursive alternatives
 Ak may have multiple alternatives that are left-
recursive:
Ak → Akα1 | Akα2 | … | Akαr
 And Ak may have multiple other alternatives:
Ak → β1 | β2 | … | βs
 So Ak may generate following strings where
β1(α1,…,αr)*
β2(α1,…,αr)*
…
59
 *
₪ The prodns with no left
recursion are same as
in P  The set of Ak prodns in P1 are
₪ Let Zk be a new variable Ak → β1 | β2 | … | βs
₪ Let G1 = (V  {Zk}), Σ, Ak → β1 Zk | β2 Zk | … | βs
P1, S) where P1 is Zk
defined as
 The set of Z prodns in P1 are

A k → A kα 1 | A kα 2 | … | A kα r Z k → α1 Z k | α2 Z k | … | αr
A k → β1 | β2 | … | βs Zk
60
Z k → α1 | α2 | … | αr
Loop
Convert to GNF
Convert to GNF A1  A 2 A 3
A1  A 2 A 3 A2  A 3 A 3
A2  A 3 A 3 A3  A 2 A 3 | a
A3  a A 3 | a There is a LOOP
Using substitution, we A2  A3  A2
can easily convert to
GNF Which gives rise to
Left recursion when we
A1  a A 3 A3 A 3 | a A3 A3 do substitution
A2  a A 3 A 3 | a A 3 A2  A 2 A3 A 3 | a A3
61
 Conversion to GNF

1. Eliminate null, unit productions &

useless symbols from the grammar G
2. Convert it to Chomsky Normal Form
(CNF) generating the language L(G′ ) =
L(G) − {ε} where
G′ = (V′ , Σ, P′ , S) in.
3. Rename the variables like A1, A2, . . .
An
starting with S = A1. Rewrite the 62
4. Checking for loop
◊ Prodns of following form don’t give rise to
loops
Ai  a X where a  Σ, X (V  Σ)*
Ai  Aj X where X (V  Σ)* and i < j
◊ In prodns having loop, carry out substitution to
get Left Recursion.
Ai  Aj X where X (V  Σ)* and i > j
5. Eliminate Left Recursion from all the Prodns
6. Do substitution in prodns not in GNF
63
 In which state are
you??
Which one is the top view of the
tower on the left?

64
Convert to GNF
Step 4: Check for Loop
S  AB A3  A1 A2 // i > j
A  BS | b
A3  A2 A3 A2 // i > j
B  SA | a
A3  A3 A1 A3 A2 | b A 3 A2
Step 1 & 2: Convert to
CNF Step 5: Eliminate Left
Already in CNF Recursion
A1  A2 A3
Step 3: Rename the A2  A3 A1 | b
variables & Rewrite the
prodns A3  A3 A1 A3 A2 | b A 3 A2 | a

S – A1 A – A2 B–
65

A3
A3  A3 A1 A3 A2 | b A3 A2 |
a A prodns are:
A3  b A 3 A2 | a
A 3  b A 3 A 2 Z3 | a Z 3

’s & 1 β1
Z prodns are:
ββ’s
2
?? Z3  A 1 A 3 A 2 Z3
Z3  A 1 A 3 A 2

66
 Substitute in A2  A3 A1 | b
5. Step 6: Do
substitution in A 2  b A 3 A 2 A 1 | b A 3 A 2 Z3 A 1 | a
prodns not in GNF Z3 A 1 | a A 1 | b

A1  A2 A3 Substitute in A1  A2 A3

A2  A3 A1 | b A 1  b A 3 A 2 A 1 A 3 | b A 3 A 2 Z3 A 1 A 3
| a Z3 A1 A3 | a A 1 A3 | b A 3
A3  b A 3 A2 | a
Substitute in Z3  A1 A3 A2 Z3
A 3  b A 3 A 2 Z3 | a Z 3
Z3  b A 3 A 2 A 1 A 3 A 3 A 2 Z3 | b A 3 A 2
Z3  A 1 A 3 A 2 Z3
Z3 A 1 A 3 A 3 A 2 Z3 | a Z 3 A 1 A 3 A 3 A 2
Z3 
All A3Aprod
1 A3 A
n
s 2are in GNF Z3 | a A 1 A 3 A 3 A 2 Z3 | b A 3 A 3 A 2 Z3
A 3  b A 3 A 2 | b A 3 A 2 Z3 | Substitute in Z3  A1 A3 A2
a Z3 | a
Z3  b A 3 A 2 A 1 A 3 A 3 A 2 | b A 3 A 2 Z3
67
A A A A |aZ A A A A |aA
5. Step 6: Do
All A3 prodns are in GNF
substitution in
prodns not in GNF A 3  b A 3 A 2 | b A 3 A 2 Z3 | a Z 3 | a
68
A1  A2 A3
A2  A3 A1 | b Substitute in A2  A3 A1 | b
A3  b A 3 A2 | a So we get 5 A2 prodns
A 3  b A 3 A 2 Z3 | a Z 3 Substitute in A1  A2 A3
Z3  A 1 A 3 A 2 Z3 So we get 5 A1 prodns
Z3  A 1 A 3 A 2 Substitute in Z3  A1 A3 A2 Z3 | A1
Total 24 A3 A2
prodns So we get 5 + 5 Z3 prodns
Equivalence of
Push Down Automata &
Context Free Languages

69
 CFG to PDA
 The PDA simulates the left-sentential forms that G
uses to generate any string w
 Let G = (V, , P, S)
 Construct PDA N that accepts L(G) by empty stack
 N = ({q}, , V  , , q, S)
 Transitions are defined as
 For each variable A
(q, ε, A) = {(q, ) | A   is a prodn in G}
 For each terminal a
(q, a, a) = {(q, ε)} 70
 Find a PDA equivalent to the grammar S  aSbb | a
Generate the string aabb and also simulate the PDA
for
N it
= ({q}, , V  , , q, S)
(q, aabb, S)
Where  = {a, b}, V = {S} ├ (q, aabb, aSbb)
&  is as following ├ (q, abb, Sbb)
├ (q, abb, abb)
1. (q, ε, S) = {(q, aSbb), ├ (q, bb, bb)
(q, a)} ├ (q, b, b)
2. (q, a, a) = {(q, ε)} ├ (q, ε, ε)

3. (q, b, b) = {(q, ε)} Stack Empty – string

71 accepted
S  aSbb  aabb
Find a PDA equivalent to the following grammar that
generates simple expressions like a * (a + b010)

E  I | E * E | E + E | (E)
I  a | b | Ia| Ib | I0 | I1
Let N be a PDA that accepts the language
generated by the given CFG
 N = ({q}, , V  , , q, E)
  = {a, b, 0, 1, (, ), *, +}
 V = {E, I) 72
1. (q, ε, E) = {(q, I), (q, E*E), (q, E+E), (q, (E))}
2. (q, ε, I) = {(q, a), (q, b), (q, Ia), (q, Ib), (q, I0),
(q, I1)}
3. (q, a, a) = {(q, ε)} E  I | E * E | E + E |
4. (q, b, b) = {(q, ε)} (E)
I  a | b | Ia| Ib | I0 |
5. (q, 0, 0) = {(q, ε)} I1
6. (q, 1, 1) = {(q, ε)}
7. (q, (, () = {(q, ε)}
8. (q, ), )) = {(q, ε)}
9. (q, *, *) = {(q, ε)} 73
True or False?

₪ L = ∅ is a language.
₪ All finite languages are regular.
₪ An infinite language can have an infinite complement.
₪ All infinite languages have infinite complements.
₪ The union of infinitely many regular languages is regular.
₪ You can always convert a PDA into an NFA for the same
language.
₪ If a CFG G is in CNF, then G is not ambiguous.
₪ Some PDAs need infinite-length descriptions, as the rules
(productions) can be an arbitrary subset of V × (V ∪ Σ)* .
74
₪ The context-free languages are closed under union.
₪
₪ Show a context-free grammar that generates L = {w ∈ {a,
b}* : the first, middle & last characters of w are
identical}.
S  aAa | bBb
A  CAC | a  1, 010, 111,
B  CBC | b
00100
Ca|b
 00000000, or, for
other examples,
₪ Consider the following grammar G:
any string of even
S→1S1|T
length.
T→1X1|X
X→0X0|1
 What are the first 4 strings in the lexicographic
enumeration of L(G)? 75
 Give an example of a string w ∈ {0, 1}+ such that |w| > 7
 PDA to CFG
We construct grammar G as follows.
G = (V, , P, S)
 V = {S}  {[q, z, q] | q, q Q, z  }
The productions in P are induced by the
moves of PDA as follows
 Rule 1:
The S productions are given by
S  [q0, Z0, q]  q  Q
76
 Rule 2:
Each move erasing a pushdown symbol given
by
(q , ε)  (q, a, z) induces the prodn
[q, z, q]  a

 Rule 3:
Each move not erasing a pushdown symbol
given by (q1, z1z2 … zm)  (q, a, z) induces
many prodns of the following form where q, q2,
…, qm can be any state in Q

77
[q, z, q]  a [q , z , q ] [q , z , q ] … [q ,
Construct a context-free grammar G which accepts
L(N), where
N = ({qo, q1}, {a, b}, {Zo, Z}, , qo, Zo, φ)
 is given by
1. (qo, b, Zo) = {(qo, ZZo)}
2. (qo, ε, Zo) = {(qo, ε)}
3. (qo, b, Z) = {(qo, ZZ)}
4. (qo, a, Z) = {(q1, Z)}
5. (q1, b, Z) = {(q1, ε)}
6. (q1, a, Z0) = {(qo, Zo)}78
 Let G = {V, {a, b}, P, S} Total no. of
variables?
 V = {S}  {q, z, q] | q, q Q, z  }
V = {S}  {[qo, Zo, qo], [qo, Zo, q1],
[qo, Z , q0], [qo, Z , q1],
[q1, Zo, qo], [q1 , Zo, q1],
[q1, Z , q0], [q1, Z , q1],

N = ({qo, q1}, {a, b}, {Zo,

79 Z}, , qo, Zo, φ)
The productions in P are induced by the moves of
PDA
Ho w
pr o m
du c a n y S
 Rule 1: S  [q0, Z0, q]  q  Q t
be t ions w
her i
e ? ll
1. S  [q0, Z0, qo]
2. S  [q0, Z0, q1]

80
 Rule 2: (q, a, z) = (q , ε) induces [q, z, q]  a

1. (qo, b, Zo) = {(qo,

ZZo)}
3. [qo, Z0, qo]  ε 2. (qo, ε, Zo) = {(qo, ε)}
3. (qo, b, Z) = {(qo, ZZ)}
4. [q1, Z, q1]  b 4. (qo, a, Z) = {(q1, Z)}
5. (q1, b, Z) = {(q1, ε)}
6. (q1, a, Z0) = {(qo, Zo)}
81
 Rule 3: (q, a, z) = (q1, z1z2 … zm)

[q, z, q]  a [q1, z1, q2] [q2, z2, q3] … [qm, zm, q]

 (qo, b, Zo) = {(qo, ZZo)}

[q0, Z0,  ]  b [q0, Z,  ] [, Z0, ]

5. [q0, Z0, q0 ]  b [q0, Z, q0 ] [q0, Z0,

q0]
6. [q0, Z0, q0 ]  b 82[q0, Z, q1 ] [q1, Z0,
q0]
 (qo, b, Z) = {(qo, ZZ)}

[q0, Z,  ]  b [q0, Z,  ] [, Z, ]

9. [q0, Z, q0 ]  b [q0, Z, q0 ] [q0, Z, q0]

10.[q0, Z, q0 ]  b [q0, Z, q1 ] [q1, Z, q0]
11.[q0, Z, q1 ]  b [q0, Z, q0 ] [q0, Z, q1]
12.[q0, Z, q1 ]  b [q0, Z, q1 ] [q1, Z, q1]

83
 (qo, a, Z) = {(q1, Z)}
[q0, Z,  ]  a [q1, Z,  ]

13.[q0, Z, q0 ]  a [q1, Z, q0 ]
14.[q0, Z, q1 ]  a [q1, Z, q1 ]

 (q1, a, Z0) = {(qo, Zo)}

[q1, Z0,  ]  a [q0, Z0,  ]
15.[q1, Z0, q0 ]  a [q0, Z0, q0 ]
16.[q1, Z0, q1 ]  a [q0,
84 Z0, q1 ]
Find the equivalent CFG

n g u age
La ted?
c e p
ac

S  [q0 Z0 q0] | [q0 Z0 q1] | [q0 Z0 q2]

[qo Z0 q0] → a [q0 X qo] [q0 Z0 q0] | a [q0 X q1] [q1 Z0 q0] | a [q0 X q2]
[q2 Z0 q0]
[q0 Z0 q1] → a [q0 X q0] [q0 Z0 q1] | a [q0 X q1] [q1 Z0 q1] | a [q0 X q2] [q2
Z0 q1]
[q0 Z0 q2] → a [q0 X q0] [q0 Z0 q2] | a [q0 X q1] [q1 Z0 q2] | a [q0 X q2] [q2
Z0 q2]
85
[q0 X q0] → a [q0 X q0] [q0 X q0] | a [q0 X q1] [q1 X q0] | a [q0 X q2] [q2 X
q0]
[q0 Z0 q0] → [q1 Z0 q0]
[q0 Z0 q1] → [q1 Z0 q1]
[q0 Z0 q2] → [q1 Z0 q2]

[q0 X q0] → [q1 X q0]

[q0 X q1] → [q1 X q1] Make a Parse Tree for the string aabb
[q0 X q2] → [q1 X q2]

[q1 X q1] → b

[q1 Z0 q2] → ε

86
 Find the equivalent CFG
We have already
designed PDA for L =
{anbmcn | n ≥ 1} • S  [q0, Z, q0] | [q0, Z, q1] | [q0, Z,
δ(q0, a, Z) = (q0, q2]
aZ) • [q0, Z, q0]  a [q0, a, q0][q0, Z, q0]
• [q0, Z, q0]  a [q0, a, q1][q1, Z, q0]
δ(q0, a, a) = (q0, • [q0, Z, q0]  a [q0, a, q2][q2, Z, q0]
aa) • [q0, Z, q1]  a [q0, a, q0][q0, Z, q1]
δ(q0, b, a) = (q1, • [q0, Z, q1]  a [q0, a, q1][q1, Z, q1]
a) • [q0, Z, q1]  a [q0, a, q2][q2, Z, q1]
• [q0, Z, q2]  a [q0, a, q0][q0, Z, q2]
δ(q1, b, a) = (q1, • [q0, Z, q2]  a [q0, a, q1][q1, Z, q2]
a) • [q0, Z, q2]  a [q0, a, q2][q2, Z, q2]
δ(q1, c, a) = (q2, • …
87
ε)
88
 ||Shri Hari ||

Module II (Part 2)
Turing machines (TM):
₪ Basic Model, Definition and Representation,
₪ Variants of Turing Machine and their equivalence,
₪ TM for Computing Integer Functions,
₪ Universal TM,
₪ Church’s Thesis,
₪ Recursive and Recursively Enumerable
Languages,
₪ Halting Problem,
₪ Introduction to Computational Complexity 89
Foundation

The theory of computation

and the practical
application it made
possible — the computer
— was developed by an
Englishman called Alan
Turing.

Oct 6, 2023 90
 Turing Machine

M = (Q, Σ, Γ, δ, q0, #, H)
 Q - The finite set of states of the finite control
 Σ - The finite set of input symbols
 Τ – The set of tape symbols; Σ  Τ
  - Q x Τ  Q x Τ x {L, R}

 q0 – The initial state

 # - Blank Symbol; #   but #  Σ

 H – Accept & Halt State; H  Q
92
Oct 6, 2023
 Computations: The Start Configuration
start state

head: on the leftmost square

q0

the input: starting from left

Note: the head cannot
move to the left of this
square!
93
Oct 6, 2023
Design a TM to recognize L = {w | w ends
with a}

a a b a # # …

Oct 6, 2023 94
 Building a TM for a Simple Language
Is L
CF a
L?
L = { anbncn | n  0 }

Examples:
Member of L: aaabbbccc
aaabbbccc
Non-Member of L: aaabbcccc
aaabbcccc

Oct 6, 2023 95
 Oct 6, 2023

The Transitions Function

aa, R
YY, R bb, R
ZZ, R
q1 bY,R
aX,R q2 transitions not
q0 specified here
XX, R cZ,L
__, R yield qreject
q3 bb, L
YY, R
aa, L
qac YY, L
__, R
q4 ZZ, L

YY, R
ZZ, R

96
 Oct 6, 2023

aa, R
bb, R
Demonstration YY, R
q1 bY,R
ZZ, R
aX,R
q2
q0
XX, R cZ,L
__, R
YY, R q3 ZZ, L
bb, L
qac
__, R YY, L
q4 aa, L

YY, R
ZZ, R

Xa b
Y Zc _ _ ...

97
Construct a TM machine for accepting the palindromes of even
length.

Some strings in L Some strings not

aabbbbaa in L
babaabab aabbbbaab
aaaa babaaabab
abababbababa baaaa
ababababbababa
Show the content of the tape initially for string
ababbaba

a b a b b 98 a b a # …
Oct 6, 2023
 Let’s now simulate the working of the TM

a b a b b a b a # …

X b a b b a b a # …

X b a b b a b # # …

X X a b b a b # # …

X X a b b a # # # …

99
Oct 6, 2023
 Let’s now simulate the working of the TM
X X X b b a # # # …

X X X b b # # # # …

X X X X b # # # # …

X X X X # # # # # …

100
Oct 6, 2023
 TRANSITIO
N GRAPH

101

Oct 6, 2023
 Transition Table

a b X #
q0 q1,X,R q4,X,R H,#,L
q1 q1,a,R q1,b,R q2,#,L
q2 q3,#,L
q3 q3,a,L q3,b,L q0,X,R
q4 q4,a,R q4,b,R q5,#,L
q5 q6,#,L
q6 q6,a,L q6,b,L q0,X,R

102
Oct 6, 2023
 Construct a TM machine for accepting the palindromes of odd
length.

Some strings in L Some strings not in

aabbabbaa L
babaaabab aabbbbaab
aaaaa babaabab
ababababababa baaaa
ababababbababa

103
Oct 6, 2023
 What change
should we do
in the TM
that accepts
all
palindromes
of even
length?
104
Oct 6, 2023
 M = ({q0,q1,q2,qf}, {0,1}, {0,1,#}, δ, q0,
#, {qf})

Describe the language L(M) if δ consists

 δ(q0, 0) = of (q
following rules.
1, 1, 0(10)*
R)
 δ(q1, 1) = (q0, 0, 01*
R)

 δ(q
δ(q01,, 0)
#) =
= (q(q0f,, #,
#, 0*11  δ(q0, 0) = (q1, 1, R)
R)
R) *  δ(q1, 1) = (q2, 0, L)
 δ(q0, 1) = (q1, #,  δ(q2, 1) = (q0, 1 ,
105
R)
Oct 6, 2023 R)
Describe a TM that decides the language
L = {w#w | w ∈ {a, b}*}

106
Oct 6, 2023
Describe a TM that decides the language
L = {w ∈ {0, 1}* | |w|0 = |w|1}

107
Oct 6, 2023
 CONFIGURATION OF A TM
As a Turing machine computes, what things
can change?

The current state,

The current tape contents, and
The current head location.
A setting of these three items is called a
c o n fi g u r a t i o n o f t h e Tu r i n g m a c h i n e .
108
Oct 6, 2023
 A Turing machine with configuration
1011q701111
If we have the following transition, then
what will be the next configuration?

 δ(q7, 0) = (q9, 1, R)

 δ(q9, 1) = (q8, 0, L)
109
Oct 6, 2023
 Describe the operation of T when started in the
Let T = (K, Σ, , S, h) be a
TM where
configuration (q0, #aaaaa)
₪ K = {q0, q1, q2}
₪ Σ = {a, #} (q0, #aaaaa)
₪ S = q0
(q1, #aaaaa)
₪  is as follows
(q2, #aaa#a)
q   (q,  )
(q0, #aaa#a)
q0 a (q1, L)
q0 # (q0, #) (q1, #aaa#a)
q1 a (q2, #) (q2, #a#a#a)
q1 # (h, #)
(q0, #a#a#a)
q2 a (q2, a)
q2 # (q0, L) (q1, #a#a#a)
( h, #a#a#a)
110
Oct 6, 2023
A language is recursively enumerable if some Turing machine
accepts it.
If w  L, then M halts in a final state
If w  L, then M halts in some non final state or loops forever

A language is recursive if some Turing machine accepts it and

halts on any input string.
If w  L, then M halts in a final state
If w  L, then M halts in some non final state

Recursively Enumerable Languages

and
Recursive Languages
111
 Surprise!
Can we augment All TMs are born equal…
the power of Turing
machines with
various
accessories?
Changing the model,
making it less or more
restrictive, does not
change the power of a
TM.

112
Oct 6, 2023
 Variants of TM

1. Turing Machine with Stay

Option
 In standard TM, the R/W head must
move either Left or right
 We can define TM with Stay-option
 : Q x   Q x  x {L, R, S}
 One can achieve the net effect of stay-
in place by moving the head off the
cell and immediately moving it back!
113
Oct 6, 2023
 … # a b # …

2. Two-way infinite Tape Turing Machine

 Infinite tape of two-way infinite tape Turing machine is unbounded
in both directions left and right.
 Let M0 with one-way infinite tape TM imitating one-way infinite tape
TM, M
 The first thing that M0 needs to do is to mark the start of its tape.
 It can move the whole string to the right every time M moves left
to the marker.
 : Q x   Q x  x {L, R}
114

Oct 6, 2023
 What is the behaviour when started on the
Given the TM tape
… # 1 0 1 0 # …
1. (0, 1, 1, 0, R)
2. (0, 0, 0, 1, R)
3. (1, 1, 1, 1, R)
… # 1 0 1 # …
4. (1, #, 1, 2, L)
5. (2, 1, 1, 2, L)
6. (2, 0, 0, 2, L)
7. (2, #, 1, 0, R)
115

Oct 6, 2023
 3. Multi Tape Turing Machine
 TM with k tapes and k
heads.
 Each tape has its own head
for reading and writing.
 Initially the input appears
on tape 1, and the others
start out blank.
 δ: Q × k → Q × k × {L,R}k
 δ(qi, a1, …, ak) = (qj, b1, …,
bk, L, R, …, L)
Oct 6, 2023 116
 4. Nondeterministic Turing Machine
 At any point in a computation the machine may proceed
according to several possibilities.
 δ: Q ×  → P(Q ×  × {L,R})
 δ(qi, a) = {(q1, b, R), (q2, c, L), …, (qm, d, R)}
 We can simulate any nondeterministic TM N with a
deterministic TM D.
 The idea behind the simulation is to have D try all possible
branches of N's nondeterministic computation.
 If D ever finds the accept state on one of these branches,
D accepts. Otherwise, D's simulation will not terminate.

119
Oct 6, 2023
 Church - Turing
Thesis
Alan Turing
Alonzo Church (1912-1954)
(1903-1995)
Some common interpretations of the thesis
are
₪ Every algorithmically computable function
is TM-computable.
₪ A function is computable iff it can be
solved by a Turing Machine
Thesis not Theorem: because we cannot prove this.. with a
counter example we could disprove it (but this has not been
125
Octdone
6, 2023yet).
 M is an algorithm for achieving some desired result if
1. M is set out in terms of a finite number of exact instructions
(each instruction being expressed by means of a finite number
of symbols);

2. M will, if carried out without error, produce the desired result

in a finite number of steps;

3. M can (in practice or in principle) be carried out by a human

being unaided by any machinery except paper and pencil;

4. M demands no insight, intuition, or ingenuity, on the part of

the human being carrying out the method.
126
Oct 6, 2023
 Design TM for 2n
L {0 | n 0}
Write an algorithm for L. If you can develop an algorithm, then you can
easily convert it into a TM

Let n =
3
0 0 0 0 0 0 0 0 # # #
127
Oct 6, 2023
 What is the
significance of this 2n
L {0 | n 0}
transition?

128

Oct 6, 2023
 TM for Computable
Functions
What is computable is Turing-computable;
TMs are algorithms (Kleene 1953).
TMs can be used to manipulate numerical
data
Each no. is represented in Unary. E.g.
<4> = 1111
A function f: N  N is computable if there
exists a TM, M, st for every integer n, M halts
129
on <n> & the number of 1s is equal to f(n)
Oct 6, 2023
 Design a TM to compute f(n) = n + 1
Consider f(3).
Write its Computation History
(S, 1, 1, S, R)
S111
(S, #, 1, H,R)
⊢ 1S11
⊢ 11S1
⊢ 111S#
⊢ 1111H#

130
Oct 6, 2023
 Design a TM to compute One’s
Complement when i/p is given in
binary
M = (Q, Σ, , , S, #, Consider f(100).
H) Write its Computation History
 Q = {S, H}
S100
 Σ = {0, 1}
⊢ 0S00
 Τ = {0, 1, #}
⊢ 01S0
  = {(S, 0, 1, S, R),
⊢ 011S#
(S, 1, 0, S, R),
⊢ 011H#
131
(S, #, #, H, R)}
Oct 6, 2023
 Design a TM to compute n  2 if n 2
f (n) 
 1 if n  2

132

Oct 6, 2023
 Given two positive integers x & y, design a TM to compute x +
y
E.g. f(2, 3) = 5
 (q0, 1) = (q0, 1,
… # 1 1 # 1 1 1 # … R)
 (q0, #) = (q1,
x y 1,R)
 (q1, 1) = (q1, 1,
… # 1 1 1 1 1 # # …
R)
 (q1, #) = (q2, #,
x+ L)
y 133  (q2, 1) = (H, #,
Oct 6, 2023 L)
 Design a TM that copies strings of 1 . i.e.
f(n) = 2n *
… # 1 1 1 # … q 0 w  Hww

… # 1 1 1 1 1 1 # …

First write an algorithm.

1. Replace every 1 by x
2. Find the rightmost x & replace it by 1
3. Travel to the right end of the current non-blank region &
write a 1 there
4. Repeat steps 2 & 3 until there are no more x’s
134
Oct 6, 2023
  (q0, 1) = (q0, x,
R)
d (q0, #) = (q1,
#,L)
d (q1, 1) = (q1, 1,
L)
 (q1, x) = (q2, 1,
R)
 (q2, 1) = (q2, 1, 135
Oct 6, 2023
R)
 What does the following TM compute?

Test for
n=ε
n = 11
1→1,R 1 → 1, R

1→□,R □→□,R
q0 q1 q2

□
□
,R

→
→

1
□

,R
→

,R
□

□→□,L □→1,L
qf q5 q4 q3

1 → 1, L 1 → 1, L

136
Oct 6, 2023
Theorem: If L is recursive language then its
complement, Lc , is also recursive
language.
Proof:
 Suppose that M decides L.
 Design a new machine M′ that behaves just like M,
but:
• If M accepts, M′ rejects.
• If M rejects, M′ accepts.
 Formally, can do this by

interchanging qacc and qrej.

139
 Then M′ decides Lc.
 Basis of Computation

Software or Hardware

Turing Machines – the formalism of

y ? algorithms
h
W (that we have studied so far)
Is an “unprogrammable”
140 piece of hardware
Oct 6, 2023
 A limitation of Turing Machines:

Turing Machines are “hardwired”

they execute only one program

Real Computers are re-programmable

141

Oct 6, 2023
 We will see TMs are also software

 There is a generic TM that can be programmed,

 in about the same way as a general purpose computer can,
 to solve any problem that can be solved by a TM.

A modern computer is also “hard wired”

But it is completely flexible

Its job is to execute the instructions stored in its memory, &

these can represent any conceivable algorithm.
142

Oct 6, 2023
 Universal TM
Input : < Description of some TM M, w >

Output: result of running M on w

Output
M
Universal Tape
for running
Turing TM-M
w Machine starting on tape
w
Oct 6, 2023 143
 A 1936 paper by Turing describes a
“Universal Computing Machine”

It is a TM Tu whose input consists essentially of

 a program
it takes the form of a string specifying some TM M
and
 A data set for the program to process
It is the second string w, interpreted as input to M.

144

Oct 6, 2023 Tu then simulates the processing of w by M

 The Halting Problem
One of the simplest problems known to be unsolvable

The Halting problem asks the question

Given a program and an input to the program,
determine if the program will eventually stop when it is
given that input.

Trial solution
Just run the program with the given input. If the program
stops, we know the program stops.
But if the program doesn't stop in a reasonable amount of
time, we cannot conclude that it won't stop.
Maybe we didn't wait long enough
145
Oct 6, 2023
Computation
al Complexity

Oct 6, 2023 146

 Travelling Salesman
Problem
Given n cities:
• n choices for a starting point.
• n-1 for the next city
• n-2 for the next city
A total of n! paths to be considered.

We notice that it doesn’t matter where we start (since we

need to make a loop).
And the cost is the same forward or backward. So we can
cut the number of paths down to:
n  1!
2 Oct 6, 2023 147
The Growth Rate of n!
2 2 12 479001600
3 6 13 6227020800
4 24 14 87178291200
5 120 15 1307674368000
6 720 16 20922789888000
7 5040 17 355687428096000
8 40320 18 6402373705728000
9 362880 19 12164510040883200
0
10 3628800 20 24329020081766400
00
11 39916800 36 3.6 1041 Oct 6, 2023 148
 Time Complexity
 The time complexity of a TM M is a
function denoting the worst-case
number of steps M takes on any input of
length n.
 By convention, n denotes the length
of the input.
 Assume we're only dealing with
deciders, so there's no need to handle
149
looping TMs. Oct 6, 2023
 The Class P
P is the class of languages that
are decidable in polynomial
time on a deterministic single-
tape Turing machine.
P TIME n k 
k

150
Oct 6, 2023
 The class P is important
because
P roughly corresponds to the class of problems that are
realistically solvable on a computer.

When a problem is in P,

o we have a method of solving it that runs in time nk for

some constant k.

o Once a polynomial time algorithm has been found for a

problem that formerly appeared to require exponential
time, some key insight into it has been gained, and further
reductions in its complexity usually follow, often to the point
of actual practical utility.
151
Oct 6, 2023
 Some Problems in P
Maximum matching:
Given a set of tasks and workers who can perform those tasks,
can all of the tasks be completed in under n hours?
Remoteness testing:
Given a graph G, are all of the nodes in G within distance at
most k of one another?
Linear programming:
Given a linear set of constraints and linear objective function, is
the optimal solution at least n?
Edit distance:
Given two strings, can the strings be transformed into one
another in at most n single-character edits? Oct 6, 2023 152
 What you can’t do in polynomial
time

Check all the

Check all the simple
subsets of a set for
paths from the start
some condition
node to the end Oct 6, 2023 153
 An Interesting
Observation
 There are (at least) exponentially many objects
of each of the preceding types.
 However, each of those objects is not very
large.
o Each simple path has length no longer than
the number of nodes in the graph.
o Each subset of a set has no more elements
than the original set.
 This brings us to our next topic...

Oct 6, 2023 154

 The Class NP
We can avoid brute-force
search in many problems
and obtain polynomial time
solutions.
However, attempts to avoid brute force in
certain other problems, including many
interesting and useful ones, haven't been
successful, and polynomial time
algorithms that solve
155 them aren't known
Oct 6, 2023
to exist.
 A Hamiltonian path in a directed graph G is a directed path
that goes through each node exactly once.

We consider a special case of this problem where the start

node and target node are fixed.

HAMPATH = {<G,s,t> | G is a directed graph with a

Hamiltonian path from s to t}

1 5
3  PATH?
s t
 HAMPATH?
4
2 6
157
Oct 6, 2023
 Does this graph have a Hamiltonian
path?

s t

158
Oct 6, 2023
 No fast
algorithm for
HAMPATH

Bu
t If I somehow
discovered such a path
(perhaps using the
exponential time
Polynomial algorithm), I could
Verifiability easily convince you of
its existence, simply by
presenting it.

Verifying the existence of a Hamiltonian path

may be much easier than determining its 159
Oct 6, 2023
existence.
 P is the class of decision problems that
can be efficiently solved.

We now define the complexity class NP that

aims to capture the set of problems whose
solutions can be efficiently verified.

160
Oct 6, 2023
 NP is the class of languages
that have polynomial time
verifiers.
The class NP is important because it contains
many problems of practical interest.

The term NP comes from nondeterministic polynomial

time and is derived from an alternative characterization by
using nondeterministic polynomial time Turing Machines.

161
Oct 6, 2023
 Some problems may not be
polynomially verifiable!!!

The complement of the HAMPATH

problem.

Even if we could determine (somehow)

that a graph did not have a Hamiltonian
path, we don't know of a way for someone
else to verify its nonexistence without
using the same exponential time
algorithm for making the determination in
the first place. 162
Oct 6, 2023
 Problems in Class NP
 P  NP
 The decision version of the travelling salesman
problem is in NP. Given an input matrix of distances
between n cities, the problem is to determine if there is a
route visiting all cities with total distance less than k. A
proof can simply be a list of the cities. Then verification
can clearly be done in polynomial time.
 The decision version of the integer factorization
problem: given integers n and k, is there a factor f with
1< f < k and f dividing n?
 The Subgraph isomorphism problem of determining
whether graph G contains a subgraph that is isomorphic
163
Oct 6,
to2023
graph H.
 The question of whether P = NP is one of the greatest
unsolved problems in theoretical computer science and
contemporary mathematics.

Most researchers
P Researchers also have
believe that the tried proving that the
two classes are not NP classes are unequal,
equal because people but that would entail
have invested showing that no fast
enormous effort to OR algorithm exists to
find polynomial time replace brute-force
algorithms for certain search. Doing so is
problems in NP, P = NP presently beyond
without success. scientific reach.

Oct 6, 2023 164

 An important advance on the P versus NP question came in the early
1970s with the work of Stephen Cook and Leonid Levin.

They discovered certain problems in NP whose individual complexity

is related to that of the entire class.

If a polynomial time algorithm exists for any of these problems, all

problems in NP would be polynomial time solvable.

These are the

NP Complete Problems
Oct 6, 2023 165
 “NP-Complete” comes from:
 Nondeterministic Polynomial
 Complete - “Solve one, Solve them all”
There are more NP-Complete problems than provably
intractable problems.

A language B is NP-complete if it satisfies two

conditions:

1. B is in NP, and

2. Every A in NP is polynomial time reducible to B.

Oct 6, 2023 166
 Language A is mapping reducible to language B, which is written A
≤M B, if there is a computable function
f: Σ*  Σ* where for every w,
w  A ↔ f(w)  B.
The function f is called the reduction of A to B.

To test whether w  A, we use the reduction f and test

whether f(w)  B.
Oct 6, 2023 167
 NP Hard
NP-hard (non-deterministic polynomial-time
hard), in computational complexity theory,
is a class of problems that are, informally,
"at least as hard as the hardest
problems in NP".

A problem H is NP-hard if and only if there

is an NP-complete problem L that is
polynomial time Turing-reducible to H (i.e., 168
Oct 6, 2023
 All NP-complete problems are NP-hard but some
NP-hard problems are known not to be NP-complete
NP-complete  NP-Hard

Oct 6, 2023 169

 When
o a decision version of a combinatorial optimization problem
is proved to belong to the class of NP-complete problems,
o then the optimization version is NP-hard.

For example,
o "is there a Hamiltonian cycle with length less than k" is
NP-complete: it is easy to determine if a proposed certificate has
length less than k.

o The optimization problem, "what is the shortest tour?", is

NP-hard, since there is no easy way to determine if a certificate is
the shortest.
Oct 6, 2023 170
 Examples of NP Hard Problems

o The decision subset sum problem, which is this: given

a set of integers, does any non-empty subset of them
add up to zero? That is a NP Hard decision problem, and
happens to be NP-complete.
o The optimization problem of finding the least-cost cyclic
route through all nodes of a weighted graph. This is
commonly known as the traveling salesman problem.

Oct 6, 2023 171

 o There are decision problems that
are NP-hard but not NP-complete,
for example the halting problem. How ?
It is easy to prove that the halting problem is
NP-hard but not NP-complete.
Prove Halting problem is not NP-complete.
When do we
say that a For a problem to be NP
problem is complete, it must be decidable
NP Complete – which halting problem is not.
?
Oct 6, 2023 172
Pumping Lemma

173
 What we have seen so far …
The following are all equivalent:
 L is a regular language.
 There is a DFA D such that ℒ (D)
= L.
 There is a NFA N such that ℒ (N)
= L.
 There is a RE, R such that ℒ (R)
= L.

There are some languages that are not Regular -

we just can’t have a DFA / RE for them
e.g. {(aa)k(bb)k | k > 0}
174
Observation:
Every language of finite size has to be
regular
 we can easily construct an NFA that
accepts every string in the language)
Therefore, every non-regular language has to
be of infinite size
 contains an infinite number ofIsstrings
the
c on v e
rse
true?
175
Take an infinite regular language  Take q to be the first state
L repeated
There exists a DFA with n states  Then, One dimensional
that accepts L projection of walk of w will be
Take string w  L with |w| ≥ n as follows
Then, at least one state is
repeated in the walk of w
First Second
occurrence occurrence
s1 s2 si s i 1 .... sj s j1 .... sk
....
q q

Unique states
176
We can write w = xyz

First Second
occurrence occurrence
s1 s2 .... si s i 1 .... sj s j1 .... sk
q q

x s1  s i y s i 1 s j z s j1  s k

177
w s1s 2 s k

s1 s2 sk
......
q ......

Repeated state in DFA

178
In DFA: w x y z y contains only first occurrence of
q
...

s j s i 1
s1 ...s i s j... sk
q 1 ...

x z
179
 What is y here?
S3 S4 S5 When we read the
5th symbol we are in
state 5 – so all new
states visited till
now.

Therefore,
|xy| ≤ n

What is x here?
S1 S2

180
 Observation 1: length |xy|  n, the no. of states in DFA
 Observation 2: length |y| ≥ 1, since there is atleast one
transition in the loop
 We do not care about
y
...
the form of string z

Unique States

sj s i 1
s1 ... s i
q Since, in xy no
state is repeated

x 181
In general: the  The string xz is
accepted if we do not
string y follow the loop
xyiz for i ≥ 0  The string xyyz is
is accepted ...
accepted if we follow
i.e the loop twice
xyiz  L  The string xyyyz is
for i ≥ 0 s j s i 1 accepted if we follow
the loop thrice
s1 ... s i s j1... sk
q ...

x z 182
Let L be a regular language. Then there exists a con-
stant n such that, for every string w  L with |w|  n,
there exist strings
x, y, z  * such that
• w = xyz with |xy|  n and
•|y|  1, (but x and z may be ε)
• Then, for all i  0, xyiz  L.

Pumping Lemma for Regular Languages

183
 Suppose you want to prove that
an infinite language L is not
regular
1. Assume the opposite: L is regular
2. The pumping lemma should hold for L
3. Use the pumping lemma to obtain a contradiction
i. Choose a particular string w  L which satisfies
the length condition |w| ≥ n
ii. Write w = xyz
iii. Show that xyiz  L for some i  1
4. Therefore, L is not regular

184
Prove that L = {akbk | k ≥ 0} is not regular.
Step 1:
Suppose L is regular & is Step 3:
accepted by a FA having n  Let i = 2
states.  w’ = xyi z where
Step 2: o w’ = ap a2q arbk
 Let w = akbk where k > n o w’ = ap +2q + r bk
w  L o w’ = ak+qbk
 We can write w = xyz where o Since q  0, w’ has
o |xy|  n more a’s than b’s
o |y| > 0  Hence w’  L
 Since k > n, we have  This is a contradiction, so L
o x = ap is not Regular
o y = aq
o z = a rb k
185
o where p + q + r = k & q
Prove that L = {w | na(w) < nb(w)} is not regular.

Let w = ak bk+1 where k > n

186
Prove that L = {ak | k is prime} is not regular.
Step 1:
Suppose L is regular & is Step 3:
accepted by a FA having n  Let i = p + 1
states.  w’ = xyi z where
Step 2: o |w’| = |xyz| + p|y|
 Let w = ap where p is prime o |w’| = p + pm = p (1 +
& m)
p>n o w’ = ap(1+m)
w  L  Hence w’  L since p (1+m)
 We can write w = xyz is not prime
where  This is a contradiction, so L
o |xy|  n is not Regular
o |y| > 0
o Let y = am where m  0
187
Prove that L is not regular Step 3:
 Let i = 2
 2
L  a k | k 1 & k is an integer   w’ = xyi z where
Which o |w’| = |xyz| + |y|
language is o |w’| = n2 + |y|
it? o |w’| > n2 {since |y|  0} --- (1)
Step 1:
Suppose L is regular & is
accepted by a FA having n o Also since |w’| = n2 + |y|
states. o |w’|  n2 + n {since |y|  n}
Step 2: o |w’| < n2 + n + n + 1 = (n + 1)2
 Let w a n2
, |w| = n2 > n ---(2)
w  L  n2 < |w’| < (n + 1)2
 We can write w = xyz  |xy2z| strictly lies b/w n2 & (n+1)2 &
where so is not a perfect square.
o |xy|  n  This is a contradiction, so L is not
Regular 188
o |y| > 0
 Prove that L is not regular Step 3:
L = { w | w  {a, b}*, w  Let i = 0
= wR 1:}  w’ = x z where
Step
o w’ = ak-qbak
Suppose L is regular & is
o w’  w’ R
accepted by a FA having n
 So, xy0z  L
states.
 This is a contradiction, so L is not
Step 2:
 Let w = akbak where k > Regular
n
w  L
 We can write w = xyz
where
o |xy|  n
o |y| > 0
 Since k > n, we have
189
o q
  Hope you enjoyed this
course
 Plz send your feedback
(both positive ones & ones
suggesting improvement) at
charul@gla.ac.in
 All the Best
 Always follow the right path
Oct 6, 2023 190
Pumping Lemma
for
Context Free Languages

191
₪ Let L be an infinite
CFL. There exists
some integer n such
that for all w in L with
|w| ≥ n,
₪ w = uvxyz with |vxy| ≤
n and |vy| > 0 such
that
₪ uvixyiz ε L for all i = 0,
1, 2, 3 .... 192
Consider the first non-terminal that is
repeated, when traversing the path from the
leaf to the
S root.
Second occurrence of non-terminal A

A
First occurrence of non-terminal A
A

195
 Derivation tree for
w = bbcaa *
for some
grammar
S  uAz
*
A  vAy
z *
A x
u
So, we can y
write
Oct 6, w
2023= uvxyz
v x 196
 Pump Zero Times

A A
A
x
u v x y z u y

198
Oct 6, 2023
 Pump Twice, Thrice …

A A

A A

u v A y z
u v x y z
v x y

199
Oct 6, 2023
 Observe
We have the production rules which lead us
to the following derivations:

 w = uvxyz with |vxy|

 SuAz ≤ n and |vy| > 0 such
that
 AvAy
 uvixyiz ε L
 Ax
for all i = 0, 1, 2, 3
Oct 6, 2023 .... 200
 Prove that L = {vv| v  (a + b)*} is not a
Proof: CFL.
 Let L be a CFL. Since L is infinite, the pumping
lemma can be applied. Let n be the constant of the
lemma.
 Let w  L & w = am bm am bm for some m > n.
o Then we can write w = uvxyz
o Such that |vy| > 0
o And |vxy| ≤ n
 We examine all the possible locations of vxy in am
201
b a b
m m m
Case 1: vxy is within the first am

k1 k2 k1  k 2 1
v a y a

m m m m
a ...... a b ...... b a ...... a b ...... b
u vx y 202
 2
Leti
= m  k1  k 2 m m m
a .......... ...... a b ...... b a ...... a b ...... b
u v2 x y2 z
a m k1 k2 b m a mb m uv 2 xy 2 z  L k1  k 2 1

However, from Pumping Lemma: uv 2 xy 2 z  L

Contradiction!!!
203
Similar analysis if vxy is in

m m m m m m m m
a b a b or a b a b

or

m m m m
a b a b 204
Case 2: v is within the first
am
y is within the first
b m
k2
v ak 1 y b k1  k 2 1

m m m m
a ...... a b ...... b a ...... a b ...... b
u v x y z
205
 2
Leti
=
m  k1 m  k2 m m
a .......... .. a b .......... .. b a ...... a b ...... b
u v 2 x
y 2 z
m k1 m k 2 m m 2 2
a b a b uv xy z  L
2 2
However, from Pumping Lemma: uv xy z  L
206
Contradiction!!!
Similar analysis if vxy overlaps

m m m m
a b a b

or

m m m m
a b a b
207
 Case 3: v overlaps the first am & bm

y is within the first bm

k1 k 2 k3
v a b y b k1, k 2 1
m m m m
a ...... a b ...... b a ...... a b ...... b
u v xy z 208
 2
Leti
=
v a k1 b k2 y b k3

m k2 k1 m  k3 m m
a ...... a b ... b a ... a b ......... b a ...... a b ...... b
u v 2 x y 2
z
m k 2 k1 m k3 m m 2 2
a b a b a b uv xy z  L
2 2
uv
However, from Pumping Lemma: xy z  L
209
Contradiction!!!
Case 4: v is within the first am
y overlaps the first am & bm

m m m m
a ...... a b ...... b a ...... a b ...... b
uv x y z

Analysis is similar to Case 3

210
There are no other cases to consider
Since | vxy |m, it is impossible
vxy to overlap: m m m m
a b a b

nor
m m m m
a b a b

nor
a mb m a mb m
211
 Prove that L = {ai bi ci | i ≥ 0} is not a CFL.
Proof:
 Let L be a CFL. Since L is infinite, the pumping
lemma can be applied. Let n be the constant of the
lemma.
 Let w  L & w = ak bk ck for some k > n.
o Then we can write w = uvxyz
o Such that |vy| > 0
o And |vxy| ≤ n
 Since |vxy| ≤ n, therefore it cannot contain all 3
212
symbols – a, b & c
  Case I: Suppose vxy contains one of the symbol in Σ.
o Suppose vxy contains at least one a.
o Then uv2xy2z will have more a’s than b’s or c’s.
o Therefore uv2xy2z does not belong to L.
o This is a contradiction of the pumping lemma.
 Case II: Suppose vxy contains two of the symbol in Σ.
o Suppose vxy contains at least one a & at least one b.
o Then uv2xy2z will have more a’s & b’s than c’s.
o Therefore uv2xy2z does not belong to L.
o This is a contradiction of the pumping lemma.
In all the cases we get a contradiction.
213
Therefore our assumption that L is context free is wrong.
Oct 6, 2023
Prove that the following languages are not CFLs.

1) L = {0i1i0i1i | i >0}

2)

214
 Prove that is not a CFL.
Proof:
 Let L be a CFL. Since L is infinite, the pumping
lemma can be applied. Let n be the constant of the
lemma.
 Let w  L and w = 02^k for some k > n.
o Then we can write w = uvxyz
o Such that |vy| > 0
o And |vxy| ≤ n
 Let v = 0p and y = 0q st p+q>0
215
 |w| = 2k
 Consider the string uvixyiz
 |uvixyiz| = |uvxyz| + |vi-1| + |yi-1|
 | uvixyiz |= 2k + (i-1) (p + q)
 Let i = 2k+1 +1
 | uvixyiz |= 2k + (2k+1 +1 -1) (p + q)
 = 2k (1 + 2 (p + q))
 So the length of uvixyiz is not a power of 2
 Therefore, uvixyiz  L
 This is a contradiction of Pumping Lemma & so L is not
216
Context free