CMSC 330: Organization of

Programming Languages

DFAs, and NFAs, and Regexps

CMSC330 Spring 2019

The story so far, and what’s next
Goal: Develop an algorithm that determines
whether a string s is matched by regex R
• I.e., whether s is a member of R’s language

Approach: Convert R to a finite automaton FA

and see whether s is accepted by FA
• Details: Convert R to a nondeterministic FA (NFA),
which we then convert to a deterministic FA (DFA),
Ø which enjoys a fast acceptance algorithm
Two Types of Finite Automata
Deterministic Finite Automata (DFA)
• Exactly one sequence of steps for each string
Ø Easy to implement acceptance check
• All examples so far

Nondeterministic Finite Automata (NFA)

• May have many sequences of steps for each string
• Accepts if any path ends in final state at end of string
• More compact than DFA
Ø But more expensive to test whether a string matches
Comparing DFAs and NFAs
NFAs can have more than one transition
leaving a state on the same symbol

a
a

DFAs allow only one transition per symbol

• I.e., transition function must be a valid function
• DFA is a special case of NFA
Comparing DFAs and NFAs (cont.)
NFAs may have transitions with empty string label
• May move to new state without consuming character

ε
e-transition

DFA transition must be labeled with symbol

• DFA is a special case of NFA
DFA for (a|b)*abb
NFA for (a|b)*abb

ba
• Has paths to either S0 or S1
• Neither is final, so rejected
babaabb
• Has paths to different states
• One path leads to S3, so accepts string
NFA for (ab|aba)*

aba
• Has paths to states S0, S1
ababa
• Has paths to S0, S1
• Need to use ε-transition
Comparing NFA and DFA for (ab|aba)*
DFA

NFA
Quiz 1: Which DFA matches this regexp?
b(b|a+b?)

A. B. C.

b b
0 1 0 1 b
0 1
b
b a a
2
b a 2
0 a,b
b a
2 3
3 4 a
b

D. None of the above

Quiz 1: Which DFA matches this regexp?
b(b|a+b?)

A. B. C.

b b
0 1 0 1 b
0 1
b
b a a
2
b a 2
0 a,b
b a
2 3
3 4 a
b

D. None of the above

Formal Definition
A deterministic finite automaton (DFA) is a
5-tuple (Σ, Q, q0, F, δ) where
• Σ is an alphabet
• Q is a nonempty set of states
• q0 Î Q is the start state
• F ⊆ Q is the set of final states
• δ : Q x Σ → Q specifies the DFA's transitions
Ø What's this definition saying that δ is?
A DFA accepts s if it stops at a final state on s
Formal Definition: Example
• Σ = {0, 1}
• Q = {S0, S1}
• q0 = S0
• F = {S1}
symbol
•
δ 0 1
S0 S0 S1
input state

S1 S0 S1
or as { (S0,0,S0),(S0,1,S1),(S1,0,S0),(S1,1,S1) }
Implementing DFAs (one-off)
cur_state = 0;
while (1) {

It's easy to build symbol = getchar();

a program which switch (cur_state) {

mimics a DFA case 0: switch (symbol) {

case '0': cur_state = 0; break;
case '1': cur_state = 1; break;
case '\n': printf("rejected\n"); return 0;
default: printf("rejected\n"); return 0;
}
break;

case 1: switch (symbol) {

case '0': cur_state = 0; break;
case '1': cur_state = 1; break;
case '\n': printf("accepted\n"); return 1;
default: printf("rejected\n"); return 0;
}
break;

default: printf("unknown state; I'm confused\n");

break;
}
}
14
Implementing DFAs (generic)
More generally, use generic table-driven DFA
given components (Σ, Q, q0, F, d) of a DFA:
let q = q0
while (there exists another symbol σ of the input string)
q := d(q, σ);
if q Î F then
accept
else reject

• q is just an integer
• Represent d using arrays or hash tables
• Represent F as a set

15
Nondeterministic Finite Automata (NFA)
An NFA is a 5-tuple (Σ, Q, q0, F, δ) where
• Σ, Q, q0, F as with DFAs
• δ ⊆ Q x (Σ È {ε}) x Q specifies the NFA's transitions
a
• Σ = {a}
• Q = {S1, S2, S3}
a ε • q0 = S1
S1 S2 S3 • F = {S3}
• δ = { (S1,a,S1), (S1,a,S2), (S2,ε,S3) }
Example

An NFA accepts s if there is at least one path via s

from the NFA’s start state to a final state
NFA Acceptance Algorithm (Sketch)
When NFA processes a string s
• NFA must keep track of several “current states”
Ø Due to multiple transitions with same label, and ε-transitions
• If any current state is final when done then accept s
Example
a
• After processing “a”
Ø NFA may be in states a ε
S1 S1 S2 S3
S2
S3
Ø Since S3 is final, s is accepted
Algorithm is slow, space-inefficient; prefer DFAs!
17
Relating REs to DFAs and NFAs
Regular expressions, NFAs, and DFAs accept
the same languages! Can convert between them
can
reduce
DFA NFA

can transform can reduce

RE

NB. Both transform and reduce are historical terms; they mean “convert”
Reducing Regular Expressions to NFAs
Goal: Given regular expression A, construct
NFA: <A> = (Σ, Q, q0, F, δ)
• Remember regular expressions are defined
recursively from primitive RE languages
• Invariant: |F| = 1 in our NFAs
Ø Recall F = set of final states

Will define <A> for base cases: σ , ε , ∅

• Where σ is a symbol in Σ
And for inductive cases: AB, A|B, A*
Reducing Regular Expressions to NFAs
Recall: NFA is (Σ, Q, q0, F, δ)
Base case: σ where
Σ is the alphabet
Q is set of states
q0 is starting state
F is set of final states
σ δ is transition relation

<σ> = ({σ}, {S0, S1}, S0, {S1}, {(S0, σ, S1)} )

Reduction
Base case: ε

<ε> = (∅, {S0}, S0, {S0}, ∅)

Base case: ∅

<∅> = (∅, {S0, S1}, S0, {S1}, ∅)

Reduction: Concatenation
Induction: AB

<A> <B>

• <A> = (ΣA, QA, qA, {fA}, δA)

• <B> = (ΣB, QB, qB, {fB}, δB)
Reduction: Concatenation
Induction: AB

<A> <B>

• <A> = (ΣA, QA, qA, {fA}, δA)

• <B> = (ΣB, QB, qB, {fB}, δB)
• <AB> = (ΣA È ΣB, QA È QB, qA, {fB}, δA È δB È {(fA,ε,qB)} )
Reduction: Union
Induction: A|B

• <A> = (ΣA, QA, qA, {fA}, δA)

• <B> = (ΣB, QB, qB, {fB}, δB)
Reduction: Union
Induction: A|B

• <A> = (ΣA, QA, qA, {fA}, δA)

• <B> = (ΣB, QB, qB, {fB}, δB)
• <A|B> = (ΣA È ΣB, QA È QB È {S0,S1}, S0, {S1},
δA È δB È {(S0,ε,qA), (S0,ε,qB), (fA,ε,S1), (fB,ε,S1)})
Reduction: Closure
Induction: A*

• <A> = (ΣA, QA, qA, {fA}, δA)

Reduction: Closure
Induction: A*

• <A> = (ΣA, QA, qA, {fA}, δA)

• <A*> = (ΣA, QA È {S0,S1}, S0, {S1},
δA È {(fA,ε,S1), (S0,ε,qA), (S0,ε,S1), (S1,ε,S0)})
Quiz 2: Which NFA matches a* ?

A. B.

C. D.
Quiz 2: Which NFA matches a* ?

A. B.

C. D.
Quiz 3: Which NFA matches a|b* ?
A.
B.

C.

D.
Quiz 3: Which NFA matches a|b* ?
A.
B.

C.

D.
Reduction Complexity
Given a regular expression A of size n...
Size = # of symbols + # of operations

How many states does <A> have?

• Two added for each |, two added for each *
• O(n)
• That’s pretty good!
Reducing NFA to DFA

can
reduce
DFA NFA

can reduce

RE
Reducing NFA to DFA
NFA may be reduced to DFA
• By explicitly tracking the set of NFA states
Intuition
• Build DFA where
Ø Each DFA state represents a set of NFA “current states”
Example
a a

a a
ε
S1 S2 S3 S1 S1, S2, S3

NFA DFA 37
Algorithm for Reducing NFA to DFA
Reduction applied using the subset algorithm
• DFA state is a subset of set of all NFA states
Algorithm
• Input
Ø NFA (Σ, Q, q0, Fn, δ)
• Output
Ø DFA (Σ, R, r0, Fd, d)
• Using two subroutines
Ø ε-closure(d, p) (and ε-closure(d, Q))
Ø move(d, p, σ) (and move(d, Q, σ))
• (where p is an NFA state)

38
ε-transitions and ε-closure
ε q
We say p →
• If it is possible to go from state p to state q by taking only
e-transitions in δ
• If $ p, p1, p2, … pn, q Î Q such that
Ø {p,ε,p1} Î δ, {p1,ε,p2} Î δ, … , {pn,ε,q} Î δ
ε-closure(δ, p)
• Set of states reachable from p using ε-transitions alone
ε
Ø Set of states q such that p → q according to δ
Ø
ε q in δ }
ε-closure(δ, p) = {q | p →
ε
Ø ε-closure(δ, Q) = { q | p Î Q, p → q in δ }
• Notes
Ø ε-closure(δ, p) always includes p
Ø We write ε-closure(p) or ε-closure(Q) when δ is clear from context
39
ε-closure: Example 1
Following NFA contains
ε
• p1 → p2 ε ε
ε p1 p2 p3
• p2 → p3
ε
• p1 → p3 a
ε ε
Ø Since p1 → p2 and p2 → p3

ε-closures
• ε-closure(p1) = { p1, p2, p3 }
• ε-closure(p2) = { p2, p3 }
• ε-closure(p3) = { p3 }
• ε-closure( { p1, p2 } ) = { p1, p2, p3 } È { p2, p3 }

40
ε-closure: Example 2
Following NFA contains ε
ε a b
• p1 → p3
ε p1 p2 p3
• p3 → p2
ε
• p1 → p2
ε ε p2 ε
Ø Since p1 → p3 and p3 →

ε-closures
• ε-closure(p1) = { p1, p2, p3 }
• ε-closure(p2) = { p2 }
• ε-closure(p3) = { p2, p3 }
• ε-closure( { p2,p3 } ) = { p2 } È { p2, p3 }
41
ε-closure Algorithm: Approach
Input: NFA (Σ, Q, q0, Fn, δ), State Set R
Output: State Set R’
Algorithm
Let R’ = R // start states
Repeat
Let R = R’ // continue from previous
Let R’ = R È {q | p Î R, (p, e, q) Î d} // new ε-reachable states
Until R = R’ // stop when no new states

This algorithm computes a fixed point

CMSC 330 Spring 2018 42

ε-closure Algorithm Example
Calculate ε-closure(d,{p1})
ε ε
R R’ p1 p2 p3

{p1} {p1} a
Let R’ = R
{p1} {p1, p2} Repeat
Let R= R’
Let R’ = R È {q | p Î R, (p, e, q) Î d}
{p1, p2} {p1, p2, p3} Until R = R’

{p1, p2, p3} {p1, p2, p3}

CMSC 330 Spring 2018 43

Calculating move(p,σ)
move(δ,p,σ)
• Set of states reachable from p using exactly one
transition on symbol σ
Ø Set of states q such that {p, σ, q} Î δ
Ø move(δ,p,σ) = { q | {p, σ, q} Î δ }
Ø move(δ,Q,σ) = { q | p Î Q, {p, σ, q} Î δ }
• i.e., can “lift” move() to a set of states Q

• Notes:
Ø move(δ,p,σ) is Ø if no transition (p,σ,q) Î δ, for any q
Ø We write move(p,σ) or move(R,σ) when δ clear from context

44
move(p,σ) : Example 1
Following NFA
• Σ = { a, b } a b
p1 p2 p3

Move a
• move(p1, a) = { p2, p3 }
• move(p1, b) = Ø move({p1,p2},b) = { p3 }
• move(p2, a) = Ø
• move(p2, b) = { p3 }
• move(p3, a) = Ø
• move(p3, b) = Ø

45
move(p,σ) : Example 2
Following NFA ε
• Σ = { a, b } a a
p1 p2 p3

Move b
• move(p1, a) = { p2 }
• move(p1, b) = { p3 } move({p1,p2},a) = {p2,p3}
• move(p2, a) = { p3 }
• move(p2, b) = Ø
• move(p3, a) = Ø
• move(p3, b) = Ø

46
NFA ® DFA Reduction Algorithm (“subset”)
Input NFA (Σ, Q, q0, Fn, δ), Output DFA (Σ, R, r0, Fd, d’)
Algorithm
Let r0 = e-closure(δ,q0), add it to R // DFA start state
While $ an unmarked state r Î R // process DFA state r
Mark r // each state visited once
For each σ Î S // for each symbol σ
Let E = move(δ,r,σ) // states reached via σ
Let e = e-closure(δ,E) // states reached via e
If e Ï R // if state e is new
Let R = R È {e} // add e to R (unmarked)
Let d’ = d’ È {r, σ, e} // add transition r→e on σ
Let Fd = {r | $ s Î r with s Î Fn} // final if include state in Fn
47
NFA ® DFA Example 1
• Start = e-closure(δ,p1) = { {p1,p3} } NFA
• R = { {p1,p3} } a b
• r Î R = {p1,p3} p1 p2 p3
• move(δ,{p1,p3},a) = {p2}
ε
Ø e = e-closure(δ,{p2}) = {p2}
Ø R = R È {{p2}} = { {p1,p3}, {p2} } DFA
Ø d’ = d’ È {{p1,p3}, a, {p2}} a
• move(δ,{p1,p3},b) = Ø {1,3} {2}

48
NFA ® DFA Example 1 (cont.)
• R = { {p1,p3}, {p2} } NFA
• r Î R = {p2} a b
• move(δ,{p2},a) = Ø p1 p2 p3
• move(δ,{p2},b) = {p3}
ε
Ø e = e-closure(δ,{p3}) = {p3}
Ø R = R È {{p3}} = { {p1,p3}, {p2}, {p3} } DFA
Ø d’ = d’ È {{p2}, b, {p3}} a b
{1,3} {2} {3}

49
NFA ® DFA Example 1 (cont.)
• R = { {p1,p3}, {p2}, {p3} } NFA
• r Î R = {p3} a b
• Move({p3},a) = Ø p1 p2 p3
• Move({p3},b) = Ø
ε
• Mark {p3}, exit loop
• Fd = {{p1,p3}, {p3}} DFA
Ø Since p3 Î Fn a b
{1,3} {2} {3}
• Done!

50
NFA ® DFA Example 2
NFA DFA

{B,D}
a

{A}

b
{C,D}

R = { {A}, {B,D}, {C,D} }

51
Quiz 4: Which DFA is equiv to this NFA?
a b A.
p0 p1 p2 a b p1,
p0 p1 p2
a
a
NFA: ε

b a
B. C.
a a p2, a b p2,
p0 p1 p0 p0 p1 p0

D. None of the above b

Quiz 4: Which DFA is equiv to this NFA?
a b A.
p0 p1 p2 a b p1,
p0 p1 p2
a
a
NFA: ε

b a
B. C.
a a p2, a b p2,
p0 p1 p0 p0 p1 p0

D. None of the above b

Actual Answer
a b
p0 p1 p2
a
NFA: ε

a
a b p2, p1,
p0 p1 p0 p0
b
a
NFA ® DFA Example 3
NFA DFA
{B,D,E} a
a a
{A,E}
b {E}
b
b
{C,D}

R = { {A,E}, {B,D,E}, {C,D}, {E} }

55
NFA ® DFA Example
NFA ® DFA Example
NFA ® DFA Practice
NFA ® DFA Practice
Subset Algorithm as a Fixed Point
Input: NFA (Σ, Q, q0, F, δ)
Output: DFA M’
Algorithm
Let q0’ = ε-closure(δ, q0)
Let F’ = {q0’} if q0’ ∩ F ≠ ∅, or ∅ otherwise
Let M’ = (Σ, {q0’}, q0’, F’, ∅) // starting approximation of DFA
Repeat
Let M = M’ // current DFA approx
For each q Î states(M), σ Î Σ // for each DFA state q and symb σ
Let s = ε-closure(δ, move(δ, q, σ)) // new subset from q
Let F’ = {s} if s ∩ F ≠ ∅, or ∅ otherwise, // subset contains final?
M’ = M’ È (∅, {s}, ∅, F’, {(q, σ, s)}) // update DFA
Until M’ = M // reached fixed point
CMSC 330 Spring 2018 60
 Redux: NFA to DFA Example 1
• q ' ε-closure(δ,p1) = {p1,p3}
• F’ = {{p1,p3}} since {p1,p3} ∩ {p3} ≠ ∅ NFA
a b
p1 p2 p3

ε
DFA
{1,3}

• M’ = { Σ, {{p1,p3}}, {p1,p3}, {{p1,p3}}, ∅ }

Q’ q 0' F’ δ'

CMSC 330 Spring 2018 61

 Redux: NFA to DFA Example 1 (cont)
• M’ = { Σ, {{p1,p3}}, {p1,p3}, {{p1,p3}}, ∅ }
• q = {p1, p3} NFA
•a=a
• s = {p2} a b
Ø since move(δ,{p1, p3},a) = {p2}
p1 p2 p3
Ø and e-closure(δ,{p2}) = {p2}
• F’ = ∅ ε
Ø Since {p2} ∩ {p3} = ∅
Ø where s = {p2} and F = {p3} DFA
a
{1,3} {2}

• M’ = M’ È ( ∅ { p }, ∅, ∅ { p p a p }
• { Σ, {{p1,p3},{p2}}, {p1,p3}, {{p1,p3}}, {({p1,p3},a,{p2})} }
Q’ q 0' F’ δ'
CMSC 330 Spring 2018 62
 Redux: NFA to DFA Example 1 (cont)
• M’ = { Σ, {{S1,S3},{S2}}, {S1,S3}, {{S1,S3}}, {({S1,S3},a,{S2})} }
• q = {S2} NFA
•a=b
• s = {S3} a b
Ø since move(δ,{S2},b) = {S3}
S1 S2 S3
Ø and e-closure(δ,{S3}) = {S3}
• F’ = {{S3}} ε
Ø Since {S3} ∩ {S3} = {S3}
Ø where s = {S3} and F = {S3} DFA
a b
{1,3} {2} {3}
• M’ = M’ È
( ∅ { S3 }, ∅ {{S3}} { S2 b S3 }
{ Σ, {{S1,S3},{S2},{S3}}, {S1,S3}, {{S1,S3},{S3}}, {({S1,S3},a,{S2}), ({S2},b,{S3})} }
Q’ q 0' F’ δ'
CMSC 330 Spring 2018 63
Analyzing the Reduction
Can reduce any NFA to a DFA using subset alg.
How many states in the DFA?
• Each DFA state is a subset of the set of NFA states
• Given NFA with n states, DFA may have 2n states
Ø Since a set with n items may have 2n subsets
• Corollary
Ø Reducing a NFA with n states may be O(2n)

NFA DFA
64
Recap: Matching a Regexp R
Given R, construct NFA. Takes time O(R)
Convert NFA to DFA. Takes time O(2|R|)
• But usually not the worst case in practice
Use DFA to accept/reject string s
• Assume we can compute d(q,σ) in constant time
• Then time to process s is O(|s|)
Ø Can’t get much faster!

Constructing the DFA is a one-time cost

• But then processing strings is fast
Closing the Loop: Reducing DFA to RE

can
reduce
DFA NFA

can transform can transform

RE
Reducing DFAs to REs
General idea
• Remove states one by one, labeling transitions with
regular expressions
• When two states are left (start and final), the
transition label is the regular expression for the DFA

68
DFA to RE example

Language over Σ = {0,1} such that every string is a

multiple of 3 in binary
Other Topics
Minimizing DFA
• Hopcroft reduction
Complementing DFA

70
Minimizing DFAs
Every regular language is recognizable by a
unique minimum-state DFA
• Ignoring the particular names of states
In other words
• For every DFA, there is a unique DFA with minimum
number of states that accepts the same language

c
a b p1 p2
p1 p2 p3
a b
c p3
71
J. Hopcroft, “An n log n algorithm for minimizing states in a finite automaton,” 1971

Minimizing DFA: Hopcroft Reduction

Intuition
• Look to distinguish states from each other
Ø End up in different accept / non-accept state with identical input
Algorithm
• Construct initial partition
Ø Accepting & non-accepting states
• Iteratively split partitions (until partitions remain fixed)
Ø Split a partition if members in partition have transitions to
different partitions for same input
• Two states x, y belong in same partition if and only if for all
symbols in Σ they transition to the same partition
• Update transitions & remove dead states
72
Splitting Partitions
No need to split partition {S,T,U,V}
• All transitions on a lead to identical partition P2
• Even though transitions on a lead to different states
P1 P2
a
S X
a
T Y

U a
a Z
V
73
Splitting Partitions (cont.)
Need to split partition {S,T,U} into {S,T}, {U}
• Transitions on a from S,T lead to partition P2
• Transition on a from U lead to partition P3

P1 P2
a
S X
a
T Y
P4
b
U a P3
Z
74
Resplitting Partitions
Need to reexamine partitions after splits
• Initially no need to split partition {S,T,U}
• After splitting partition {X,Y} into {X}, {Y} we need to split
partition {S,T,U} into {S,T}, {U}

b
P1 a P2
S X
a b
P4 T
a Y
U P3
75
Minimizing DFA: Example 1
DFA a

a b
S T R

b
Initial partitions

Split partition

76
Minimizing DFA: Example 1
DFA a

a b
S T R

b
Initial partitions a
• Accept {R} = P1 b
P2 P1
• Reject { S, T } = P2
Split partition? → Not required, minimization done
• move(S,a) = T ∈ P2 – move(S,b) = R ∈ P1
• move(T,a) = T ∈ P2 – move (T,b) = R ∈ P1
77
Minimizing DFA: Example 2
a
a b
S T R

78
Minimizing DFA: Example 2
DFA a
P2 a b P1
S T R

b P3 DFA
Initial partitions already
• Accept {R} = P1 minimal
• Reject { S, T } = P2
Split partition? → Yes, different partitions for B
• move(S,a) = T ∈ P2 – move(S,b) = T ∈ P2
• move(T,a) = T ∈ P2 – move (T,b) = R ∈ P1
79
Complement of DFA
Given a DFA accepting language L
• How can we create a DFA accepting its complement?
• Example DFA
Ø Σ = {a,b}

82
Complement of DFA
Algorithm
• Add explicit transitions to a dead state
• Change every accepting state to a non-accepting state
& every non-accepting state to an accepting state
Note this only works with DFAs
• Why not with NFAs?

83
Summary of Regular Expression Theory
Finite automata
• DFA, NFA
Equivalence of RE, NFA, DFA
• RE → NFA
Ø Concatenation, union, closure
• NFA → DFA
Ø e-closure & subset algorithm
DFA
• Minimization, complementation

84