Finite Automata

Reading: Chapter 2

1
 Finite Automaton (FA)
• Informally, a state diagram that comprehensively captures all
possible states and transitions that a machine can take while
responding to a stream or sequence of input symbols
• Recognizer for “Regular Languages”

• Deterministic Finite Automata (DFA)

– The machine can exist in only one state at any given time
• Non-deterministic Finite Automata (NFA)
– The machine can exist in multiple states at the same time

2
 Deterministic Finite Automata -
Definition
• A Deterministic Finite Automaton (DFA) consists of:
– Q ==> a finite set of states
– ∑ ==> a finite set of input symbols (alphabet)
– q0 ==> a start state
– F ==> set of accepting states
– δ ==> a transition function, which is a mapping between Q
x ∑ ==> Q
• A DFA is defined by the 5-tuple:
– {Q, ∑ , q0,F, δ }

3
 What does a DFA do on reading an
input string?
• Input: a word w in ∑*
• Question: Is w acceptable by the DFA?
• Steps:
– Start at the “start state” q0
– For every input symbol in the sequence w do
• Compute the next state from the current state, given the current
input symbol in w and the transition function
– If after all symbols in w are consumed, the current state is
one of the accepting states (F) then accept w;
– Otherwise, reject w.

4
 Regular Languages
• Let L(A) be a language recognized by a DFA A.
– Then L(A) is called a “Regular Language”.

• Locate regular languages in the Chomsky

Hierarchy

5
 The Chomsky Hierachy
• A containment hierarchy of classes of formal languages

Regular Context-
(DFA) Context- Recursively-
free
sensitive
(PDA) enumerable
(LBA)
(TM)

6
 Example #1
• Build a DFA for the following language:
– L = {w | w is a binary string that contains 01 as a substring}
• Steps for building a DFA to recognize L:
– ∑ = {0,1}
– Decide on the states: Q
– Designate start state and final state(s)
– δ: Decide on the transitions:
• “Final” states == same as “accepting states”
• Other states == same as “non-accepting states”

7
 Regular expression: (0+1)*01(0+1)*

DFA for strings containing 01

• What makes this DFA deterministic? • Q = {q0,q1,q2}

1 0 0,1 • ∑ = {0,1}
• start state = q0
start 0 1
q0 q1 q2 • F = {q2}
Accepting • Transition table
state symbols
0 1
q0 q1 q0
states

• What if the language allows q1 q1 q2

empty strings? *q2 q2 q2
8
Construct DFA to accept (0+1)*

1
Construct DFA to accept 00(0+1)*

0
0 0

1 1
1

0 1
Give a DFA for Σ={0,1} and Strings that have an odd
number of 1’s and any number of 0’s
Give a DFA for Σ={0,1} that accepts any string that aab
as substring

12
Give a DFA for Σ={0,1} that accepts any string that
aababb as substring

13
 Example #2
Clamping Logic:
– A clamping circuit waits for a ”1” input, and turns on forever. However,
to avoid clamping on spurious noise, we’ll design a DFA that waits for
two consecutive 1s in a row before clamping on.
• Build a DFA for the following language:
L = { w | w is a bit string which contains the substring 11}
• State Design:
– q0 : start state (initially off), also means the most recent input was not
a1
– q1: has never seen 11 but the most recent input was a 1
– q2: has seen 11 at least once

14
 Example #3
• Build a DFA for the following language:
L = { w | w is a binary string that has even
number of 1s and even number of 0s}
• ?

15
Extension of transitions (δ) to Paths (δ)
• δ (q,w) = destination state from state q on
input string w

• δ (q,wa) = δ (δ(q,w), a)

– Work out example #3 using the input sequence

w=10010, a=1:

• δ (q0,wa) = ?

16
 Language of a DFA
A DFA A accepts string w if there is a path from
q0 to an accepting (or final) state that is
labeled by w

• i.e., L(A) = { w | δ(q0,w)  F }

• I.e., L(A) = all strings that lead to an accepting

state from q0

17
 Non-deterministic Finite Automata
(NFA)
• A Non-deterministic Finite Automaton (NFA)
– is of course “non-deterministic”
• Implying that the machine can exist in more than one
state at the same time
• Transitions could be non-deterministic

1 qj
qi … • Each transition function therefore
1 maps to a set of states
qk
18
 Non-deterministic Finite Automata
(NFA)
• A Non-deterministic Finite Automaton (NFA) consists
of:
– Q ==> a finite set of states
– ∑ ==> a finite set of input symbols (alphabet)
– q0 ==> a start state
– F ==> set of accepting states
– δ ==> a transition function, which is a mapping between Q
x ∑ ==> subset of Q
• An NFA is also defined by the 5-tuple:
– {Q, ∑ , q0,F, δ }

19
 How to use an NFA?
• Input: a word w in ∑*
• Question: Is w acceptable by the NFA?
• Steps:
– Start at the “start state” q0
– For every input symbol in the sequence w do
• Determine all possible next states from all current states, given the
current input symbol in w and the transition function
– If after all symbols in w are consumed and if at least one of the current
states is a final state then accept w;
– Otherwise, reject w.

20
 Regular expression: (0+1)*01(0+1)*

NFA for strings containing 01

Why is this non-deterministic?

• Q = {q0,q1,q2}
0,1 0,1 •  = {0,1}
• start state = q0
start 0 1
q0 q1 q2 • F = {q2}
Final • Transition table
state symbols
0 1
What will happen if at state q1 q0 {q0,q1} {q0}
states

an input of 0 is received? q1 Φ {q2}

*q2 {q2} {q2}
21
Note: Omitting to explicitly show error states is just a matter of design convenience
(one that is generally followed for NFAs), and
i.e., this feature should not be confused with the notion of non-determinism.

What is an “error state”?

• A DFA for recognizing the key word “while”

w h i l e
q0 q1 q2 q3 q4 q5

Any other input symbol

• An NFA for the same purpose:q err

Any symbol

w h i l e
q0 q1 q2 q3 q4 q5

Transitions into a dead state are implicit 22

 Example #2
• Build an NFA for the following language:
L = { w | w ends in 01}
• ?
• Other examples
– Keyword recognizer (e.g., if, then, else, while, for,
include, etc.)
– Strings where the first symbol is present
somewhere later on at least once

23
 Extension of δ to NFA Paths
• Basis: δ (q,) = {q}

• Induction:
– Let δ (q0,w) = {p1,p2…,pk}
– δ (pi,a) = Si for i=1,2...,k

– Then, δ (q0,wa) = S1 U S2 U … U Sk

24
 Language of an NFA
• An NFA accepts w if there exists at least one
path from the start state to an accepting (or
final) state that is labeled by w
• L(N) = { w | δ(q0,w) ∩ F ≠ Φ }

25
 Advantages & Caveats for NFA
• Great for modeling regular expressions
– String processing - e.g., grep, lexical analyzer

• Could a non-deterministic state machine be

implemented in practice?
– Probabilistic models could be viewed as extensions of non-deterministic state
machines
(e.g., toss of a coin, a roll of dice)
• They are not the same though
– A parallel computer could exist in multiple “states” at the same time

26
 Technologies for NFAs

• Micron’s Automata Processor (introduced in 2013)

• 2D array of MISD (multiple instruction single data) fabric
w/ thousands to millions of processing elements.
• 1 input symbol = fed to all states (i.e., cores)
• Non-determinism using circuits
• http://www.micronautomata.com/

27
 Differences: DFA vs. NFA
But, DFAs and NFAs are equivalent in their power to capture langauges !!

• DFA • NFA
1. All transitions are deterministic 1. Some transitions could be non-
– Each transition leads to exactly one deterministic
state – A transition could lead to a subset
2. For each state, transition on all of states
possible symbols (alphabet) should 2. Not all symbol transitions need to
be defined be defined explicitly (if undefined
3. Accepts input if the last state will go to an error state – this is just
visited is in F a design convenience, not to be
4. Sometimes harder to construct confused with “non-determinism”)
because of the number of states 3. Accepts input if one of the last
5. Practical implementation is feasible states is in F
4. Generally easier than a DFA to
construct
5. Practical implementations limited
but emerging (e.g., Micron
automata processor)

28
 Equivalence of DFA & NFA
• Theorem:
– A language L is accepted by a DFA if and only if it is
Should be accepted by an NFA.
•
true for
any L
Proof:
1. If part:
• Prove by showing every NFA can be converted to an equivalent
DFA (in the next few slides…)

2. Only-if part is trivial:

• Every DFA is a special case of an NFA where each state has
exactly one transition for every input symbol. Therefore, if L is
accepted by a DFA, it is accepted by a corresponding NFA.

29
 Proof for the if-part
• If-part: A language L is accepted by a DFA if it is
accepted by an NFA
• rephrasing…
• Given any NFA N, we can construct a DFA D such that
L(N)=L(D)
• How to convert an NFA into a DFA?
– Observation: In an NFA, each transition maps to a subset of
states
– Idea: Represent:
each “subset of NFA_states”  a single “DFA_state”

Subset construction 30
 NFA to DFA by subset construction

• Let N = {QN,∑,δN,q0,FN}
• Goal: Build D={QD,∑,δD,{q0},FD} s.t. L(D)=L(N)
• Construction:
1. QD= all subsets of QN (i.e., power set)
2. FD=set of subsets S of QN s.t. S∩FN≠Φ
3. δD: for each subset S of QN and for each input
symbol a in ∑:
• δD(S,a) = U δN(p,a)

p in s

31
 Idea: To avoid enumerating all of
power set, do
NFA to DFA construction: Example “lazy creation of states”

• L = {w | w ends in 01}
1 0
NFA: DFA: 0 1
0,1 [q0] [q0,q1] [q0,q2]
0
0 1 1
q0 q1 q2

δD 0 1 δD 0 1
δN 0 1
Ø Ø Ø [q0] [q0,q1] [q0]
q0 {q0,q1} {q0} [q0] {q0,q1} {q0} [q0,q1] [q0,q1] [q0,q2]
q1 Ø {q2} [q1] Ø {q2} *[q0,q2] [q0,q1] [q0]
*q2 Ø Ø *[q2] Ø Ø
[q0,q1] {q0,q1} {q0,q2}
*[q0,q2] {q0,q1} {q0} 0. Enumerate all possible subsets
*[q1,q2] Ø {q2} 1. Determine transitions
*[q0,q1,q2] {q0,q1} {q0,q2} 2. Retain only those states
reachable from {q0}
32
 NFA to DFA: Repeating the example using LAZY
CREATION
• L = {w | w ends in 01}
1 0
NFA: DFA: 0 1
0,1 [q0] [q0,q1] [q0,q2]
0
0 1 1
q0 q1 q2

δN 0 1 δD 0 1
q0 {q0,q1} {q0} [q0] [q0,q1] [q0]
q1 Ø {q2} [q0,q1] [q0,q1] [q0,q2]
*q2 Ø Ø *[q0,q2] [q0,q1] [q0]

Main Idea:
Introduce states as you go
(on a need basis)
33
 Correctness of subset construction

Theorem: If D is the DFA constructed from NFA N

by subset construction, then L(D)=L(N)
• Proof:
– Show that δD({q0},w) ≡ δN(q0,w} , for all w
– Using induction on w’s length:
• Let w = xa
• δD({q0},xa) ≡ δD( δN(q0,x}, a ) ≡ δN(q0,w}

34
 A bad case where
#states(DFA)>>#states(NFA)
• L = {w | w is a binary string s.t., the kth symbol from
its end is a 1}

– NFA has k+1 states

– But an equivalent DFA needs to have at least 2k states

(Pigeon hole principle)

– m holes and >m pigeons
• => at least one hole has to contain two or more pigeons

35
 Applications
• Text indexing
– inverted indexing
– For each unique word in the database, store all locations
that contain it using an NFA or a DFA
• Find pattern P in text T
– Example: Google querying
• Extensions of this idea:
– PATRICIA tree, suffix tree

36
 A few subtle properties of DFAs and
NFAs
• The machine never really terminates.
– It is always waiting for the next input symbol or making transitions.
• The machine decides when to consume the next symbol from the input
and when to ignore it.
– (but the machine can never skip a symbol)
• => A transition can happen even without really consuming an input symbol
(think of consuming  as a free token) – if this happens, then it becomes
an -NFA (see next few slides).
• A single transition cannot consume more than one (non-) symbol.

37
 FA with -Transitions
• We can allow explicit -transitions in finite automata
– i.e., a transition from one state to another state without
consuming any additional input symbol
– Explicit -transitions between different states introduce
non-determinism.
– Makes it easier sometimes to construct NFAs
Definition:  -NFAs are those NFAs with at least one
explicit -transition defined.
•  -NFAs have one more column in their transition
table
38
 Example of an -NFA
L = {w | w is empty, or if non-empty will end in 01}
0,1

0 1
• -closure of a state q,
q0 q1 q2 ECLOSE(q), is the set of all

states (including itself)
start q’0 that can be reached from
q by repeatedly making an
δE 0 1
 arbitrary number of -
*q’0 Ø Ø {q’0,q0}
ECLOSE(q’0) transitions.
q0 {q0,q1} {q0} {q0} ECLOSE(q0)
q1 Ø {q2} {q1} ECLOSE(q1)
*q2 Ø Ø {q2} 39
ECLOSE(q2)
 To simulate any transition:
Step 1) Go to all immediate destination states.
Step 2) From there go to all their -closure states as well.
Example of an -NFA
L = {w | w is empty, or if non-empty will end in 01}
0,1

0 1
Simulate for w=101:
q0 q1 q2
 q0’
 
start q’0
q0’ q0
1 1
q0
δE

0 1 Ø
x 0
ECLOSE(q’0)
*q’0 Ø Ø {q’0,q0} q1
q0 {q0,q1} {q0} {q0} ECLOSE(q0) 1
q1 Ø {q2} {q1} q2
*q2 Ø Ø {q2} 40
 To simulate any transition:
Step 1) Go to all immediate destination states.
Step 2) From there go to all their -closure states as well.
Example of another -NFA
0,1 Simulate for w=101:
?
0 1
q0 q1 q2
  1

start q’0 q3

δE 0 1

*q’0 Ø Ø {q’0,q0,q3}
q0 {q0,q1} {q0} {q0,q3}
q1 Ø {q2} {q1}
*q2 Ø Ø {q2}
q3 Ø {q2} {q3}

41
 Equivalency of DFA, NFA, -NFA
• Theorem: A language L is accepted by some -
NFA if and only if L is accepted by some DFA

• Implication:
– DFA ≡ NFA ≡ -NFA
– (all accept Regular Languages)

42
 Eliminating -transitions
Let E = {QE,∑,δE,q0,FE} be an -NFA
Goal: To build DFA D={QD,∑,δD,{qD},FD} s.t. L(D)=L(E)
Construction:
1. QD= all reachable subsets of QE factoring in -closures
2. qD = ECLOSE(q0)
3. FD=subsets S in QD s.t. S∩FE≠Φ
4. δD: for each subset S of QE and for each input symbol a∑:
• Let R= U δE(p,a) // go to destination states

• δD(S,a) = U ECLOSE(r) // from there, take a union

p in s of all their -closures

r in R

Reading: Section 2.5.5 in book 43

 Example: -NFA  DFA
L = {w | w is empty, or if non-empty will end in 01}
0,1

0 1
q0 q1 q2

start q’0

δE 0 1
 δD 0 1

*q’0 Ø Ø {q’0,q0} *{q’0,q0}

q0 {q0,q1} {q0} {q0} …
q1 Ø {q2} {q1}
*q2 Ø Ø {q2} 44
 Example: -NFA  DFA
L = {w | w is empty, or if non-empty will end in 01}
0,1 0
0
0 1 {q0,q1}
q0 q1 q2 0 1 {q0,q2}
 0
1
start start {q’0, q0} 1
q’0 q0 1
ECLOSE union
δE 0 1
 δD 0 1

*q’0 Ø Ø {q’0,q0} *{q’0,q0} {q0,q1} {q0}

q0 {q0,q1} {q0} {q0} {q0,q1} {q0,q1} {q0,q2}
q1 Ø {q2} {q1} {q0} {q0,q1} {q0}
*q2 Ø Ø {q2} *{q0,q2} {q0,q1} {q0} 45
 Summary
• DFA
– Definition
– Transition diagrams & tables
• Regular language
• NFA
– Definition
– Transition diagrams & tables
• DFA vs. NFA
• NFA to DFA conversion using subset construction
• Equivalency of DFA & NFA
• Removal of redundant states and including dead states
• -transitions in NFA
• Pigeon hole principles
• Text searching applications

46