Intro

CIT 596 – Theory of Computation 1
Theory of Computation: An Introduction
Traditionally, the study of theory of computation comprises three cen-

tral areas:
automata, computability, and complexity.
These areas are linked by the question:
What are the fundamental capabilities and limitations of computers?
In this course, we will study the fundamentals of these three central ar-
eas of theory of computation.
c Marcelo Siqueira — Spring 2005

The central question of complexity theory is
What makes some problems computationally hard and others easy?.
Remarkably, we still have not been able to answer this question!
An important research achievement, though, was the definition of a

scheme to classify problems according to their “level of difficulty” (diffi-
culty is precisely expressed in this scheme).
What does a computer scientist do when faced with a “hard” computa-

tional problem?
Cryptography: sometimes, hardness is very desirable!


The central question of computability theory is
Which problems can be solved by computers and which ones cannot?
Even before the invention of “real computers”, Kurt Gödel, Alan Turing,
and Alonzo Church discovered that certain basic problems cannot be
solved by computers!
For instance, computers cannot solve the problem of determining whether

a mathematical statement is true or false.
The fact that this discovery is older than the birth of real computers sug-
gests that we do not have to study semicondutors or integrated circuits
in order to understand “computation”.
Actually, we can do that by using “abstract models” of computers.

Automata theory deals with the definitions and properties of “abstract

models” of computers.
These abstract models are models of computation that can be defined

mathematically.
Some of these models are (believed to be) as powerful as real comput-

ers, whereas others can be defined more simply and are less powerful.
The simplest model of computation is the finite automaton or finite-

state machine.
This simple model has several applications in computer science, includ-

ing text processing, compilers, and interface design.

Another important model is the pushdown automaton, which is more

powerful than the finite automaton.
This model provides a natural way of determining whether the sintax

of a statement in a high-level programming language is correct, and
therefore it is heavily used in compiler construction and computational
linguistics.
Finally, there is the Turing machine, named after Alan Turing, the En-
glish mathematician who invented it.
There is general agreement that Turing machines are able to carry out
any algorithm whatsoever.

How do these models of computation work?
These models actually solve decision problems, i.e., computational

problems for which every specific instance can be answered “yes” or
“no”.
For instance, “Given a positive integer n, is it a prime?”
The input value n is encoded as a string of digits, and a computation

that solves the problem starts with this string and ends with a “yes” or
“no”.
This encoding process is no different from what actually happens with

the input data in real computers. Everything is converted into strings of
0’s and 1’s, isn’t it?

Not all computational problems are decision problems. However, in the

case of a problem requiring an answer more complicated than “yes” or
“no”, there is often a decision problem whose solution is comparable to
that of the original.
For instance, if f is a function, being able to answer the question “Given

x and y, is y = f (x)?” is tantamount to being able to compute f (x) for
an arbitrary x.
Furthermore, the more powerful computational models are able to deal

with more general kinds of problems.
An extremely interesting point is the relationship between models of

computation and languages.

We can think of a decision problem as a language recognition prob-

lem.
For instance, for the primality decison problem (“Given a positive integer
n, is n a prime?”), we can view ALL strings for which our model answers
“yes” as a “language”.
So, our decision problem can be viewed as the one of determining

whether an arbitrary string is in the language recognized by the model
of computation.
In our example, the recognition problem is to determine whether a given

string (enconding a number) belongs to the language of ALL strings rep-
resenting prime numbers.

Distinct models of computation recognize (resp. solves) languages (resp.

decision problems) of different complexity.
Finite automata recognize regular languages.
Pushdown automata recognize context-free languages.
Turing machines recognize recursively enumerable languages.
Regular languages are simpler than context-free languages, which in

turn are simpler than recursively enumerable languages. This seems
very natural! Why?

We can also study “languages” from a generation point of view, rather

than recognition.
From the generation point of view, we use a formalism known as a gram-

mar.
A grammar contains a set of rules that can be applied to generate all

strings in a language.
Grammars are extremely useful for specifying and representing pro-

gramming languages.
In order to build a compiler for a programming language, we must spec-

ify the language in some way. Grammars are pretty handy and often
used for that.


As you may expect, we also classify grammars according the complexity
of the languages they can generate.
Regular languages are generated by regular grammars.
Context-free languages are generated by context-free grammars.
Recursively enumerable languages are generated by unrestricted gram-

mars.
This hierarchy of grammars and languages was created by Noam Chom-

sky, and it is known as Chomsky hierarchy.
Yeah, this is the same Noam Chomsky you may have heard about in
the past four years within a completely different context... He is a former
PhD student of UPenn!

Intro

Uploaded by

Copyright:

Available Formats

Intro

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Intro

Uploaded by

Copyright:

Available Formats

CIT 596 – Theory of Computation 1

Theory of Computation: An Introduction

Traditionally, the study of theory of computation comprises three cen-

automata, computability, and complexity.

These areas are linked by the question:

What are the fundamental capabilities and limitations of computers?

c Marcelo Siqueira — Spring 2005

Theory of Computation: An Introduction

The central question of complexity theory is

What makes some problems computationally hard and others easy?.

Remarkably, we still have not been able to answer this question!

An important research achievement, though, was the definition of a

What does a computer scientist do when faced with a “hard” computa-

Cryptography: sometimes, hardness is very desirable!

c Marcelo Siqueira — Spring 2005

Theory of Computation: An Introduction

Which problems can be solved by computers and which ones cannot?

For instance, computers cannot solve the problem of determining whether

Actually, we can do that by using “abstract models” of computers.

c Marcelo Siqueira — Spring 2005

Theory of Computation: An Introduction

Automata theory deals with the definitions and properties of “abstract

These abstract models are models of computation that can be defined

Some of these models are (believed to be) as powerful as real comput-

The simplest model of computation is the finite automaton or finite-

This simple model has several applications in computer science, includ-

c Marcelo Siqueira — Spring 2005

Theory of Computation: An Introduction

Another important model is the pushdown automaton, which is more

This model provides a natural way of determining whether the sintax

c Marcelo Siqueira — Spring 2005

Theory of Computation: An Introduction

How do these models of computation work?

These models actually solve decision problems, i.e., computational

For instance, “Given a positive integer n, is it a prime?”

The input value n is encoded as a string of digits, and a computation

This encoding process is no different from what actually happens with

c Marcelo Siqueira — Spring 2005

Theory of Computation: An Introduction

Not all computational problems are decision problems. However, in the

For instance, if f is a function, being able to answer the question “Given

Furthermore, the more powerful computational models are able to deal

An extremely interesting point is the relationship between models of

c Marcelo Siqueira — Spring 2005

Theory of Computation: An Introduction

We can think of a decision problem as a language recognition prob-

So, our decision problem can be viewed as the one of determining

In our example, the recognition problem is to determine whether a given

c Marcelo Siqueira — Spring 2005

Theory of Computation: An Introduction

Distinct models of computation recognize (resp. solves) languages (resp.

Finite automata recognize regular languages.

Pushdown automata recognize context-free languages.

Turing machines recognize recursively enumerable languages.

Regular languages are simpler than context-free languages, which in

c Marcelo Siqueira — Spring 2005

Theory of Computation: An Introduction

We can also study “languages” from a generation point of view, rather

From the generation point of view, we use a formalism known as a gram-

A grammar contains a set of rules that can be applied to generate all