Chapter 7 & 8

Computability
and complexity theory
Computability Theory
Computability theory answers:
What problems can computers solve?

 Why we want to know if a problem is unsolvable?

 to realize that the problem must be simplified or altered
before you can find an algorithmic solution.
Complexity theory
Measuring Complexity
Suppose that we have a decider D for some
language L.

How might we measure the complexity of D?

Number of states.
Size of tape alphabet.
Size of input alphabet.
Amount of tape required.
Number of steps required. Or ?
Time complexity
 A step of a Turing machine is one event where the
TM takes a transition.
 Running a TM on different inputs might take a
different number of steps.
 The number of steps a TM takes on some
input is sensitive to
The structure of that input. T
he length of the input.
How can we come up with a consistent
measure of a machine's runtime?
 Time Complexity
The time complexity of a TM M is a function (typically
denoted f(n)) that measures the worst-case number of
steps M takes on any input of length n.

By convention, n denotes the length of the input.

Example: L = { w Σ* | w has the same number of 0s and 1s }
Solution: M = “On input w: –
Scan across the tape until a 0 or 1 is found.
If none are found, accept.
If one is found, continue scanning until a matching 1 or 0 is found.
If none is found, reject.
Otherwise, cross off that symbol and repeat.”
What is the time complexity of M?
Example
Scan across the tape until a 0 or 1 is found. At most n steps
If none are found, accept. At most 1 step
If one is found, continue scanning until ….. At most n step
a matching 1 or 0 is found.
If none is found, reject. ………………… At most 1 step
Otherwise, cross off that symbol and
go back to start of tape………………………………….. At most n step
repeat.” …………………………………………………….n/2
Accept………………………..atmost 1 step

Hence f(n) = n/2(n+n) + 1 = n2 + 1

An Easier Approach
In complexity theory, we rarely need an exact value
for a TM's time complexity.
Usually, we are curious with the long-term growth
rate of the time complexity.

For example,
if the time complexity is 3n + 5, then doubling the length of the string roughly
doubles the worst-case runtime.
If the time complexity is 2n – n2 , since 2n grows much more quickly than n2 , for
large values of n, increasing the size of the input by 1 doubles the worst-case
running time
Big-O Notation

Ignore everything except the dominant growth

term, including constant factors.

Examples:
4n + 4 = O(n)
137n + 271 = O(n)
n 2 + 3n + 4 = O(n2)
2 n + n 3 = O(2n)
137 = O(1)
n 2 log n + log5 n = O(n2 log n)
Big-O Notation, Formally

Let f : ℕ → ℕ and g : ℕ → ℕ.
Then f(n) = O(g(n)) iff there exist constants c ℝ
and n0 ℕ such that
For any n ≥ n0 , f(n) ≤ cg(n)

Intuitively, as n gets “large” (greater than n0 ), f(n)

is bounded from above by some multiple
(determined by c) of g(n)
 Properties of Big-O Notation
Theorem: If f 1 (n) = O(g1 (n)) and f 2 (n) = O(g2 (n)), then f
1 (n) + f 2 (n) = O(g1 (n) + g2 (n)).
Intuitively: If you run two programs one after another, the big-O of the result is
the big-O of the sum of the two runtimes.
Theorem: If f 1 (n) = O(g1 (n)) and f 2 (n) = O(g2 (n)), then f
1 (n)f 2 (n) = O(g1 (n)g2 (n)).
Intuitively: If you run one program some number of times, the big-O of the
result is the big-O of the program times the big-O of the number of iterations.

This makes it substantially easier to analyze time

complexity, though we do lose some precision.
Classes of problems
P Vs. NP
Efficiency
 What does it mean for an algorithm/ TM to be efficient?
 We define running time as the function of input length n
 When an algorithm has a running time O(n2) it means that
for a long enough inputs the algorithm takes no more than
quadratic time.
 An algorithm is efficient if its running time is polynomial.
More precisely a running time is polynomial if it is O(nc)
for some constant c.
 Polynomial: n2, n100,nlogn
 Non polynomial: 2n,n!,2logn
NP-Hard and NP-Complete problems

• P = the set of problems that are solvable in polynomial

time.

• If the problem has size n, the problem should be solved in

nO(1).
• NP = the set of decision problems solvable in
nondeterministic polynomial time.
Decision Problems
• The output of these problems is a YES or NO answer.
• Example: is N prime? Do sequences x and y have common subsequences of length>k?
• For many problems there is an associated decision problem.

• Nondeterministic refers to the fact that a solution can be

guessed out of polynomially many options in O(1) time.
• If any guess is a YES instance, then the nondeterministic
algorithm will make that guess.
• In this model of nondeterminism, we can assume that all
guessing is done first.
• This is equivalent to finding a polynomial-time verifier of
polynomial-size certificates for YES answers.
NP

Given a solution to an instance of a decision problem, we

want to verify if it actually is a solution (is this sequence
actually a subsequence of x and y?)

15
NP

Given a solution to an instance of a decision problem, we want to

verify if it actually is a solution (is this sequence actually a
subsequence of x and y?)
There are many decision problems where we can efficiently
verify solutions, but can’t efficiently find them.

Example: Hamiltonian Cycle: In the graph G is there a simple cycle that

goes through every vertex?

16
NP

Given a solution to an instance of a decision problem, we

want to verify if it actually is a solution (is this sequence
actually a subsequence of x and y?)
There are many decision problems where we can efficiently
verify solutions, but can’t efficiently find them.
Example: Hamiltonian Cycle: In the graph G is there a simple
cycle that goes through every vertex?
There is no known poly-time algorithm for this but it is easy
to verify if a given cycle is a Hamiltonian cycle.

17
NP

Given a solution to an instance of a decision problem, we

want to verify if it actually is a solution (is this sequence
actually a subsequence of x and y?)
There are many decision problems where we can efficiently
verify solutions, but can’t efficiently find them.
Example: Hamiltonian Cycle: In the graph G is there a simple
cycle that goes through every vertex?
There is no known poly-time algorithm for this but it is easy
to verify if a given cycle is a Hamiltonian cycle.
NP is the class of decision problems where if the answer is
YES then there is a short “solution” which can be easily
verified.

18
Maximum Clique
7
17 8

16 1 9
6 2

15 10
5 3
4
14 11
13 12

Given a graph G and a number K, are there K vertices in G

that are all pairwise adjacent (this is called a clique)?

19
Maximum Clique
7
17 8

16 1 9
6 2

15 10
5 3
4
14 11
13 12

Given a graph G and a number K, are there K vertices in G

that are all pairwise(very 2 of w/c are adjacent) adjacent
(this is called a clique)?
Easy to verify since given the vertices we only need check that
they are all adjacent.
Useful for finding groups of mutual friends in social networks.
20
3-Coloring

1
10 7
5 2

4 3

9 8

Given a graph G, can we assign 3 colors to its vertices so that

any pair of adjacent vertices have different colors?

21
3-Coloring

1
10 7
5 2

4 3

9 8

Given a graph G, can we assign 3 colors to its vertices so that

any pair of adjacent vertices have different colors?
Easy to verify a coloring by examining all edges so it is in NP.
Useful for allocating transmission frequencies to radio stations
to avoid interference.
22
Partition Problem

1 5 2 4 3 7 1 5 9 4 3 8 10 2 6

1 5 2 4 3 7 ?
Given n numbers, can they be partitioned into 2 sets such
that the sums of the numbers in the sets are equal

23
Partition Problem

1 5 2 4 3 7 1 5 9 4 3 8 10 2 6

1 5 2 4 3 7 ?
Given n numbers, can they be partitioned into 2 sets such
that the sums of the numbers in the sets are equal
Easy to verify given the two sets, so it is in NP.

24
P v NP

Recall that for a decision problem in NP, if the answer is yes

for a given input then the “solution” can be verified efficiently

25
P v NP

Recall that for a decision problem in NP, if the answer is yes for
a given input then the “solution” can be verified efficiently
But can we compute the solution efficiently?

26
P v NP

Recall that for a decision problem in NP, if the answer is yes for
a given input then the “solution” can be verified efficiently
But can we compute the solution efficiently?
We don’t know! This isthe P =? NP question.

27
Hard Problems

?
One approach to settling P = NP: Identify the “hardest”
problems in NP and focus on solving them in poly-time

28
Hard Problems

?
One approach to settling P = NP: Identify the “hardest”
problems in NP and focus on solving them in poly-time
But how do we know which problems are hard?

29
Hard Problems

?
One approach to settling P = NP: Identify the “hardest”
problems in NP and focus on solving them in poly-time
But how do we know which problems are hard?
Idea: Reductions

30
Reductions

Input x to A
We can reduce problem A to problem B if we
can use a solution to B to solveA.
reduction f
f(x)
Algorithm for B

Output for A

31
Reductions

Input x to A
We can reduce problem A to problem B if we
can use a solution to B to solveA.
reduction f
Example: Problem A is finding the median of n
f(x) elements and Problem B is sorting n elements.
Algorithm for B

Output for A

32
Reductions

Input x to A
We can reduce problem A to problem B if we
can use a solution to B to solveA.
reduction f
Example: Problem A is finding the median of n
f(x) elements and Problem B is sorting n elements.
A reduces to B since we can take the input
Algorithm for B
to A, sort it using the solution to B, and
then recover the solution to A by looking at
the middle element in sorted order.
Output for A

33
Reductions cont.

The median and sorting example illustrates reductions, but it

is a bad example since medians can be computed directly
faster than sorting.

34
Reductions cont.

The median and sorting example illustrates reductions, but it

is a bad example since medians can be computed directly
faster than sorting.
However if we go the other way we can use the median finding
algorithm to make an efficient sorting algorithm.

35
Reductions cont.

The median and sorting example illustrates reductions, but it

is a bad example since medians can be computed directly
faster than sorting.
However if we go the other way we can use the median finding
algorithm to make an efficient sorting algorithm.
Sort n elements, compute the median using the black box for A
Use the median as a pivot like in quicksort and recurse.
We will always have a perfect partition so the algorithm will be
efficient.

36
Reduction Definition

Input to A
We need a more rigorous definition of a
reduction to identify the hardest problems in NP.
f (x)

Algorithm for B

Output for A

37
Reduction Definition

Input to A
We need a more rigorous definition of a reduction
to identify the hardest problems in NP.
f (x) We say Decision Problem A reduces to Decision
problem B if there is a function f mapping
inputs of A to inputs of B such that:
If x is a YES input for A, then f (x) is a YES
Algorithm for B input for B
IF x is a NO input for A, then f (x) is a NO
input for B
Output for A

38
Reduction Definition

Input to A
We need a more rigorous definition of a reduction
to identify the hardest problems in NP.
f (x) We say Decision Problem A reduces to Decision
problem B if there is a function f mapping
inputs of A to inputs of B such that:
If x is a YES input for A, then f (x) is a YES
Algorithm for B input for B
IF x is a NO input for A, then f (x) is a NO
input for B
Output for A The reduction is f itself.

39
Poly-Time Reductions

If f (x ) can be computed in polynomial time, we say it is a

polynomial-time reduction.
Note: If f is a polynomial-time reduction, then |f (x )| is
polynomial in the length of x.

40
Poly-Time Reductions

If f (x ) can be computed in polynomial time, we say it is a

polynomial-time reduction.
Note: If f is a polynomial-time reduction, then |f (x )| is
polynomial in the length of x.
Median finding and sorting are not decision problems, but
otherwise the median finding to sorting reduction fits this
definition.
What is f (x)? Is it in computable poly-time?
However, the other direction does not fit since we repeatedly
use the median finding algorithm

41
Poly-Time Reduction Implications

What does it mean if there is a poly-time reduction from A to

B?

42
Poly-Time Reduction Implications

What does it mean if there is a poly-time reduction from A to

B?
If B is solvable in poly time, then so is A. Just map the input
to A to an input to B, and use the method for solving B
(recall that if f (x ) and g (x ) are polynomials then f (g (x )) is
also a polynomial).

43
Poly-Time Reduction Implications

What does it mean if there is a poly-time reduction from A to

B?
If B is solvable in poly time, then so is A. Just map the input
to A to an input to B, and use the method for solving B
(recall that if f (x ) and g (x ) are polynomials then f (g (x )) is
also a polynomial).
If A is not solvable in polynomial time, then neither is B.
This statement is actually equivalent to the original one.

44
Example

Suppose we know that if one could travel faster than the

speed of light, then one could travel back in time.

45
Example

Suppose we know that if one could travel faster than the

speed of light, then one could travel back in time.
Using our language, the problem of traveling back to the past
reduces to the problem of traveling faster than the speed of
light

46
Example

Suppose we know that if one could travel faster than the

speed of light, then one could travel back in time.
Using our language, the problem of traveling back to the past
reduces to the problem of traveling faster than the speed of
light
If we manage to build a faster-than-light vehicle, then we can
go back to the past
But if we prove that is impossible to travel back in time, then
we immediately know it is impossible to build a
faster-than-light vehicle.

47
NP −Completeness Definition

Suppose we have a decision problem A in NP such that for

every problem in NP there is a polynomial time reduction to
A.

48
NP −Completeness Definition

Suppose we have a decision problem A in NP such that for

every problem in NP there is a polynomial time reduction to
A.
If we knew A is solvable in polynomial time, then every
problem in NP can be solved in polynomial time.
Equivalently, A P = P = NP.

49
NP −Completeness Definition

Suppose we have a decision problem A in NP such that for

every problem in NP there is a polynomial time reduction to
A.
If we knew A is solvable in polynomial time, then every
problem in NP can be solved in polynomial time.
Equivalently, A P = P = NP.
In a sense, A is a “hardest” problem in NP.
We say a problem A is NP-complete if
A NP
Every problem in NP reduces to A

50
Satisfiability

How could we ever show that every problem in NP reduces

to a problem A?

51
Satisfiability

How could we every show that every problem in NP reduces

to a problem A?
Cook and Levin did exactly this for the Satisfiability problem.
Satisfiability: Given a boolean formula, is there a way to set
the variables such that the formula evaluates to true?

52
Satisfiability

53
Satisfiability

How could we every show that every problem in NP reduces

to a problem A?
Cook and Levin did exactly this for the Satisfiability problem.
Satisfiability: Given a boolean formula, is there a way to set
the variables such that the formula evaluates to true?
For example, ((¬a¬ v ¬ b v c ) x (a v c) x(¬c
¬ v b)) v (¬a
¬ x bx
c)
is satisfied by a 1, b 0, c 0.
Cook-Levin uses the fact that every problem in NP has a
poly-time verifier to construct a formula that is satisfiable if
and only if there is a “solution” that the verifier will accept.

54
Properties of Reductions

Input to A f (x) Algorithm for B Output for A

Input to B g(x) Algorithm for C Output for B

Input to A f (x) g(x) Algorithm for C Output for A

Polynomial-time reductions are transitive: If A reduces to B

and B reduces to C, then A reduces to C

55
Properties of Reductions

Input to A f (x) Algorithm for B Output for A

Input to B g(x) Algorithm for C Output for B

Input to A f (x) g(x) Algorithm for C Output for A

Polynomial-time reductions are transitive: If A reduces to B

and B reduces to C, then A reduces to C
After Cook-Levin, to show a problem X is NP-complete we
need only show that X NP and that Satisfiability or another
NP-complete problem reduces to X .

56