Algorithm Analysis and Design

Approximation Algorithm
 Overview
•
Approximation Algorithms:
•
Well, your boss understands it's a hard problem, but he still wants you to do something about it. Is
there a way to get a solution to be "good enough" without using a huge amount of computer time
delaying the orders?

•
There are three broad approaches to handing NP-Complete or NP-Hard problems in practice:

 Stick with small problems, where the total execution time for an optimal solution is not bad. Your
boss rejects this as it would limit the configuration options the company offers.

 Find special cases of the problem that can be solved in polynomial time (e.g., 2-CNF rather than 3-
CNF). It requires that we know more about the structure of the problem. Sometimes we don't know
much about it.

 Find near-optimal solutions with approximation algorithms. Your boss thinks it just might work:
since the problem is hard, customers won't realize you haven't given them the optimal solution as
long as a lot of their requests are met. This is the approach we'll examine today.
  Approximation Algorithms

•
Up to now, the best algorithm for solving an NP-
complete problem requires exponential time in
the worst case. It is too time-consuming.
•
To reduce the time required for solving a problem,
we can relax the problem, and obtain a feasible
solution “close” to an optimal solution.
 Definitions
•
Let C be the cost of a solution found for a problem of size n
and C* be the optimal solution for that problem.

•
Then we say an algorithm has an approximation ratio of ρ(n) if

–
C/C* ≤ ρ(n) for minimization problems: the factor by which the
actual solution obtained is larger than the optimal solution.

–
C*/C ≤ ρ(n) for maximization problems: the factor by which the
optimal solution is larger than the solution obtained.
•
Both of these statements can be written together as
follows:

•
The ratio is never less than 1 (perfect performance).
•
An algorithm that has an approximation ratio of ρ(n) is
called a ρ(n)-approximation algorithm.
•
For example, if the ratio is 2 we have a 2-approximation
algorithm, meaning it gives solutions that never cost more
than twice that of optimal if it is a minimization problem, or
never provide less than half the optimal value if it is a
maximization problem.
•
An approximation scheme is a parameterized approximation
algorithm that takes an additional input ε > 0. And for any
fixed ε, there is a (1+ε)-approximation algorithm.

•
ε is how much slack you will allow away from the optimal
1-"approximation".

•
For example, if ε = 1 we have a 2-approximation scheme.

•
An approximation scheme is a polynomial approximation
scheme if for any fixed ε > 0 the scheme runs in time
polynomial in input size n.
•
Examples:

–
Vertex Cover Approximations

–
TSP Approximations
 • Vertex Cover Approximations
•
A vertex cover of an undirected graph G = (V, E) is a
subset V′ ⊆ V such that if (u, v) ∈ E then u ∈ V′ or v
∈ V′ or both.

•
The optimization version of the Vertex Cover
Problem is to find a vertex cover of minimum size in
G.

•
The corresonding decision problem is NP-Complete,
so the optimization problem is NP-Hard,.
 Approx-Vertex-Cover
•
Vertex Cover can be approximated by the following surprisingly simple
algorithm, which iterately chooses an edge that is not covered yet and covers
it using both incident vertices:
•
Example:

•
Input Graph:

•
Suppose then that edge {b, c} is chosen. The two incident
vertices are added to the cover and all other incident edges
are removed from consideration:
•
Iterating now for edges {e, f} and then {d, g}:

•
The resulting vertex cover is shown on the left and the optimal
vertex on the right (the lighter colored vertices are in the cover:
 Analysis
•
How good is the approximation? We can show that the
solution is within a factor of 2 of optimal.

•
Theorem: Approx-Vertex-Cover is a polynomial time 2-
approximation algorithm for Vertex Cover.

•
Proof: Vertex Cover: The algorithm constructs a vertex
cover because it loops until every edge in E has been
covered.
•
Polynomial: The algorithm has O(|E|) iterations of the loop, and (using aggregate analysis,
Topic 15) across all loop iterations, O(|V|) vertices are added to C. Therefore it is O(E + V),
so is polynomial.

•
2-Approximation: It remains to be shown that the solution is no more than twice the size of
the optimal cover. We'll do so by finding a lower bound on the optimal solution C*.

•
Let A be the set of edges chosen in line 4 of the algorithm. Any vertex cover must cover at
least one endpoint of every edge in A.

•
No two edges in A share a vertex, because for any two edges one of them must be selected
first (line 4), and all edges sharing vertices with the selected edge are deleted from E′ in line
6, so are no longer candidates on the next iteration of line 4. Therefore, in order to cover A,
the optimal solution C* must have at least as many vertices as edges in A:
| A | ≤ | C* |
•
Since each execution of line 4 picks an edge for which neither endpoint is yet in
C and line 5 adds these two vertices to C, then we know that
|C| = 2|A|

•
Therefore:
| C | ≤ 2 | C* |
That is, |C| cannot be larger than twice the optimal, so is a 2-approximation
algorithm for Vertex Cover.

•
This is a common strategy in approximation proofs: we don't know the size of
the optimal solution, but we can set a lower bound on the optimal solution and
relate the obtained solution to this lower bound.
 • TSP Approximations
(Approximate using MST)
•
Travelling Salesman Problem has Naive and
Dynamic Programming Solutions.
–
Both infeasible solutions
–
No polynomial time solution
–
NP Hard problem

•
There are approximate algorithms to solve the
problem though. The approximate algorithms work
only if the problem instance satisfies Triangle-
Inequality.
•
Triangle-Inequality: The least distant path to
reach a vertex j from i is always to reach j directly
from i, rather than through some other vertex k (or
vertices), i.e., dis(i, j) is always less than or equal to
dis(i, k) + dist(k, j). The Triangle-Inequality holds in
many practical situations.

•
When the cost function satisfies the triangle
inequality, we can design an approximate algorithm
for TSP that returns a tour whose cost is never more
than twice the cost of an optimal tour. The idea is to
use Minimum Spanning Tree (MST). Following is the
MST based algorithm.
 Algorithm

1) Let 1 be the starting and ending point for salesman.

2) Construct MST from with 1 as root using Prim’s

3) List vertices visited in preorder walk of the

• In this case, the approximate algorithm produces the optimal tour, but it may not
produce optimal tour in all cases.
•
How is this algorithm 2-approximate?

•
The cost of the output produced by the above algorithm is
never more than twice the cost of best possible output. Let us
see how is this guaranteed by the above algorithm.

Let us define a term full walk to understand this. A full walk is

lists all vertices when they are first visited in preorder, it also li
st vertices when they are returned after a subtree is visited in pr
eorder. The full walk of above tree would be 1-2-1-4-1-3-1.
•
Following are some important facts that prove the 2-approximateness.
1) The cost of best possible Travelling Salesman tour is never less than the
cost of MST. (The definition of MST says, it is a minimum cost tree that
connects all vertices).
2) The total cost of full walk is at most twice the cost of MST (Every edge of
MST is visited at-most twice)
3) The output of the above algorithm is less than the cost of full walk. In
above algorithm, we print preorder walk as output. In preorder walk, two or
more edges of full walk are replaced with a single edge. For example, 2-1 and
1-4 are replaced by 1 edge 2-4. So if the graph follows triangle inequality,
then this is always true.

•
From the above three statements, we can conclude that the cost of output
produced by the approximate algorithm is never more than twice the cost of
best possible solution.

•
We have discussed a very simple 2-approximate algorithm for the travelling
salesman problem. There are other better approximate algorithms for the
problem. For example Christofides algorithm is 1.5 approximate algorithm.