Computational Complexity Theory

Lecture 1: Introduction to Computational Complexity

1. Philosophical Foundations of Complexity Theory

Computational complexity theory is a branch of theoretical computer science that focuses on understanding the inherent difficulty of
computational problems. The goal is to classify problems according to the resources they require to solve, such as time, space, or memory.
Philosophically, complexity theory seeks to answer questions like:

What problems can be solved efficiently?

What does it mean for a problem to be computationally difficult?

Can we improve the efficiency of algorithms, or are certain problems fundamentally hard?

This area of study is crucial as it informs us of the limits of computation and helps prioritize which problems can be feasibly tackled by
computers in practice.

2. Motivations for Studying Computational Complexity

From a computer science perspective, the motivation for studying computational complexity comes from the practical need to understand how
algorithms perform as the size of the input grows.

Efficiency: We aim to design algorithms that can solve problems quickly, even with large inputs.

Limits of Computation: Complexity theory helps us identify problems that cannot be solved efficiently, even with powerful machines.

Classifying Problems: Understanding how problems relate to one another in terms of complexity can provide insight into their solvability.

3. Computational Models

A computational model defines how a computation is performed. The most widely used model is the Turing Machine (TM), which serves as the
theoretical foundation for computation.

Turing Machine (TM): A Turing machine is an abstract mathematical model that consists of:

A tape divided into cells, each capable of holding a symbol.

A tape head that can read and write symbols on the tape and move left or right.

A finite set of states and a transition function that dictates the behavior of the machine based on its current state and the symbol under
the tape head.

The Turing machine is capable of performing any computation that can be mechanically executed, making it a universal model of
computation.

4. Running Time of a Turing Machine

The running time of a Turing machine refers to the number of steps it takes to complete its computation, typically measured as a function of
the size of the input. A step is a single operation, such as moving the tape head or writing a symbol.

For an input of size n, the running time is denoted as T (n), which represents the number of steps required to solve the problem for inputs
of size n.

1/77
 The focus is on asymptotic behavior, where we consider the running time for large inputs (i.e., as n grows to infinity).

5. Time-Constructible Functions

A function f (n) is said to be time-constructible if there exists a Turing machine that can compute the function within O(f (n)) time.

Example of a time-constructible function: The function f (n) = n2 is time-constructible, as a Turing machine can compute n2 in quadratic
time.

Example of a non-time-constructible function: A function like f (n) = 2n might not be time-constructible in a reasonable time frame for
large n, though this depends on the specific context or machine model being used.

6. Time-Bounded Classes

We define complexity classes based on how much time a Turing machine uses to solve a problem. The running time of a machine can be
bounded by a polynomial or exponential function of the input size. These time-bounded classes help in classifying problems according to their
difficulty.

Polynomial Time (P): A problem is in the class P if there exists a deterministic Turing machine that can solve the problem in polynomial
time, i.e., O(nk ) for some constant k , where n is the size of the input.

Exponential Time: Problems that require exponential time, i.e., O(2n ), are considered harder to solve as the input size grows.

7. The Complexity Class P

The class P consists of decision problems (problems with a yes/no answer) that can be solved by a deterministic Turing machine in polynomial
time.

Examples of problems in P:

Sorting: Given a list of numbers, sort them in ascending order. Sorting algorithms like Merge Sort or Quick Sort have polynomial time
complexity, i.e., O(n log n).

Shortest Path: Finding the shortest path in a graph can be done in polynomial time using algorithms such as Dijkstra’s Algorithm.

8. The Complexity Class NP

The class NP consists of decision problems for which a proposed solution (certificate) can be verified by a deterministic Turing machine in
polynomial time.

Certificate-based Definition of NP: A problem is in NP if, for any "yes" instance of the problem, there exists a certificate (or proof) that can
be verified in polynomial time. The certificate is a proposed solution to the problem, and the verification process checks if the solution is
correct.

Examples of problems in NP:

Traveling Salesman Problem (TSP): Given a set of cities, is there a tour that visits each city exactly once and returns to the origin with
total distance less than or equal to a given number? A solution (tour) can be verified quickly, though finding it may take exponential
time.

Subset Sum: Given a set of numbers, is there a subset whose sum is equal to a given target? The verification of a subset is easy, as the
sum can be computed in polynomial time.

2/77
9. Non-Deterministic Turing Machines (NDTM)

A non-deterministic Turing machine (NDTM) is a theoretical model where the machine has the ability to make "guesses" or "choices" at each
step. Unlike a deterministic machine, which has a single transition for each state and symbol, an NDTM can transition to multiple possible states
for a given input.

Interpretation of NDTM: The machine explores all possible choices simultaneously. If one of the computational paths leads to an accepting
state, the input is accepted.

The class NP can also be defined using NDTMs, where the problem is solvable in non-deterministic polynomial time, meaning that an NDTM
can solve the problem in polynomial time by making the right choices at each step.

10. Equivalence Between Certificate-Based and Machine-Based Definitions of NP

It can be shown that the certificate-based and machine-based definitions of NP are equivalent. That is, a problem is in NP if and only if it can be
solved by a non-deterministic Turing machine in polynomial time.

Certificate-based: The problem is in NP if a solution (certificate) can be verified in polynomial time.

Machine-based: The problem is in NP if it can be solved by a non-deterministic Turing machine in polynomial time.

The equivalence arises because a non-deterministic Turing machine can be thought of as effectively "guessing" a valid certificate and verifying it
in polynomial time.

Summary

Computational complexity theory explores the inherent difficulty of problems based on resources like time.

The primary model of computation is the Turing machine, and the running time of a TM is used to classify problems.

P is the class of problems solvable in polynomial time, while NP consists of problems for which solutions can be verified in polynomial time.

Non-deterministic Turing machines and certificates offer an alternate but equivalent definition of NP.

Lecture 2: NP-Completeness

1. Notion of NP-Completeness

NP-completeness is a concept used to classify decision problems within the class NP. A problem is NP-complete if it satisfies two conditions:

1. The problem is in NP: This means that a solution to the problem can be verified in polynomial time by a deterministic Turing machine.

2. The problem is NP-hard: This means that every problem in NP can be reduced to this problem in polynomial time. In other words, if we
could solve this problem in polynomial time, we could solve every problem in NP in polynomial time as well.

The significance of NP-completeness lies in its potential to reveal problems that are, in some sense, the "hardest" within NP. If any NP-complete
problem can be solved efficiently (in polynomial time), then all problems in NP can also be solved efficiently, implying P = NP .

2. Polynomial-Time Many-One Reductions

To show that one problem is NP-hard, we often use polynomial-time many-one reductions. A reduction is a process that transforms instances
of one problem into instances of another problem. Specifically:

A many-one reduction (or Karp reduction) from problem A to problem B means that there is a polynomial-time algorithm that takes an
instance of problem A and transforms it into an instance of problem B in such a way that the answer to the transformed instance of B is
the same as the answer to the original instance of A.

Mathematically, if we want to reduce A to B , we need a function f such that:

3/77
 f takes an instance x of problem A and outputs an instance f (x) of problem B .
The answer to x (whether it’s "yes" or "no") is the same as the answer to f (x).

In the context of NP-completeness, we use many-one reductions to show that if we can solve problem B in polynomial time, then we can also
solve problem A in polynomial time.

3. Properties of Polynomial-Time Reductions

The properties of polynomial-time many-one reductions are crucial for establishing NP-completeness:

Transitivity: If problem A reduces to problem B in polynomial time, and problem B reduces to problem C in polynomial time, then
problem A reduces to problem C in polynomial time. This allows us to chain reductions and extend NP-completeness results.

Formally, if A ≤p B and B ≤p C , then A ≤p C .

​ ​ ​

Closure under Polynomial-Time Reductions: The class NP is closed under polynomial-time reductions. That is, if problem A is NP-complete,
and problem B reduces to A in polynomial time, then problem B is also NP-complete.

Transitivity and Reduction Chain: By transitivity, we can create chains of reductions to prove that a problem is NP-complete. If we know
that a problem P is NP-complete, we can show that another problem Q is NP-complete by demonstrating that Q reduces to P .

4. Definition of Boolean Formulae

A Boolean formula is a logical expression composed of Boolean variables, logical operators (AND, OR, NOT), and possibly parentheses to
indicate the order of operations. A Boolean formula evaluates to either true or false.

Example: A Boolean formula like (x1 ​ ∨ ¬x2 ) ∧ (x3 ∨ x4 ) is a conjunction of two clauses, where each clause is a disjunction (OR) of literals,
​ ​ ​

and the literals are either variables xi or their negations.

​

Boolean formulae are fundamental to many problems in complexity theory, particularly in the Satisfiability (SAT) problem.

5. The SAT Problem

The Satisfiability (SAT) problem is the problem of determining whether a given Boolean formula has an assignment of truth values to its
variables such that the entire formula evaluates to true. In formal terms:

Given a Boolean formula φ with variables x1 , x2 , ..., xn , does there exist an assignment of truth values to the variables such that φ
​ ​ ​

evaluates to true?

The SAT problem is a decision problem, and the answer is either "yes" (if such an assignment exists) or "no" (if no such assignment exists).

Example of a SAT instance: The formula (x1 ​ ∨ x2 ) ∧ (¬x1 ∨ x3 ) is satisfiable because we can assign x1 = true, x2 = false, x3 = true to
​ ​ ​ ​ ​

make the formula true.

6. SAT is in NP

The SAT problem is in NP because, for a given assignment of truth values (a certificate), we can verify in polynomial time whether it satisfies the
given Boolean formula.

To verify a certificate, we simply substitute the truth values into the formula and check if it evaluates to true. This verification process clearly
takes polynomial time in the size of the formula, as each variable in the formula is evaluated in constant time, and the number of variables
and clauses is polynomially related to the size of the formula.

Thus, SAT is in NP, as we can verify solutions in polynomial time.

4/77
7. SAT is NP-Hard

The SAT problem is NP-hard, which means that any problem in NP can be reduced to SAT in polynomial time. This was first shown by Stephen
Cook in his seminal paper in 1971, where he proved that SAT was NP-complete.

To show that SAT is NP-hard, we need to demonstrate that any NP problem can be reduced to SAT. The idea is to take an arbitrary problem Q ∈
NP and construct a Boolean formula φ such that:
If Q has a "yes" answer, then φ is satisfiable.

If Q has a "no" answer, then φ is not satisfiable.

Thus, solving SAT would allow us to solve any NP problem, implying that SAT is NP-hard.

8. Proof that SAT is NP-Hard

To start the proof that SAT is NP-hard, we must reduce a known NP-complete problem to SAT. A common starting point is to reduce the 3-SAT
problem (a special case of SAT where each clause has exactly three literals) to SAT. The general strategy involves encoding the constraints of the
original problem into a Boolean formula that captures the essence of the problem's logic.

Step 1: Select a Known NP-Complete Problem. We choose a problem like 3-SAT, which is already known to be NP-complete.

Step 2: Construct a Polynomial-Time Reduction. Show that any instance of 3-SAT can be transformed into an instance of SAT.

Step 3: Prove the Equivalence. Prove that if we can solve SAT efficiently, we can solve the original 3-SAT problem efficiently.

This reduction forms the foundation for proving that SAT is NP-hard.

Summary

NP-completeness involves both membership in NP and NP-hardness, where NP-hard problems can be reduced to a given problem.

Polynomial-time many-one reductions are used to show NP-completeness, with properties like transitivity and closure under reductions.

The SAT problem is central in NP-completeness theory, and its satisfiability can be verified in polynomial time, placing it in NP.

SAT is NP-hard, meaning that any problem in NP can be reduced to SAT, making it one of the hardest problems in NP. The proof of NP-
hardness is often initiated by reducing another NP-complete problem to SAT.

Lecture 3: SAT is NP-Complete

1. Completion of NP-Hardness Proof for SAT

In the previous lecture, we introduced the concept of NP-completeness and showed that the Satisfiability (SAT) problem is NP-hard. In this
lecture, we will complete the proof by demonstrating that SAT is both in NP and NP-hard, which establishes SAT as NP-complete.

SAT is in NP: As discussed earlier, SAT is in NP because, for any candidate solution (truth assignment), we can verify in polynomial time
whether the Boolean formula evaluates to true.

SAT is NP-Hard: We began the process of proving that SAT is NP-hard by showing that any NP problem can be reduced to SAT. In this lecture,
we will complete the reduction from 3-SAT to SAT and prove the equivalence of the two problems.

The reduction from 3-SAT to SAT is done by recognizing that the 3-SAT problem is a specific, restricted version of the SAT problem, and we show
that solving the general SAT problem also allows us to solve 3-SAT efficiently.

5/77
2. SAT Polynomial-Time Reduces to 3SAT

To show that SAT is NP-complete, we need to demonstrate that SAT can be polynomial-time reduced to 3-SAT. 3-SAT is a restricted version of SAT
where every clause in the formula has exactly three literals.

3-SAT Problem: Given a Boolean formula φ in conjunctive normal form (CNF) where each clause has exactly three literals, determine
whether there is an assignment of truth values to the variables that satisfies φ.

Reduction from SAT to 3-SAT: We now show that any general SAT instance can be reduced to an equivalent 3-SAT instance in polynomial
time. Given a Boolean formula φ in SAT (in CNF form) where some clauses may have more or fewer than three literals, we can transform φ
into a new formula φ′ such that:

1. Each clause in φ′ contains exactly three literals.

2. φ and φ′ are satisfiable if and only if each other is satisfiable.

This transformation is done using auxiliary variables. If a clause has more than three literals, we introduce new variables to break the clause
into multiple smaller clauses, each with exactly three literals. If a clause has fewer than three literals, we add dummy variables (such as
introducing additional variables that are always true) to ensure that each clause has exactly three literals.

Example: If we have a clause like (x1 ​ ∨ x2 ∨ x3 ∨ x4 ), we can introduce a new auxiliary variable y and split the clause into two smaller
​ ​ ​

clauses:

(x1 ∨ x2 ∨ y) ∧ (¬y ∨ x3 ∨ x4 )
​ ​ ​ ​

This transformation preserves satisfiability. If φ was satisfiable, the corresponding transformed formula φ′ will also be satisfiable, and vice
versa.

Through this process, we have reduced a general SAT instance to a 3-SAT instance in polynomial time.

3. Why Stop at 3?

The choice of 3 in 3-SAT is significant and serves several purposes in complexity theory:

Generalization: While the SAT problem can have clauses of any size, the 3-SAT problem is a specific and well-defined case that still retains
the essential difficulty of the general SAT problem. This makes it a convenient benchmark for proving NP-completeness.

Uniformity: By restricting the number of literals per clause to three, we can create a more uniform problem that is still representative of all
SAT instances. The restriction to three literals does not simplify the problem significantly; in fact, 3-SAT is just as hard as the general SAT
problem, making it a natural candidate for NP-completeness proofs.

Reduction to 3-SAT: Many NP-complete problems, including SAT, can be reduced to 3-SAT. If a problem is NP-complete, any instance of that
problem can be reduced to an instance of 3-SAT, which helps establish the broad applicability of 3-SAT as a key problem in NP-completeness.

Historical Significance: The restriction to 3 literals was introduced by Stephen Cook in his original proof of NP-completeness. He showed
that 3-SAT is NP-complete, meaning that any problem in NP can be reduced to 3-SAT. This was a landmark result in complexity theory.

4. Consequences of the SAT to 3-SAT Reduction

The reduction from SAT to 3-SAT has several important consequences:

1. NP-Completeness of 3-SAT: Since SAT is NP-complete and we have shown that SAT can be reduced to 3-SAT in polynomial time, it follows
that 3-SAT is also NP-complete. This means that any problem in NP can be reduced to 3-SAT in polynomial time.

2. Generalization of NP-Hardness: The result shows that 3-SAT is not only NP-hard, but it is the first NP-complete problem to be identified.
From this result, many other problems in computer science were shown to be NP-complete by reducing them to 3-SAT, including many
combinatorial optimization problems.

3. Canonical NP-Complete Problem: 3-SAT has become the canonical NP-complete problem, and many complexity theorists use it as a
starting point for proving the NP-completeness of other problems. This means that once we show a problem reduces to 3-SAT, we have
effectively demonstrated its NP-hardness.

6/77
5. Summary

We completed the NP-hardness proof for SAT by showing that SAT can be polynomial-time reduced to 3-SAT.

This reduction uses auxiliary variables to ensure that every clause in the SAT formula has exactly three literals.

The restriction to 3 literals in 3-SAT is not arbitrary: it preserves the NP-completeness of the problem while simplifying its form, making it
easier to work with in proofs and reductions.

3-SAT is NP-complete, and this result is foundational in the study of computational complexity.

The reduction from SAT to 3-SAT plays a pivotal role in understanding the complexity of Boolean satisfiability and serves as a stepping stone for
proving the NP-completeness of many other problems.

Lecture 4: More on NP-Completeness

1. Directed Hamiltonian Path Problem is NP-Complete

We begin by showing that the Directed Hamiltonian Path Problem is NP-complete. The Hamiltonian Path Problem is the problem of finding a
path in a directed graph that visits every vertex exactly once.

Problem Definition: Given a directed graph G = (V , E), does there exist a path in G that visits every vertex exactly once? In the directed
version, the graph is directed, meaning the edges have a specific direction.

NP Membership: The problem is in NP because a potential solution can be verified in polynomial time. If a Hamiltonian path is given as a
certificate, we can check whether this path visits all the vertices exactly once and whether it is a valid directed path in the graph.

NP-Hardness: To prove that the Directed Hamiltonian Path Problem is NP-hard, we perform a reduction from a known NP-complete
problem to the Directed Hamiltonian Path Problem. A natural candidate for this reduction is the Hamiltonian Path Problem in undirected
graphs, which is known to be NP-complete. By converting an undirected graph to a directed graph (while preserving the structure), we show
that solving the Directed Hamiltonian Path Problem is at least as hard as solving the general Hamiltonian Path Problem.

Conclusion: Since the Directed Hamiltonian Path Problem is both in NP and NP-hard, it is NP-complete.

2. The Class coNP

The class coNP is the complement of NP. It contains decision problems for which a "no" answer can be verified in polynomial time by a
deterministic Turing machine.

coNP Definition: A decision problem L belongs to coNP if its complement L is in NP. In other words, if for a problem L in coNP, there exists
a polynomial-time verifier for the "no" instances of the problem, then L is in coNP.

Example: Consider the tautology problem. Given a Boolean formula φ, does φ evaluate to true for all possible truth assignments (i.e., is it a
tautology)? This is the complement of the SAT problem (which is in NP). If we could efficiently verify that a formula is not a tautology (i.e.,
there is a truth assignment that makes it false), this would mean that the tautology problem is in coNP.

3. Complete Problem (SAT) in NP and coNP

SAT is a central problem in the theory of NP-completeness, and it is one of the best-known NP-complete problems. SAT is in NP, meaning that a
solution to a given Boolean formula can be verified in polynomial time. Since SAT is the first problem proven to be NP-complete, it serves as a
benchmark for other NP problems.

CoNP and SAT: The class coNP contains problems where "no" answers are verifiable in polynomial time, and while SAT itself is in NP, it is not
known to be in coNP. The complement of SAT, which asks whether a Boolean formula is unsatisfiable, is not trivially in coNP, and there is no
known polynomial-time verifier for unsatisfiability.

Challenges: One of the key open questions in complexity theory is whether NP = coNP. If NP were equal to coNP, then every problem for
which we can efficiently verify "yes" answers (in NP) would also have an efficiently verifiable "no" answer (in coNP), and vice versa. However,

7/77
 proving or disproving NP = coNP is extremely difficult and remains one of the central challenges in computational complexity.

4. Why Showing that NP = coNP is Hard

The difficulty in proving that NP = coNP stems from the fact that showing a polynomial-time verifier for the complement of a problem requires
deep insights into the structure of the problem and the relationships between different complexity classes. If NP = coNP, it would imply that for
every NP problem (for which a solution can be verified efficiently), there is also a complementary problem where the absence of a solution can
be verified efficiently.

However, several key points make this difficult:

1. Lack of Evidence: There is currently no known example of a problem in NP whose complement is in NP. This suggests that the two classes,
NP and coNP, might be distinct, but proving this rigorously has proven to be elusive.

2. Probabilistic and Non-Deterministic Approaches: Many NP problems, including SAT, involve non-determinism or probabilistic reasoning,
making it difficult to establish complementary verification procedures that fit within the deterministic framework of coNP.

3. Intuitive Beliefs: While it has not been proven, many complexity theorists believe that NP does not equal coNP. This belief is based on the
fact that we have been unable to find efficient verification procedures for the complements of NP-complete problems like SAT.

5. Exponential Classes EXP and NEXP

The EXP and NEXP classes are extensions of NP and coNP that deal with exponential time complexity.

EXP (Exponential Time Complexity): The class EXP contains decision problems that can be solved by a deterministic Turing machine in time
O(2p(n) ), where p(n) is a polynomial in the size of the input n. These are problems that require exponential time to solve in the worst case.
NEXP (Nondeterministic Exponential Time Complexity): The class NEXP contains decision problems that can be solved by a
nondeterministic Turing machine in time O(2p(n) ). NEXP extends NP in the sense that it allows for nondeterminism and exponential time.

Known Relations: The known relationships between EXP and NEXP are as follows:

1. NP ⊆ EXP: Every problem in NP can be solved in exponential time, so EXP is a superset of NP.
2. coNP ⊆ NEXP: Similarly, every problem in coNP can be solved in nondeterministic exponential time, so NEXP is a superset of coNP.
3. EXP ⊆ NEXP: Since nondeterministic Turing machines are more powerful than deterministic ones, NEXP is a superset of EXP.
4. EXP =
 NEXP: It is widely believed that EXP is not equal to NEXP, though this has not been proven.

6. Summary

The Directed Hamiltonian Path Problem was proven to be NP-complete by showing it is both in NP and NP-hard.

The class coNP is the complement of NP, containing problems where "no" answers can be verified in polynomial time.

The relationship between NP and coNP is a major open question in complexity theory, and proving or disproving NP = coNP remains a
difficult task.

EXP and NEXP are classes of problems solvable in exponential time, with NEXP being a more powerful class than EXP.

The known relations between these classes help us understand the hierarchy of computational complexity and the difficulty of various
problems.

In summary, NP-completeness continues to be central in computational complexity, and understanding the relationships between NP, coNP, EXP,
and NEXP provides important insights into the limits of what can be computed efficiently.

Lecture 5: Hierarchy Theorems

8/77
1. Padding Technique and Hierarchy Theorems

The padding technique is a method used in computational complexity to demonstrate that lower-order complexity classes can collapse into
higher-order classes. Specifically, this technique is useful in showing that any potential collapse of complexity classes at lower levels (such as P
and NP) would imply a collapse of higher classes as well.

Padding: In the context of time complexity, padding refers to the technique of artificially increasing the size of an input to a problem to
ensure that it belongs to a particular complexity class. This technique is used to create new problems that are more "difficult" by adding
additional irrelevant information to the input, thus making the problem harder to solve.

Collapse Implication: If a collapse occurs in a lower-order complexity class (e.g., P = NP), then it would imply that higher-order complexity
classes (e.g., PSPACE or EXPTIME) would also collapse. This suggests that the inherent structure of complexity classes is very sensitive to
small collapses, and proving a collapse in a lower class could lead to surprising consequences in higher classes.

The padding technique is often used to construct problems that require specific levels of complexity. This ensures that any complexity class
collapse must respect the hierarchy of problems from simpler to more complex classes.

2. Time Hierarchy Theorem

The Time Hierarchy Theorem is a key result in computational complexity theory, stating that there are problems that require strictly more time
to solve than others within a given class. More specifically, it asserts that there are decision problems that can be solved in time O(f (n)) for a
certain function f (n), but cannot be solved in time O(g(n)) for any smaller function g(n), where f (n) grows faster than g(n).

Formal Statement: There exists a language L that can be decided by a deterministic Turing machine in time O(f (n)), where f (n) is a
time-constructible function, but no deterministic Turing machine can decide L in time O(g(n)), for any function g(n) that grows slower
than f (n).

Proof Outline Using Diagonalization: The proof of the Time Hierarchy Theorem relies on a technique called diagonalization, a method of
constructing a problem that cannot be solved by any Turing machine within a specified time bound.

1. Constructing the Language: Given a function f (n) that is time-constructible (i.e., the function grows sufficiently fast), we define a
language L that encodes the behavior of Turing machines that run within a specific time bound. The idea is to use a diagonalization
process to define a problem that can be solved by a machine running in time O(f (n)), but cannot be solved by any machine running in
time O(g(n)), where g(n) is slower than f (n).

2. Diagonalization Argument: The diagonalization argument ensures that the constructed language L is distinct from any language
accepted by a machine running in time O(g(n)). This is done by systematically constructing an input that forces the machine to behave
differently from all others in terms of its time complexity.

3. Conclusion: By this construction, we show that there exist languages that are beyond the reach of machines with lower time bounds,
thus establishing that there are strict hierarchies of time complexities.

3. Oracle Turing Machines and Complexity Classes

An oracle Turing machine (OTM) is a theoretical machine that has access to an oracle — a hypothetical device that can instantly answer certain
decision problems. This allows the OTM to solve certain problems in a single step that would normally require many steps in a conventional
Turing machine.

Oracle Machines: An oracle Turing machine is a standard Turing machine augmented with an oracle. The oracle can be thought of as a
black box that can be queried to provide answers to a specific problem. These queries are resolved in one computational step, regardless of
the complexity of the problem.

Oracle and Complexity Classes: Oracle machines help us explore the relationship between different complexity classes. By choosing
specific oracles, we can construct scenarios where classes like P and NP behave differently, or even appear to be the same.

If an oracle for a specific problem is added to a Turing machine, it may increase the power of the machine, allowing it to solve problems
it otherwise could not. However, the oracle itself is assumed to be non-computable in most cases.

By defining oracles with specific properties, we can simulate scenarios where known complexity class inclusions or separations (such as
P ⊆ NP ) may or may not hold.

9/77
4. Results on Oracles and the P vs NP Question

A key result involving oracle Turing machines is that there exist oracles for which P = NP and oracles for which P =
 NP .
Oracle with P = NP: There exists an oracle A such that P A
= N P , meaning that if the oracle A is available, the class of problems solvable
A

in polynomial time with access to the oracle (denoted P ) is the same as the class of problems that can be solved in nondeterministic
A

polynomial time with access to the oracle (denoted N P A ).

This result shows that there is a scenario (involving a specific oracle) where the P vs NP question resolves in favor of equality. However, this
oracle does not provide a concrete solution to the general P vs NP question in the real world.

Oracle with P ≠ NP: Conversely, there exists an oracle B such that P B = N P B , meaning that for some oracle B , the class of problems

solvable in polynomial time with the oracle is strictly smaller than the class of problems solvable in nondeterministic polynomial time with
the oracle.

This result indicates that the relationship between P and NP can depend on the specific oracle, and thus cannot be fully resolved by
considering oracles alone.

Implications: These results suggest that the P vs NP question is independent of the choice of oracle and that the resolution of this question
may require deeper insights into the structure of complexity classes beyond the use of oracles.

5. Summary

The padding technique is used to show that collapses in lower complexity classes can imply collapses in higher classes, demonstrating the
sensitivity of class structures.

The Time Hierarchy Theorem proves that there exist problems that cannot be solved faster than a given time bound, using a
diagonalization argument.

Oracle Turing Machines are used to explore the relationship between complexity classes and show that there exist oracles where P = NP
and others where P  NP .
=
These results emphasize the complexity of the P vs NP question and show that oracles provide valuable insights but are not definitive
solutions to the problem.

In conclusion, the study of hierarchy theorems and oracle machines reveals important properties of computational complexity, but the question
of P vs NP remains unresolved and deeply intertwined with the structure of computational classes.

Lecture 6: Introduction to Space Complexity

1. Introduction to Space Complexity

Space complexity refers to the amount of memory or space a computational problem requires to solve, typically measured in terms of the size
of the input. This contrasts with time complexity, which measures the number of steps or operations a machine performs.

Space Complexity Function: The space complexity of an algorithm is typically expressed as a function of the input size, denoted as S(n),
where n is the size of the input. For example, an algorithm that uses linear space in the size of the input would have a space complexity of
O(n).
Importance of Space Complexity: In many practical scenarios, memory is a limited resource, and an algorithm's ability to efficiently use
space is as important as its ability to perform computations in a reasonable amount of time. Therefore, understanding and analyzing space
complexity is crucial for designing algorithms that work in resource-constrained environments.

Space Complexity vs. Time Complexity: While time complexity focuses on how the number of operations increases as the input size grows,
space complexity concerns itself with how much additional memory is needed for a computation. A problem could have a polynomial time
complexity but exponential space complexity, or vice versa, and understanding these trade-offs is fundamental to algorithm design.

10/77
2. Machine Model: Work Tape and Space Usage

In the context of space complexity, we consider a Turing machine model, specifically focusing on how much space is used by the machine
during its execution.

Work Tape: In a standard Turing machine, there are several components:

Input tape: This is where the input to the problem is stored. The space used by the input tape is not counted towards the space
complexity, as it represents the problem instance.

Work tape: This is the tape used by the Turing machine to perform computations. The space used on the work tape is the space that we
are concerned with when analyzing space complexity.

The space complexity of a Turing machine is primarily determined by how much space is required on the work tape during computation, as
the input is given and is typically not modified.

Space Usage: The machine’s space usage is tracked by counting the total number of cells used on the work tape at any point during its
computation. This gives us a way to define and measure space complexity.

3. Deterministic and Non-Deterministic Space Classes

Space complexity can be categorized into deterministic and non-deterministic space classes, depending on the nature of the computational
model.

Deterministic Space Complexity (DSPACE): This refers to the space complexity of problems that can be solved by a deterministic Turing
machine, where the machine’s next move is determined by its current state and the symbol it reads on the tape. For example:

DSPACE(f(n)): This represents the class of problems that can be solved by a deterministic Turing machine using O(f (n)) space, where
f (n) is some function of the input size n.
Non-Deterministic Space Complexity (NSPACE): This refers to the space complexity of problems that can be solved by a non-deterministic
Turing machine. A non-deterministic Turing machine has the ability to make multiple computational "guesses" at each step, allowing it to
explore many possibilities in parallel.

NSPACE(f(n)): This represents the class of problems that can be solved by a non-deterministic Turing machine using O(f (n)) space.

Space Hierarchy Theorem: Similar to the Time Hierarchy Theorem, the Space Hierarchy Theorem states that there exist problems that
require strictly more space to solve than others. More specifically, there are problems that can be solved using O(f (n)) space but
cannot be solved using any smaller space bound O(g(n)), where f (n) grows faster than g(n).

4. Configuration Graphs: Definition, Size, Properties

A configuration graph is a graphical representation of all possible states a Turing machine can reach during its computation. This graph is
useful in understanding space complexity because it helps us visualize how space usage evolves during the execution of the machine.

Definition of Configuration Graph: A configuration of a Turing machine is a snapshot of its current state, the tape contents, and the
position of the tape head. The configuration graph is a directed graph where:

Each node represents a unique configuration of the machine.

There is an edge from one configuration to another if the Turing machine can move from the first configuration to the second in one
step.

Size of the Configuration Graph: The number of nodes in the configuration graph corresponds to the number of distinct configurations the
Turing machine can enter during its computation. Since the Turing machine operates in a finite state space and uses a finite tape, the
number of distinct configurations is finite, though it can be large.

The size of the configuration graph is influenced by the number of states of the Turing machine, the size of the tape, and the set of
possible symbols that can appear on the tape.

Properties of Configuration Graphs:

Reachability: The configuration graph has a reachability property. If a configuration can be reached from another, it means there is a
sequence of moves that leads from one configuration to the other.

11/77
 Space Complexity as Graph Reachability: One of the key insights from studying configuration graphs is that space complexity can be
reduced to the graph reachability problem. The space complexity of a Turing machine can be understood as the problem of
determining whether there is a reachable configuration in the graph using a given amount of space.

Bounding Space Complexity: The space complexity of a machine is bounded by the number of reachable configurations, as each
configuration requires a fixed amount of space.

5. Reducing Space Complexity to Graph Reachability Problem

The process of reducing space complexity to graph reachability involves viewing the space usage of a Turing machine as a graph traversal
problem. By examining the configuration graph of the machine, we can analyze the space required to reach a particular configuration.

Graph Reachability: The problem of determining whether a configuration can be reached from a given initial configuration is a reachability
problem in a graph. The size of the graph and the number of reachable configurations directly correspond to the space complexity of the
Turing machine.

Space Complexity as Reachability:

In the worst case, the space complexity of a Turing machine can be proportional to the number of reachable configurations in its
configuration graph.

The space required to store the configuration graph is directly tied to the space complexity, as each configuration consumes a fixed
amount of space on the tape.

6. Summary

Space complexity is concerned with how much memory is required to solve a problem, and it is measured in terms of the space used on the
work tape of a Turing machine.

The distinction between deterministic and non-deterministic space classes allows us to categorize problems based on whether they can be
solved using deterministic or non-deterministic Turing machines.

Configuration graphs help visualize the space usage of a Turing machine by representing all possible configurations and the transitions
between them. The space complexity can be analyzed by studying these graphs, as the number of reachable configurations determines the
space usage.

Space complexity can be reduced to the graph reachability problem, providing a powerful tool for understanding and analyzing space-
efficient computations.

In conclusion, space complexity is an important aspect of algorithm design, and understanding its relationship to configuration graphs and
reachability is essential for analyzing the resource usage of algorithms.

Lecture 7: Savitch's Theorem

1. Revisited Configuration Graphs

Configuration graphs are a crucial concept in understanding space complexity, as they represent all the possible states that a Turing machine
can enter during its computation. In this lecture, we revisit these graphs in the context of space complexity classes and computational problems,
particularly focusing on reachability problems.

Configuration Graphs: A configuration graph of a Turing machine consists of nodes representing the machine's configurations, where each
node is a unique combination of the machine's state, tape contents, and tape head position. There is a directed edge between two nodes if
the machine can transition from one configuration to the other in one step.

Space Complexity and Reachability: The space complexity of a Turing machine can be analyzed by examining the configuration graph. The
problem of determining if a particular configuration is reachable within a certain space bound is equivalent to the reachability problem in
the graph. This problem is central to many of the space complexity classes that we will define in this lecture.

12/77
2. The Path Problem (Reachability Problem in Graphs)

The path problem (or reachability problem) involves determining whether there exists a path between two nodes in a graph. This is a
fundamental problem in graph theory and has direct implications for space complexity.

Path Problem: Given a directed graph G with nodes V and edges E , and two nodes u and v , the path problem asks whether there exists a
directed path from node u to node v .

Space Complexity of Path Problem: The path problem can be solved in various space classes, depending on the computational resources
available. In particular, the space complexity of solving the path problem gives us insights into the space classes L, NL, PSPACE, and
NPSPACE, which we will explore in detail in the next sections.

3. Space Hierarchy Theorem and Proof Idea

The Space Hierarchy Theorem asserts that there are problems that require strictly more space to solve than others, given a certain space
bound. More formally, it states that for any time-constructible function f (n), there exists a problem that can be solved in O(f (n)) space, but
not in O(g(n)) space for any g(n) that grows slower than f (n).

Proof Idea: The proof of the Space Hierarchy Theorem typically uses a diagonalization argument, similar to the one used in the Time
Hierarchy Theorem. The idea is to construct a language that requires more space to solve than any problem solvable in a smaller space
bound. This is done by leveraging the fact that the configuration graph of a Turing machine grows exponentially with respect to the amount
of space used, and we can build a language that is distinct from all others that use less space.

Key Insight: The space hierarchy implies that there is a strict separation between different space classes, meaning that more space allows
us to solve strictly more complex problems.

4. Savitch's Theorem

Savitch's Theorem is a fundamental result in space complexity theory, which relates the deterministic space complexity of a problem to its non-
deterministic space complexity. Specifically, Savitch's Theorem shows that the class NL (Nondeterministic Logspace) is contained within L
(Deterministic Logspace).

Statement of Savitch's Theorem: For any language L that can be decided by a non-deterministic Turing machine using O(s(n)) space,
there is a deterministic Turing machine that can decide L using O(s(n)2 ) space.

In other words, NL is contained within PSPACE, and non-determinism does not give more space-efficient computations in terms of
asymptotic space bounds.

Proof Outline:

The proof of Savitch's Theorem involves a simulation of a non-deterministic machine by a deterministic machine. The idea is to "guess"
the non-deterministic choices and then verify whether the path of configurations can be traversed within the available space.

By using a dynamic programming approach and storing partial solutions at each stage, a deterministic machine can simulate the non-
deterministic machine while using only a quadratic increase in space.

Implications: This result is significant because it implies that NPSPACE = PSPACE, meaning that the use of non-determinism does not
increase the space complexity of a problem beyond deterministic space bounds. It also shows that non-deterministic space classes are
fundamentally weaker than might have been expected.

5. Space Classes and Their Relations

Savitch's Theorem leads us to the definition and exploration of several important space complexity classes, including L, NL, PSPACE, and
NPSPACE, as well as the relations between them.

13/77
 L (Logspace): The class L consists of decision problems that can be solved by a deterministic Turing machine using logarithmic space, i.e.,
O(log n) space. Problems in L are generally considered space-efficient.
NL (Nondeterministic Logspace): The class NL consists of decision problems that can be solved by a non-deterministic Turing machine
using logarithmic space. Savitch’s Theorem states that N L ⊆ PSP ACE , meaning non-deterministic logspace problems are also solvable
in polynomial space.

PSPACE: The class PSPACE consists of decision problems that can be solved by a deterministic Turing machine using polynomial space. This
includes a wide range of problems that are feasible for real-world computations but require more space than can be managed by
algorithms in L or NL.

NPSPACE: The class NPSPACE consists of decision problems solvable by a non-deterministic Turing machine in polynomial space. By
Savitch's Theorem, NPSPACE is equivalent to PSPACE, meaning that non-deterministic space does not provide any additional power over
deterministic space when considering polynomial space.

Known Relations:

L ⊆ N L ⊆ PSP ACE = NPSP ACE .

From Savitch's Theorem, we know that non-deterministic space classes (like NL) are contained within deterministic polynomial space
classes (like PSPACE).

Further, it is known that PSPACE contains L and NL, but whether L equals NL remains an open question in complexity theory.

6. Logspace Reductions

A logspace reduction is a type of reduction where the function transforming one problem instance to another can be computed using
logarithmic space.

Definition: A logspace reduction from problem A to problem B means that there exists a function that transforms any instance of A into
an instance of B in logarithmic space. If we can solve problem B in logarithmic space, we can solve problem A in logarithmic space as well.

Motivation for Studying Logspace Reductions:

Logspace reductions are important because they allow us to study the relative difficulty of problems in terms of space. By reducing one
problem to another using logspace, we can infer that if one problem is logspace reducible to another, then solving the second problem
efficiently in space implies efficient space solutions for the first problem as well.

Logspace reductions also help in characterizing complexity classes. For example, NL is closed under logspace reductions, meaning that
if a problem in NL can be reduced to another problem in NL via a logspace reduction, the resulting problem is also in NL.

Key Application: Logspace reductions are used extensively in the study of NL, L, and other space classes, helping to determine the
boundaries between classes and providing insights into the relative power of different types of space-bounded computation.

7. Summary

Savitch’s Theorem establishes that non-deterministic logspace problems can be solved deterministically in polynomial space, leading to the
equivalence NPSP ACE = PSP ACE .
We explored important space complexity classes, such as L, NL, PSPACE, and NPSPACE, and derived their relationships based on Savitch’s
Theorem.

The Path problem (reachability in graphs) is closely related to space complexity, as it illustrates the practical implications of space-bounded
computation.

Logspace reductions are a key tool for understanding and comparing space complexity classes, enabling us to categorize problems based
on their space requirements.

In conclusion, Savitch’s Theorem and the relationships between space complexity classes provide important insights into the power and
limitations of space-bounded computation, helping to delineate the boundaries between different classes of problems.

Lecture 8: Immerman-Szelepscenyi Theorem

14/77
1. Properties of Logspace Reductions

Logspace reductions are fundamental tools in complexity theory, particularly in the study of space-bounded classes like L and NL. These
reductions help us understand how problems relate to each other in terms of their space complexity.

Logspace Reduction: A logspace reduction from problem A to problem B is a transformation function that maps instances of A to
instances of B using logarithmic space. Specifically, if a problem A can be reduced to problem B in logspace, we say that A is logspace
reducible to B (denoted as A ≤L B ).
​

Transitivity: Logspace reductions are transitive, meaning that if A ≤L B and B ≤L C , then A ≤L C . This property is crucial because it
​ ​ ​

allows us to chain reductions between problems, creating a hierarchy of problems that are logspace reducible to each other.

Closure of L Under Logspace Reductions: The class L (Deterministic Logspace) is closed under logspace reductions. This means that if a
problem A is in L and there exists a logspace reduction from A to problem B , then B must also belong to L. This closure property is
important for proving that problems in L are equivalent or reducible to each other under logspace bounds.

2. Path Problem is NL-Complete

The path problem is a fundamental problem in graph theory and space complexity. It is defined as the problem of determining whether there
exists a path between two vertices in a directed graph.

NL-Complete Problem: The path problem is NL-complete, which means that:

The path problem is in NL (Non-deterministic Logspace).

The path problem is as hard as any problem in NL. Specifically, any problem in NL can be logspace reduced to the path problem. This
establishes the path problem as a representative "hard" problem within the NL class.

Path and Path Complements: The complement of the path problem, denoted as Path, is the problem of determining whether there is no
path between two vertices in a graph. This problem is also in NL, as we will explore with the Immerman-Szelepscenyi Theorem.

3. Immerman-Szelepscenyi Theorem (1988)

The Immerman-Szelepscenyi Theorem provides a groundbreaking result in the study of NL and its closure properties. It shows that the
complement of any problem in NL is also in NL.

Statement of the Theorem: The Immerman-Szelepscenyi Theorem states that NL is closed under complement. More formally:

NL = NL

In other words, if a problem is in NL, its complement is also in NL.

Implication of the Theorem: This result has profound implications for the class NL. Before the Immerman-Szelepscenyi Theorem, it was not
clear whether NL was closed under complementation. The theorem establishes that for every NL problem, there is an efficient way to
compute its complement using logarithmic space. This makes NL a "closed" class in the sense that its problems can be complemented
without leaving the class.

Proof Idea: The proof of the Immerman-Szelepscenyi Theorem uses a non-deterministic algorithm to verify the existence of a path in a
graph, and it shows how to construct a non-deterministic machine that can verify the complement of the path problem (i.e., checking for the
non-existence of a path) using the same amount of space.

The general idea is to use a technique called backtracking to simulate the "complementary" computation. Essentially, if a non-
deterministic machine can verify the existence of a path in logarithmic space, a similar machine can be constructed to verify the non-
existence of a path by backtracking through the graph's configurations.

Key Consequences:

NL is a closed class under complementation, meaning that the class is robust to operations such as negation.

Since the complement of the path problem is in NL, this implies that Path and its complement, Path¯, are both in NL.

15/77
4. Relations Between NL and Other Complexity Classes

The Immerman-Szelepscenyi Theorem has important consequences for the relationships between NL and other complexity classes.

NL and L: We know that L is contained within NL because deterministic logspace computations are a subset of non-deterministic logspace
computations. However, it remains an open question whether L is strictly smaller than NL.

L ⊆ NL
NL and PSPACE: It is also known that NL is contained within PSPACE, since polynomial space can simulate non-deterministic logspace
computations. However, the exact relationship between NL and PSPACE is still not fully understood, although it is generally accepted that
PSPACE is strictly larger than NL.

N L ⊆ PSP ACE
Complements and Nondeterminism: The result that NL is closed under complementation challenges the earlier assumption that non-
deterministic classes behave differently from deterministic ones in terms of complementation. It suggests that non-determinism does not
provide additional power for complementing problems, at least for problems in NL.

5. Motivations and Applications of the Immerman-Szelepscenyi Theorem

Simplification of the Complexity Landscape: Before the Immerman-Szelepscenyi Theorem, the relationship between NL and its
complement was unclear, and many problems were left in a state of ambiguity regarding their complexity. This theorem clarified that NL is
closed under complementation, simplifying the complexity landscape of problems in NL.

Practical Relevance: The result has practical applications in the design of algorithms for problems that are NL-complete. Since we know that
NL is closed under complementation, we can apply similar techniques to solve problems that involve finding paths as well as ensuring there
are no paths in graphs (or their complements).

Theoretical Relevance: The theorem is fundamental in the theoretical study of space-bounded complexity classes, providing insight into the
limitations and strengths of NL as a class. It also suggests that NL is computationally more powerful than initially thought, as it can handle
both positive and negative answers to path-related problems in logarithmic space.

6. Summary

Logspace reductions are transitive and closed under reduction, meaning that problems in L and NL can be related through logspace
reductions.

The path problem is NL-complete, meaning it is both in NL and as hard as any problem in NL.

The Immerman-Szelepscenyi Theorem establishes that NL is closed under complementation, meaning that if a problem is in NL, its
complement is also in NL.

This theorem has important implications for the space complexity classes, particularly for NL, simplifying the understanding of problems in
this class and their relationship with other space classes like L and PSPACE.

In conclusion, the Immerman-Szelepscenyi Theorem is a crucial result that enhances our understanding of non-deterministic logspace and its
closure properties, while establishing the equivalence between a problem and its complement within NL.

Lecture 9: Polynomial Hierarchy

1. Completed Proof of Immerman-Szelepscenyi Theorem

In the previous lecture, we introduced the Immerman-Szelepscenyi Theorem, which states that NL is closed under complement. This means
that if a problem is in NL, its complement is also in NL.

Proof Overview: The proof of the theorem used a non-deterministic machine to simulate the complement of the path problem by
leveraging backtracking techniques. The main idea was to construct a non-deterministic machine that, when given the complement of a

16/77
 graph reachability problem, could verify the non-existence of a path using logarithmic space, just as a machine could verify the existence of
a path.

Conclusion: The theorem establishes that NL is closed under complement, which has profound implications for the space complexity
landscape. This closure property is essential for simplifying many results in space complexity theory and understanding the behavior of
problems within NL.

2. Motivation for Studying the Polynomial Hierarchy

The Polynomial Hierarchy (PH) is an important structure in computational complexity theory that generalizes the well-known classes P, NP, and
coNP. It provides a more granular understanding of the relationship between problems that can be solved by deterministic and non-
deterministic machines within polynomial time, but with varying levels of complexity.

Need for Generalization: The study of the polynomial hierarchy arose from the desire to understand the intricacies of problems that lie
beyond P and NP. While NP captures decision problems solvable by non-deterministic polynomial-time algorithms, there are many
problems that do not neatly fit into the P or NP categories. The polynomial hierarchy provides a natural framework for classifying such
problems.

Richness of the Polynomial Hierarchy: The polynomial hierarchy organizes problems into different levels based on the number of
alternations between existential and universal quantifiers needed to describe the solution. These alternations reflect the computational
complexity of solving these problems, with each level increasing in computational power.

Applications: The polynomial hierarchy is useful in many areas of computational complexity, including algorithm design, proof complexity,
cryptography, and the study of approximation algorithms. It helps us understand the complexity of problems beyond NP and coNP,
enabling a more nuanced classification of computational tasks.

3. Definition of the Polynomial Hierarchy

The polynomial hierarchy is a hierarchy of complexity classes that generalizes NP, coNP, and P through alternating quantifiers. It is built by
adding alternating existential and universal quantifiers to the standard polynomial-time decision process.

Basic Idea: The polynomial hierarchy is defined using a sequence of classes, each corresponding to a different level of alternation between
existential and universal quantifiers in the decision process. The classes are defined in terms of oracle machines that can query a "black
box" corresponding to a problem in a lower class of the hierarchy.

Formal Definition:

Σ₁^P (Existential Level): The class of decision problems solvable by a non-deterministic Turing machine in polynomial time, with a single
existential quantifier. This corresponds to the class NP, where the machine can guess a solution and verify it in polynomial time.

Π₁^P (Universal Level): The class of decision problems solvable by a deterministic Turing machine in polynomial time, with a single
universal quantifier. This corresponds to the class coNP, where the machine must check that no solution exists.

Σ₂^P (Second Level): The class of decision problems that can be solved by a non-deterministic Turing machine in polynomial time with
an existential quantifier followed by a universal quantifier. This corresponds to problems like Quantified Boolean Formula (QBF), where
one must check the existence of a solution, then verify that for all possible configurations, the solution holds.

Π₂^P (Second Universal Level): The class of decision problems that can be solved by a deterministic Turing machine in polynomial time
with a universal quantifier followed by an existential quantifier. These problems require checking the worst-case scenario for all
configurations before verifying the existence of a valid solution.

Higher Levels: The polynomial hierarchy continues with higher levels of alternating quantifiers, with each level adding one more
alternation between existential and universal quantifiers.

More formally, the k-th level of the polynomial hierarchy can be defined recursively as follows:

Σᵏ^P: The class of problems solvable by a non-deterministic machine that makes polynomial-time queries to a Σᵏ⁻¹^P oracle.

Πᵏ^P: The class of problems solvable by a deterministic machine that makes polynomial-time queries to a Σᵏ⁻¹^P oracle.

17/77
 Polynomial Hierarchy for k ≥ 3: Higher levels of the hierarchy involve further alternations between existential and universal quantifiers.
For example:

Σ₃^P: Problems solvable by a non-deterministic machine with three alternating quantifiers: existential, universal, and existential.

Π₃^P: Problems solvable by a deterministic machine with three alternating quantifiers: universal, existential, and universal.

General Structure: The hierarchy can be visualized as a tree, with each level ΣP
k and Πk forming a branch of the tree. The levels form a
​
P
​

nested structure, where each class is contained within the class of the next higher level.

4. Importance of the Polynomial Hierarchy

Generalization of Known Classes: The polynomial hierarchy generalizes NP, coNP, and P, providing a framework that can classify a wider
range of problems based on the complexity of their decision processes. For example, Σ₁^P is NP, Π₁^P is coNP, and P forms the base level of
the hierarchy.

Separations Between Classes: One of the central questions in complexity theory is whether different levels of the polynomial hierarchy are
distinct from one another. If Σₖ^P = Σₖ⁺¹^P, it would suggest that the hierarchy collapses at that level, and the computational power of the
hierarchy would be limited to a lower level. However, proving such collapses is challenging, and it is widely believed that the hierarchy does
not collapse at any finite level, meaning each level represents a distinct class of problems.

Applications to Algorithms and Complexity: Understanding the polynomial hierarchy is critical for designing efficient algorithms, as
problems in higher levels of the hierarchy may require more complex approaches. For example, algorithms for problems in Σ₂^P or Π₂^P are
typically more complex than those for problems in NP or coNP, often requiring more sophisticated techniques such as probabilistic
algorithms or approximations.

Quantum Computing: In recent years, the polynomial hierarchy has also found applications in the study of quantum complexity classes.
Certain problems that are hard for classical computers may be easier for quantum computers, and understanding where these problems lie
within the polynomial hierarchy can help in determining the relative power of quantum versus classical computations.

5. Summary

The Polynomial Hierarchy (PH) is a hierarchy of complexity classes that generalizes NP, coNP, and P by introducing multiple levels of
alternating existential and universal quantifiers.

The hierarchy is defined recursively, with each level representing a more complex problem that requires additional alternations between
existential and universal quantifiers.

The study of the polynomial hierarchy helps us understand the relationships between problems of different complexity and provides a
framework for classifying problems that lie beyond the standard P, NP, and coNP classes.

The structure of the hierarchy provides insight into whether the classes collapse and into the fundamental nature of computational
complexity across different levels of decision-making power.

The polynomial hierarchy is a cornerstone of complexity theory, with profound implications for algorithm design, problem classification, and the
theoretical understanding of computational limits.

Lecture 10: A PSPACE Complete Problem

1. PSPACE-Complete Problem: TQBF

The True Quantified Boolean Formula (TQBF) problem is a fundamental problem in the study of PSPACE complexity, and it is an example of a
PSPACE-complete problem. A problem is PSPACE-complete if it is both in PSPACE and every problem in PSPACE can be reduced to it in
polynomial space.

TQBF Definition: The TQBF problem is the decision problem of determining whether a fully quantified Boolean formula is true. A fully
quantified Boolean formula is a Boolean formula in which the variables are preceded by either existential or universal quantifiers, and all
variables are explicitly quantified. The problem is to decide whether the formula evaluates to true for some assignment of the variables.

18/77
 The formula has the form:
Q1 x1 Q2 x2 … Qn xn φ(x1 , x2 , … , xn )
​ ​ ​ ​ ​ ​ ​ ​ ​

where Qi is either an existential quantifier (∃) or a universal quantifier (∀), and φ(x1 , x2 , … , xn ) is a Boolean expression involving
​ ​ ​ ​

the variables x1 , x2 , … , xn .
​ ​ ​

Why TQBF is PSPACE-Complete: TQBF is known to be PSPACE-complete because:

1. TQBF is in PSPACE: The problem can be solved using polynomial space. A deterministic machine can evaluate a fully quantified Boolean
formula by simulating the evaluation of each quantifier while keeping track of the polynomial space used during the computation.

2. TQBF is PSPACE-hard: Every problem in PSPACE can be reduced to TQBF in polynomial space. This follows from the fact that PSPACE is
closed under polynomial space reductions and TQBF is a canonical problem for this class.

2. Levels of the Polynomial Hierarchy

The Polynomial Hierarchy (PH) is a layered structure of complexity classes that generalizes NP and coNP by introducing multiple levels of
quantifiers, and each level corresponds to a different alternation of existential and universal quantifiers in a decision process.

Definition of Σᵢ^P and Πᵢ^P:

Σᵢ^P (Existential Level): The class of decision problems solvable by a non-deterministic Turing machine in polynomial time, with i
alternating quantifiers, starting with an existential quantifier. Specifically, for Σᵢ^P, the machine can choose a value for some variables
(existential quantifier), then for all possible values of other variables (universal quantifier), and this alternation continues for i levels.

Πᵢ^P (Universal Level): The class of decision problems solvable by a deterministic Turing machine in polynomial time, with i alternating
quantifiers, starting with a universal quantifier. Here, the machine must check for all possible solutions before verifying the existence of
a valid one.

Hierarchy Structure: The polynomial hierarchy is structured as follows:

Σ₁^P is NP, the class of decision problems solvable by a non-deterministic polynomial-time machine with an existential quantifier.

Π₁^P is coNP, the class of decision problems solvable by a deterministic polynomial-time machine with a universal quantifier.

Σ₂^P and Π₂^P represent problems that alternate between existential and universal quantifiers (two levels of alternation).

Higher levels Σᵢ^P and Πᵢ^P continue this pattern for more alternations between existential and universal quantifiers.

In essence, the Σᵢ^P classes are associated with non-deterministic computation with i alternating quantifiers, while the Πᵢ^P classes are
associated with deterministic computation and the same number of alternating quantifiers.

3. Complete Problems for the Polynomial Hierarchy

Each level of the polynomial hierarchy has associated complete problems that encapsulate the computational difficulty of the entire level.

Σᵢ^P-complete Problems: These are problems that are complete for the class Σᵢ^P under polynomial-time many-one reductions. For
example:

Σ₁^P-complete: The SAT (Satisfiability) problem is Σ₁^P-complete because it is both in NP and every problem in NP can be reduced to it.

Σ₂^P-complete: The Quantified Boolean Formula (QBF) problem is Σ₂^P-complete. It generalizes SAT by allowing multiple levels of
quantifiers and is a canonical problem for the second level of the polynomial hierarchy.

Σᵢ^P-complete: For higher levels, problems like k-QBF (where there are k alternating quantifiers) serve as Σᵢ^P-complete problems.

Πᵢ^P-complete Problems: Similarly, there are Πᵢ^P-complete problems, which are complete for the class Πᵢ^P. These problems typically
require universal quantifiers at the beginning of the alternation sequence.

4. Conditions Under Which the Polynomial Hierarchy Collapses

19/77
The polynomial hierarchy is assumed to be a strict hierarchy, with each level representing a distinct complexity class. However, under certain
conditions, the hierarchy may collapse, meaning that the higher levels of the hierarchy are equivalent to lower levels.

Collapse to NP or coNP: If Σᵢ^P equals Πᵢ^P for some level i, then the hierarchy collapses at level i. This would imply that higher levels of the
hierarchy do not provide additional computational power, effectively reducing the entire polynomial hierarchy to a single class like NP or
coNP.

NP = PSPACE: Another hypothetical collapse would occur if NP were equal to PSPACE. This would suggest that problems in NP could be
solved in polynomial space, leading to a collapse of the hierarchy at the level of Σ₁^P (since NP corresponds to the first level of the
hierarchy).

Oracle Separation: There are known oracles (theoretical "black-box" machines) where P ≠ NP but P = NP for certain restricted classes, and
similarly, NP = PSPACE in some oracle worlds. These separations are important in understanding the conditions under which the polynomial
hierarchy could collapse.

Practical Collapse: While there is no proof that the polynomial hierarchy collapses in general, certain results in complexity theory suggest
that if certain famous problems, such as NP-complete or coNP-complete, were to be solved efficiently, then the polynomial hierarchy would
collapse.

5. Containment of the Polynomial Hierarchy in PSPACE

One of the most important results in complexity theory is the containment of the entire polynomial hierarchy within PSPACE.

Containment of PH in PSPACE: It is known that PH is contained within PSPACE. This result follows from the fact that any problem in the
polynomial hierarchy can be solved using polynomial space. This containment holds because, at each level of the polynomial hierarchy, the
decision process involves alternating quantifiers that can be simulated with a polynomial amount of space.

Implication: Since PSPACE contains Σᵢ^P and Πᵢ^P for all i, this result shows that problems that lie in the polynomial hierarchy are not
necessarily more complex than problems in PSPACE. In particular, this containment suggests that PSPACE is a very powerful class,
encompassing problems that would otherwise appear to be harder at higher levels of the hierarchy.

6. Summary

TQBF is a PSPACE-complete problem, meaning that it is both in PSPACE and that all other problems in PSPACE can be reduced to it in
polynomial space.

The Polynomial Hierarchy is a layered structure that generalizes NP, coNP, and P by introducing multiple levels of quantifiers. Problems at
each level of the hierarchy are classified as Σᵢ^P and Πᵢ^P, depending on the alternation between existential and universal quantifiers.

There are Σᵢ^P-complete and Πᵢ^P-complete problems that capture the computational difficulty of each level of the hierarchy.

The hierarchy may collapse under certain conditions, such as when Σᵢ^P equals Πᵢ^P or when certain famous problems like NP or PSPACE
collapse.

The Polynomial Hierarchy is contained within PSPACE, indicating that problems in the hierarchy can be solved using polynomial space,
highlighting the strength of PSPACE as a complexity class.

In conclusion, the study of PSPACE and the Polynomial Hierarchy deepens our understanding of the structure of complexity classes, providing
important insights into the relationships between decision problems and the resources required to solve them.

Lecture 11: More on Polynomial Hierarchy

1. Complete Problems for Σᵢ^P and Πᵢ^P

As mentioned earlier, the Polynomial Hierarchy (PH) consists of multiple levels, each corresponding to a different alternating quantifier
structure. Each level in the hierarchy has associated complete problems that are representative of the complexity of the problems at that level.

Σᵢ^P-complete Problems: These problems are complete for the class Σᵢ^P, meaning that every problem in Σᵢ^P can be polynomial-time
reduced to these problems.

20/77
 Example: The Quantified Boolean Formula (QBF) problem is Σᵢ^P-complete for appropriate values of i. Specifically, QBF with i
alternating quantifiers corresponds to the class Σᵢ^P.

These problems involve a sequence of alternating existential and universal quantifiers, starting with an existential quantifier.

Πᵢ^P-complete Problems: Similarly, there are Πᵢ^P-complete problems that are complete for the class Πᵢ^P. These problems require
universal quantifiers at the beginning of the alternation sequence, followed by alternating existential quantifiers.

Example: A QBF problem with i alternating quantifiers, where the first quantifier is universal, is Πᵢ^P-complete.

Reduction to Complete Problems: A problem is complete for Σᵢ^P or Πᵢ^P if all other problems in that class can be reduced to it in
polynomial time. Such complete problems serve as the canonical problems that capture the computational complexity of the corresponding
class.

2. Why PH is Not Believed to Have a Complete Problem

While the polynomial hierarchy has complete problems at each level, it is not believed to have a single "complete" problem for the entire
hierarchy. There are several reasons for this belief:

No Known Single Problem: The polynomial hierarchy generalizes NP and coNP, which are already known to have complete problems (like
SAT for NP and TAUT for coNP). However, for higher levels of the polynomial hierarchy, no single problem has been identified that captures
the complexity of the entire hierarchy. This is in part due to the alternating quantifiers in the hierarchy, which makes it difficult to pinpoint
one problem that encompasses the entire structure.

Complexity of Alternating Quantifiers: Each level of the polynomial hierarchy adds complexity by alternating between existential and
universal quantifiers. As the number of quantifier alternations increases, it becomes increasingly difficult to reduce all problems at that level
to a single "canonical" problem. Thus, no one problem can encapsulate the computational difficulty of the entire hierarchy.

Interdependence Between Levels: The levels of the polynomial hierarchy are interdependent, with higher levels depending on the
structure and complexity of lower levels. This makes the notion of a single problem that represents the entire hierarchy problematic.
Problems at higher levels often involve combinations of problems from lower levels, adding to the complexity.

Lack of Closure: While each level in the hierarchy has complete problems under polynomial-time reductions, these classes are not
necessarily closed under operations like complementation or intersection in a way that would naturally lead to a single problem capturing
the complexity of all levels. This suggests that the hierarchy is too complex to have a complete problem for the entire class.

3. Alternating Turing Machines (ATMs)

To formalize the alternating quantifiers that define the polynomial hierarchy, we use a computational model known as Alternating Turing
Machines (ATMs).

Definition: An Alternating Turing Machine (ATM) is a generalization of a Non-Deterministic Turing Machine (NDTM), where the machine
can make choices that depend on alternating between existential and universal states. In other words, it can behave in a non-deterministic
manner at some points (choosing from multiple possible transitions) and deterministically at other points (ensuring that all possible
transitions lead to an accepting state).

In an ATM, each configuration of the machine can be either existential or universal.

Existential configurations allow the machine to accept if any of the possible next configurations leads to acceptance.

Universal configurations require that all possible next configurations lead to acceptance for the machine to accept.

Acceptance Criterion:

An ATM accepts an input if there is a sequence of configurations that leads to an accepting state. This sequence must satisfy the
alternating quantifier pattern of the computation.

The acceptance condition is thus defined by the alternation of existential and universal states. At existential configurations, the machine
accepts if one of the possible next states leads to acceptance, while at universal configurations, it accepts only if all possible next states
lead to acceptance.

Running Time:

21/77
 The running time of an ATM is defined in terms of the time complexity of the machine's computation. Like other Turing machines,
ATMs are typically analyzed with respect to their polynomial time complexity, which is determined by the number of computational
steps needed to reach an accepting configuration.

The time complexity of an ATM is polynomial if the machine runs in polynomial time on all inputs. This defines the class of problems
that can be solved by ATMs in polynomial time, which corresponds to the polynomial hierarchy.

For example:

Σ₁^P problems are solvable by an ATM with only existential quantifiers (i.e., a non-deterministic machine).

Σ₂^P problems involve alternating existential and universal quantifiers, and so on.

4. Running Time of Alternating Turing Machines

The running time of an Alternating Turing Machine (ATM) is the number of steps it takes to reach an accepting or rejecting configuration. The
complexity of this machine is determined by the length of the computation and how it alternates between existential and universal states.

Polynomial Time Alternating Turing Machines: An ATM that can decide a problem in polynomial time has a running time that is bounded
by a polynomial function in the size of the input. This means the ATM halts after at most polynomially many steps.

Polynomial Hierarchy and Alternating Turing Machines:

Σᵢ^P problems are solvable by an ATM that alternates i times between existential and universal quantifiers.

Πᵢ^P problems are similarly solvable by an ATM but start with a universal quantifier and then alternate existential and universal
quantifiers.

The polynomial time complexity of an ATM is significant because it directly relates to the corresponding level of the polynomial hierarchy. If a
problem can be solved by an ATM in polynomial time, it belongs to the appropriate class in the polynomial hierarchy.

5. Summary

Complete Problems: Each level of the Polynomial Hierarchy has canonical complete problems that represent the complexity of that level.
However, the hierarchy as a whole does not have a single complete problem due to the increasing complexity introduced by alternating
quantifiers and the interdependencies between levels.

Alternating Turing Machines (ATMs): These machines generalize the concept of non-deterministic machines and are essential for defining
the polynomial hierarchy. ATMs alternate between existential and universal states, and their acceptance criteria and running time are
fundamental to understanding the complexity of problems in the hierarchy.

Running Time: The running time of ATMs is polynomial if the machine halts after a polynomial number of steps, which defines the
complexity classes Σᵢ^P and Πᵢ^P at different levels of the polynomial hierarchy.

In conclusion, while each level of the polynomial hierarchy has complete problems, the overall hierarchy does not have a single complete
problem, and alternating Turing machines are a powerful tool for analyzing the complexity of these problems.

Lecture 12: Alternating Turing Machines

1. Alternating Time Class ATIME

The Alternating Time (ATIME) class is a complexity class that generalizes the idea of non-deterministic time to alternating Turing machines
(ATMs). While non-deterministic machines can make arbitrary guesses at certain points in their computation, alternating Turing machines can
alternate between existential and universal configurations.

Definition: A problem is in the class ATIME(T(n)) if there exists an alternating Turing machine (ATM) that decides the problem in O(T (n))
time, where T (n) is a function of the input size n. Here, the machine has alternating existential and universal transitions, which can be
thought of as making a series of alternating existential (or non-deterministic) choices and universal (or deterministic) checks.

Existential Configurations: A configuration where the machine accepts if any of the possible transitions lead to an accepting state.

22/77
 Universal Configurations: A configuration where the machine accepts only if all possible transitions lead to an accepting state.

Significance: The class ATIME(T(n)) extends the computational power of non-deterministic time by allowing the machine to alternate
between existential and universal states, providing a richer framework for modeling complex computational problems.

2. ATMs with Limited Number of Alternations

In the context of alternating Turing machines, we often restrict the number of alternations between existential and universal states to
understand more refined complexity classes.

Limited Alternations: The number of alternations in an ATM can be bounded, and this restriction leads to different classes within the ATIME
hierarchy. For example, when the number of alternations is limited to a constant, the machine behaves similarly to a non-deterministic
machine.

Classes with Limited Alternations:

ΣᵢTIME(T(n)): This class consists of problems that can be solved by an alternating Turing machine with i alternations between existential
and universal configurations, running in T (n) time. The number of alternations is fixed for a given level i of the hierarchy.

ΠᵢTIME(T(n)): This class consists of problems solvable by an alternating Turing machine with i alternations, starting with a universal
configuration, running in T (n) time.

Alternating Turing Machines and Time Complexity:

A machine with fewer alternations can solve simpler problems (i.e., problems that are computationally less complex), while increasing
the number of alternations adds to the machine's power, allowing it to solve more complex problems.

3. Defining the Polynomial Hierarchy Using ATMs

The Polynomial Hierarchy (PH) is often defined using alternating Turing machines. In particular, the levels of the hierarchy correspond to
machines that alternate between existential and universal configurations, with the number of alternations increasing at each level.

Defining Σᵢ^P and Πᵢ^P using ATMs:

Σᵢ^P: The class Σᵢ^P consists of problems that can be solved by an ATM with i alternations, starting with an existential quantifier, running
in polynomial time.

Πᵢ^P: The class Πᵢ^P consists of problems that can be solved by an ATM with i alternations, starting with a universal quantifier, running
in polynomial time.

Using this definition, we can characterize the polynomial hierarchy entirely in terms of alternating Turing machines, with the classes Σᵢ^P and
Πᵢ^P corresponding to different levels of the hierarchy, based on the number of quantifiers and the alternations between existential and
universal quantifiers.

Implication: This provides a formal framework for understanding the relationships between different complexity classes and offers insights
into how the complexity of a problem can grow as the number of alternations increases.

4. AP = PSPACE and Related Theorems

The class AP (Alternating Polynomial Time) is defined as the class of problems solvable by an alternating Turing machine in polynomial time.
There is an important result in computational complexity, known as AP = PSPACE, which shows that alternating polynomial time is equivalent in
power to PSPACE.

AP = PSPACE: This result states that problems solvable by an alternating Turing machine in polynomial time are exactly the same as
problems solvable by a deterministic Turing machine using polynomial space. This is a key result in space complexity theory, showing that
alternating time complexity can be reduced to space complexity.

Intuition: The result makes use of the fact that alternating Turing machines, even with polynomial time constraints, can simulate space-
bounded computations. This equivalence also suggests that the power of alternating computation is tied to the resources required for

23/77
 the computation (in terms of space).

Other Related Theorems: There are similar results that relate alternating time complexity to space complexity, and these results are crucial
for understanding the interactions between different computational resources such as time and space.

5. Oracle-Based Definition of the Polynomial Hierarchy

An oracle is a theoretical device that allows a machine to make arbitrary queries to an external source that provides answers to specific
questions. The use of oracles is essential for defining complexity classes in terms of relative power.

Oracle Turing Machines: An oracle Turing machine is a Turing machine that can make queries to an oracle. The answers to these queries
are provided instantaneously, and the machine uses the oracle’s answers to help solve its problem.

Oracle-Based Definition of PH: The polynomial hierarchy can be defined in terms of oracle Turing machines. The basic idea is that each
level of the polynomial hierarchy corresponds to a machine that makes queries to an oracle of a lower level in the hierarchy.

For example, Σᵢ^P and Πᵢ^P can be defined as the class of problems solvable by an oracle Turing machine with access to an oracle that
solves problems from a lower level of the hierarchy, with the oracle being queried during the computation.

Oracle Separations: One of the key insights from oracle-based complexity theory is that there exist oracles for which P = NP and oracles for
which P ≠ NP. This highlights the importance of oracles in separating different complexity classes and understanding their relationships.

6. NP ∪ coNP ⊆ PSAT

A significant result in complexity theory is the containment of NP ∪ coNP within PSAT, where PSAT is the problem of determining the
satisfiability of a quantified Boolean formula (QBF). This result is derived from the structure of the polynomial hierarchy.

PSAT: The problem of PSAT involves determining whether a quantified Boolean formula is true. This problem generalizes SAT and is PSPACE-
complete, meaning it is solvable using polynomial space.

Containment: The containment NP ∪ coNP ⊆ PSAT indicates that the union of NP and coNP problems can be reduced to the PSPACE-
complete problem PSAT. This is significant because it demonstrates that the problems in NP and coNP can be tackled using polynomial
space resources, which is the same space required for solving PSAT.

7. Summary

ATIME(T(n)): The class of problems solvable by an alternating Turing machine in T (n) time, where the machine alternates between
existential and universal configurations.

ΣᵢTIME(T(n)) and ΠᵢTIME(T(n)): These classes represent problems solvable by alternating Turing machines with i alternations, with ΣᵢTIME
starting with an existential configuration and ΠᵢTIME starting with a universal configuration.

Polynomial Hierarchy Using ATMs: The polynomial hierarchy is naturally defined using alternating Turing machines, with Σᵢ^P and Πᵢ^P
corresponding to problems solvable by ATMs with i alternations.

AP = PSPACE: Alternating polynomial time is equivalent to PSPACE, indicating that alternating time complexity can be captured by space
complexity.

Oracle-Based Definition of PH: The polynomial hierarchy can be defined in terms of oracle Turing machines, with each level of the hierarchy
corresponding to queries to an oracle from a lower level.

NP ∪ coNP ⊆ PSAT: The union of NP and coNP is contained within the PSPACE-complete problem PSAT, showing the relationship between
these complexity classes and their connection to space-bounded problems.

This lecture emphasizes the importance of alternating Turing machines and oracles in defining and understanding the complexity classes of the
polynomial hierarchy.

Lecture 13: Equivalence of Quantifiers and Oracle-Based Definitions of Polynomial Hierarchy

24/77
1. Introduction to Equivalence of Quantifiers and Oracle-Based Definitions of Polynomial Hierarchy

In this lecture, we explore the equivalence between the quantifier-based and oracle-based definitions of the polynomial hierarchy (PH).
Specifically, we will prove the equivalence Σᵖ₂ = NPSAT and provide an introduction to Boolean circuits as they relate to complexity theory.

2. Recap: Polynomial Hierarchy and Quantifiers

The Polynomial Hierarchy (PH) is a hierarchy of complexity classes that generalizes the classes P, NP, and coNP, extending them to multiple
levels of alternations between existential and universal quantifiers. The levels of the PH are typically defined as follows:

Σᵢ^P: Problems solvable by an alternating Turing machine with i alternations, starting with an existential quantifier, and running in
polynomial time.

Πᵢ^P: Problems solvable by an alternating Turing machine with i alternations, starting with a universal quantifier, and running in polynomial
time.

The goal of this lecture is to demonstrate that the second level of the polynomial hierarchy, Σᵖ₂, can be defined both using alternating
quantifiers and using oracle Turing machines, and these two definitions are equivalent.

3. Proof of Σᵖ₂ = NPSAT

To begin the discussion, we will prove that Σᵖ₂ = NPSAT, where NPSAT refers to the problem of Non-deterministic Quantified Boolean Formula
(QBF) satisfiability.

Σᵖ₂ (Second Level of the Polynomial Hierarchy): The class Σᵖ₂ is defined as problems that can be solved by an alternating Turing machine
with two layers of alternations (one existential and one universal), running in polynomial time.

Specifically, problems in Σᵖ₂ can be formulated as:

∃x1 ∀x2 ϕ(x1 , x2 )

​ ​ ​ ​

where ϕ(x1 , x2 ) is a Boolean formula that can be evaluated in polynomial time, and x1 and x2 represent different sets of variables. The
​ ​ ​ ​

quantifier alternation starts with an existential quantifier for x1 , followed by a universal quantifier for x2 .
​ ​

NPSAT (Non-deterministic Quantified Boolean Formula): The NPSAT problem involves determining the satisfiability of a quantified
Boolean formula, where the formula is of the form:

∃x1 ∀x2 ∃x3 ⋯ ϕ(x1 , x2 , …)

​ ​ ​ ​ ​

This problem is in NPSAT because it is solvable by a non-deterministic Turing machine that guesses the values of the existential variables,
checks the correctness of the universal variables, and finally evaluates the Boolean formula.

Equivalence: We now demonstrate that Σᵖ₂ is equivalent to NPSAT. The key idea is to show that for a Σᵖ₂ problem, we can construct an
NPSAT problem and vice versa.

From Σᵖ₂ to NPSAT: Given a Σᵖ₂ problem ∃x1 ∀x2 ϕ(x1 , x2 ), we can transform this into an NPSAT problem by treating the existential
​ ​ ​ ​

and universal quantifiers as alternating decisions within a non-deterministic framework. Specifically, the non-deterministic machine can
guess values for x1 , check for all possible values of x2 , and evaluate the Boolean formula ϕ(x1 , x2 ).
​ ​ ​ ​

From NPSAT to Σᵖ₂: Conversely, any NPSAT problem can be expressed as a problem in Σᵖ₂. This is because non-determinism in NPSAT is
captured by the existential quantifier in Σᵖ₂, and universal quantifiers are used to verify the correctness of the non-deterministic
choices.

This establishes the equivalence between Σᵖ₂ and NPSAT, showing that the second level of the polynomial hierarchy can be understood both
through alternating quantifiers and non-deterministic QBF satisfiability.

4. Introduction to Boolean Circuits

25/77
With the equivalence of quantifiers and oracle-based definitions of the polynomial hierarchy established, we now turn our attention to Boolean
circuits, which play a key role in complexity theory, especially in the study of problems in classes like P, NP, and PSPACE.

Boolean Circuit: A Boolean circuit is a computational model consisting of a network of gates that perform Boolean operations on inputs.
Each gate in the circuit performs a basic Boolean operation (such as AND, OR, or NOT) on one or more input bits and produces an output
bit.

Structure: A Boolean circuit consists of:

Inputs: Binary variables (0 or 1).

Gates: Boolean operations (AND, OR, NOT, etc.).

Outputs: Result of the computations based on the circuit configuration.

Size of a Boolean Circuit: The size of a Boolean circuit is the number of gates in the circuit. A circuit can be characterized by its depth (the
maximum length of a path from an input to the output) and its size (the total number of gates).

Boolean Circuit Complexity: The complexity of a problem can be analyzed in terms of the size of the smallest Boolean circuit that solves the
problem. A problem is said to be in the complexity class AC^0 if it can be solved by a constant-depth Boolean circuit with a polynomial
number of gates. If a Boolean circuit for a problem requires logarithmic depth and polynomial size, it may belong to classes like NC^1 or
NC^2.

Boolean Circuits in Complexity Theory: Boolean circuits provide a concrete model for parallel computation. They allow us to study the
parallel time complexity of a problem, as the gates can be evaluated simultaneously in parallel. This model is useful for analyzing problems
that are solvable in P or NP, as many problems can be reduced to circuit satisfiability problems.

Circuit Satisfiability (CIRC-SAT): This is the problem of determining whether a given Boolean circuit has a satisfying assignment (i.e.,
whether there exists an input that makes the output true). This problem is known to be NP-complete, and its analysis leads to
important results in parallel complexity and computational complexity theory.

5. Summary

Equivalence of Σᵖ₂ and NPSAT: We have proven that the second level of the polynomial hierarchy, Σᵖ₂, is equivalent to NPSAT, the problem
of non-deterministic QBF satisfiability. This equivalence highlights the close relationship between the quantifier-based and oracle-based
definitions of the polynomial hierarchy.

Boolean Circuits: Boolean circuits are a powerful computational model that allows us to study parallel computations and the complexity of
problems in classes like P and NP. The size and depth of a Boolean circuit can be used to analyze the efficiency of solving problems in these
classes.

The topics covered in this lecture lay the foundation for understanding the deeper connections between quantifiers, alternating Turing
machines, and the circuit complexity model in computational theory.

Lecture 14: Boolean Circuits

1. Introduction to Boolean Circuits

In this lecture, we explore the computational model of Boolean circuits, discussing their definition, complexity, and relation to the class P/poly.
We also introduce the notion of circuit depth, families of circuits, and the P-uniform circuit families. Understanding Boolean circuits is crucial as
they help us analyze parallel computation and the relationship between circuit complexity and decision problems in P and other complexity
classes.

2. Definition of Boolean Circuits

A Boolean circuit is a computational model consisting of a directed acyclic graph, where each node represents a gate that performs a Boolean
operation, and edges represent the flow of data. The inputs to the circuit are binary variables, and the output is the result of the Boolean

26/77
operations.

Gates: The gates in a Boolean circuit perform basic logical operations like AND, OR, and NOT. A gate has one or more input edges and one
output edge.

Inputs: Binary variables or constants (0 or 1) provided to the circuit.

Outputs: The result of the computations, typically a single Boolean value (True/False or 1/0).

Size of a Circuit: The size of a Boolean circuit is determined by the number of gates in the circuit.

Depth of a Circuit: The depth of a Boolean circuit is the longest path from any input to the output, which corresponds to the number of
layers of gates through which data passes. Depth is a measure of how many steps (or "levels") the circuit takes to produce an output from
the input.

3. Families of Boolean Circuits

A family of Boolean circuits is a sequence of Boolean circuits Cn such that each circuit in the sequence solves a particular problem for input size
​

n. The circuits in the family are indexed by the input size, and for a given input size n, the circuit Cn computes the desired function or decision
​

problem for inputs of length n.

Formal Definition: A family of Boolean circuits {Cn } is a sequence of circuits where:

​

Each circuit Cn has n input variables.

​

Cn solves the decision problem for inputs of size n, i.e., Cn (x) = f (x), where f is the Boolean function computed by the family.
​ ​

Class SIZE(T(n)): This class consists of Boolean circuit families where the size of the circuit Cn is bounded by some function T (n), that is, the
​

circuit has at most T (n) gates. For example, SIZE(n^k) refers to circuits whose size is bounded by a polynomial function in n.

4. Depth of a Boolean Circuit

The depth of a Boolean circuit is the length of the longest path from an input to the output, measured in terms of the number of gates in the
path. This measure reflects the parallel complexity of the circuit, i.e., how many layers of parallel computation are required to compute the
output.

Shallow Circuits: If the depth of a circuit is small, it can be computed in parallel efficiently, leading to problems in complexity classes such as
NC (Nick's Class), which includes problems that can be solved in polylogarithmic time using a polynomial number of processors.

Deep Circuits: If the depth is large, the circuit requires more sequential steps to compute the output, which may lead to a higher
computational cost in terms of time.

5. The Class P/poly

The class P/poly consists of decision problems that can be solved by a family of Boolean circuits of polynomial size. Formally, a language L is in
P/poly if there exists a family of Boolean circuits {Cn } such that:
​

For each input length n, there is a circuit Cn that computes the characteristic function of L, i.e., Cn (x)
​ ​ = 1 if x ∈ L and Cn (x) = 0 if x ∈
​ /
L.
The circuit Cn has at most polynomial size (i.e., ∣Cn ∣
​ ​ = O(nk ) for some constant k ) and is computed in polynomial time.
Key Properties of P/poly:

Non-uniformity: Unlike P, which requires a single deterministic algorithm to compute a problem for any input size, P/poly allows for a
different circuit for each input size. This means that solutions to problems in P/poly can be computed using non-uniform resources.

Relation to P: It is clear that P is a subset of P/poly because polynomial-time algorithms correspond to circuit families of polynomial
size. However, the converse is not necessarily true. P/poly allows non-uniformity, while P is restricted to uniform algorithms.

Example of Languages in P/poly:

27/77
 Many undecidable languages, such as the halting problem, may be included in P/poly. While these languages are undecidable in a
general sense (i.e., no single Turing machine can decide them for all inputs), there exist circuit families that can "decide" these
problems in a non-uniform manner.

6. P-Uniform Circuit Families

A circuit family {Cn } is P-uniform if there exists a polynomial-time Turing machine that, for each input size n, outputs the description of the
​

circuit Cn . In other words, the circuits in the family are uniformly constructed by a polynomial-time algorithm.
​

Definition: A family of Boolean circuits {Cn } is P-uniform if there exists a polynomial-time Turing machine M such that for every n, M
​

outputs the description of the circuit Cn on input n.

​

Class of Languages Accepted by P-Uniform Circuit Families: The class of languages accepted by P-uniform circuit families coincides with P.
This means that problems solvable by P-uniform circuit families are exactly the problems solvable by polynomial-time algorithms.

Reasoning: The key observation here is that a P-uniform circuit family can be efficiently constructed by a polynomial-time machine,
and the family can solve any problem in P. This is because, given that the circuits are uniform, there exists a polynomial-time algorithm
(a Turing machine) that produces the corresponding circuit for any input size, allowing the problem to be solved in polynomial time.

7. Summary

Boolean Circuits: Boolean circuits are a model of computation that uses logical gates to process binary input and produce output. They are
defined by their size (number of gates) and depth (longest path from input to output).

P/poly: The class P/poly contains languages that can be solved by polynomial-size Boolean circuits, possibly with non-uniformity. It includes
problems solvable by families of circuits that may differ for different input sizes.

P-uniform Circuit Families: The class of languages accepted by P-uniform circuit families coincides with the class P, indicating that
uniformity (polynomial-time constructibility of circuits) is sufficient for problems in P to be solved using circuits.

This lecture connects circuit complexity to classical computational classes like P and NP, laying the foundation for further exploration of non-
uniform and uniform computation models.

Lecture 15: Shannon's Theorem and Karp-Lipton-Sipser Theorem

1. L-Uniform Circuit Families

Before diving into the core topics of this lecture, we review the concept of L-uniform circuit families. A circuit family is said to be L-uniform if
there exists a logarithmic-time deterministic Turing machine that, for each input size n, can output a description of the circuit Cn . The
​

uniformity condition here is weaker than P-uniformity but still ensures that circuits are constructed in a systematic manner for all input sizes.

L-uniformity allows for a logarithmic-time construction of circuits, implying that the machine producing the circuit descriptions runs in
O(log n) time. This is a much more relaxed constraint than polynomial-time construction.

2. Shannon's Theorem

Shannon's Theorem (also known as the Exponential Size Requirement Theorem) is a fundamental result in circuit complexity. It asserts that
there exist Boolean functions that cannot be computed by any Boolean circuit family of polynomial size. In other words, there are problems for
which the best possible circuit size grows exponentially with the input size.

Theorem Statement (Shannon's Theorem): There exists a family of Boolean functions fn on n-bit inputs such that any Boolean circuit
​

computing fn requires at least 2Ω(n) gates.

​

Implications:

28/77
 This theorem shows that not all problems can be solved efficiently using polynomial-size circuits. It highlights the inherent
difficulty of certain problems in the context of circuit complexity.

Specifically, this theorem is used to demonstrate that certain Boolean functions, which appear to have no easy structure, cannot be
compressed into smaller circuits, emphasizing that there is a size barrier in computational complexity for some problems.

Proof Outline: The proof of Shannon’s theorem involves showing that the number of possible Boolean functions grows exponentially in
n
the size of the input (specifically, 22 functions for n-bit inputs). For any polynomial-size circuit, there are a limited number of Boolean
functions it can compute. Since the number of Boolean functions far exceeds the number of polynomial-size circuits, there must exist
functions for which no polynomial-size circuit can exist.

3. Karp-Lipton-Sipser Theorem

The Karp-Lipton-Sipser Theorem is a significant result in computational complexity that provides an interesting relationship between
complexity classes, particularly the classes NP and the Polynomial Hierarchy (PH).

Theorem Statement: If NP ⊆ P/poly, then PH = ΣP2 .

 ​

Implications:

NP not in P/poly implies that the Polynomial Hierarchy (PH) collapses to its second level ΣP
2 , i.e., the classes Σ2 and Π2 become
​
P
​
P
​

equal to ΣP
1 and Π1 in this case.
​
P
​

This result essentially suggests that the assumption that NP problems cannot be solved by polynomial-size circuits (which would
imply non-uniformity) has profound consequences on the structure of the Polynomial Hierarchy.

The collapse of the Polynomial Hierarchy is a strong and surprising result. It means that if NP problems cannot be solved by
polynomial-size circuits, the complexity of higher levels of the hierarchy is not distinct and collapses to the second level.

Proof Outline:

The proof relies on the concept of oracle machines. The idea is to use a non-uniform oracle to reduce problems in higher levels of the
Polynomial Hierarchy to problems in NP and show that such reductions imply that PH collapses to ΣP
2. ​

The basic idea behind the proof is that if NP is not contained in P/poly, then the behavior of oracle machines for higher levels of the
hierarchy (like ΣP
2 ) is influenced in such a way that it causes the collapse of the hierarchy to the second level.
​

4. Conclusion

This lecture covered two important results in complexity theory: Shannon's Theorem and the Karp-Lipton-Sipser Theorem. Shannon's theorem
demonstrates the exponential growth of circuit size for certain Boolean functions, while the Karp-Lipton-Sipser theorem provides a deep insight
into the relationship between NP and the Polynomial Hierarchy. Specifically, it shows that if NP is not contained in P/poly, then the Polynomial
Hierarchy collapses to its second level, which has significant consequences for our understanding of the computational complexity of decision
problems.

These theorems are foundational in complexity theory, providing crucial insights into the limitations of computational models and the hierarchy
of complexity classes.

Lecture 16: Bounded Depth Circuit Classes

1. Introduction to Bounded Depth Circuit Classes

In this lecture, we delve into the circuit complexity classes NC (Nick's Class) and AC (Alternating Class), which are concerned with Boolean circuits
of bounded depth. These classes are important because they reflect the parallel complexity of problems. We also examine the hierarchy within
these classes, the placement of specific problems like Parity in NC1, and the Nonuniform Hierarchy Theorem.

29/77
2. Circuit Classes NC and AC

NC (Nick’s Class): The class NC consists of decision problems that can be solved by Boolean circuits with polylogarithmic depth and
polynomial size. Formally, a language L is in NC if there exists a family of Boolean circuits {Cn } such that:
​

The circuit Cn has at most polynomial size.

​

k
The circuit depth is O(log n) for some constant k , i.e., the longest path from input to output is logarithmic in the input size.

NC1 is a subset of NC where the circuits have constant depth (i.e., the number of layers of gates is bounded by a constant, independent of
the input size).

AC (Alternating Class): The class AC consists of problems that can be solved by Boolean circuits with alternating gates and bounded depth.
The key difference between AC and NC is the use of alternating gates, which combine AND and OR gates in a more flexible way.

AC0: This is the simplest class in AC, consisting of Boolean circuits with constant depth and unbounded fan-in (i.e., the number of
inputs to a gate can be arbitrarily large). The class is further restricted by using only AND, OR, and NOT gates.
i
AC1, AC2, etc.: These classes are defined in terms of the depth of the circuits, where circuits in ACi have depth O(log n).
Relation Between NC and AC:

NC1 is contained in AC0, as both have constant-depth circuits. However, AC0 circuits may have more flexibility in terms of gate
connections (unbounded fan-in).

AC classes are typically considered more powerful than NC classes because alternating circuits (used in AC) allow for more complex
combinations of gates compared to NC circuits.

3. Hierarchy within NC and AC Classes

The classes NC and AC have internal hierarchies based on the depth of the circuits. The NC hierarchy and AC hierarchy are defined as follows:
k
NCk: This class consists of decision problems solvable by circuits of depth O(log n) and size polynomial in n.
NC1 is the class of problems solvable by constant-depth circuits (depth O(1)).
2
NC2, NC3, ...: These classes are for problems solvable by circuits of depth O(log n), O(log3 n), ..., respectively.
k
ACk: This class consists of problems solvable by circuits of depth O(log n), but with alternating gates.
AC0 consists of problems solvable by constant-depth circuits with unbounded fan-in gates.

AC1, AC2, ... are for alternating circuits with progressively higher depths.

Relation between NC and AC Hierarchies:

AC0 is a subset of NC1 because NC1 circuits are constant-depth circuits, while AC0 circuits are also constant-depth but with unbounded
fan-in.

Higher levels of AC are generally considered to be more powerful than corresponding levels of NC because of the ability to use
alternating gates.

4. Parity is in NC1

The Parity problem (determining whether an input has an odd or even number of 1’s) is a well-known problem in circuit complexity. It turns out
that Parity is in NC1, meaning it can be solved by a constant-depth circuit with polynomial size.

Reasoning: The Parity function can be computed by a circuit with a series of XOR gates, which are sufficient for computing the parity of a
sequence of bits. XOR gates can be arranged in layers such that the depth of the circuit is constant, and the size of the circuit is polynomial.

5. Nonuniform Hierarchy Theorem

30/77
The Nonuniform Hierarchy Theorem is a fundamental result in circuit complexity that extends the classical hierarchy theorems (which relate
time complexity classes based on resource bounds) to the realm of nonuniform computation. It states that:

For all c, there exists a language L ∈ Σ4 such that L ∈

​ / SIZE(nc ).
This means that for sufficiently large c, there are languages in the Polynomial Hierarchy (specifically at the Σ4 -level) that require
​

circuits of exponential size. This result is significant because it shows the non-existence of a uniform circuit class that can solve all
problems in higher levels of the Polynomial Hierarchy using polynomial-size circuits.

Implications: The Nonuniform Hierarchy Theorem demonstrates the exponential growth of the complexity of certain problems as we move
to higher levels in the Polynomial Hierarchy. It asserts that certain problems cannot be solved by small, nonuniform circuits, reinforcing the
difference in complexity between different levels of the hierarchy.

6. Conclusion

This lecture covered the important classes NC and AC, focusing on the hierarchy within these classes and their relationships. We saw how Parity
fits within the NC1 class, and we explored the profound implications of the Nonuniform Hierarchy Theorem. These results are foundational in
the study of circuit complexity and provide insights into the parallel complexity of problems as well as the limitations of bounded-depth circuits
in solving higher-complexity problems within the Polynomial Hierarchy.

NC and AC Hierarchies: These classes represent problems solvable by circuits of bounded depth and polynomial size, with AC being
generally more powerful due to its use of alternating gates.

Nonuniform Hierarchy Theorem: This theorem underscores the limitations of nonuniform circuits in solving complex problems in the
higher levels of the Polynomial Hierarchy, showing that size limitations lead to exponential growth in circuit complexity for certain
problems.

Lecture 17: Kannan's Theorem

1. Introduction

In this lecture, we conclude our discussion on circuit complexity and focus on Kannan's Theorem, an important result in the area of
computational complexity that has implications for circuit lower bounds and hardness of problems. Kannan’s theorem addresses the
complexity of certain problems and provides a key result related to circuit families and their ability to compute functions efficiently.

2. Kannan’s Theorem

Kannan's Theorem deals with lower bounds for certain problems in the context of circuit complexity. It shows that there exist Boolean functions
that require large circuit sizes to compute, even when the circuits are allowed to have certain special properties or types of computation.

Theorem Statement: There exists a Boolean function f for which any circuit that computes f requires a size of at least 2Ω(n) , even if the
circuit is restricted to certain types of gates or certain computational models.

Implications:

The theorem emphasizes that not all functions can be computed efficiently, even if the circuits are constrained to specific types or
structures (such as restricted gates or bounded depth). This provides evidence for the inherent difficulty of certain problems in the
context of circuit complexity.

It can be used to establish lower bounds for specific classes of functions that cannot be efficiently computed by polynomial-size
circuits, strengthening our understanding of the hardness of problems in these classes.

3. Application of Kannan's Theorem

31/77
 Circuit Lower Bounds: One of the main applications of Kannan’s theorem is in proving lower bounds for circuit classes. It helps show that
for certain Boolean functions or problems, no matter how the circuits are constructed (even with specific restrictions), the size of the circuit
required will always be exponentially large.

Complexity of Computation: Kannan’s theorem demonstrates that there are fundamental limits to what can be computed efficiently using
Boolean circuits. It is used to prove hardness results for certain computational problems by showing that no polynomial-size circuit can
compute them.

Hardness of Functions: The theorem provides a way to establish the hardness of certain functions, i.e., the impossibility of solving the
problem using small circuits. This is particularly useful in complexity theory to prove that certain problems are inherently hard to compute.

4. Proof Outline of Kannan's Theorem

While the full proof of Kannan's Theorem is technical, we outline the basic steps and ideas behind it:

Step 1: Defining the Function: The first step is to define the Boolean function f for which we want to establish a lower bound. This function
is typically chosen to be complex enough to require a large circuit size, but not so simple as to be trivially computable by small circuits.

Step 2: Circuit Size Bound: Next, we analyze the size of circuits that can compute this function. The key idea is that no matter how the circuit
is structured (with respect to gate types, circuit depth, etc.), the size must grow at least exponentially with the input size n. This establishes
the lower bound.

Step 3: Contradiction with Polynomial-Time Assumptions: Kannan's Theorem typically involves a contradiction argument. Assuming that
there exists a polynomial-size circuit for the function f , we derive an inconsistency with known results from complexity theory or circuit
theory. This contradiction establishes that no such circuit can exist, and therefore the circuit size must be exponentially large.

5. Connection to Circuit Complexity Classes

Kannan’s theorem provides a deep connection between lower bounds in circuit complexity and the computational difficulty of problems in
various complexity classes:

P and Circuit Complexity: Kannan's theorem helps establish the limitations of polynomial-size circuits in computing certain problems. This
is directly related to the class P, which consists of problems solvable by deterministic polynomial-time algorithms. By showing that certain
functions require large circuits, it reinforces the distinction between polynomial-time computation and circuit complexity.

NP and Circuit Hardness: The theorem can also be applied to problems in the class NP, showing that certain problems in NP cannot be
efficiently solved by small circuits, reinforcing the separation between classes like P and NP.

6. Conclusion

In this lecture, we wrapped up our discussion on circuit complexity and introduced Kannan’s Theorem, which provides a key result in the study
of circuit lower bounds. Kannan’s Theorem illustrates that there are Boolean functions that require exponentially large circuits to compute,
establishing inherent limits to the power of certain circuit families.

Kannan's Theorem strengthens our understanding of the complexity of specific problems and helps us prove hardness results for
computational problems.

The theorem’s proof method, involving contradiction and lower bounds, is a powerful tool for understanding the limitations of circuit-based
computation and provides insights into the relationship between different complexity classes in computational complexity theory.

Lecture 18: Probabilistic Complexity

1. Introduction to Probabilistic Computations

Probabilistic computations represent a broad class of algorithms and models of computation where randomness plays a central role in the
decision-making process. Unlike deterministic computations, where each computation step is fixed, probabilistic computations involve

32/77
decisions based on random choices, introducing an element of uncertainty into the algorithm's behavior.

This lecture introduces probabilistic Turing machines (PTMs), defines key probabilistic complexity classes, and explores the relationships
between these classes and other well-known classes such as P, NP, and PSPACE.

2. Probabilistic Turing Machines (PTMs)

A probabilistic Turing machine (PTM) is a variant of the standard Turing machine (TM), where at certain points in its computation, the machine
can make a probabilistic choice between different transitions (e.g., moving to different states or writing different symbols). These transitions are
determined by a probability distribution rather than a fixed set of rules. The machine's configuration is thus influenced by randomness at
specific stages of the computation.

Formal Definition: A probabilistic Turing machine can be described by:

A finite set of states, including the start state and halting states.

A tape alphabet, which includes the symbols that can be written on the tape.

A set of transition rules that determine both the next state and the symbol to be written on the tape, with probabilistic choices at some
points.

A probability distribution over these choices (i.e., the randomness that governs the transitions).

Acceptance Criterion: For a PTM, the acceptance of an input is determined by the probability that the machine reaches an accepting state.
The machine can either:

Accept the input if the probability of accepting is greater than a certain threshold (usually 1/2 or 2/3).

Reject the input if the machine reaches a rejecting state with high probability.

3. Difference Between NDTMs and PTMs

Both non-deterministic Turing machines (NDTMs) and probabilistic Turing machines (PTMs) allow for branching in the computation process,
but they differ in how they make choices:

NDTMs (Non-Deterministic Turing Machines): In NDTMs, the machine at each step can choose between multiple possible transitions, but
these choices are not probabilistic. The machine "branches" into several computational paths, and it is said to accept the input if at least one
computation path leads to an accepting state.

PTMs (Probabilistic Turing Machines): In PTMs, instead of deterministically branching into different paths, the machine makes probabilistic
choices. At each branching step, there is a probability distribution over which path to take. A PTM is said to accept the input if, with high
probability (greater than 1/2 or a similar threshold), it reaches an accepting state.

4. Classes of Probabilistic Complexity

Several important complexity classes are defined based on probabilistic Turing machines, particularly regarding the amount of time the machine
takes to complete its computation.

BPTIME(T(n)): The class BPTIME(T(n)) consists of decision problems that can be solved by a probabilistic Turing machine in time T (n),
where n is the size of the input. Specifically, this class contains problems for which there exists a PTM that, with high probability (greater
than 1/2), will correctly accept or reject an input in O(T (n)) time.

The time complexity of the machine is bounded by a function T (n), where T (n) is typically a polynomial or other function of the input
size.

PTMs can make probabilistic choices during the computation, and the machine’s decision is based on the majority of the computational
paths.

BPP (Bounded-Error Probabilistic Polynomial Time): The class BPP consists of problems that can be solved by a probabilistic Turing
machine in polynomial time with a bounded error. A problem is in BPP if there exists a PTM that, on any input of size n,:

33/77
 Runs in polynomial time O(nk ) for some constant k .

Accepts a yes instance with probability at least 2/3 and rejects a no instance with probability at least 2/3.

This definition means that the machine has a high probability of making the correct decision, with the error probability being limited to
at most 1/3.

5. Relationship of BPP with Other Complexity Classes

The class BPP is related to several other well-known complexity classes, especially in the context of deterministic and non-deterministic
computation.

BPP vs. P:

P is the class of problems that can be solved by a deterministic Turing machine in polynomial time. Since PTMs are a generalization of
deterministic TMs (i.e., they allow probabilistic choices), we have:

P ⊆ BPP

This implies that all problems solvable in polynomial time by a deterministic machine can also be solved by a PTM in polynomial time.
However, it is not known whether BPP = P, and this remains one of the central questions in complexity theory.

BPP vs. NP:

NP is the class of problems that can be solved by a non-deterministic Turing machine in polynomial time. Since NDTMs and PTMs both
allow for non-deterministic choices, it is generally believed that:

NP ⊆ BPP

This implies that if a PTM can solve a problem in polynomial time, it is possible to check a certificate for that problem with high
probability, akin to how NP problems are handled using certificates.

BPP vs. PSPACE:

PSPACE is the class of problems that can be solved by a deterministic Turing machine using polynomial space. Since BPP focuses on
time complexity (polynomial time) while PSPACE focuses on space complexity, there is no known direct inclusion between these classes.
However, it is generally believed that:

BPP ⊆ PSP ACE

This is because polynomial-time probabilistic machines are often able to simulate polynomial-space machines in many cases, though
the exact relationship is still an open problem.

6. Conclusion

In this lecture, we introduced the concept of probabilistic computations and the corresponding model of probabilistic Turing machines
(PTMs). We defined the complexity classes BPTIME(T(n)) and BPP, and explored their relationships with other complexity classes such as P, NP,
and PSPACE.

BPTIME(T(n)) describes problems solvable by probabilistic TMs within a given time bound.

BPP represents the class of problems solvable in polynomial time with bounded error probability.

We also discussed the relationships between BPP, P, NP, and PSPACE, emphasizing that while P is a subset of BPP, the equality of these
classes is still an open question.

Understanding probabilistic complexity and the role of randomness in computation is crucial for modern computational theory, especially in
fields like cryptography, randomized algorithms, and approximation algorithms.

Lecture 19: StrongBPP and WeakBPP

1. Introduction

34/77
In this lecture, we introduce the StrongBPP and WeakBPP classes, which are variations of the BPP (Bounded-Error Probabilistic Polynomial
Time) class. These classes are used to study the power of probabilistic computation and the impact of different types of error bounds and
randomness on the complexity of decision problems. We will define these classes formally, describe their computational characteristics, and
explore the differences between them.

2. BPP Recap

Before defining StrongBPP and WeakBPP, let's briefly recall the class BPP:

BPP (Bounded-Error Probabilistic Polynomial Time): The class of problems that can be solved by a probabilistic Turing machine in
polynomial time with a bounded error probability.

The PTM must accept "yes" instances with probability at least 2/3 and "no" instances with probability at least 2/3. The error probability
for both cases is at most 1/3.

This class captures decision problems where randomness is used to reduce the computational complexity while ensuring a high
probability of correctness.

3. WeakBPP

WeakBPP is a subclass of BPP that allows for a weaker error bound compared to BPP. Specifically, WeakBPP permits a higher error probability
for both acceptance and rejection than BPP.

Definition of WeakBPP: A decision problem is in WeakBPP if there exists a probabilistic Turing machine that solves the problem in
polynomial time, with an error probability that can be arbitrarily close to 1/2, but still less than 1/2.

In other words, a PTM can accept a "yes" instance with probability at least 1/2 + ϵ (for some small constant ϵ) and reject a "no" instance
with probability at least 1/2 + ϵ, where ϵ can be arbitrarily small but not equal to 0.

The key distinction from BPP is that WeakBPP allows for the error probability to be closer to 1/2 (as opposed to the strict 1/3 bound in
BPP).

Properties of WeakBPP:

The error probability for both "yes" and "no" instances is still bounded, but the bound is weaker than in BPP.

Problems in WeakBPP are still solvable in polynomial time by a PTM, but the guarantee of correctness is less stringent than in BPP.

Example: A problem where the PTM gives correct results with probability 1/2 + ϵ, where ϵ is a small constant, can be classified as
WeakBPP.

4. StrongBPP

StrongBPP is another subclass of BPP, but in this case, the error probability is much lower than that in WeakBPP. The machine in StrongBPP
must ensure a much stronger correctness guarantee.

Definition of StrongBPP: A decision problem is in StrongBPP if there exists a probabilistic Turing machine that solves the problem in
polynomial time, with an error probability that is exponentially small (e.g., 2−n , where n is the input size).

The PTM must accept a "yes" instance with probability 1 − δ for some small constant δ , and reject a "no" instance with probability 1 − δ
, where δ decays exponentially as the input size grows.

The error probability is therefore significantly smaller than in BPP, ensuring a stronger guarantee of correctness.

Properties of StrongBPP:

StrongBPP guarantees an exponentially small error probability, which provides a much more reliable solution compared to BPP or
WeakBPP.

Like BPP, problems in StrongBPP are solvable in polynomial time, but the reliability of the solution is significantly higher.

35/77
 Example: A problem where the PTM gives correct results with probability 1 − 2−n (i.e., exponentially small error probability) would be in
StrongBPP.

5. Relationships Between BPP, StrongBPP, and WeakBPP

BPP lies between WeakBPP and StrongBPP in terms of error probability.

WeakBPP ⊆ BPP ⊆ StrongBPP

WeakBPP allows for larger error probabilities compared to BPP, while StrongBPP ensures exponentially smaller error probabilities.

Error Bound Comparisons:

In BPP, the error probability is bounded by 1/3.

In WeakBPP, the error probability is bounded but can be arbitrarily close to 1/2.

In StrongBPP, the error probability is exponentially small, making it the most reliable among the three.

6. Conclusion

In this lecture, we introduced the complexity classes StrongBPP and WeakBPP, which are variants of the BPP class of problems.

WeakBPP allows for a weaker error probability, where the machine's decision can be wrong with a probability close to 1/2.

StrongBPP provides a stronger guarantee of correctness, with exponentially small error probabilities, leading to higher reliability compared
to BPP and WeakBPP.

Understanding the differences between these classes is important in the study of probabilistic computation and randomized algorithms, as
these models provide insights into how randomness can be used to improve computation while managing error probability.

Lecture 20: One-Sided and Zero-Sided Error Probabilistic Complexity Classes

1. Introduction

In this lecture, we introduce the concept of one-sided and zero-sided error probabilistic Turing machines (PTMs) and explore the complexity
classes associated with them. These classes are extensions of the classical probabilistic classes such as BPP (bounded-error probabilistic
polynomial time) and provide stricter error bounds.

We will define these new classes, investigate their properties, and show their interrelationships. Specifically, we will focus on the classes RP,
coRP, ZPP, and demonstrate that ZPP = RP ∩ coRP.

2. PTMs with One-Sided Error

A probabilistic Turing machine with one-sided error operates similarly to a regular BPP machine, except that it only allows error on one type of
input (either "yes" or "no" instances), while the machine must always give the correct answer for the other type.

Definition of One-Sided Error PTM: A probabilistic Turing machine has one-sided error if:

For a "yes" instance, the machine always accepts with probability 1.

For a "no" instance, the machine rejects with probability at least 1/2, but may sometimes incorrectly accept (with probability less than
1/2).

The class of decision problems solvable by such machines in polynomial time is denoted as RP (Randomized Polynomial time with one-sided
error).

Class RP (Randomized Polynomial time with one-sided error):

36/77
 A problem L is in RP if there exists a probabilistic Turing machine that, for an input of size n:

Accepts a "yes" instance with probability 1 (i.e., always correctly accepts).

Rejects a "no" instance with probability at least 1/2, and if it accepts, it does so with probability strictly less than 1/2 (i.e., one-sided
error).

The running time of the machine is bounded by a polynomial function of n.

Properties of RP:

RP allows for errors only on "no" instances (i.e., false positives), but guarantees that "yes" instances are always accepted correctly.

The error probability is bounded, and the machine runs in polynomial time.

RP is typically used to model problems where we are more concerned with avoiding false positives rather than ensuring absolute
correctness for all instances.

3. PTMs with Zero-Sided Error

A zero-sided error probabilistic Turing machine is a machine where the error probability for both "yes" and "no" instances is zero, meaning that
the machine gives the correct answer with certainty for both types of input.

Definition of Zero-Sided Error PTM: A probabilistic Turing machine has zero-sided error if:

For a "yes" instance, the machine always accepts.

For a "no" instance, the machine always rejects.

However, the machine may have a small chance of running indefinitely (i.e., does not halt) for certain inputs.

The class of decision problems solvable by such machines in polynomial time is denoted as ZPP (Zero-sided error Probabilistic Polynomial time).

Class ZPP (Zero-sided error Probabilistic Polynomial time):

A problem L is in ZPP if there exists a probabilistic Turing machine that:

Accepts a "yes" instance with probability 1.

Rejects a "no" instance with probability 1.

Always halts on every input, with no chance of error.

The machine runs in polynomial time.

Properties of ZPP:

ZPP guarantees zero-sided error, meaning the machine makes no mistakes for both "yes" and "no" instances.

It is also zero-error in the sense that there are no false positives or false negatives.

ZPP can be thought of as the class of problems where randomness is used to ensure efficient computation without error.

4. Relationships Between RP, coRP, and ZPP

We now explore the relationships between RP, coRP, and ZPP.

Class coRP:

The class coRP is defined similarly to RP, but with the roles of "yes" and "no" instances reversed.

A decision problem L is in coRP if there exists a probabilistic Turing machine that:

For a "no" instance, the machine always rejects.

For a "yes" instance, the machine rejects with probability at least 1/2, but may sometimes incorrectly accept.

Thus, coRP allows for one-sided error on "yes" instances, but guarantees correct rejection for "no" instances.

The Relationship ZPP = RP ∩ coRP :

37/77
 It can be shown that:

ZPP = RP ∩ coRP
RP and coRP both allow for one-sided error, but in opposite directions (one-sided error on "no" instances for RP, and one-sided error on
"yes" instances for coRP). The intersection of these classes, ZPP, guarantees zero-sided error for both "yes" and "no" instances.

Proof Outline:

If a problem is in RP, there exists a probabilistic Turing machine with one-sided error for "no" instances (false positives), but always
accepts "yes" instances correctly.

Similarly, for coRP, the machine rejects "yes" instances with probability at least 1/2 but accepts "no" instances correctly.

Combining these two machines, one can construct a machine that always halts and correctly decides the problem with zero-sided
error (no incorrect answers for either "yes" or "no" instances). This machine will run in polynomial time, and thus the problem is in
ZPP.

5. Conclusion

In this lecture, we defined and explored the complexity classes RP, coRP, and ZPP, focusing on the differences in the allowed error probabilities
for probabilistic Turing machines.

RP allows one-sided error for "no" instances, with the machine always correctly accepting "yes" instances.

coRP allows one-sided error for "yes" instances, with the machine always correctly rejecting "no" instances.

ZPP, the intersection of RP and coRP, guarantees zero-sided error, meaning the machine always gives the correct answer for both "yes"
and "no" instances with no possibility of error.

The class ZPP is of particular interest because it combines the strengths of RP and coRP, providing a framework for problems that are solved
with certainty but still allow the use of randomness for efficiency.

Lecture 21: Error Reduction for BPP

1. Introduction

In this lecture, we will explore techniques for error reduction in BPP (Bounded-error Probabilistic Polynomial Time) machines. These methods
are fundamental in probabilistic computation, as they allow us to reduce the error probability of a probabilistic algorithm while maintaining the
efficiency of the machine.

We will also discuss the relationship between BPP and the class P/poly, and show that BPP ⊆ P/poly. This result will help us understand the
limitations and power of probabilistic computation in the context of nonuniform complexity.

2. BPP and Error Reduction

The key idea behind error reduction for BPP is to reduce the error probability of a probabilistic algorithm by repeatedly running the algorithm
multiple times. This is a standard technique in randomized algorithms and is based on the fact that repeated trials of a probabilistic machine can
lead to a lower error probability.

Definition of BPP:

A problem is in BPP if there exists a probabilistic Turing machine that solves the problem in polynomial time such that:

The machine accepts a "yes" instance with probability at least 2/3.

The machine rejects a "no" instance with probability at least 2/3.

The error probability for both "yes" and "no" instances is at most 1/3.

Error Reduction:

38/77
 Suppose we have a BPP algorithm with error probability ϵ (i.e., it makes an error with probability at most ϵ). If we repeat this algorithm
independently k times, the combined error probability can be reduced.

Independent Trials: If the algorithm is run k times, each with error probability ϵ, then the error probability of the majority decision (i.e.,
deciding based on the majority of the k outcomes) is:

Error Probability = ϵk
By choosing k large enough, we can make the error probability arbitrarily small. Specifically, if we choose k such that the error
probability ϵk is less than any desired value δ , then the error probability of the resulting algorithm is reduced to δ .

Reducing Error to an Arbitrary Small Value: Repeating the algorithm k = O(log(1/δ)) times ensures that the error probability is
reduced to δ . The total running time of the algorithm will still be polynomial because:

The number of repetitions is logarithmic, and each repetition takes polynomial time.

Hence, the overall running time remains polynomial.

Conclusion: This shows that error reduction in BPP can significantly reduce the error probability while preserving the polynomial time
complexity of the algorithm. By running the probabilistic algorithm multiple times, we can achieve any desired small error probability.

3. BPP ⊆ P/poly

Now we turn to the result that BPP is contained within the class P/poly, the class of problems that can be solved by polynomial-size
nonuniform circuits.

Class P/poly:

P/poly is the class of decision problems that can be solved by polynomial-size circuit families. These circuit families are nonuniform
because each circuit can be constructed independently for each input size n, without requiring any general algorithmic pattern.

A language L is in P/poly if there exists a sequence of Boolean circuits Cn of size at most p(n) (for some polynomial p) such that:
​

For every input x of length n, Cn (x) correctly decides whether x is in L.

​

Theorem: BPP ⊆ P/poly

We now show that BPP is contained within P/poly.

Suppose we have a BPP algorithm that solves a problem with an error probability of at most 1/3 and runs in polynomial time. As
mentioned earlier, by repeating the BPP algorithm k = O(log(1/δ)) times, we can reduce the error probability to δ for any desired
value of δ .

Circuit Representation of BPP: For each repetition, the probabilistic machine performs a set of operations based on a random string of
bits. The randomness can be encoded in a circuit family, where each circuit in the family represents the decision of the machine based
on the random choices for that particular repetition.

The circuits for each repetition are of polynomial size, and the total size of the circuit family is bounded by a polynomial in n and k .
Hence, the resulting circuit family has polynomial size and can be seen as a member of P/poly.

Conclusion: Since BPP algorithms can be represented by polynomial-size circuit families that decide the problem with high probability,
we conclude that BPP ⊆ P/poly.

4. Implications of BPP ⊆ P/poly

The inclusion BPP ⊆ P/poly has significant implications:

Circuit Complexity: The fact that BPP is contained in P/poly suggests that probabilistic algorithms can be efficiently implemented using
nonuniform circuit families. This result connects probabilistic complexity with circuit complexity.

Nonuniform Complexity: The result also implies that problems that can be solved by BPP can be solved by circuits of polynomial size, which
is a nonuniform model of computation. This contrasts with uniform models like P, where the algorithm must be the same for all inputs.

Potential Insights into P vs NP: While BPP ⊆ P/poly holds, it is important to note that this result does not imply that BPP = P or even that
BPP ⊆ P. However, it suggests that there is a strong connection between probabilistic polynomial time and polynomial-size nonuniform

39/77
 circuits.

5. Conclusion

In this lecture, we explored the concept of error reduction for BPP algorithms and showed that by repeating a BPP algorithm a polynomial
number of times, we can reduce the error probability to arbitrarily small values while maintaining polynomial time complexity.

We also showed that BPP ⊆ P/poly, meaning that BPP problems can be solved by polynomial-size nonuniform circuits. This result connects
probabilistic complexity with circuit complexity and offers important insights into the nature of probabilistic computation and its relation to
nonuniform models.

Through error reduction and the relationship with P/poly, we gained a deeper understanding of the computational power and limitations of
probabilistic polynomial-time algorithms.

Lecture 22: BPP in PH and Logspace Randomized Classes

1. Introduction

In this lecture, we will examine the placement of BPP within the Polynomial Hierarchy (PH) and introduce two logspace randomized classes:
BPL (bounded-error logarithmic space) and RL (randomized logarithmic space). We will also explore the undirected reachability problem within
RL, which provides a concrete example of a problem solvable by logspace randomized algorithms.

2. BPP in the Polynomial Hierarchy (PH)

Polynomial Hierarchy (PH):

The Polynomial Hierarchy is a hierarchy of complexity classes that generalize NP and coNP. It is structured as a hierarchy of alternating
quantifiers, where the quantifiers range over existential and universal statements, with alternating complexity at each level.

The levels of PH are denoted as ΣP

k and Πk , where:
P
​ ​

ΣPk involves k alternating existential and universal quantifiers, beginning with an existential quantifier.
​

ΠPk involves k alternating existential and universal quantifiers, beginning with a universal quantifier.
​

The first two levels of PH are the classes NP and coNP:

ΣP1 = NP
​

ΠP1 ​ = coNP
BPP in the Polynomial Hierarchy:

It is known that BPP is contained in the second level of the Polynomial Hierarchy. Specifically, we have:

BPP ⊆ ΣP2 ∪ ΠP2

​ ​

This result follows from the fact that BPP can be seen as a problem that involves probabilistic choices (existential quantifiers) and
deterministic computation (universal quantifiers). By mapping these probabilistic choices and deterministic checks to alternating
quantifiers, BPP fits naturally into the second level of PH.

Formal Argument:

A BPP machine operates by choosing a random string and then using a deterministic algorithm to verify the solution. This process
can be framed as an alternating quantifier sequence, where the existential quantifier corresponds to the choice of random bits, and
the universal quantifier corresponds to the verification of the solution.

Therefore, the class BPP is contained within ΣP

2 ​ ∪ ΠP2 , the second level of the polynomial hierarchy. This result gives us a deeper
​

understanding of where BPP fits in relation to other complexity classes, particularly in the context of higher-order complexity.

40/77
3. Logspace Randomized Classes

We now turn our attention to the logspace randomized classes: BPL and RL.

BPL (Bounded-error Logspace):

BPL is the class of decision problems solvable by a randomized logarithmic space machine with bounded error. The machine operates
in logarithmic space but has access to randomness.

A problem is in BPL if there exists a probabilistic Turing machine that uses logarithmic space and solves the problem with bounded
error (error probability less than 1/3).

The space complexity constraint ensures that the algorithm works efficiently even in the presence of randomness.

RL (Randomized Logspace):

RL is the class of decision problems solvable by a randomized logarithmic space machine with one-sided error. The machine uses
logarithmic space, and the error is only on the reject side (it may never incorrectly accept a "no" instance).

Specifically, a problem is in RL if there exists a probabilistic Turing machine that operates in logarithmic space and accepts a "yes"
instance with high probability, while rejecting a "no" instance with certainty.

Undirected Reachability in RL:

One of the key problems solvable in RL is undirected reachability. The problem asks whether there exists a path between two vertices
in an undirected graph.

Undirected Reachability is solvable in RL because it can be reduced to a process of randomly selecting paths in the graph and
verifying whether a path exists between the given vertices using logarithmic space. The error is one-sided: if there is no path, the
machine will correctly reject, and if there is a path, the machine will accept with high probability.

Algorithm:

The RL algorithm for undirected reachability works by maintaining a set of visited vertices in logarithmic space and performing
random walks on the graph.

At each step of the random walk, the machine uses its randomness to choose a vertex to explore, marking it as visited.

If the algorithm reaches the target vertex, it accepts. Otherwise, it continues the walk with a probabilistic approach to ensure that, if
a path exists, the algorithm will likely find it.

Key Property:

Since RL machines use logarithmic space, they cannot store all the vertices they visit in memory. Instead, they employ randomized
techniques such as random walks or Markov chains to probabilistically explore the graph while managing space efficiently.

4. Relationships Between BPP, RL, and Other Complexity Classes

BPP vs RL:

We can observe that BPP is more powerful than RL because BPP allows for both one-sided and two-sided error, while RL has only one-
sided error.

Additionally, BPP may use polynomial space, while both BPL and RL are restricted to logarithmic space.

BPP vs P:

BPP is believed to be strictly more powerful than P, as the randomness in BPP allows it to solve certain problems that cannot be solved
deterministically in polynomial time. However, the exact relationship remains unknown.

BPP in the Context of RL:

Since RL is contained within BPP (i.e., RL ⊆ BPP), it is clear that BPP can solve any problem that is solvable in RL. However, RL
operates in logarithmic space, which provides a more space-efficient approach to solving problems like undirected reachability.

Logspace Randomized Classes:

The classes BPL and RL provide important insights into the space complexity of randomized algorithms. Despite using only logarithmic
space, these classes are able to solve nontrivial problems like reachability in undirected graphs, demonstrating the power of
randomness in space-constrained environments.

41/77
5. Conclusion

In this lecture, we explored the placement of BPP within the Polynomial Hierarchy and demonstrated that BPP ⊆ \Sigma^P_2 ∪ \Pi^P_2,
indicating its relationship with higher levels of the polynomial hierarchy. We also introduced the logspace randomized classes BPL and RL,
highlighting their ability to solve problems with logarithmic space while using probabilistic techniques. In particular, we examined the
undirected reachability problem within RL, showing how randomness can be used to efficiently explore graphs in space-constrained settings.

These results offer deep insights into the relationship between randomized and deterministic complexity classes and highlight the efficiency of
randomized algorithms in restricted computational settings.

Lecture 23: Valiant-Vazirani Theorem - I

1. Introduction

In this lecture, we discuss the Valiant-Vazirani Theorem and its significance in probabilistic complexity theory, particularly in relation to RP
(Randomized Polynomial Time) machines. The lecture will cover the success amplification technique for RP machines, and we will introduce the
Valiant-Vazirani Theorem, which provides a mechanism for converting an NP problem into a one-sided error problem that can be solved in RP.

2. Success Amplification of RP Machines

Randomized Polynomial Time (RP):

An RP machine is a probabilistic Turing machine that runs in polynomial time and has one-sided error. Specifically:

If the answer is yes (the instance belongs to the language), the machine accepts with probability at least 12 .
​

If the answer is no (the instance does not belong to the language), the machine always rejects (with probability 1).

Error Probability:

The machine may fail to accept a "yes" instance with probability less than 12 , and this is what makes the machine non-deterministic.
​

However, the error is one-sided: there is no chance of mistakenly accepting a "no" instance.

Success Amplification:

Success amplification is a technique used to reduce the error probability of a randomized algorithm.

The basic idea is to repeat the randomized algorithm multiple times and take a majority vote or a consensus decision on the outcome.
By repeating the process, we can amplify the probability of success.

If an RP machine has an error probability of 12 , running the machine k times and taking a majority vote will reduce the error probability
​

to 21k . Thus, as
​ k → ∞, the error probability becomes arbitrarily small.
Formally:

Given an RP machine with error probability 12 , we can run it k times independently. The error probability of the resulting algorithm
​

is:

k
1
P (error) = ( )
2
​

As k increases, this error probability decreases exponentially.

Importance of Success Amplification:

Success amplification makes RP machines more reliable by reducing their error probability. This technique plays a crucial role in
algorithms for problems that are not necessarily in P but can be solved by randomized methods with arbitrarily high success
probability.

42/77
3. The Valiant-Vazirani Theorem

The Valiant-Vazirani Theorem is a foundational result in complexity theory that helps transform problems in NP into problems that can be
solved in RP. It essentially shows that NP problems with a certain structure can be reduced to problems that can be solved in RP by applying
randomness.

Statement of the Theorem:

The Valiant-Vazirani Theorem asserts that if there is a NP-complete problem, such as SAT, then there is a reduction from this problem
to a specific problem in RP. More precisely:

For every NP-complete problem, there exists a polynomial-time reduction to a one-sided error problem that can be solved in RP.
Proof Outline:

The theorem provides a method for transforming an arbitrary NP problem into a form that can be solved by a randomized machine
with one-sided error.

The process involves constructing a certificate for an NP problem and then using this certificate to create a randomized solution. The
machine will be able to verify the certificate with high probability, while ensuring that it never mistakenly accepts an invalid certificate
(hence the one-sided error).

Implication:

This result is important because it implies that NP problems can be transformed into problems that can be solved in RP, provided that
the solution involves probabilistic checks. This transformation makes certain problems solvable with randomized algorithms,
significantly improving the computational efficiency of certain NP problems.

4. Valiant-Vazirani Theorem: The Basic Idea

To understand the theorem more clearly, let's break down the core idea:

From NP to RP:

Consider an NP problem such as SAT, which involves deciding whether a given Boolean formula is satisfiable.

The theorem asserts that any NP-complete problem can be transformed into a problem where the answer can be verified with high
probability by a randomized algorithm.

The Approach:

Given an NP-complete problem, the idea is to represent the problem in such a way that a probabilistic machine can guess a solution
and verify it in polynomial time.

The machine can perform a series of random trials (e.g., guessing a solution) and verify each guess using a deterministic procedure. If
the guess is correct, the machine will accept; otherwise, it will reject. Since the machine can check the guess efficiently, the process will
ensure the answer is correct with high probability.

5. Applications and Implications of the Valiant-Vazirani Theorem

Reduction to One-Sided Error:

The Valiant-Vazirani Theorem highlights that NP problems, which are traditionally hard to solve deterministically, can be approached
using randomized methods with guaranteed correctness in one direction. Specifically, it shows that for many NP problems, there is a
way to use randomness to solve them in polynomial time with one-sided error.

Implication for NP-Complete Problems:

The theorem gives a powerful insight into the relationship between NP and RP. It tells us that many problems in NP can be solved by
probabilistic algorithms if we allow for one-sided error. This does not contradict the fact that NP-complete problems are hard; instead,
it shows that randomized methods can efficiently verify solutions to NP-complete problems.

Complexity Implications:

43/77
 The Valiant-Vazirani Theorem also bridges a gap between NP and RP, making it clear that probabilistic approaches can sometimes
bypass the inherent hardness of NP-complete problems. It suggests that for many NP-complete problems, randomness can be used to
check solutions efficiently, though the error probability can be reduced to arbitrarily small levels using techniques like success
amplification.

6. Conclusion

In this lecture, we introduced the concept of success amplification for RP machines and discussed how it can be used to reduce error
probabilities for randomized algorithms. We then explored the Valiant-Vazirani Theorem, which shows how problems in NP can be reduced to
problems solvable in RP with one-sided error. This theorem plays a crucial role in our understanding of the relationship between NP and RP,
highlighting how randomized algorithms can solve NP-complete problems efficiently using probabilistic techniques.

Lecture 24: Valiant-Vazirani Theorem - II

1. Introduction

In this lecture, we continue our discussion on the Valiant-Vazirani Theorem and expand its scope to other problems and complexity classes. We
specifically focus on the application of the theorem to USAT (Unambiguous SAT), introduce the classes #P and ⊕P, and explore the #SAT
problem, which is complete for the class #P. We also discuss the closure properties of the ⊕ quantifier under Boolean operations such as AND,
OR, and NOT.

2. Valiant-Vazirani Theorem for USAT

USAT (Unambiguous SAT):

USAT refers to the unambiguous satisfiability problem. Unlike the traditional SAT problem, where a Boolean formula can have multiple
satisfying assignments, USAT asks whether a Boolean formula has exactly one satisfying assignment. If there is exactly one satisfying
assignment, the formula is said to be unambiguously satisfiable.

Application of the Valiant-Vazirani Theorem:

The Valiant-Vazirani Theorem, as applied to USAT, asserts that we can transform USAT into a problem solvable by a probabilistic
algorithm in RP. The crux of this is that the machine can guess an assignment and verify whether it is the unique satisfying assignment
with high probability, ensuring correctness with one-sided error.

Relation to RP:

While the standard SAT problem is NP-complete, USAT provides a way to reduce this problem to one where a randomized algorithm
can check the uniqueness of the solution efficiently. This result shows that USAT can be viewed as a problem in RP with one-sided error,
similar to how the Valiant-Vazirani theorem applies to SAT.

3. The Class #P

Definition of #P:

The class #P consists of counting problems associated with NP problems. Formally, a problem is in #P if it can be reduced to counting
the number of accepting paths of a nondeterministic polynomial-time Turing machine. These problems involve computing the number
of accepting computations for a given input.

#P Completeness:

The most prominent complete problem for #P is #SAT, which involves counting the number of satisfying assignments for a given
Boolean formula. If a formula is satisfiable, #SAT computes how many distinct satisfying assignments exist.

Example of #P Problems:

44/77
 #SAT: Given a Boolean formula, count the number of satisfying assignments.

#MATCHING: Count the number of perfect matchings in a bipartite graph.

#PATH: Count the number of paths in a graph between two nodes that satisfy certain constraints.

Computational Complexity of #P:

Problems in #P are generally considered harder than NP because they involve counting solutions, not just verifying the existence of
one. While NP is about decision problems, #P involves counting the total number of solutions, which typically requires more
computational resources.

Relation to #SAT:

#SAT is the canonical #P-complete problem, meaning it is both in #P and any other problem in #P can be reduced to #SAT in
polynomial time. This establishes the fundamental role of #SAT in the class.

4. The Class ⊕P

Definition of ⊕P:

The class ⊕P (also known as the Parity class) consists of problems that involve counting the number of accepting paths of a
nondeterministic polynomial-time Turing machine mod 2. Specifically, for a given input, the problem asks whether the number of
accepting computations is even or odd. The machine can compute this parity efficiently in polynomial time.

Example of ⊕P Problems:

⊕SAT: The problem asks whether the number of satisfying assignments for a Boolean formula is even or odd.

⊕MATCHING: Determines whether the number of perfect matchings in a graph is even or odd.

⊕PATH: Determines whether the number of paths between two nodes in a graph is even or odd.

Computational Complexity of ⊕P:

The class ⊕P is similar to #P, but it focuses on determining the parity of the count of solutions, rather than the exact count. Problems in
⊕P are often harder than problems in NP because counting is more complex than simple decision-making.

5. #SAT is Complete for #P

#SAT as #P-Complete:

#SAT, the problem of counting the number of satisfying assignments of a Boolean formula, is the canonical #P-complete problem. This
means:

#SAT is in #P.

Every problem in #P can be polynomial-time reduced to #SAT.

The #P-completeness of #SAT shows that counting the number of satisfying assignments for Boolean formulas is one of the hardest
problems in #P. Solving #SAT is computationally difficult because it involves counting, which is generally harder than deciding whether
a single solution exists (as in NP).

6. Closure of the ⊕ Quantifier Under Boolean Operations

Closure Under AND, OR, and NOT:

⊕P is closed under certain Boolean operations. This means that the parity of the number of accepting paths can be manipulated using
standard Boolean operations:

AND Operation: If A and B are in ⊕P, then A ∧ B is also in ⊕P. The parity of the intersection of solutions in ⊕P problems can be
computed efficiently by combining the parities of the individual solutions.

45/77
 OR Operation: If A and B are in ⊕P, then A ∨ B is also in ⊕P. The parity of the union of solutions in ⊕P problems can be computed in
the same way.

NOT Operation: If A is in ⊕P, then ¬A (the negation of A) is also in ⊕P. The parity of the number of accepting paths for the
complement of a problem can also be efficiently computed.

Implications:

The closure of ⊕P under these operations means that we can combine problems in ⊕P using these basic Boolean operations without
leaving the class. This property makes ⊕P a robust class in complexity theory, as we can build more complex problems from simpler
ones while preserving the complexity.

7. Conclusion

In this lecture, we extended the discussion of the Valiant-Vazirani Theorem to the USAT problem, and introduced the classes #P and ⊕P, which
deal with counting and parity problems associated with nondeterministic polynomial-time Turing machines. We discussed the #P-completeness
of #SAT and the closure properties of the ⊕ quantifier under Boolean operations. These concepts form the foundation for understanding the
complexity of counting problems and their relationship with decision problems in NP and RP.

Lecture 25: Amplified Version of Valiant-Vazirani Theorem

1. Introduction

In this lecture, we will discuss an amplified version of the Valiant-Vazirani Theorem as it applies to ⊕SAT, and introduce Toda's Theorem,
including its statement and proof outline. We will also explore the relationship between the quantifiers ∃, ∀, ⊕, and BP, which are used to
characterize different classes of problems in computational complexity.

2. Amplified Version of Valiant-Vazirani Theorem for ⊕SAT

⊕SAT:

⊕SAT is the problem of determining whether the number of satisfying assignments to a Boolean formula is even or odd. Unlike the
traditional SAT problem, which asks whether there exists at least one satisfying assignment, ⊕SAT focuses on the parity of the number
of satisfying assignments.

Amplified Valiant-Vazirani Theorem:

The amplified version of the Valiant-Vazirani theorem for ⊕SAT shows that this problem can be solved by a probabilistic algorithm in
the class BPP (bounded-error probabilistic polynomial time) under the following conditions:

A probabilistic machine is allowed to sample randomly and check whether the number of satisfying assignments is even or odd.

The machine can be amplified to reduce the error probability to be arbitrarily small.

Key Idea: The theorem essentially states that for problems like ⊕SAT, where the output involves counting modulo 2, we can amplify the
probability of correctness by performing multiple trials. After repeated trials, the error probability decreases, making the probabilistic
algorithm reliable.

Relation to BPP:

Since ⊕SAT involves counting modulo 2, it belongs to ⊕P, a complexity class that is harder than NP but can be solved efficiently with
probabilistic algorithms under the Valiant-Vazirani framework.

By applying the amplified version of the theorem, ⊕SAT can be classified as a problem solvable in BPP, thereby demonstrating a
connection between the class of problems that can be solved probabilistically and those that require counting solutions in ⊕P.

46/77
3. Toda's Theorem

Statement of Toda's Theorem:

Toda's Theorem is a deep result in computational complexity that establishes a surprising relationship between PH (the polynomial
hierarchy) and #P (the class of counting problems). The theorem states that:

PH = P#P

This means that the entire polynomial hierarchy can be simulated by a P-machine (deterministic polynomial-time machine) that has access
to an oracle for #P problems.

Explanation of the Result:

The polynomial hierarchy, PH, is a hierarchy of complexity classes that generalizes NP and coNP through multiple levels of alternating
existential and universal quantifiers. #P is the class of counting problems related to nondeterministic polynomial-time decision
problems.

Toda’s theorem shows that the entire polynomial hierarchy is contained within P with access to a #P oracle. In other words, any problem
in PH can be solved in polynomial time with access to a counting oracle.

This result has profound implications, as it shows that counting problems, like those in #P, are powerful enough to capture the
complexity of the entire polynomial hierarchy.

4. Proof Outline of Toda's Theorem

Oracle-Based Approach:

The proof of Toda’s theorem involves showing how to simulate the decision problems of the polynomial hierarchy using a #P oracle. By
using #P oracles, one can determine the number of accepting paths of nondeterministic Turing machines, which is crucial for
simulating the existential and universal quantifiers in the polynomial hierarchy.

Simulation of Polynomial Hierarchy:

PH problems are typically expressed as alternating quantifiers (existential and universal), and the goal is to replace these quantifiers
with #P-based computations.

A P machine can use the #P oracle to count solutions at each level of the hierarchy and decide whether to accept or reject based on the
results.

Reduction of PH to P with #P Oracle:

NP can be viewed as a special case of PH, where the hierarchy consists of a single existential quantifier. By repeatedly applying the
oracle-based approach at higher levels of the hierarchy, Toda’s theorem shows that every level of PH can be reduced to P with a #P
oracle.

Implications:

The theorem implies that the computational power of the polynomial hierarchy is equivalent to the computational power of P with
access to counting problems. This connects the P class with the more complex classes in the polynomial hierarchy, providing a unifying
result that has profound consequences for our understanding of the complexity of counting and decision problems.

5. The Quantifiers ∃, ∀, ⊕, and BP

Existential Quantifier (∃):

The existential quantifier, ∃, appears in the definition of NP problems, where we are interested in whether there exists at least one
satisfying assignment or accepting computation. Problems like SAT are NP-complete because they ask whether an existential solution
exists.

Universal Quantifier (∀):

The universal quantifier, ∀, appears in the definition of coNP problems, where we ask whether all computations lead to a "no" answer
(e.g., for the complement of SAT, the UNSAT problem). It is also used in higher levels of the polynomial hierarchy.

47/77
 ⊕ Quantifier:

The ⊕ quantifier corresponds to the class ⊕P, where the problem is concerned with counting the number of accepting paths modulo 2.
This is used in problems like ⊕SAT, where we are interested in whether the number of satisfying assignments is even or odd.

BP (Bounded-Error Probabilistic Polynomial Time):

BP is a class of problems that can be solved by probabilistic polynomial-time algorithms with a bounded error probability. This class
encompasses problems that can be solved by algorithms like those based on the Valiant-Vazirani theorem, which apply probabilistic
methods to efficiently determine the answer to counting problems such as ⊕SAT.

6. Conclusion

In this lecture, we discussed the amplified version of the Valiant-Vazirani Theorem for ⊕SAT, which shows that such counting problems can be
solved by probabilistic machines with high accuracy. We then introduced Toda's Theorem, which demonstrates the equivalence between the
polynomial hierarchy and P with access to #P oracles, revealing the deep connections between decision and counting problems. Finally, we
examined the relationship between various quantifiers (∃, ∀, ⊕) and the class BP, which are essential for understanding the structure and
power of different complexity classes.

2/2

Lecture 26: Toda's Theorem I

1. Introduction

In this lecture, we will begin the proof of Toda’s Theorem, which shows a deep connection between the polynomial hierarchy (PH) and
probabilistic counting classes. Specifically, we will prove the inclusion:

PH ⊆ BP#P ⋅ P

This result demonstrates that the entire polynomial hierarchy is contained in a class that is a combination of BP (bounded-error probabilistic
polynomial time), #P (counting problems), and P (deterministic polynomial time).

The proof of this theorem is critical because it highlights how counting problems can provide a powerful tool for simulating problems in the
polynomial hierarchy.

2. Recall: The Polynomial Hierarchy (PH)

The polynomial hierarchy (PH) is a hierarchy of complexity classes that generalizes NP and coNP. It consists of alternating existential and
universal quantifiers and is defined by the following levels:

ΣₙP (Existential levels): These are decision problems where the first part of the problem is existential (there exists a satisfying solution),
followed by a universal quantifier for the subsequent levels.

ΠₙP (Universal levels): These are decision problems where the first part of the problem is universal (for all solutions, something holds),
followed by an existential quantifier at the next level.

Each level of the hierarchy can be described by alternating quantifiers, and the total complexity of a decision problem in PH depends on the
number of alternations between existential and universal quantifiers.

3. Recall: The Classes BP, #P, and P

BP (Bounded-Error Probabilistic Polynomial Time): This is the class of decision problems solvable by probabilistic polynomial-time Turing
machines with bounded error probability. That is, a problem belongs to BP if it can be solved in polynomial time with the help of
randomization and has a probability of error that can be made arbitrarily small by running the algorithm multiple times.

48/77
 #P (Counting Problems): This is the class of counting problems associated with NP problems, where the goal is to count the number of
accepting paths of a nondeterministic machine rather than deciding whether there exists at least one accepting path. #P problems are
generally considered harder than NP problems.

P (Deterministic Polynomial Time): This is the class of decision problems that can be solved in polynomial time by a deterministic Turing
machine. It is the class of problems that are efficiently solvable.

4. The Statement of Toda’s Theorem (Part I)

Toda’s theorem states that:

PH ⊆ BP#P ⋅ P

This inclusion means that any problem in the polynomial hierarchy can be solved using a combination of:

BP: a probabilistic machine that operates in polynomial time,

#P: a counting oracle for problems that involve counting solutions (such as counting the number of satisfying assignments to a Boolean
formula),

P: deterministic polynomial-time computation.

This result is significant because it implies that the power of counting problems (in #P) can be used to simulate all problems in the polynomial
hierarchy. The inclusion shows that the complexities involved in PH problems are not beyond what can be handled by probabilistic machines
with access to a counting oracle.

5. Proof Outline

We will break down the proof into several steps:

Step 1: Simulating Existential Quantifiers Using BP and #P

At the first level of PH, we have an existential quantifier that represents a search for a solution (such as a satisfying assignment). We can
simulate this existential quantifier using a probabilistic machine in BP. The machine guesses a solution and verifies it probabilistically.

A probabilistic machine in BP can guess a solution (like a satisfying assignment) and then use a #P oracle to check the number of accepting
paths (solutions) for the given guess.

If the number of solutions is non-zero, the machine accepts; otherwise, it rejects.

Step 2: Simulating Universal Quantifiers Using BP and #P

At the second level, we have a universal quantifier, which requires us to verify that a condition holds for all possible solutions. This is more
challenging because the machine must check that a property holds for all guesses of the existential quantifier.

A probabilistic machine in BP can use a #P oracle to count the solutions at each level of the computation. If the number of solutions is non-
zero, the machine moves to the next stage.

The probabilistic machine can verify all solutions and reject if any solution fails to satisfy the condition.

Step 3: Simulating Alternating Quantifiers Using BP and #P

In higher levels of the PH, we alternate between existential and universal quantifiers. The key observation here is that we can use BP machines
with access to a #P oracle to simulate this alternation:

At each level, the probabilistic machine guesses a solution and then uses the #P oracle to verify whether all possible solutions meet the
required conditions.

This allows the machine to handle the alternation of existential and universal quantifiers.

Step 4: Generalizing the Approach

By repeatedly applying the probabilistic and counting oracle-based approach at each level of alternation, we can simulate any decision problem
in PH within the BP-based class.

49/77
Thus, by combining BP, #P, and P, we can simulate the entire polynomial hierarchy.

6. Conclusion

In this part of the proof of Toda's Theorem, we showed that the polynomial hierarchy is contained in the class BP raised to the power of #P and
combined with P. This demonstrates that counting problems (in #P) can be used to solve problems in the polynomial hierarchy, including
problems with alternating existential and universal quantifiers.

The next step in the proof will extend this result and explore the implications for the complexity of counting and decision problems in PH.

This concludes the first part of the proof of Toda’s Theorem. Further steps will delve into the implications of this result and how it helps in
understanding the relationship between counting and decision problems across different levels of the polynomial hierarchy.

Lecture 27: Toda's Theorem II

1. Introduction

In this lecture, we continue the proof of Toda’s Theorem, building upon the results established in the previous lecture. Specifically, we aim to
prove that:

BP#P ⋅ P ⊆ P #P

This inclusion shows that a probabilistic polynomial-time machine with access to a #P oracle (a counting oracle) is equivalent in power to a
deterministic polynomial-time machine with access to the same #P oracle. We will explore the connections between probabilistic and
deterministic machines in the context of counting problems and demonstrate how Toda’s theorem helps to unify these different computational
models.

2. Review: Toda’s Theorem and the Current Objective

To recap, Toda's Theorem (in its full form) asserts that the entire polynomial hierarchy (PH) can be simulated by a probabilistic polynomial-time
machine with access to a #P oracle. More formally:

PH ⊆ BP#P ⋅ P

In the previous lecture, we established one direction of this inclusion:

PH ⊆ BP#P ⋅ P

Now, we turn our attention to proving the reverse inclusion:

BP#P ⋅ P ⊆ P #P

This result demonstrates that the combination of probabilistic polynomial-time machines and #P oracles is no more powerful than a
deterministic polynomial-time machine with a #P oracle.

3. Background on BP and #P

Before diving into the proof, let's recall the definitions of the key classes involved:

BP (Bounded-Error Probabilistic Polynomial Time): This is the class of decision problems solvable by a probabilistic Turing machine in
polynomial time with bounded error probability. A machine in BP can make probabilistic decisions during computation and can be made to
err with arbitrarily small probability by running the machine multiple times.

50/77
 #P (Counting Problems): This is the class of problems where the goal is to count the number of accepting paths of a nondeterministic
polynomial-time machine. For example, given a nondeterministic polynomial-time decision problem, #P asks how many accepting
computations there are, instead of simply asking whether there exists one accepting computation.

P (Deterministic Polynomial Time): This is the class of problems solvable by a deterministic Turing machine in polynomial time. Problems in
P are considered to be "efficiently solvable" because they can be solved in time polynomial in the size of the input.

4. The Core Idea of the Proof

The goal is to show that BP with access to a #P oracle and a deterministic polynomial-time machine with access to a #P oracle are equivalent in
computational power. To establish this, we will simulate the probabilistic polynomial-time machine with a #P oracle using a deterministic
polynomial-time machine with a #P oracle.

Step 1: Simulating Probabilistic Machines Using Deterministic Machines

The key idea is to simulate the probabilistic computation in BP using a deterministic machine with access to a #P oracle. We use the following
observation:

A probabilistic machine in BP operates by randomly guessing values and performing computations based on those guesses.

Since #P can count the number of accepting paths of a nondeterministic machine, it can be used to verify whether a guess made by a
probabilistic machine is correct (or whether there is any solution at all).

To simulate a probabilistic machine, we can essentially "unroll" the probabilistic choices. Specifically, we use the #P oracle to count the number
of possible choices at each step, allowing us to simulate the probabilistic decision process deterministically.

Step 2: Simulating Randomness with a Deterministic Oracle Machine

The power of randomness in BP is used to guess solutions in an efficient manner. However, this guessing process can be deterministically
simulated by querying the #P oracle.

A deterministic machine can simulate the probabilistic machine by exhaustively considering all possible outcomes of the randomness used
by the probabilistic machine.

The #P oracle helps the deterministic machine to verify how many solutions there are to a particular guess, effectively simulating the
"randomness" of a probabilistic machine.

In this way, a deterministic machine can simulate the probabilistic guessing process, ensuring that it explores all possibilities and arrives at the
same result.

Step 3: Complexity Considerations

The critical observation here is that the deterministic machine, using a #P oracle, can simulate the probabilistic machine within polynomial time.
Since #P oracles operate in polynomial time, this process does not cause any exponential blowup in the time complexity of the machine.

Thus, we conclude that a deterministic polynomial-time machine with a #P oracle can simulate a probabilistic polynomial-time machine with a
#P oracle.

5. Conclusion of the Proof

Having demonstrated how a deterministic machine with a #P oracle can simulate a probabilistic machine with a #P oracle, we conclude that:

BP#P ⋅ P ⊆ P #P

Thus, we have established the second inclusion in Toda’s theorem. This completes the proof that BP with access to a #P oracle is no more
powerful than a deterministic polynomial-time machine with access to a #P oracle.

6. Implications of the Result

51/77
This result is important because it shows that, in terms of computational power, the addition of probabilistic polynomial-time computation does
not increase the complexity of problems when a counting oracle (#P) is available. The main takeaway is that #P oracles can simulate the
probabilistic guessing process deterministically in polynomial time.

In the context of Toda’s theorem, this means that the entire polynomial hierarchy (PH) is contained within the class BP raised to #P and
combined with P.

This concludes the second part of the proof of Toda’s Theorem. The full theorem demonstrates a profound connection between the polynomial
hierarchy, probabilistic machines, and counting problems, highlighting the interplay between these different complexity classes.

Lecture 28: Permanent and Determinant Functions

1. Introduction

In this lecture, we will introduce two fundamental functions in algebra and complexity theory: the permanent and the determinant. These
functions are defined for square matrices and play crucial roles in various fields of mathematics, particularly in combinatorics, linear algebra,
and complexity theory.

The determinant function has been widely studied and is known for its central role in linear algebra and its connection to computational
complexity. On the other hand, the permanent function, though similar in form, is computationally more challenging and has important
implications in the theory of #P-complete problems.

We will explore both functions in detail, define them rigorously, and discuss their complexity, focusing on their relationships and significance in
computational complexity theory.

2. The Determinant Function

Definition of Determinant

The determinant is a scalar value that can be computed from a square matrix, and it provides important information about the matrix, including
whether the matrix is invertible. For a square matrix A = [aij ] of size n × n, the determinant of A, denoted by det(A), is defined recursively
​

as:

n
det(A) = ∑ sgn(σ) ∏ ai,σ(i)
​ ​ ​

σ∈Sn ​ i=1

where:

Sn is the set of all permutations of the set {1, 2, … , n},

​

sgn(σ) is the sign of the permutation σ , and

ai,σ(i) is the element of the matrix at position (i, σ(i)).
​

The determinant function is computed by summing over all permutations of the columns of the matrix, multiplying the matrix entries
accordingly and applying the sign of each permutation.

Properties of the Determinant

The determinant is a multilinear function.

det(A) is zero if the rows (or columns) of A are linearly dependent.

det(A) is unchanged under row or column swaps but changes sign if two rows or columns are swapped.
The determinant is multiplicative: det(AB) = det(A) ⋅ det(B) for square matrices A and B .

Determinant in Computational Complexity

52/77
Computing the determinant of a matrix is a well-studied problem in computational complexity. The determinant function can be computed in
polynomial time using Gaussian elimination or other matrix factorization techniques, and it is known to belong to the class P (problems solvable
in polynomial time). The determinant can be computed in O(n3 ) time for an n × n matrix using standard algorithms.

3. The Permanent Function

Definition of Permanent

The permanent of a square matrix A = [aij ], denoted perm(A), is defined similarly to the determinant, but without the alternating signs:
​

n
perm(A) = ∑ ∏ ai,σ(i) ​ ​ ​

σ∈Sn i=1
​

The only difference between the permanent and the determinant is the absence of the sign function sgn(σ). The permanent is therefore a sum
over all permutations of the columns of the matrix, but all terms are added without alternating signs.

Properties of the Permanent

Like the determinant, the permanent is a multilinear function.

The permanent is not invariant under row or column swaps, and it does not change sign when rows or columns are swapped.

The permanent of a matrix is generally more difficult to compute than the determinant.

Permanent in Computational Complexity

Unlike the determinant, which can be computed in polynomial time, the permanent function is much more computationally challenging.
Computing the permanent is known to be #P-complete, meaning that it is as hard as any problem in #P (the class of counting problems). The
difficulty of computing the permanent lies in the fact that there are no efficient algorithms for it, and it involves a sum over all permutations of
the matrix entries, which is computationally expensive.

For an n × n matrix, computing the permanent is believed to be much harder than computing the determinant, and it is suspected that there is
no polynomial-time algorithm for the permanent unless P = NP, a major open question in computational complexity.

The complexity of the permanent makes it a central problem in the theory of #P-completeness. Many problems that involve counting solutions
to combinatorial problems can be reduced to the computation of the permanent of a matrix.

4. Relation Between Determinant and Permanent

Although the determinant and permanent functions are similar in form, they differ significantly in their computational complexity:

The determinant can be computed efficiently in polynomial time (in P).

The permanent is #P-complete, which is believed to be a much harder class of problems, and there is no known polynomial-time algorithm
to compute it.

These differences highlight a critical distinction between problems in P (decidable in polynomial time) and #P (counting problems, which are
believed to be intractable).

One of the key results in complexity theory is that if we could compute the permanent of a matrix in polynomial time, we could solve all
problems in #P in polynomial time, which would imply that P = NP, a conjecture that is widely believed to be false.

5. Applications of the Permanent

The permanent function has important applications in various fields of mathematics and computer science:

Combinatorics: The permanent is related to the counting of perfect matchings in bipartite graphs, which has applications in statistical
physics, graph theory, and combinatorial optimization.

53/77
 Quantum Computing: The permanent appears in the context of quantum computing, particularly in the study of the Feynman path
integral and in quantum complexity theory.

Complexity Theory: The permanent is a central example of a #P-complete problem and serves as a benchmark for understanding the
difficulty of counting problems.

6. Conclusion

In this lecture, we introduced the determinant and permanent functions, two important functions in linear algebra and computational
complexity. While the determinant can be computed efficiently in polynomial time, the permanent is known to be #P-complete, making it
significantly more difficult to compute. The complexity of the permanent has far-reaching implications in both theoretical computer science and
combinatorics, and it continues to be a topic of active research in computational complexity theory.

Lecture 29: Permanent is Hard for #P

1. Introduction

In this lecture, we will focus on the permanent function and its computational difficulty. Specifically, we will demonstrate that the permanent is
hard for the class #P, which is the class of counting problems associated with nondeterministic polynomial-time computations.

We will establish the fact that computing the permanent of a matrix is a #P-complete problem, which means that it is among the hardest
problems in the #P class. This result highlights the inherent difficulty of counting problems and illustrates the distinction between problems in P
(decidable in polynomial time) and #P (counting problems, believed to be intractable).

2. #P Class Overview

Before diving into the specifics of the permanent, let’s first review the #P class.

#P is the class of problems that count the number of accepting paths of a nondeterministic Turing machine (NDTM) for a given input.
Formally, a problem is in #P if it can be described as counting the number of solutions to a decision problem in NP.

For example:

The #SAT problem, which asks for the number of satisfying assignments of a Boolean formula, is in #P.

More generally, #P contains counting problems that correspond to various combinatorial problems, such as counting the number of
perfect matchings in a bipartite graph or counting the number of solutions to a system of linear equations.

#P-complete problems are those that are both in #P and are at least as hard as any other problem in #P. This means that any problem in #P
can be reduced to a #P-complete problem in polynomial time.

3. Permanent Function

Let us now recall the definition of the permanent.

Given an n × n matrix A = [aij ], the permanent of A, denoted by perm(A), is defined as:

​

n
perm(A) = ∑ ∏ ai,σ(i) ​ ​ ​

σ∈Sn i=1
​

where:

Sn is the set of all permutations of the indices {1, 2, … , n},

​

n
∏i=1 ​ ai,σ(i) is the product of the matrix entries corresponding to the permutation σ , and
​

The sum runs over all possible permutations of the columns.

54/77
The permanent is similar to the determinant, except that the sign function sgn(σ) is omitted. This results in a non-alternating sum over all
permutations of the matrix entries.

4. #P-Completeness of the Permanent

We now turn to the core result of this lecture: that the permanent function is #P-complete.

Permanent is in #P: The permanent function is in #P because it can be expressed as a counting problem. Specifically, it counts the number
of perfect matchings in a bipartite graph, which is a well-known problem in #P. For an n × n matrix, the permanent can be computed by
summing the products of entries corresponding to each permutation, which can be viewed as counting the number of different ways to
assign values to the matrix entries, akin to counting perfect matchings.

#P-completeness of the Permanent: To show that the permanent is #P-complete, we need to prove that any problem in #P can be reduced
to the problem of computing the permanent of a matrix in polynomial time.

Proof Sketch:

It is known that the #SAT problem (counting the number of satisfying assignments of a Boolean formula) is #P-complete.

A key result is that #SAT can be reduced to computing the permanent of a matrix, specifically a matrix that encodes the satisfiability of
the Boolean formula. This reduction is done by constructing a matrix whose entries encode the possible assignments to variables, and
whose permanent counts the number of satisfying assignments.

Since #SAT is #P-complete, and #SAT can be reduced to the permanent, it follows that the permanent is #P-complete.

This result shows that computing the permanent is at least as hard as the hardest counting problems in #P, and there is no known polynomial-
time algorithm for computing the permanent, unless P = NP (which is widely believed not to be the case).

5. Complexity Implications

The #P-completeness of the permanent has several important consequences:

Intractability: Since the permanent is #P-complete, it is believed to be intractable in general. There is no known polynomial-time algorithm
for computing the permanent, and it is unlikely that such an algorithm exists unless P = NP, a hypothesis that is widely regarded as false.

Reduction from Other Problems: The fact that the permanent is #P-complete implies that any problem in #P can be polynomially reduced
to the permanent. This means that any counting problem in #P is at least as hard as computing the permanent.

Connection to Other Problems: The permanent is related to many other important problems in combinatorics, such as counting the
number of perfect matchings in bipartite graphs and counting the number of solutions to certain systems of equations. The #P-
completeness of the permanent provides a deeper understanding of the complexity of these problems.

6. Applications of the Permanent

The difficulty of computing the permanent makes it central in many areas of computational theory:

Combinatorial Counting: The permanent is used in counting problems such as counting perfect matchings, which are important in graph
theory and combinatorics.

Quantum Computing: The permanent also appears in quantum mechanics, particularly in the study of quantum interference and the
Feynman path integral. It has been used in quantum algorithms, providing a link between complexity theory and quantum computing.

Statistical Physics: In statistical physics, the permanent arises in the calculation of partition functions and in the study of statistical models,
especially in problems involving random matrices and random graphs.

7. Conclusion

55/77
In this lecture, we showed that the permanent is #P-complete, meaning it is one of the hardest problems in the class #P. We demonstrated that
the permanent is a counting problem equivalent in difficulty to #SAT, a well-known #P-complete problem. As a result, computing the permanent
is computationally intractable in general, unless P = NP. The #P-completeness of the permanent makes it a central problem in both complexity
theory and combinatorics, with significant implications for our understanding of counting problems and their relationship to other complexity
classes.

Lecture 30: Interactive Proofs

1. Introduction

In this lecture, we introduce the concept of interactive proofs and explore their relevance in the context of computational complexity.
Interactive proofs represent a powerful class of problems that lie beyond the traditional NP class, and provide a framework for probabilistic
verification of computational statements. We will explore the concept of an interactive proof system, discuss its relationship with NP, and
introduce the class IP[k], which generalizes the concept to multiple rounds of interaction.

2. Interactive Proof Systems

An interactive proof system involves two parties:

A prover who tries to convince the other party (the verifier) of the truth of a statement, typically by sending messages.

A verifier who checks the messages and either accepts or rejects the statement, based on the interaction.

Formally, an interactive proof system consists of a sequence of messages exchanged between the verifier and the prover. The verifier’s role is to
ensure that the prover is not attempting to deceive it, typically using randomness. The verifier will accept the statement if it believes the prover’s
messages are consistent with the truth, and reject otherwise.

A key feature of an interactive proof system is that the verifier only interacts with the prover a limited number of times, and the verifier can
make decisions based on probabilistic reasoning.

3. Formal Definition of Interactive Proofs

An interactive proof system consists of the following components:

A prover P , who has unlimited computational power.

A verifier V , who is a probabilistic polynomial-time machine. The verifier performs the computation, and interacts with the prover.

The proof system is defined by the following rules:

1. Interaction: The verifier sends messages to the prover, and the prover responds to each message. The verifier’s responses are probabilistic,
based on the prover’s replies.

2. Completeness: If the statement being proved is true, the prover can convince the verifier with high probability, no matter how the verifier
behaves.

3. Soundness: If the statement being proved is false, the prover cannot convince the verifier to accept the false statement, except with very
low probability.

Formally, we define an interactive proof for a language L as follows:

Completeness: If x ∈ L, there exists a strategy for the prover such that the verifier accepts x with probability at least 1 − ϵ, where ϵ is
small.

Soundness: If x ∈
/ L, no prover can convince the verifier to accept x with probability greater than ϵ, where ϵ is small (typically less than 1/2
).

56/77
4. **The Class IP

The class IP (for Interactive Polynomial time) is defined as the set of decision problems for which there exists an interactive proof system with a
polynomial-time verifier and an arbitrary prover. The verifier has access to randomness and runs in polynomial time, while the prover is
unbounded in computational power.

The class IP can be understood as a generalization of NP. Specifically:

If a problem is in NP, it can be solved by an interactive proof system where the verifier is deterministic and the interaction is essentially a
single round of communication.

If a problem is in IP, it can be solved by an interactive proof system with multiple rounds of interaction, using probabilistic reasoning on the
part of the verifier.

This means that:

NP can be viewed as a special case of IP, where the verifier’s behavior is deterministic and there is only one round of communication
between the prover and the verifier.

A key result in the theory of interactive proofs is that IP = PSPACE, meaning that any problem that can be solved using polynomial space can also
be solved using an interactive proof system. This result was a significant breakthrough in computational complexity theory, as it showed a
surprising equivalence between two seemingly unrelated complexity classes.

5. Interactive Proofs with a Deterministic Verifier

When the verifier in an interactive proof system is deterministic, the class of problems solvable by such a system is NP.

In this case, the verifier simply checks the validity of the prover’s claims without using randomness. The interaction is typically limited to a
single round where the prover presents a certificate (or witness) and the verifier checks it deterministically.

This is essentially the certificate-based definition of NP, where the verifier can efficiently check the correctness of a solution, given a
proposed certificate (or solution).

Thus, IP with a deterministic verifier is equivalent to NP. In this scenario, the prover is allowed to provide a solution, but the verifier must verify
the correctness of the solution deterministically.

6. Probabilistic Verifier and the Class IP [k]

Next, we consider the case where the verifier is probabilistic. In this case, the verifier uses randomness to decide whether to accept or reject the
prover’s claim.

We can generalize the concept of interactive proofs to multiple rounds of interaction, where the verifier’s decisions are influenced by several
rounds of communication.

The class IP [k] is a hierarchy of classes, where the number of rounds of interaction between the prover and the verifier is bounded by k .
Specifically:

IP[1] is the class of problems that can be solved by an interactive proof system with a single round of interaction, which is equivalent to NP.

IP[2] involves two rounds of interaction.

In general, IP[k] involves k rounds of interaction.

The class IP [k] can be thought of as generalizing NP with increasing levels of interaction and probabilistic reasoning. The class IP [k]
encompasses problems that may require more rounds of interaction and probabilistic checks before a conclusion can be drawn.

7. IP and PSPACE

A key result in the theory of interactive proofs is the equivalence of the class IP with PSPACE:

57/77
 IP = PSPACE.

This result shows that interactive proofs, which involve multiple rounds of communication and probabilistic checks, can be simulated by
polynomial-space computations. In other words, any problem that can be solved with an interactive proof system can also be solved using
polynomial space, regardless of the number of rounds involved.

This equivalence implies that the power of interactive proofs extends beyond what we previously thought, as the space complexity of a problem
does not increase with the number of rounds of interaction.

8. Applications and Importance

Interactive proofs and the class IP have profound implications in theoretical computer science, particularly in the study of computational
complexity and cryptography. Some important points include:

Cryptography: Interactive proofs provide a natural framework for designing cryptographic protocols, such as zero-knowledge proofs, where
one party can prove a statement without revealing any additional information.

Verification of Hard Problems: Interactive proofs allow the verification of computational problems that are hard to solve deterministically,
such as NP-complete problems. The verifier does not need to compute the solution, but only verify that the prover’s claims are consistent
with the truth.

Understanding of PSPACE: The equivalence of IP and PSPACE helps us understand the complexity of problems that require polynomial
space, providing a deeper insight into the relationship between space-bounded and interaction-based computation.

9. Conclusion

In this lecture, we introduced the concept of interactive proofs, a framework for probabilistically verifying the truth of statements via interaction
between a prover and a verifier. We showed that IP (the class of problems solvable by interactive proofs) contains NP as a special case, and that
IP = PSPACE. This result has important implications for understanding the complexity of problems in PSPACE and for designing cryptographic
protocols such as zero-knowledge proofs. The introduction of probabilistic verifiers and multiple rounds of interaction expands our
understanding of what can be verified efficiently, highlighting the power and versatility of interactive proofs.

1/2

Lecture 31: Graph Non-Isomorphism is in IP [2]

1. Introduction

In this lecture, we delve into the complexity of the Graph Non-Isomorphism problem and demonstrate that it belongs to the class IP [2]. We
will also discuss the importance of having private coins versus public coins in probabilistic verification, which plays a key role in the design of
interactive proof systems. Finally, we will introduce and explore the classes AM[k] and MA, which generalize the concept of interactive proofs
and have important implications for probabilistic complexity classes.

2. Graph Non-Isomorphism

The Graph Non-Isomorphism problem is a decision problem where, given two graphs G1 and G2 , the task is to determine whether or not they
​ ​

are isomorphic. Two graphs are isomorphic if there is a one-to-one correspondence between their vertices such that the edges are preserved.

This problem is known to be hard in terms of computational complexity. In particular, it is not known whether Graph Non-Isomorphism belongs
to NP, nor is it known to be NP-complete. However, we can show that it belongs to the class IP [2], which indicates that the problem can be
solved using an interactive proof system with two rounds of interaction.

58/77
3. Graph Non-Isomorphism is in IP [2]

The goal in this section is to prove that the Graph Non-Isomorphism problem is contained in IP [2]. This means that we can design an
interactive proof system for the problem where the verifier uses two rounds of interaction with the prover.

To prove this, we construct an interactive proof system for Graph Non-Isomorphism. The steps involved in the interactive proof system are as
follows:

1. First Round (Commitment Phase):

The verifier randomly selects one of the two graphs G1 or G2 .

​ ​

The verifier sends this selection to the prover, who then provides a commitment to either the isomorphism or non-isomorphism of the
graphs, depending on the truth of the instance.

2. Second Round (Challenge Phase):

The verifier sends a random challenge to the prover, and the prover responds with a proof that either confirms or denies the
isomorphism of the graphs.

The key to the efficiency of this interactive proof system is that it uses randomness in both rounds, which is why the verifier’s decision is
probabilistic. The verifier can then accept or reject the graphs based on the responses it receives.

This interaction occurs in two rounds, making it clear that Graph Non-Isomorphism belongs to the class IP [2].

4. Importance of Private Coins vs. Public Coins

In the context of interactive proof systems, there is an important distinction between private coins and public coins. These refer to the type of
randomness used by the verifier:

Private coins: The verifier generates randomness independently for each round of interaction with the prover.

Public coins: The verifier’s randomness is shared between all participants, including the prover.

The distinction becomes crucial when considering the efficiency and power of the interactive proof system.

1. Private Coins: The verifier can select its random choices independently for each round. This provides more flexibility and randomness in the
system, which can make the proof system stronger and more powerful. Using private coins often leads to more efficient interactive proofs
with stronger soundness guarantees.

2. Public Coins: The randomness is shared between the verifier and the prover. This creates a more constrained system where both parties use
the same random values. Public coin protocols are often referred to as zero-knowledge proofs, where the prover does not reveal any
additional information other than the validity of the statement.

The distinction between private and public coins is important because it determines the complexity and power of the corresponding
probabilistic proof systems.

5. The Class AM [k] and MA

Next, we introduce the classes AM[k] and MA, which are generalizations of the interactive proof concept. These classes provide a richer
framework for interactive proofs, where the number of rounds of interaction and the type of verifier’s randomness vary.

AM[k]: The class AM (standing for Arthur-Merlin) generalizes IP by allowing for a polynomial-time verifier to interact with an unbounded
prover. The class AM[k] refers to the use of k rounds of interaction, where each round involves the verifier sending a message to the prover
and receiving a response. AM[k] is known for its use in polynomial-time interactive proofs.

MA: The class MA (standing for Merlin-Arthur) is a variant of AM, where the verifier only performs one round of interaction, and the prover
provides a solution to the problem. This class is often used in the context of one-round interactive proofs.

Both classes provide powerful ways of capturing the complexity of decision problems that cannot easily be solved deterministically. These
classes have implications for the study of probabilistic computation and interactive proofs, allowing us to classify a wider range of problems.

59/77
6. Key Properties of the Classes AM [k] and M A

AM[k] allows for multiple rounds of interaction between the prover and verifier, with each round involving probabilistic checks by the
verifier.

MA, on the other hand, is a single round protocol where the prover provides a solution to the verifier, and the verifier checks the solution
using randomness.

Relation to Other Complexity Classes: Both AM[k] and MA lie within the realm of probabilistic complexity classes. They generalize
traditional classes like NP and coNP and can be used to classify problems that are not easily solvable within conventional frameworks.

7. Conclusion

In this lecture, we demonstrated that the Graph Non-Isomorphism problem is contained in the class IP [2], meaning that it can be solved using
an interactive proof system with two rounds of interaction. We discussed the importance of private coins versus public coins in interactive
proofs, highlighting their impact on the power and efficiency of proof systems. Finally, we introduced the classes AM[k] and MA, which provide
powerful generalizations of interactive proofs with multiple rounds and differing interaction structures. These classes play a significant role in
our understanding of probabilistic complexity and help us classify a broader range of problems in computational complexity theory.

Lecture 32: Set Lower Bound Protocol and Graph Non-Isomorphism is in AM

1. Introduction

In this lecture, we will discuss the Set Lower Bound Protocol, which is a technique used to establish lower bounds in complexity theory, and
demonstrate that the Graph Non-Isomorphism problem belongs to the class AM. We will explore how interactive proof systems can be used to
classify problems based on their complexity, particularly focusing on Arthur-Merlin protocols, where the interaction is key to the problem’s
complexity characterization.

2. Set Lower Bound Protocol

The Set Lower Bound Protocol is a method used in the context of interactive proof systems and complexity theory to demonstrate lower
bounds for certain problems, typically in terms of the number of rounds or the size of the verifier’s computation. The protocol involves
establishing a lower bound on the computational resources required to solve a particular problem, often by showing that no efficient solution
exists without violating fundamental properties such as the class of problems being considered.

The Set Lower Bound Protocol typically proceeds as follows:

A set of problems is considered that belong to a certain complexity class, and the goal is to show that there is no efficient (e.g., polynomial-
time) solution for these problems within the class unless a certain lower bound is met.

The protocol often works by using a reduction argument: if an algorithm exists for solving a problem efficiently, it would be possible to
solve a known hard problem (or class of problems) within the same bound, which leads to a contradiction.

For example, if solving the Graph Non-Isomorphism problem in a particular class required exponential time, but we could use this solution to
solve a problem in NP , we would be forced to conclude that NP is solvable in a time that contradicts known assumptions about complexity.

By establishing such lower bounds, the Set Lower Bound Protocol helps us understand the inherent difficulty of certain computational
problems, which in turn aids in classifying them within the correct complexity class.

3. Graph Non-Isomorphism is in AM

60/77
We now show that the Graph Non-Isomorphism problem is contained in the class AM, where the class AM refers to the Arthur-Merlin
protocol, a type of interactive proof system. This is a class of problems that can be verified probabilistically by an interactive protocol between a
prover (who has unlimited computational power) and a verifier (who has polynomial-time computational power).

Arthur-Merlin protocols consist of multiple rounds of interaction between the prover and verifier. In the case of AM , the verifier is a
probabilistic polynomial-time machine that interacts with the prover, and the prover is unbounded but has the goal of convincing the verifier
that the graph non-isomorphism holds.

In the first round, the verifier sends a random challenge to the prover. The prover’s response is a commitment to a particular statement
about the graphs (either isomorphic or not).

In the second round, the verifier uses the response from the prover and computes whether the claimed property holds. The verifier may
also challenge the prover with further randomness to confirm the prover’s claim.

The interaction is probabilistic, meaning that the verifier accepts the prover’s claim with high probability if the graph non-isomorphism property
holds. The advantage of using the Arthur-Merlin framework is that it allows us to verify the correctness of a statement about the graphs in
polynomial time, even if the problem itself is hard.

The Graph Non-Isomorphism problem fits into the AM class because it can be expressed as an interactive proof where the verifier uses
randomness and interaction to verify the correctness of the prover's claim. The protocol works in such a way that the verifier can be convinced
of the correctness of the answer with high probability after a constant number of rounds.

4. How AM Protocol for Graph Non-Isomorphism Works

The interactive proof system for Graph Non-Isomorphism in the AM class follows this process:

1. Verifier’s Challenge:

The verifier randomly selects one of the two graphs, G1 or G2 , and sends this choice to the prover.
​ ​

2. Prover’s Response:

The prover then responds with a statement about the relationship between the graphs. If G1 and G2 are non-isomorphic, the prover
​ ​

can provide a response that asserts the difference between the graphs. The prover's response is generated in such a way that the
verifier can check it using the randomness provided in the challenge.

3. Verification:

The verifier checks whether the prover’s response is consistent with the non-isomorphism of the graphs. If it is, the verifier accepts the
response; otherwise, it rejects it. The verifier’s decision process is probabilistic, and if the prover is truthful, the verifier will accept the
response with high probability.

The Graph Non-Isomorphism problem is not NP-complete, but it can be solved using the AM framework. This shows that Graph Non-
Isomorphism is a problem that can be efficiently verified with interaction, even though it does not fall into the traditional NP class.

5. Properties of AM Protocols

Interaction: An Arthur-Merlin protocol requires a number of interaction rounds, and each round involves the verifier sending a message
and the prover responding with a commitment. The number of rounds is polynomial, which makes the protocol feasible for problems in the
AM class.

Randomness: The verifier’s decisions are probabilistic. In many cases, the verifier uses randomness to ensure that the proof is convincing
with high probability.

Polynomial-time Verifier: The verifier in an AM protocol is a probabilistic polynomial-time machine, meaning that the verification process is
efficient.

Soundness: If the prover is dishonest or tries to trick the verifier, the probability of the verifier accepting an incorrect proof is small. This
provides the soundness guarantee in interactive proof systems.

61/77
6. Conclusion

In this lecture, we discussed the Set Lower Bound Protocol, which helps establish lower bounds for problems in interactive proof systems, and
demonstrated that Graph Non-Isomorphism is in the class AM. By using an interactive proof system with two rounds of interaction, we showed
how the Graph Non-Isomorphism problem can be verified probabilistically, even though it is not NP-complete. The class AM provides a
powerful framework for classifying problems that require interaction between the prover and verifier, and the Graph Non-Isomorphism
problem is a key example of such a problem.

Through these discussions, we have explored the complexity of interactive proof systems and how they help classify problems that are beyond
the reach of traditional deterministic or probabilistic algorithms. The concept of interaction plays a crucial role in expanding the boundaries of
computational complexity theory.

Lecture 33: MA is in AM

1. Introduction

In this lecture, we will establish that MA is a subset of AM and explore the implications of this inclusion. We will also discuss the consequences of
assuming that the Graph Isomorphism problem is NP-complete, including the potential collapse of the Polynomial Hierarchy to the second
level. This result sheds light on the relationship between different interactive proof classes and their implications for the structure of complexity
classes.

2. MA and AM

MA (Merlin-Arthur) and AM (Arthur-Merlin) are both classes of interactive proof systems where the interaction between the prover (Merlin) and
the verifier (Arthur) plays a key role in verifying the correctness of a solution.

MA: In an MA protocol, the verifier is a probabilistic polynomial-time machine, and the prover (Merlin) sends a single message (a certificate)
after receiving a challenge from the verifier. The verifier then checks whether the prover's response is valid.

AM: In an AM protocol, the interaction between the verifier and the prover consists of multiple rounds, where the verifier sends a series of
challenges, and the prover responds in a way that convinces the verifier of the correctness of the claim.

The key difference between MA and AM is the number of interaction rounds. MA has only one round of interaction, whereas AM allows multiple
rounds of interaction.

We will show that MA is contained in AM, i.e., M A ⊆ AM .

3. Proof: MA is in AM

To prove that MA is in AM, we observe that an MA protocol can be transformed into an AM protocol by simply extending the number of rounds
of interaction. The one-round MA protocol can be turned into a two-round AM protocol as follows:

1. Verifier’s Challenge: In the first round of the AM protocol, the verifier sends a challenge to the prover (just like in the MA protocol).

2. Prover’s Response: In the second round, the prover sends the response (the certificate) to the verifier.

3. Verification: The verifier checks the response provided by the prover, which was the exact behavior in the MA protocol.

Thus, a protocol with a single round of interaction (MA) can be viewed as a protocol with two rounds (AM). Since AM allows multiple rounds, the
MA protocol is subsumed by the AM class.

Therefore, we conclude that:

M A ⊆ AM

62/77
4. Implications of Graph Isomorphism being NP-Complete

Next, we examine the consequences of assuming that the Graph Isomorphism problem is NP-complete. The Graph Isomorphism problem is a
well-known problem in computational complexity, where the task is to determine whether two given graphs are isomorphic (i.e., can be
transformed into each other by renaming vertices).

If we assume that Graph Isomorphism is NP-complete, it would have significant implications for the Polynomial Hierarchy (PH):

Polynomial Hierarchy (PH) is a hierarchy of complexity classes that generalizes NP, coNP, and other classes by introducing alternating
quantifiers. The hierarchy consists of different levels such as Σkp and Πkp , where k represents the level of alternations between existential
​ ​

and universal quantifiers.

If Graph Isomorphism is NP-complete, we can derive the following consequence:

P H = Σ2p ​

This means that the Polynomial Hierarchy collapses to the second level, and this would imply that higher levels of the hierarchy are not
necessary to describe the complexity of problems in PH. This would dramatically simplify our understanding of the structure of PH.

The collapse of the Polynomial Hierarchy to Σ2p is a significant result because it shows that if Graph Isomorphism is NP-complete, the classes at
​

higher levels of the Polynomial Hierarchy (such as Σ3p , Π3p , etc.) do not offer additional power beyond the second level.
​ ​

5. Graph Isomorphism and the Collapse of PH

Let us explore why assuming that Graph Isomorphism is NP-complete leads to the collapse of PH:

If Graph Isomorphism is NP-complete, it would be a problem in NP that is as hard as any other problem in NP.

By a result in complexity theory, if there exists a problem in NP that is NP-complete and also belongs to a class Σkp for some k , it implies that
​

all higher levels of the Polynomial Hierarchy collapse to Σ2p .

​

Thus, assuming Graph Isomorphism is NP-complete causes the Polynomial Hierarchy to collapse to Σ2p , which is a surprising and significant
​

result. This would suggest that the additional power offered by higher levels of the hierarchy is redundant and not needed to solve problems in
NP.

6. Conclusion

In this lecture, we showed that MA is a subset of AM, meaning that every problem in the MA class can be solved using an AM protocol.
Additionally, we explored the consequences of assuming that Graph Isomorphism is NP-complete, which leads to the collapse of the
Polynomial Hierarchy to its second level. This result demonstrates the delicate interdependencies between different complexity classes and
highlights the importance of understanding the structural relationships between them.

By understanding these concepts, we gain deeper insights into the complexity of problems and the theoretical limits of computational systems,
as well as the implications of making assumptions about the hardness of certain problems like Graph Isomorphism.

Lecture 34: Sumcheck Protocol - I

1. Introduction to the Sumcheck Protocol

The Sumcheck Protocol is a powerful interactive proof technique that is used to verify the correctness of a sum over a high-dimensional space.
It is widely used in the context of proving the hardness of certain problems and is a fundamental technique in complexity theory. This protocol
allows for the verification of a sum, which is often encountered in the context of combinatorial optimization, counting problems, and the class
#P.

The Sumcheck Protocol is a generalization of the IP (Interactive Proof) class, which was discussed earlier in the course. The protocol is
particularly useful when the sum to be checked can be represented as a multivariate polynomial, and the goal is to verify the correctness of this
sum efficiently.

63/77
In this lecture, we will introduce the Sumcheck Protocol and explain its underlying technique, known as arithmetization, which transforms a
sum into a form that can be more easily verified by an interactive proof.

2. The Sumcheck Protocol Overview

The Sumcheck Protocol is designed to verify the sum of a function over a large set of variables. Given a function f : {0, 1}n → R, the goal is to
verify that the sum of f (x) over all possible inputs x ∈ {0, 1}n is correct. This sum is often expressed as:

S = ∑ f (x) ​

x∈{0,1}n

The challenge is to verify that this sum is correct without having to explicitly calculate it, which is computationally infeasible for large n. The
Sumcheck Protocol enables a verifier to interact with a prover and verify the sum efficiently.

3. Arithmetization

The key technique used in the Sumcheck Protocol is called arithmetization. The purpose of arithmetization is to transform the sum into a
polynomial that can be checked via the protocol. Specifically, the idea is to replace the sum over binary inputs x ∈ {0, 1}n with a polynomial in
one variable.

Arithmetization works by creating a polynomial representation of the function f (x), which maps each input x to a value in the field. This is
done by interpolating the function using polynomials and reducing the problem of summing over all binary inputs to evaluating polynomials at
specific points.

The process of arithmetization is illustrated in the following steps:

1. Define the polynomial: First, a polynomial p(x1 , x2 , ..., xn ) is constructed that encodes the function f (x). The idea is to represent f (x) as
​ ​ ​

a polynomial whose evaluation over all inputs x ∈ {0, 1} corresponds to the sum we want to check.
n

2. Reduction to lower dimensions: The polynomial can then be reduced to a lower-dimensional problem by iteratively evaluating it along each
variable. This process reduces the complexity of the sum-checking problem and allows for the sum to be verified with fewer computational
resources.

4. The Sumcheck Protocol in Action

The Sumcheck Protocol involves a series of interactions between a prover and a verifier. Here is a general outline of how the protocol works:

1. Prover's commitment: The prover commits to a function f (x) and provides an initial value of the sum, which is the sum over all possible
binary inputs to f (x).

2. Verifier's challenge: The verifier sends a challenge to the prover, asking for a specific piece of information about the polynomial
representation of f (x). This is where the arithmetization process comes into play. The verifier requests the prover to reveal a certain part of
the polynomial that corresponds to a specific evaluation of the sum.

3. Interactive process: The verifier and the prover engage in multiple rounds of communication, with the prover revealing portions of the
polynomial and the verifier verifying the consistency of these portions.

4. Final verification: After several rounds, the verifier is able to confirm whether the sum is correct based on the information provided by the
prover.

5. Properties of the Sumcheck Protocol

The Sumcheck Protocol has several important properties:

64/77
 1. Efficiency: The protocol reduces the verification of a sum over all inputs to a more manageable process by turning the problem into a
sequence of polynomial evaluations. This allows the verifier to check the sum with significantly fewer computations than a brute-force
approach.

2. Completeness: If the prover is honest and the sum is indeed correct, the verifier will always be convinced that the sum is correct after the
interactive process.

3. Soundness: If the sum is incorrect, the verifier will reject the claim with high probability. The protocol is designed to detect errors even if the
prover attempts to deceive the verifier.

4. Interaction rounds: The number of rounds of interaction in the Sumcheck Protocol is proportional to the number of variables in the
function f (x). This makes the protocol efficient in terms of the number of rounds, but it still relies on the complexity of the problem to
determine how many rounds are needed.

6. Applications of the Sumcheck Protocol

The Sumcheck Protocol has important applications in several areas of computational complexity and theoretical computer science:

1. #P-Complete Problems: The protocol is often used to prove that problems are #P-complete, which means they are at least as hard as
counting problems. Examples of such problems include counting the number of solutions to a constraint satisfaction problem.

2. Approximation Algorithms: The Sumcheck Protocol can be used to design approximation algorithms for counting problems. By efficiently
verifying partial sums, the protocol can help in approximating difficult counting problems.

3. Interactive Proofs and Zero-Knowledge Proofs: The Sumcheck Protocol is a powerful tool in the study of interactive proofs and zero-
knowledge proofs. It allows for the verification of the correctness of a sum without revealing sensitive information about the underlying
problem.

7. Conclusion

In this lecture, we introduced the Sumcheck Protocol and its underlying technique of arithmetization. The Sumcheck Protocol provides an
efficient way to verify the correctness of sums over large sets, making it a valuable tool in complexity theory, particularly for #P-complete
problems. The protocol is an example of how interactive proofs can be used to verify mathematical statements without the need to explicitly
compute everything, offering a powerful approach to computational problems involving sums and counting.

In the next lecture, we will continue our discussion of the Sumcheck Protocol and explore further applications and refinements of this
technique in complexity theory.

Lecture 35: Sumcheck Protocol - II (Proof Overview)

1. Overview of the Sumcheck Protocol

In this lecture, we will continue our discussion of the Sumcheck Protocol and provide a detailed proof overview. The Sumcheck Protocol is a
crucial tool in complexity theory, particularly in proving hardness results for #P-complete problems. In the previous lecture, we discussed the
basic ideas behind the protocol, including its use of arithmetization to transform a sum into a polynomial. Today, we will focus on the formal
proof of the protocol’s correctness, completeness, soundness, and the mechanics of the interaction between the prover and verifier.

2. Restating the Problem

The central goal of the Sumcheck Protocol is to verify the sum:

S = ∑ f (x) ​

x∈{0,1}n

65/77
where f (x) is a function defined on n-dimensional binary vectors. The goal is to check the value of this sum without directly computing it, which
is computationally infeasible for large n. The function f (x) is assumed to be computable, but its direct evaluation for all possible inputs would
require exponential time.

Instead of evaluating this sum directly, we use the Sumcheck Protocol to verify its correctness interactively, utilizing the notion of
arithmetization and iterative polynomial reductions.

3. Protocol Description: Detailed Steps

The protocol consists of the following steps:

1. Initial Commitment:

The prover commits to the sum S , which is the sum of the function f (x) over all binary vectors x ∈ {0, 1}n . The prover then computes
a polynomial representation of f (x) and sends this polynomial (or a commitment to it) to the verifier.

2. Arithmetization:

The polynomial corresponding to the function f (x) is arithmetized, which means it is transformed into a polynomial in a single
variable. The polynomial represents the sum over all possible inputs and can be checked incrementally by the verifier.

3. Verifier’s Challenge:

The verifier sends a challenge to the prover. This challenge typically involves asking the prover to reveal specific values or coefficients of
the polynomial at certain evaluation points.

4. Iterative Rounds:

In each round, the prover is asked to reveal part of the polynomial corresponding to a lower-dimensional sum. The verifier checks the
correctness of the revealed polynomial segments at each step. This process is repeated for multiple rounds, progressively reducing the
problem's dimensionality until the sum is completely verified.

5. Final Acceptance:

After a series of rounds, if the prover has provided the correct answers for all challenges, the verifier accepts the sum as correct.
Otherwise, the verifier rejects the sum.

4. Completeness of the Sumcheck Protocol

Completeness refers to the condition where, if the prover is honest and the sum S is correct, the verifier will always accept the sum.

Proof of Completeness: If the prover follows the protocol correctly and the sum S is indeed the correct sum of f (x), the verifier will always
be able to verify the sum in the final round. This is because the verifier receives consistent and verifiable information at each round of
interaction.

The polynomial commitment from the prover encodes the correct sum, and since the verification process checks progressively smaller
portions of the polynomial, the verifier will always accept if the sum is correct. The completeness property holds as long as the prover
correctly commits to the function and follows the protocol.

5. Soundness of the Sumcheck Protocol

Soundness ensures that if the sum S is incorrect, the verifier will reject the sum with high probability.

Proof of Soundness: The verifier rejects any incorrect sum with a high probability due to the randomness involved in the challenges at each
round. The verifier essentially checks whether the polynomial committed to by the prover is consistent with the sum being claimed. If the
sum is incorrect, at least one round will result in an inconsistency, leading the verifier to reject the claim.

The soundness is based on the fact that, with each round, the polynomial's evaluation becomes progressively more constrained, making it
harder for the prover to deceive the verifier. If the prover attempts to cheat, they must provide an incorrect polynomial evaluation at some
point, which will be caught by the verifier.

66/77
 The probability of the verifier making a mistake (i.e., accepting an incorrect sum) can be made arbitrarily small by repeating the protocol for
a sufficient number of rounds. This is achieved by using probabilistic methods in the verification process.

6. Analysis of the Protocol's Complexity

The Sumcheck Protocol reduces the problem of summing over all binary vectors x ∈ {0, 1}n to a series of polynomial evaluations. The number
of rounds in the protocol is proportional to the number of variables in the function f (x), meaning that the protocol has complexity that is
exponential in the number of variables. However, the number of computations in each round is significantly less than the number of
computations needed for a direct sum over all inputs.

Time Complexity: Each round involves checking a lower-dimensional sum, which is more computationally feasible than directly computing
the sum over all inputs. Therefore, the time complexity of the protocol is more efficient than a brute-force approach, but still depends on the
size of the input.

Space Complexity: The space complexity of the protocol is also reduced compared to direct summation because the verifier does not need
to store all inputs or the entire sum at once. Instead, the verifier only stores the necessary parts of the polynomial for checking at each
round.

7. Correctness of Arithmetization and Polynomial Evaluation

A key part of the Sumcheck Protocol is the arithmetization of the function f (x). The correctness of this transformation is critical to ensuring
that the protocol works as intended.

The arithmetization step ensures that the sum can be expressed as a polynomial, and this polynomial has the desired properties that make
it suitable for interactive proof techniques.

During the protocol, each query made by the verifier corresponds to a specific evaluation of the polynomial. As the rounds progress, the
degree of the polynomial decreases, and the problem becomes easier to handle.

8. Conclusion

In this lecture, we have presented a detailed overview of the Sumcheck Protocol, including its structure, proof of completeness, soundness, and
its interaction rounds. The protocol enables efficient verification of sums over large sets by transforming the problem into a series of polynomial
evaluations. The Sumcheck Protocol plays a crucial role in complexity theory, especially in the study of #P-complete problems and interactive
proofs.

In the next lecture, we will explore further applications of the Sumcheck Protocol and delve into more advanced topics related to interactive
proofs and the connections between different complexity classes.

Lecture 36: Parity not in AC0 - I (The Proof by Razborov and Smolensky)

1. Introduction to the Problem

In this lecture, we will discuss the landmark result by Razborov and Smolensky regarding the complexity of the parity function. The parity
function is defined as:

n
Parity(x1 , x2 , … , xn ) = ⨁ xi
​ ​ ​ ​ ​

i=1

where ⊕ denotes the XOR operation. The parity function outputs 1 if the number of 1's in the input is odd, and 0 otherwise. The primary result
of this lecture is the proof that the parity function is not in the class AC0.

67/77
2. Complexity Classes: AC0 and its Significance

Before we dive into the proof, let’s recall some basic definitions. The class AC0 consists of families of Boolean circuits that:

Have constant depth.

Have unbounded fan-in AND, OR, and NOT gates.

Can be constructed in polynomial size.

AC0 is a very limited class of circuits with respect to depth, and the class is known for its constant-depth nature, where the number of layers of
gates (or the circuit depth) is fixed and does not grow with the size of the input.

One of the major results in computational complexity is to understand which Boolean functions can be computed by such constant-depth
circuits. While AC0 is quite weak in terms of computational power, it can still compute many interesting functions. However, the parity function
has long been known to be difficult to compute within AC0 circuits. Razborov and Smolensky's work formalized this intuition with a rigorous
proof.

3. Razborov's Result: Parity is Not in AC0

Razborov's proof, which was given in 1985, shows that the parity function cannot be computed by constant-depth circuits of polynomial size,
i.e., Parity ∉ AC0. The strategy behind the proof is to prove a lower bound on the size of any AC0 circuit that computes the parity function.

The key idea in Razborov's approach involves communication complexity and spectral analysis. Specifically, Razborov used the Fourier
analysis of Boolean functions to establish that any AC0 circuit computing the parity function would have to be extremely large. Fourier analysis
plays a central role in understanding the structure of Boolean functions and their representations in different classes of circuits.

3.1. Fourier Representation of Boolean Functions

A Boolean function f (x1 , … , xn ) can be represented as a sum of Fourier basis functions:

​ ​

f (x1 , … , xn ) = ∑ f^(S)χS (x1 , … , xn )

​ ​ ​ ​ ​ ​ ​

S⊆[n]

where χS (x)
​
= ∏i∈S xi , and f^(S) are the Fourier coefficients of f . The Fourier coefficients capture the contribution of each subset of
​ ​ ​

variables to the function. The idea is that any constant-depth circuit can be viewed in terms of its Fourier expansion, and Razborov showed that
for the parity function, the Fourier coefficients behave in a way that any AC0 circuit would require a large number of terms to compute it.

3.2. Spectral Proof Strategy

Razborov's proof shows that the Fourier coefficients of the parity function decay in such a way that they cannot be efficiently captured by
constant-depth circuits. More specifically, he shows that the constant-depth circuits in AC0 cannot compute the parity function efficiently
because the parity function has a very high influence on the input variables, which cannot be handled by shallow circuits. This high influence
implies that the non-zero Fourier coefficients of the parity function are too large for the function to be computed by constant-depth circuits.

4. Smolensky's Contribution

While Razborov’s result used Fourier analysis, Smolensky made another important contribution using polynomial representations. Smolensky
extended Razborov's result to the class AC0[p], which consists of circuits with constant depth and unbounded fan-in gates, where the gates are
restricted to computing mod p operations. Smolensky's proof shows that parity is not in AC0[p] for any prime p.

4.1. Smolensky's Use of Modulo p Gates

Smolensky's proof is based on a more sophisticated use of polynomial identities and the fact that the parity function is hard to represent using
circuits of bounded depth, even when the gates are allowed to compute modulo p operations. Smolensky used algebraic techniques to show
that any constant-depth circuit with modulo gates cannot compute the parity function efficiently.

The key insight was that the parity function cannot be expressed as a low-degree polynomial over finite fields when restricted to a constant
number of layers of gates. Smolensky's result extends the understanding of the parity function's computational complexity to a broader range
of circuit classes.

68/77
5. Implications of Razborov and Smolensky’s Result

The main takeaway from Razborov and Smolensky's work is that parity is a hard function for constant-depth circuits. This result has profound
implications in the theory of circuit complexity. It establishes that certain natural functions, such as parity, are inherently difficult for circuits
with very limited resources (such as depth). This result provides a separation between the complexity of different classes of circuits.

Additionally, the result led to an explosion of interest in understanding the limitations of constant-depth circuits and motivated the development
of further lower bounds for other Boolean functions and more general circuit classes.

6. Conclusion

In this lecture, we have discussed the proof by Razborov and Smolensky that the parity function is not in AC0. Razborov’s use of Fourier
analysis and Smolensky's extension to AC0[p] circuits both show that constant-depth circuits are incapable of efficiently computing the parity
function, highlighting the limitations of shallow circuits in terms of computational power.

In the next lecture, we will continue our exploration of circuit complexity, looking into other important lower bounds and separations between
circuit classes.

Lecture 37: Parity Not in AC0 - II

1. Review of Previous Lecture

In the previous lecture, we explored Razborov's and Smolensky's proofs showing that the parity function is not in the class AC0. We discussed
how Fourier analysis and polynomial representations were used to establish that constant-depth circuits cannot efficiently compute the parity
function. Specifically, Razborov’s proof relied on Fourier analysis to show that the parity function requires a large number of terms in its Fourier
expansion, making it impossible for AC0 circuits to compute it in constant depth. Smolensky extended this result to AC0[p], proving that parity is
not computable by constant-depth circuits with modulo p gates.

In this lecture, we will delve deeper into the complexity of the parity function and explore further aspects of the Razborov-Smolensky proof, as
well as their implications for circuit complexity theory.

2. Summary of the Parity Function

Recall that the parity function outputs the XOR of all the input bits, i.e.,
n
Parity(x1 , x2 , … , xn ) = ⨁ xi
​ ​ ​ ​ ​

i=1

This function is a natural Boolean function that is simple to define but surprisingly difficult to compute using shallow circuits. The primary result
in our discussion is that parity cannot be computed by constant-depth circuits with polynomial size. This leads us to study the circuit
complexity of the parity function in detail.

3. In-Depth Exploration of Razborov’s Proof

Razborov's proof that parity is not in AC0 relies on Fourier analysis and communication complexity. To better understand the depth of this
result, we will discuss the key steps in his proof, which are crucial for the broader understanding of lower bounds in circuit complexity.

3.1. Fourier Analysis of Boolean Functions

The idea behind Fourier analysis in Boolean functions is to express a Boolean function as a sum of Fourier basis functions, each of which
corresponds to a particular subset of the input variables. A general Boolean function f (x1 , x2 , … , xn ) can be expressed as:
​ ​ ​

69/77
 f (x1 , … , xn ) = ∑ f^(S)χS (x1 , … , xn )
​ ​ ​ ​ ​ ​ ​

S⊆[n]

where χS (x1 , … , xn )
​ ​ ​
= ∏i∈S xi and f^(S) are the Fourier coefficients of f . The Fourier coefficients f^(S) give us the weight of each term in
​ ​ ​ ​

the function’s Fourier expansion.

The parity function has a particularly unique Fourier expansion. Its Fourier coefficients are large for many subsets of variables, which implies
that the function requires a high degree of influence from many input bits. Razborov showed that the Fourier coefficients of the parity function
decay too slowly to be efficiently captured by constant-depth circuits. Specifically, the parity function’s influence on the input bits is large
enough that the Fourier expansion cannot be represented using only a small number of terms, which is what constant-depth circuits would
require.

3.2. Lower Bound via Communication Complexity

Razborov used the communication complexity approach to argue that any constant-depth circuit that computes the parity function requires a
large number of gates. The communication complexity approach models the computation of the parity function as a communication game
between two players, where each player has part of the input, and they must communicate to compute the result. The complexity of the game is
related to how hard it is to compute the parity function using a constant-depth circuit.

Razborov demonstrated that the communication complexity of computing the parity function is large, implying that no small AC0 circuit can
compute it efficiently.

4. Smolensky’s Extension to AC0[p]

Smolensky extended the result to the class AC0[p], which includes constant-depth circuits with unbounded fan-in gates that compute modulo p
operations. He showed that parity is also not in AC0[p] for any prime p.

4.1. Polynomial Representations of Boolean Functions

Smolensky’s proof used polynomial representations of Boolean functions. By using algebraic techniques, he showed that parity cannot be
represented as a low-degree polynomial over finite fields when restricted to a constant number of layers of gates, even if the gates are allowed
to compute modulo p operations. Specifically, Smolensky showed that parity requires a polynomial of high degree, which AC0[p] circuits cannot
efficiently compute because they are limited to constant-depth circuits.

4.2. Impact of Modulo Gates

The key difference between AC0 and AC0[p] lies in the operations performed by the gates. While AC0 allows only AND, OR, and NOT gates,
AC0[p] allows gates that compute modulo p operations. However, even with the enhanced expressiveness of modulo gates, the parity function
still cannot be computed efficiently by constant-depth circuits. This result significantly extends the limitations of AC0 circuits.

5. Implications of the Results

Razborov and Smolensky’s results have profound implications for the theory of circuit complexity:

1. Separation of Circuit Classes: These results provide a clear separation between the computational power of different circuit classes. The
parity function, despite being a simple Boolean function, cannot be computed by circuits with constant depth, even with the addition of
modulo gates. This illustrates the limitations of shallow circuits.

2. Lower Bounds in Circuit Complexity: The proof provides a lower bound for the size of any circuit computing the parity function. This lower
bound is crucial in understanding the power of constant-depth circuits and circuit complexity more generally.

3. Focus on Fourier Analysis: Fourier analysis has become an essential tool in the study of Boolean functions and circuit complexity. The fact
that Razborov used Fourier analysis to prove the result highlights the importance of this method in proving lower bounds.

4. Wider Applications: These results also serve as a stepping stone for proving similar lower bounds for other Boolean functions, leading to a
deeper understanding of the computational power of circuits with bounded depth.

6. Conclusion

70/77
In this lecture, we continued our exploration of Razborov’s and Smolensky’s results showing that the parity function is not in AC0. We
discussed in greater detail the Fourier analysis and communication complexity techniques used in Razborov’s proof, as well as Smolensky’s
extension to AC0[p] circuits. These results provide a significant insight into the limitations of constant-depth circuits and have wide-ranging
implications for circuit complexity theory.

In the next lecture, we will explore other important results in circuit complexity, including further separations between circuit classes and new
techniques for proving lower bounds.

Lecture 38: Circuits with Counters and Introduction to Communication Complexity

1. Circuit Lower Bounds: ACC0

In this lecture, we begin with a discussion of circuit lower bounds specifically focusing on ACC0 circuits. These are circuits in the class AC0, but
with additional functionality that includes counters. The introduction of counters significantly increases the computational power of these
circuits compared to plain AC0 circuits.

1.1. Definition of ACC0 Circuits

The class ACC0 consists of circuits that can perform AND, OR, NOT, and modulo gates (with a constant number of gates) plus counters. A
counter in this context refers to a register that counts up to a certain value (modular arithmetic) and is updated during the computation. More
formally, the circuits are allowed to use gates that perform modulo p addition for some fixed prime p, and these modulo gates can be used
together with AND, OR, and NOT gates.

These circuits are still restricted to constant depth but are allowed more computational power than the simple gates of AC0. The inclusion of
counters provides more flexibility in counting operations and performing modular arithmetic efficiently, which is particularly important in
certain types of Boolean function computations.

1.2. Circuit Complexity of Parity

We have previously shown that the parity function is not computable in AC0. However, when we move to ACC0, things change slightly. ACC0
circuits with counters are significantly more powerful than regular AC0 circuits. In particular, parity becomes easier to compute in ACC0 circuits
because counters allow the circuit to maintain and compute a running sum modulo 2.

Even though ACC0 circuits are more powerful than AC0, they still have their limitations. The class ACC0 is still shallow compared to other circuit
classes like NC or P/poly, which can compute more complex functions, but it serves as an important intermediate class in the study of circuit
complexity.

1.3. Circuit Lower Bound Results for ACC0

Despite the added power of counters, ACC0 circuits still cannot compute all Boolean functions efficiently. For instance, the parity function and
certain majority functions can be computed by ACC0 circuits, but there are functions in AC0 and NC1 that ACC0 circuits cannot compute. These
results are typically proven by using Fourier analysis and communication complexity tools to show that certain Boolean functions require a
larger number of gates or deeper circuits than those allowed by ACC0 circuits.

In general, the lower bound results in ACC0 circuits illustrate the boundaries of constant-depth circuits and provide insight into the power of
counters in computation.

2. Introduction to Communication Complexity

The next part of the lecture introduces the communication complexity model, which is an important tool in the study of computational
problems that involve multiple parties (or players) exchanging information. In this model, the complexity of a problem is measured by the
amount of communication required between two or more parties to solve a computational problem.

2.1. The Communication Complexity Model

In the communication complexity model, we have two parties, Alice and Bob, who want to compute a function f (x, y), where x is known to
Alice and y is known to Bob. The parties communicate by sending bits to each other, and the goal is to minimize the total number of bits
exchanged while still correctly computing the function f (x, y).

The communication complexity of a function is the minimum number of bits that need to be exchanged in the worst-case scenario to compute
the function. This can be represented as a function of the input size, and the goal is to develop lower bounds for the communication complexity
of various problems.

71/77
2.2. Types of Communication Protocols

There are several types of communication protocols used in communication complexity, which vary in terms of the number of rounds of
communication, the number of bits exchanged in each round, and the type of interaction allowed. Some of the most common types of protocols
include:

1. Deterministic Protocols: In this model, Alice and Bob follow a predetermined strategy to compute f (x, y). This strategy is fixed in advance
and does not change based on the communication.

2. Probabilistic Protocols: In this case, Alice and Bob use randomness in their communication to compute f (x, y). Probabilistic protocols
typically have a probability of error, but this can be reduced by repeating the protocol a certain number of times.

3. Non-Deterministic Protocols: These allow the parties to compute the function using non-deterministic steps. Non-deterministic
communication complexity is generally harder to analyze than deterministic or probabilistic communication complexity.

4. Interactive Protocols: These protocols allow for multiple rounds of communication, with each round allowing the parties to exchange bits.
They can be more flexible than single-round protocols and are often used for problems requiring more complex computations.

2.3. Example: Parity Function

To provide a concrete example, let us consider the parity function in the context of communication complexity. Suppose Alice knows the value
of x = (x1 , x2 , … , xn ) and Bob knows the value of y = (y1 , y2 , … , ym ). The goal is to compute the parity of the entire combined string
​ ​ ​ ​ ​ ​

(x1 , x2 , … , xn , y1 , y2 , … , ym ), i.e., the XOR of all the bits.

​ ​ ​ ​ ​ ​

The communication complexity of the parity function involves analyzing how many bits Alice and Bob must exchange to compute the XOR of the
entire input. It turns out that computing parity requires a significant amount of communication if the bits are distributed between Alice and Bob.
The exact number of bits exchanged depends on the protocol used, but the key takeaway is that even simple functions like parity can require a
substantial amount of communication when the input is split between two parties.

2.4. Lower Bound on Communication Complexity

One of the fundamental goals of communication complexity is to establish lower bounds for the number of bits that must be exchanged. For
instance, parity has a well-established lower bound in the communication complexity model, and it requires a significant number of rounds or
bits to compute when the input is divided between Alice and Bob.

In general, the communication complexity of Boolean functions often provides insight into their circuit complexity. Many lower bounds for
circuit complexity can be translated into communication complexity results, and vice versa.

3. Applications of Communication Complexity

Communication complexity is not just an abstract tool—it has real-world applications in fields such as:

1. Distributed Computing: In distributed systems, multiple machines may need to communicate to solve a problem. Communication
complexity helps in designing algorithms that minimize the communication cost between nodes.

2. Cryptography: In cryptographic protocols, secure communication between parties is a crucial aspect. Understanding the communication
complexity of various cryptographic protocols helps in minimizing the amount of data that needs to be exchanged.

3. Parallel Computation: Communication complexity is also useful in designing parallel algorithms, where processes need to exchange data to
achieve a goal efficiently.

4. Conclusion

In this lecture, we explored ACC0 circuits, which are a class of constant-depth circuits enhanced with counters, and discussed their
computational power compared to AC0 circuits. We also introduced the concept of communication complexity, a model that focuses on the
amount of communication needed between parties to compute a function. We examined how the parity function serves as a useful example for
understanding communication complexity.

In the next lecture, we will delve further into complexity classes and explore additional results in communication complexity, along with their
implications for circuit complexity and distributed computing.

72/77
Lecture 39: Communication Complexity - I

1. Introduction to Communication Complexity

In this lecture, we begin with an exploration of communication complexity using the fooling set method. Communication complexity is a
powerful tool for analyzing problems that involve multiple parties (Alice and Bob) who need to communicate in order to compute a function
f (x, y), where x is known to Alice and y is known to Bob.
We begin by considering the Equality Problem (EQ), which asks whether two strings x and y are identical:

EQ(x, y) = {
1 if x = y
0 if x 
=y
​ ​

The goal of this lecture is to show a communication lower bound for the Equality Problem (EQ) using the fooling set method, which will allow
us to understand the minimum number of bits that must be exchanged to solve the EQ problem.

2. The Fooling Set Method

The fooling set method is a technique used to prove lower bounds on communication complexity. The idea is to define a set of input pairs that
"fool" any protocol that attempts to solve a problem by trying to compute the function f (x, y). Specifically, we show that no protocol can
compute the function efficiently unless it can handle all pairs in the fooling set correctly.

2.1. Definition of a Fooling Set

A fooling set for a communication protocol is a set of input pairs (x1 , y1 ), (x2 , y2 ), … , (xk , yk ) such that the following condition holds:
​ ​ ​ ​ ​ ​

For any pair (xi , yi ) in the fooling set, the protocol must send at least one bit of communication.
​ ​

Furthermore, for any two distinct pairs (xi , yi ) and (xj , yj ), the communication protocol cannot simultaneously give the same answer for
​ ​ ​ ​

both pairs unless it can distinguish them correctly.

The fooling set essentially captures the "difficulty" of the communication problem by emphasizing that, in order to solve the problem, the
parties must distinguish between certain input pairs correctly.

3. Equality Problem and the Fooling Set

Let us now apply the fooling set method to the Equality Problem (EQ). The task is to determine whether two input strings x and y are identical.

3.1. Fooling Set for the Equality Problem

To create a fooling set for the EQ problem, we consider the following input pairs:

S = {(x, y) : x =
 y and ∣x∣ = ∣y∣ = n}

In other words, the set S contains all pairs of strings x and y of length n where x  y . A fooling set for EQ must ensure that for any pair (x, y),
=
a communication protocol must exchange enough information to distinguish between all possible pairs (x, y).

3.2. Size of the Fooling Set

The set S consists of pairs of strings of length n, and since x  y , the number of such distinct pairs is at least 2n−1 . This gives a lower bound on
=
the size of the fooling set. Intuitively, if Alice and Bob were to use a protocol that requires less than n bits of communication, they would not be
able to distinguish between all the pairs in this fooling set. Hence, any protocol solving the EQ problem must communicate at least n bits in the
worst case.

Therefore, the communication complexity of the Equality Problem (EQ) is at least n, meaning that the minimum number of bits required to
solve the EQ problem is n. This gives the lower bound:

C(EQ) ≥ n

This result shows that n bits of communication are necessary to determine whether two strings are equal, thus proving a communication lower
bound for the EQ problem.

73/77
4. Conclusion

In this lecture, we applied the fooling set method to prove that the communication complexity of the Equality Problem (EQ) is at least n. This
method allowed us to construct a fooling set that demonstrates the need for n bits of communication in the worst case.

In the next lecture, we will explore additional communication problems and apply similar techniques to derive lower bounds for their
communication complexity. We will also discuss further applications of the fooling set method and other tools in communication complexity.

Lecture 40: Communication Complexity - II

1. Introduction to the Tiling Method

In this lecture, we continue our exploration of communication complexity by introducing the tiling method, a technique that provides another
approach for proving lower bounds on the communication complexity of a function. We will see how this method is applied to various problems
and how it extends the concepts we discussed in the previous lecture.

2. Overview of Communication Complexity

As a quick recap, communication complexity measures the minimum amount of communication required for two parties, Alice and Bob, to
compute a function f (x, y) where:

Alice knows the input x

Bob knows the input y

The goal is to compute f (x, y) with minimal communication

The task is to understand how much information Alice and Bob need to exchange to solve the problem. The tiling method is a powerful
technique for proving lower bounds on the communication complexity of certain functions.

3. The Tiling Method for Lower Bounds

The tiling method is based on the idea of tiling a grid or matrix, where each tile corresponds to a communication step between Alice and Bob.
This method is particularly useful for problems where the inputs x and y have a structure that can be represented as a grid, matrix, or graph.
We will use this method to establish lower bounds for certain problems by showing that any protocol that computes the function must
communicate a certain number of bits, which can be derived from the properties of the tiling.

4. Tiling Method Applied to a Communication Problem

To illustrate the tiling method, let us consider a simple communication problem. We will show how the tiling method can be used to lower bound
the communication complexity of a function.

4.1. Tiling Problem Setup

Consider the Tiling Problem, where the goal is to determine whether a given grid of tiles can be fully tiled with the available set of tile shapes.
For simplicity, we consider a 2-dimensional grid where tiles from a given set must be placed in such a way that certain adjacency constraints are
satisfied. The problem can be formulated as:

Alice knows a portion of the grid (say, part of the rows)

Bob knows another portion of the grid (say, part of the columns)

Together, they must determine whether the grid can be tiled in accordance with the rules

74/77
4.2. Using Tiling for Lower Bound Proof

To lower bound the communication complexity of the tiling problem, we argue that any protocol for solving the problem must be able to handle
all possible placements of tiles that respect the adjacency constraints. The number of possible ways to arrange the tiles grows exponentially with
the size of the grid. Therefore, any communication protocol must have enough bits of communication to handle this combinatorial explosion of
possibilities.

We construct a tiling grid representing the possible arrangements of tiles, with each tile corresponding to a specific communication step. Each
distinct arrangement of tiles requires a different sequence of communication steps, and we show that this necessitates a large number of bits
exchanged between Alice and Bob. The communication complexity is thus lower bounded by the number of possible valid tiling arrangements.

5. Generalizing the Tiling Method

The tiling method can be generalized to a wide range of communication problems. The key idea is to represent the problem as a grid or matrix
of communication steps, where each possible configuration of the inputs x and y requires a distinct set of communication steps. By analyzing
the size of the grid and the number of possible configurations, we can derive lower bounds on the communication complexity.

5.1. Applications to Other Problems

The tiling method has been applied to a variety of problems in communication complexity, including:

Equality Problem (EQ): We saw in the previous lecture that the communication complexity of the Equality Problem can be lower bounded
using a fooling set. The tiling method provides an alternative approach to establishing this lower bound.

Set Intersection Problems: The tiling method can be used to lower bound the communication complexity of set intersection problems by
representing the sets as grids and analyzing the number of configurations that require distinct communication steps.

Matrix Multiplication: For problems like matrix multiplication, the tiling method can be applied to represent the matrix elements and the
steps required to compute the product, allowing us to derive lower bounds on the communication complexity.

6. Conclusion

In this lecture, we introduced the tiling method for proving lower bounds on communication complexity. This method is a powerful tool that
allows us to handle problems with structured inputs by representing them as grids or matrices, and analyzing the number of communication
steps required to compute the function.

We illustrated the method using the Tiling Problem and showed how it can be applied to a variety of communication problems, including the
Equality Problem and Set Intersection Problems.

In the next lecture, we will explore further applications of the tiling method and discuss more advanced techniques in communication
complexity.

Lecture 41: PCP Theorem

1. Introduction to the PCP Theorem

In this final lecture, we discuss the Probabilistically Checkable Proofs (PCP) Theorem, a central result in computational complexity that has
profound implications for both approximability and hardness of approximation. The PCP Theorem connects the complexity class NP to the
class of problems for which solutions can be verified probabilistically using small amounts of information. We will explore two perspectives of
the PCP Theorem:

Locally and probabilistically checkable proof systems (PCP as a verification model)

Hardness of approximation (PCP as a tool for proving inapproximability)

We will also look at the class PCP(r(n), q(n)), the concept of approximation algorithms for the MAX optimization problem, and the equivalence
between these two views.

75/77
2. PCP as a Locally and Probabilistically Checkable Proof System

The PCP Theorem originated from the study of verifiable proofs. It defines a model of proof systems where:

Proofs are probabilistically checkable: A verifier can check the correctness of a proof by inspecting only a few bits of the proof at random,
and the verifier is allowed to make probabilistic decisions during the checking process.

Local checks: The verifier only needs to check a small portion of the proof rather than reading it entirely, ensuring that the correctness of
the entire proof can be determined with high probability by checking just a small portion of it.

The key idea of this proof system is that instead of requiring the verifier to read the entire proof to determine if the solution is correct, the
verifier can check it by reading a constant number of random bits from the proof. The probability of correctness is high if the proof is valid, and
low if the proof is invalid.

2.1. Formulation of the PCP Class

A PCP(r(n), q(n)) system is defined as follows:

r(n): The number of random bits that the verifier reads.

q(n): The number of positions in the proof that the verifier queries (the number of bits it checks).

A problem is in PCP(r(n), q(n)) if there exists a proof system where the verifier can decide whether a solution is correct by reading at most r(n)
random bits and querying at most q(n) bits of the proof. The PCP Theorem asserts that NP problems can be transformed into a PCP system with
specific parameters.

3. PCP as Hardness of Approximation

While the PCP Theorem provides an efficient verification method for proofs, it also has deep implications for approximation problems.
Specifically, it shows that certain optimization problems are hard to approximate within a certain factor unless P = NP.

The classic example is the MAX-CUT problem. Given a graph, the goal is to partition the vertices into two sets to maximize the number of edges
between the sets. A simple approximation algorithm for this problem gives a 1/2 approximation (i.e., it guarantees at least half of the maximum
number of edges in the optimal cut).

However, the PCP Theorem implies that approximating the MAX-CUT problem within any constant factor better than 1/2 is NP-hard, unless P =
NP. This type of result can be generalized to a wide range of optimization problems.

3.1. Hardness of Approximation and PCP

The PCP Theorem provides a framework for proving the hardness of approximation. Specifically, it shows that many problems that are NP-hard
to solve exactly are also hard to approximate within certain factors. This is a critical insight in complexity theory, as it demonstrates the
limitations of approximation algorithms for NP-hard problems.

4. PCP(r(n), q(n)) and NP = PCP(log n, 1)

A central result of the PCP Theorem is that NP = PCP(log n, 1), meaning that any problem in NP can be verified using a probabilistically
checkable proof system where the verifier reads logarithmic number of random bits and queries only one bit of the proof.

This result is surprising because it shows that the entire complexity class NP, which traditionally requires polynomial time verification, can be
reduced to a probabilistic model with much fewer resources. The proof of this result involves a transformation of NP-complete problems into
problems that can be verified using the PCP framework.

5. GNI ∈ PCP(poly, 1)

The Graph Non-Isomorphism (GNI) problem is a fundamental problem in computational complexity, and it is known that GNI is in PCP(poly, 1).
This means that there exists a PCP system for the GNI problem where the verifier queries only one bit of the proof, but the number of random
bits it reads is polynomial in the size of the input.

76/77
This result is part of a broader class of problems that are in PCP(poly, 1) and indicates that even problems like GNI, which are believed to be
outside of NP (due to their difficulty in finding efficient algorithms), can still be expressed in the PCP framework.

6. Hardness of Approximation View of the PCP Theorem

The second major view of the PCP Theorem is its role in hardness of approximation. The theorem shows that certain optimization problems are
not only difficult to solve exactly but also difficult to approximate within a reasonable factor. Specifically, the theorem establishes a connection
between the PCP model and approximation hardness results for a large class of optimization problems, including MAX-CUT, MAX-3SAT, and
many others.

6.1. Hardness of Approximation Results

Using the PCP framework, we can prove that for many NP-hard problems, no approximation algorithm can perform better than a certain
approximation ratio unless P = NP. This has major implications for both algorithm design and understanding the limits of computation.

7. Equivalence of the Two Views

The two views of the PCP Theorem—verification model and hardness of approximation—are deeply connected. The theorem shows that the
existence of efficient probabilistically checkable proofs can be used to prove the hardness of approximating NP-hard problems. In fact, the
hardness of approximation results often follow directly from the properties of locally and probabilistically checkable proof systems.

This equivalence is one of the key insights of the PCP Theorem, and it has profound implications for both theoretical computer science and
practical algorithm design.

8. Conclusion

In this final lecture, we covered the PCP Theorem from two perspectives:

1. PCP as a locally and probabilistically checkable proof system: The theorem provides a way to verify solutions efficiently using only a small
number of queries to the proof.

2. PCP as a tool for proving the hardness of approximation: The theorem shows that many NP-hard problems are also hard to approximate,
with no approximation algorithm performing better than a certain factor unless P = NP.

The PCP Theorem is one of the most powerful results in computational complexity, providing both a new way to think about NP problems and a
framework for understanding the limits of approximation algorithms.

This concludes our course on computational complexity. Thank you for your attention throughout the lectures.

77/77