Matrix Chain Multiplication (MCM)

Problem:
Given a chain  A1 , A2 ,..., An  of n matrices,
where for i = 0,1,…,n, matrix Ai has
dimension pi-1pi, fully parenthesize the
product A1 A2 ... An in a way that
minimizes the number of scalar
multiplications.

Paris 1
 Matrix Chain Multiplication
• Suppose we have a sequence or chain
A1, A2, …, An of n matrices to be
multiplied
– That is, we want to compute the product
A1A2…An

• There are many possible ways

(parenthesizations) to compute the
product
Paris 2
Matrix Chain Multiplication …contd

• Example: consider the chain A1, A2, A3, A4 of

4 matrices
– Let us compute the product A1A2A3A4
• There are 5 possible ways:
1. (A1(A2(A3A4)))
2. (A1((A2A3)A4))
3. ((A1A2)(A3A4))
4. ((A1(A2A3))A4)
5. (((A1A2)A3)A4)
Paris 3
Background: Matrix Multiplication
• To compute the number of scalar
multiplications necessary, we must know:
– Algorithm to multiply two matrices
– Matrix dimensions

• Let A be ap q matrix
B be a q r matrix.
Then the complexity is
p q r
Paris 4
 Algorithm: Multiply 2 Matrices
Input: Matrices Ap×q and Bq×r (with dimensions p×q and q×r)
Result: Matrix Cp×r resulting from the product A·B

MATRIX-MULTIPLY(Ap×q , Bq×r)
1. for i ← 1 to p
2. for j ← 1 to r
3. C[i, j] ← 0
4. for k ← 1 to q
5. C[i, j] ← C[i, j] + A[i, k] · B[k, j]
6. return C
Scalar multiplication in line 5 dominates time to compute C
Number of scalar multiplications = pqr
Paris 5
Example: Complexity to Multiply

• Example: Consider three matrices A10100,

B1005, and C550
• There are 2 ways to parenthesize
– ((AB)C) = D105 · C550
• AB  10·100·5=5,000 scalar multiplications Total:
7,500
• DC  10·5·50 =2,500 scalar multiplications
– (A(BC)) = A10100 · E10050
• BC  100·5·50=25,000 scalar multiplications
• AE  10·100·50 =50,000 scalar multiplications
Total:
75,000
Paris 6
 Recall: MCM Problem

• Given a chain of matrices A1, A2, …, An, where for

i = 1, 2, …, n matrix Ai has dimensions pi-1x pi, fully

parenthesize the product A1  A2  An in a way

that minimizes the number of scalar multiplications.
A1  A2  Ai  Ai+1
 An
p0 x p1 p1 x p2 pi-1 x pi pi x pi+1 pn-1 x pn
Paris 7
 Dynamic Programming Approach
• Step 1: The structure of an optimal solution
– Let us use the notation Ai..j for the matrix that
results from the product Ai Ai+1 … Aj
– An optimal parenthesization of the product
A1A2…An splits the product between Ak and
Ak+1 for some integer k where1 ≤ k < n
– First compute matrices A1..k and Ak+1..n ; then
multiply them to get the final matrix A 1..n

Paris 8
Dynamic Programming Approach …contd

Optimal

(( A1 A2 ... Ak )( Ak 1 Ak 2 ... An ))

Combine

Paris 9
Dynamic Programming Approach …contd
• Step 2: Recursive definition of the value
of an optimal solution
– Define m[i, j] = Minimum number of scalar
multiplications necessary to compute Ai..j
Ai.. j  Ai Ai 1 .. A j

– Goal m[1, n] = Minimum cost to compute A1..n

Paris 10
Dynamic Programming Approach …contd
– Assume that the optimal parenthesization
splits the product Ai Ai+1  Aj at k (i  k <
j)
– Ai..j = (Ai Ai+1…Ak)·(Ak+1Ak+2…Aj)
pi-1pkpj

m[i, k]
= Ai…k Ak+1…jm[k+1,j] for i  k < j

Paris 11
Dynamic Programming Approach …contd
– Cost of computing Ai..j = cost of computing Ai..k +
cost of computing Ak+1..j + cost of multiplying Ai..k
and Ak+1..j

m[i, k] + m[k+1, j] + p i-1pkpj

m[i, j] = min # of min # of # of multiplications
multiplications multiplications to compute Ai…
to compute Ai…k to compute Ak+1…j kAk+1…j

– Cost of multiplying Ai..k and Ak+1..j is pi-1pk pj

– m[i, i ] = 0 for i=1,2,…,n

Paris 12
Dynamic Programming Approach …contd
– But… optimal parenthesization occurs at
one value of k among all possible i ≤ k < j
– Check all these and select the best one

0 if i=j
m[i, j ] =
min {m[i, k] + m[k+1, j ] + pi-1pk pj } if i<j
i ≤ k< j

Paris 13
Dynamic Programming Approach …contd
• To keep track of how to construct an
optimal solution, we use a table s

• s[i, j ] = value of k at which Ai Ai+1 … Aj is

split for optimal

parenthesization

Paris 14
Dynamic Programming Approach …contd
• Step 3: Computing the Optimal Costs

 Algorithm:
– First computes costs for chains of length l=1
– Then for chains of length l=2,3, … and so on
– Computes the optimal cost bottom-up

Paris 15
 Example: Step 1
• n = 5, p = (10, 5, 1, 10, 2, 10)
– [10×5]×[5×1]×[1×10]×[10×2]×[2×10]

1 2 3 4 5 1 2 3 4 5

1 1

2 2

3 3

4 4

5 5

m(i,j), i ≤ j s(i,j), i ≤ j 16
Paris
 Example: Step 2
– p = [10×5]×[5×1]×[1×10]×[10×2]×[2×10]
– m(i,i) = 0

1 2 3 4 5 1 2 3 4 5

1 0 1

2 0 2

3 0 3

4 0 4

5 0 5

Paris 17
 Example: Step 3
– p = [10×5]×[5×1]×[1×10]×[10×2]×[2×10]
– m(i,i+1) = pi-1pipi+1
– s(i,i+1) = i
1 2 3 4 5 1 2 3 4 5

1 0 50 1 1

2 0 50 2 2

3 0 20 3 3

4 0 200 4 4

5 0 5

Paris 18
 Example: Step 4
– p = [10×5]×[5×1]×[1×10]×[10×2]×[2×10]
– m(i,i+2) = min{ m(i,i) + m(i+1,i+2) + pi-1pipi+2,
m(i,i+1) + m(i+2,i+2) + pipi+1pi+2}
1 2 3 4 5 1 2 3 4 5

1 0 50 1 1

2 0 50 2 2

3 0 20 3 3

4 0 200 4 4

5 0 5

Paris 19
 Example: Step 5
– p = [10×5]×[5×1]×[1×10]×[10×2]×[2×10]
– m(2,4) = min{ m(2,2) + m(3,4) + p1p2p4,
m(2,3) + m(4,4) + p1p3p4}
– m(3,5) = …
1 2 3 4 5 1 2 3 4 5

1 0 50 150 1 1 2

2 0 50 30 2 2 2

3 0 20 40 3 3 4

4 0 200 4 4

5 0 5

Paris 20
 Example: Step 6
– p = [10×5]×[5×1]×[1×10]×[10×2]×[2×10]
– m(i,i+3) = min{ m(i,i) + m(i+1,i+3) + pi-1pipi+3,
m(i,i+1) + m(i+2,i+3) + pi-1pi+1pi+3,
m(i,i+2) + m(i+3,i+3) + pi-1pi+2pi+3}
1 2 3 4 5 1 2 3 4 5

1 0 50 150 90 1 1 2 2

2 0 50 30 90 2 2 2 2

3 0 20 40 3 3 4

4 0 200 4 4

5 0 5

Paris 21
 Example: Step 7
– p = [10×5]×[5×1]×[1×10]×[10×2]×[2×10]
– m(i,i+4) = min { m(i,i) + m(i+1,i+4) + pi-1pipi+4,
m(i,i+1) + m(i+2,i+4) + pi-1pi+1pi+4,
m(i,i+2) + m(i+3,i+4) + pi-1pi+2pi+4,
m(i,i+3) + m(i+4,i+4) + pi-1pi+3pi+4}
1 2 3 4 5 1 2 3 4 5

1 0 50 150 90 190 1 1 2 2 2

2 0 50 30 90 2 2 2 2

3 0 20 40 3 3 4

4 0 200 4 4

5 0 5

m(i,j), i ≤ j s(i,j), i ≤ j 22
Paris
Dynamic Programming Approach …contd
Algorithm: Compute the Optimal Cost
Input: Array p containing matrix dimensions
Output: Minimum-cost table m and split table s

MATRIX-CHAIN-ORDER(p)
1 n  length[p] –1
2 for i  1 to n
3 m[i, i]  0
4 for l  2 to n
5 for i  1 to n – l + 1
6 ji+l–1
7 m[i, j]  
8 for k  i to j – 1
9 q  m[i, k] + m[k+1, j]+ pi-1pkpj
10 if q < m[i, j]
11 m[i, j]  q
12 s[i, j]  k
13 return m and s
Paris 23
Dynamic Programming Approach …contd
• Step 4: Constructing an Optimal Solution

 The optimal solution can be constructed

from the split table s
– Each entry s[i, j ] = k shows where to split
the product Ai Ai+1 … Aj for the minimum cost

Paris 24
 Example: Step 1.1
• Optimal multiplication sequence
– s(1,5) = 2
▶ A15 = A12×A35
1 2 3 4 5 1 2 3 4 5

1 0 50 150 90 190 1 1 2 2 2

2 0 50 30 90 2 2 2 2

3 0 20 40 3 3 4

4 0 200 4 4

5 0 5

m(i,j), i ≤ j s(i,j), i ≤ j 25
Paris
 Example: Step 1.2
– A15 = A12×A35
– s(1,2) = 1 ▶ A12 = A11×A22
→ A15 = (A11×A22)×A35
1 2 3 4 5 1 2 3 4 5

1 0 50 150 90 190 1 1 2 2 2

2 0 50 30 90 2 2 2 2

3 0 20 40 3 3 4

4 0 200 4 4

5 0 5

Paris 26
 Example: Step 1.3
– A15 = (A11×A22)×A35
– s(3,5) = 4 ▶ A35 = A34×A55
→ A15 = (A11×A22)×(A34×A55)
1 2 3 4 5 1 2 3 4 5

1 0 50 150 90 190 1 1 2 2 2

2 0 50 30 90 2 2 2 2

3 0 20 40 3 3 4

4 0 200 4 4

5 0 5

Paris 27
 Example: Step 1.4
– A15 = (A11×A22)×(A34×A55)
– s(3,4) = 3 ▶ A34 = A33×A44
→ A15 = (A11×A22)×((A33×A44)×A55)
1 2 3 4 5 1 2 3 4 5

1 0 50 150 90 190 1 1 2 2 2

2 0 50 30 90 2 2 2 2

3 0 20 40 3 3 4

4 0 200 4 4

5 0 5

m(i,j), i ≤ j s(i,j), i ≤ j 28
Paris
Dynamic Programming Approach …contd

Algorithm: Constructing an Optimal Solution

Paris 29
 Practice Example (Book: Cormen; Page-376)

• Show how to multiply A1 30 35  p0 p1

this matrix chain
optimally
A2 35 15  p1 p2
A3 15 5  p2 p3
• Solution: A4 5 10  p3 p4
– Minimum cost 15,125
– Optimal parenthesization A5 10 20  p4 p5
((A1(A2A3))((A4 A5)A6)) A6 20 25  p5 p6

Paris 30
The m and s table computed by
MATRIX-CHAIN-ORDER

Paris 31
Practice Example: Solution Sample

m[2,5]=
min{
m[2,2]+m[3,5]+p1p2p5=0+2500+351520=13000,
m[2,3]+m[4,5]+p1p3p5=2625+1000+35520=7125,
m[2,4]+m[5,5]+p1p4p5=4375+0+351020=11374
}
=7125

Paris 32
Thank You

Paris 33
Stay Safe

Paris 34