Scribd
Upload
711 views

Convex Optimization Cheatsheet

This document defines key concepts in convex optimization, including: 1) A function f is convex if its epigraph is a convex set. It provides several equivalent definitions of convexity in terms of inequalities involving f. 2) A differentiable function f is L-Lipschitz continuous if its gradient f' is L-Lipschitz, which places an upper bound on how much f' can change with respect to its inputs. 3) A differentiable function f is μ-strongly convex if it satisfies quadratic growth conditions involving its gradient and inputs, with stronger implications than just convexity.

Uploaded by

Eugenio Morocho Cayamcela
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
711 views

Convex Optimization Cheatsheet

This document defines key concepts in convex optimization, including: 1) A function f is convex if its epigraph is a convex set. It provides several equivalent definitions of convexity in terms of inequalities involving f. 2) A differentiable function f is L-Lipschitz continuous if its gradient f' is L-Lipschitz, which places an upper bound on how much f' can change with respect to its inputs. 3) A differentiable function f is μ-strongly convex if it satisfies quadratic growth conditions involving its gradient and inputs, with stronger implications than just convexity.

Uploaded by

Eugenio Morocho Cayamcela
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

CONVEX OPTIMIZATION CHEAT SEET

MARK SCHMIDT

See Nesterovs book for proofs of the below.


We say that a function f is convex if for all x and y on its domain and all 0 1 we have

f (x + (1 )y) f (x) + (1 )f (y).

If f is differentiable, equivalent definitions are that

f (y) f (x) + hf 0 (x), y xi,


hf 0 (x) f 0 (y), x y)i 0.

If f is twice-differentiable, an equivalent definition is that

2 f (x)  0.

For a differentiable convex f , the following conditions are equivalent to the condition that the gradient f 0 is
L-Lipschitz continuous:

kf 0 (x) f 0 (y)k Lkx yk


L
f (y) f (x) + hf 0 (x), y xi + kx yk2
2
1
f (y) f (x) + hf 0 (x), y xi + kf 0 (x) f 0 (y)k2
2L
hf 0 (x) f 0 (y), x yi Lkx yk2
1
hf 0 (x) f 0 (y), x yi kf 0 (x) f 0 (y)k2
L
(1 )L
f (x + (1 y)) f (x) + (1 )f (y) kx yk2
2
(1 ) 0
f (x + (1 )y) f (x) + (1 )f (y) kf (x) f 0 (y)k2 .
2L
You can define Lipschitz continuity under a different norm, and in this case the first condition becomes
kf 0 (x) f 0 (y)kq Lkx ykp where k kp and k kq are dual norms. For all the other inequalities, you replace
all instances of kx yk with kx ykp and kf 0 (x) f 0 (y)k with kf 0 (x) f 0 (y)kq .
For twice-differentiable f , any of the above are equivalent (under the Euclidean norm) to

2 f (x)  LI.

The following conditions are equivalent to the condition that a differentiable f is -strongly convex:

x 7 f (x) kxk2 is convex
2

f (y) f (x) + hf 0 (x), y xi + kx yk2
2
hf 0 (x) f 0 (y), x yi kx yk2
(1 )
f (x + (1 )y)) f (x) + (1 )f (y) kx yk2 .
2
1
The following are not equivalent to -strong convexity but are implied by it:
kf 0 (x) f 0 (y)k kx yk
1
f (y) f (x) + hf 0 (x), y xi + kf 0 (x) f 0 (y)k2
2
1 0
hf 0 (x) f 0 (y), x yi
kf (x) f 0 (y)k2 .

For a twice-differentiable f strong-convexity is equivalent to:
2 f (x)  I.
If f is -strongly convex and f 0 is L-Lipschitz continuous then we have
L 1
hf 0 (x) f 0 (y), x yi kx yk2 + kf 0 (x) f 0 (y)k2 .
+L +L

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy