ECE 5314: Power System Operation & Control

Lecture 10: Power System State Estimation

Vassilis Kekatos

R2 A. Gomez-Exposito, A. J. Conejo, C. Canizares, Electric Energy Systems: Analysis and

Operation, Chapter 4.

R1 A. J. Wood, B. F. Wollenberg, and G. B. Sheble, Power Generation, Operation, and Control,

Wiley, 2014, Chapter 9.

Lecture 11 V. Kekatos 1
Power system monitoring

Critical for

• situational awareness

• contingency analysis

• load forecasting

• economic operations

• billing

Lecture 11 V. Kekatos 2
Power system state estimation (PSSE)

Problem: given meter readings and grid parameters, find actual state v

Figure: Left: actual state. Right: Measurements in red; use only {P12 , P32 } to find
state (θ̂1 , θ̂2 ) via PF equations; estimated quantities in blue.

Why PSSE and not PF?

1. measurements are noisy
2. electric quantities are not measured at all grid locations
Lecture 11 V. Kekatos 3
Measurement model

• System state: vector of voltages v in polar or rectangular coordinates

zm = hm (v) + m , m = 1, . . . , M

• Function hm (v) can be linear or non-linear

• M : number of measurements

• m noise of m-th measurement

• Zero-injection buses handled via constraints pn = qn = 0 or In = 0

• If actual measurements are not enough, operators may use

pseudo-measurements (scheduled generation, historical loads)

Lecture 11 V. Kekatos 4
Introduction to estimation theory

• If m is random and v deterministic, then zm = hm (v) + m is random

2
2

e.g., if m ∼ N 0, σm =⇒ zm ∼ N hm (v), σm

(zm − hm (v))2
 
1
p(zm ) = √ · exp − 2
2
2πσm 2σm

• If {m }M M
m=1 are independent, then {zm }m=1 are independent

M
Y
p(z; v) = p(zm ; v)
m=1

z is measured; hm (·)’s and σm ’s are known; and v is unknown

• For fixed v, the function p(z; v) is the likelihood of observing z

Lecture 11 V. Kekatos 5
Maximum Likelihood Estimator (MLE)

Find the v that maximizes the likelihood function for the observed z
M
Y
max p(z; v) = p(zm ; v)
v
m=1

Equivalently [why?], minimize the negative log-likelihood function

M
X
min − log p(zm ; v)
v
m=1

2
For Gaussian noise m ∼ N (0, σm ), the MLE becomes
M
X 1
v̂ = arg min 2
(zm − hm (v))2
v
m=1
2σm

First-order optimality conditions

M
X
∇v log p(zm ; v) = 0
m=1

Lecture 11 V. Kekatos 6
Weighted least-squares (WLS)

• Statistical formulation of PSSE by seminal work of Schweppe

• Linear or non-linear weighted least-squares problem

M
X 1
v̂WLS = arg min 2
(zm − hm (v))2
v
m=1
2σm

• Non-convex problem unless measurement model is linear

v̂ := arg min kz − h(v)k2

v

>
with h(v) := [h1 (v) . . . hM (v)]

zm hm (v)
• Let us ignore weights wlog; replace zm → σm
and hm (v) → σm

F. C. Schweppe et al, “Power System Static-State Estimation, Part I-III,” IEEE Trans. Power
Apparatus and Systems, Vol. PAS-89, No. 1, Jan. 1970.
Lecture 11 V. Kekatos 7
Phasor measurement units (PMUs)

PMUs are linear functions of the state v (if state in rectangular coordinates)!

• Measurement of voltage at bus n

zk = Vn + k = e>
n v + k

• Measurement of line current

bc
Imn = ymn (Vm − Vn ) + j mn
2
Vm
zk = h>
k v + k

P
• Measurement of injection current Im = n∼m Imn

zk = h>
k v + k

• Vectors hk ’s are known

A. Phadke – J. Thorp, ‘Synchronized phasor measurements and their applications.’ Springer, 2008.

Lecture 11 V. Kekatos 8
Linear WLS estimator
• Collect measurement vectors hk ’s as rows of matrix H

z = Hv + 

complex- or real-valued model H is CM ×N or R2M ×2N

• MLE is the (Weighted) Least-Squares estimator found by solving

min kz − Hvk22
v

(unconstrained) convex quadratic program

• Minimizer is unique if H is full column-rank

v̂WLS = (H> H)−1 Hz

need at least M ≥ N ; typically M ∼ 1.7 − 2.3N

Lecture 11 V. Kekatos 9
SCADA measurements

Supervisory Control And Data Acquisition (SCADA) system measurements:

– line power flows {Pmn , Qmn }

– line current magnitudes {|Imn |}

– bus voltage magnitudes {Vn }

– bus power injections {pn , qn }

• Functions hm ’s for SCADA are nonlinear; quadratic if v in Cartesian

• MLE on SCADA data yields a non-linear WLS fit (non-convex problem)

v̂WLS := arg min kz − h(v)k22

v

Lecture 11 V. Kekatos 10
Gauss-Newton method

• Applies to non-linear LS; do not confuse with Newton-Raphson’s method

• Approximate h(v) with its first-order Taylor’s series expansion at vk

h(v) ' ĥk (v) = h(vk ) + Jk (v − vk ), Jk : Jacobian matrix

• Gauss-Newton updates: vk+1 = arg minv kz − ĥk (v)k22

• Since ĥk (v) is linear, GN solves a sequence of linear LS problems

vk+1 = vk + (J>
k Jk )
−1 >
Jk (z − h(vk ))

• GN method converges to a local minimum

Lecture 11 V. Kekatos 11
Solvers using QR decomposition*

Inverting J>
k Jk is sensitive to its condition number

Solvers based on the QR decomposition are numerically more robust

 
Rk
• QR decomposition: Jk = Qk R̃k = Qk  
0
for orthogonal Q>
k Qk = IM and upper triangular Rk

• Find the minimizer of

vk+1 = arg min kzk − Jk vk22 = kQ> 2

k zk − R̃k vk2
v

by solving a linear system with upper triangular form

• Upper part of error vector can be made zero

• Same complexity (due to QR), yet better numerical properties

Lecture 11 V. Kekatos 12
Semidefinite program relaxation

• As in PF and OPF, introduce V = vvH

• Measurements are quadratic functions of v; linear functions of V

hm (V) = Tr(Mm V) + m , m = 1, . . . , M

• Express PSSE using SDP variable

M
X
min (zm − Tr(Mm V))2
V
m=1

s.to V  0, rank(V) = 1

• Drop rank constraint to get a convex problem

H. Zhu and G. B. Giannakis, “Power system nonlinear state estimation using distributed
semidefinite programming,” IEEE J. of Sel. Topics in Signal Processing, Vol. 8, No. 6, Dec. 2014.

Lecture 11 V. Kekatos 13
More on semidefinite program relaxation

• Problem transformed to SDP via epigraph trick and Schur’s complement

M
X
min tm
V,t
m=1
 
tm zm − Tr(Mm V)
s.to    0, m = 1, . . . , M
zm − Tr(Mm V) 1

V0

• Relaxation is NOT exact for noisy measurements

• Heuristics based on minimizer V̂ can give good state estimates

– Select v̂ as eigenvector corresponding to largest eigenvalue of V̂
– Draw random states v̂ ∼ CN (0, V̂) and keep the one with the smallest fit

R. Madani, A. Ashraphijuo, J. Lavaei, and R. Baldick, “Power System State Estimation with a
Limited Number of Measurements,” in Proc. IEEE Conf. on Decision and Control, Dec. 2016.

Lecture 11 V. Kekatos 14
Fast decoupled solver

To avoid evaluating Jk and inverting J>

k Jk

• resort to similar tricks as in power flow solvers

• decoupling between P − θ and Q − V

• approximate (J>
k Jk )
−1
at flat voltage profile v = 1 + j0

A. Abur and A. Gomez-Exposito, “Power system state estimation: theory and implementation,”
CRC Press, 2004.

Lecture 11 V. Kekatos 15
SCADA vs. PMU measurements

SCADA:

• readings every 4 secs

• no grid-wide angles due to lack of timing {|Vm |, Pm , Qm , Pmn , Qmn , Imn }

• non-linear model: z = h(v) + 

• MLE via non-convex problem: v̂ = arg minv kz − h(v)k22

PMUs:

• 30 readings per second

• time-synchronized via GPS signaling: complex currents and voltages

• linear model (v in rectangular coordinates): z = Hv + 

• MLE via convex problem: v̂ = arg minv kz − Hvk22 = (H> H)−1 Hz

Lecture 11 V. Kekatos 16
Circuit breakers

Devices used for protection and grid reconfiguration

CBs are zero-impedance elements

• closed CB15,16 : V15 = V16

• open CB18,19 : I18,19 = 0

Bus-branch model (left); bus section-switch model (right)

Lecture 11 V. Kekatos 17
Generalized state estimation

• Network topology processor collects circuit breaker (CB) statuses

• Topology errors easily detected, but hardly identified by PSSE

• Generalized state estimation (GSE) seeks state and grid topology

x̂ = arg min 1
2
kz − Hxk22
x

s.to Cx = 0

• State augmented to include section voltages and CB currents

• CB statutes effect linear equality constraints to ensure identifiability

• Solution via linear system

    
H> H C> x H> z
  = 
C 0 λ 0

Lecture 11 V. Kekatos 18
Observability (identifiability) analysis

Given measurement set and grid parameters, assess state identifiability. If

non-identifiable, find observable islands (maximally connected subgrids).

Q: Why bother? A: To select locations for pseudo-measurements

Observable: if different states v1 6= v2 yield different outputs h(v1 ) 6= h(v2 )

• runs online to cope with meter failures, meter delays, and grid changes

• uses the DC power flow model (pairs of active-reactive meters)

X
Pm = bmn (θm − θn )
n6=m

• numerical and topological observability

K. A. Clements, P. R. Krumpholz, and G. W. Davis, “PSSE with measurement deficiency: An

observability/measurement placement algorithm,” IEEE Trans. Power App. Syst., July 1983.
A. Monticelli, “Electric power system state estimation,” Proc. of the IEEE, Feb. 2000.

Lecture 11 V. Kekatos 19
Numerical observability

• Recall from DC grid model (state is θ):

p = A> X−1 Aθ and f = X−1 Aθ

• If measurements z = Hθ are only some pn ’s and f` ’s,

then H = SA for a wide matrix S ∈ RM ×L

• In general, system z = Hθ is identifiable iff H is full column-rank

• But, in PSSE there is an angle shift ambiguity: Aθ = A(θ + c1)

• Power system is observable iff null(H) = null(A) = {c1, c ∈ R}

• If not, for u ∈ null(H), the non-zeros of Au define unobservable lines

• Systematic removal of unobservable branches reveals observable islands

Lecture 11 V. Kekatos 20
Topological observability
• Graph-theoretic approach that studies the sparsity pattern of H rather

than the values of its entries

• Builds spanning tree (forest) by selecting branches (lines)
– a flow measurement is assigned to its branch
– an injection measurement can be assigned to an incident branch
– at the end, reject injection meas. if they have a non-selected incident
branch that does not form a loop with existing branches

• Different algorithms implement the above

[R2] Gomez-Exposito, Conejo, Canizares, Electric Energy Systems, Chapter 4.6

Lecture 11 V. Kekatos 21
Topological vs. numerical observability

• If a system is numerically observable, it is also topologically observable

• Converse does not hold

• Counter-example: shown system with x = y

• topologically is observable
• but numerically, it is not

A. Monticelli, “Electric power system state estimation,” Proc. of the IEEE, Feb. 2000.

Lecture 11 V. Kekatos 22
Bad data

Sources: time-skews, uncalibrated meters, reverse wiring

• Preprocessing (polarity and range tests)

• Assume linear model (DC, linearized AC, or PMU measurements)

z = Hx + , H ∈ RM ×N

• Least-squares estimator (LSE) solves minx 1

2
kz − Hxk22

−1
• LSE: x̂ = H> H H> z

• LSE residual: r := z − Hx̂ = P⊥ ⊥

H z = PH 

where PH := H(H> H)−1 H> is a projection matrix onto range(H) and

P⊥
H := IM − PH is a projection matrix onto range(H)
⊥

• For  ∼ N (0, I), it holds that r ∼ N (0, P⊥ ⊥

H ), rank(PH ) = M − N

Lecture 11 V. Kekatos 23
Revealing outliers

Chi-squared test (detection test)

• Because r ∼ N (0, P⊥ 2 2
H ), then krk2 ∼ χM −N

• Find α for which Pr(krk22 ≤ α) ≥ 0.95

• If krk22 > α = F −1 (0.95), declare bad data

χ2
M −N

Largest normalized residual test (LNRT) (identification test)

√
• ri / Pii ∼ N (0, 1), Pii is the i-th diagonal entry of P⊥
H

|ri |
• If maxi √ > t, measurement zi for maximizing i is deemed bad
Pii

• Remove bad datum and re-compute LSE

• LSE can be computed efficiently using the matrix inversion lemma

Lecture 11 V. Kekatos 24
Least Absolute Deviations (LAV) for robustness

• LSE is sensitive to outliers: single bad datum can deteriorate estimate

• Least-median of squares (LMS) is an NP-hard problem

x̂LMS := arg min med(zi − h>

i x)
2
x i

• Least-absolute deviations (LAV) can be expressed as a linear program

x̂LAV := arg min kz − Hxk1 ⇐⇒ min t> 1

x x,t

s.to − t ≤ z − Hx ≤ t
• LAV with SDP relaxation for robust nonlinear PSSE

min t> 1
V0,t

s.to − tm ≤ zm − Tr(Mm V) ≤ tm , m = 1, . . . , M

Lecture 11 V. Kekatos 25
Huber estimator

Huber’s and quadratic functions

Huber estimator: combination of LS and LAV

M
X
x̂H := arg min hλ (zi − h>
i x)
x
i=1

L. Mili, M. G. Cheniae, N. S. Vichare, and P. J. Rousseeuw, “Robust state estimation based on

projection statistics of power systems,” IEEE Trans. on Power Systems, Vol. 11, No. 2, May 1996.

Lecture 11 V. Kekatos 26
Reformulating Huber’s estimator

• Convex problem that can be reformulated as

min 1
2
kz − Hx − ok22 + λkok1
x,o

• Interpretation via compressed sensing and the outlier model

z = Hx +  + o

V. Kekatos and G. B. Giannakis, “Distributed Robust Power System State Estimation,” IEEE
Trans. on Power Syst., Vol. 28, No. 2, May 2013.

Lecture 11 V. Kekatos 27
Critical measurements

Measurement i is critical if once removed from measurement set, the power

system becomes unobservable

• Claim: column i of P⊥ ⊥
H is zero; hence, ri = 0 (r = PH )

|ri |
• For critical measurement: undefined √ , impossible cross-validation
Pii

• Bad data processing vulnerable to critical measurements

• Multiple corrupted readings: communication link failures, cyber-attacks

O. Kosut, L. Jia, R. J. Thomas, and L. Tong, “Malicious data attacks on the smart grid,” IEEE
Trans. on Smart Grid, Vol. 2, No. 4, Dec. 2011.
Lecture 11 V. Kekatos 28
Cyber attacks

• Numbers of cyber-incidents on power SCADA: 3 (2009), 25 (2011)

• Challenged by increased sensing and networking
• GPS spoofing, generator controls, CB tripping

• Cyber-attacks on PSSE (situational awareness, billing, trading)

z = Hx +  + a

compromised meters at non-zero entries of a

• Stealth attacks can arbitrarily mislead PSSE by a = Hv

• Deleting related rows of H deems the system unobservable

another perspective of multiple critical measurements

Y. Liu, P. Ning, and M. K. Reiter, “False data injection attacks against state estimation in electric
power grids,” ACM Trans. Info. and System Security, May 2011.

Lecture 11 V. Kekatos 29
Topics not covered

• Dynamic state estimation

• Decentralized solvers

• Attacks to energy markets

• Phasor measurement units (internal estimation algorithms)

• Measurement placement

• Estimating line and transformer parameters

Lecture 11 V. Kekatos 30