Circuit Design

• Design Procedure
1. Step 1 (Specification): Concretely restate the goals for the design.
2. Step 2 (Strategy): Describe your strategy (often in the form of a block diagram) to achieve
your goal.
3. Step 3 (Implementation): Implement the components described in your strategy.
4. Step 4 (Verification): Check that your design from Step 3 does what you specified in Step
1.
• “Almost” Current Source (Vs + negative feedback op-amp)

Locationing and Trilateration

• Inner Product
– Commutative: h~x, ~y i = h~y , ~xi
– hc~x, ~y i = ch~x, ~y i
~ zi = h~x, ~y i + h~x, ~zi
– Linearity: h~x, y +
• Orthogonal ⇐⇒ Inner product = 0
p p
• Euclidean Norm: k~xk2 = x21 + · · · + x2n = h~x, ~xi
– Non-negativity, Zero Vector, Scalar Multiplication, Triangle Inequality
• h~x, ~y i = k~xk k~y k cos θ =⇒ |h~x, ~y i| ≤ k~xk k~y k The Cauchy-Schwarz inequality
• The projection of ~y onto ~x (with ~u = ~x / k~xk) is:
D ~x E ~x h~y , ~xi
proj~x ~y = h~y , ~ui~u = ~y , = ~x
k~xk k~xk h~x, ~xi

∞
P
• Linear cross-correlation: corr(~x, ~y )[k] = ~x[i] ~y [i − k]
i=−∞

−1
NP
• Circular cross-correlation with period N : circcorr(~x, ~y )[k] = ~x[i] ~y [(i − k)N ]
i=0
T
⇐⇒ circcorr(~x, ~y )[k] = ~x, ~y (k) = ~y (k) ~x

~x, ~y (0)
 
 ~x, ~y (1) 
Thus, circcorr(~x, ~y ) =   = C~y ~x where C~y is the matrix containing all of the shifted
 
..
 . 
~x, ~y (N −1)
versions of ~y in its rows.
• corr(~x, ~y ) 6= corr(~y , ~x), and thus, circcorr(~x, ~y ) 6= circcorr(~y , ~x)
• Autocorrelation: a correlation between a signal and itself, i.e. circcorr(~x, ~x)

Least Squares
• Error vector: ~e = A~x − ~b

1
 • Minimizing k~ek2 , the optimal solution (in least squares sense) is: ~x̃ = (AT A)−1 AT ~b

OMP
• Setup:
– Let the number of unique codes be m. Each code is length n. We represent each of the m
codes with a vector s~i where i ∈ {0, . . . , m − 1}. Each code can potentially carry a message
ai along with it. Then the measurement at the receiver is:
(τ ) (τ ) (τ
m−1 )
~y = a0~s0 0 + a1~s1 1 + · · · + am−1~sm−1

We will assume that the number of codes that are being broadcast at the same time is very
small (for example, 10). If a code is not being broadcast, it’s coefficient ai = 0. Therefore,
there are at most k non-zero ai ’s.
• Inputs:
– A set of m codes, each of length n, i.e. S = {s~0 , s~1 , . . . , sm−1
~ }
– An n-dimensional received signal vector, i.e. ~y
– The sparsity level k of the signal, i.e. the number of codes with non-zero coefficients.
– Some threshold, th. When k~ek < th, the signal is assumed to contain only noise.
• Outputs:
– A set of codes that were identified, F , which will contain at most k elements.
– A vector ~x containing the coefficients of the codes (a1 etc.), which will be of length k or
less.
– An n-dimensional residual ~e
• Procedure:
– Initialize the following values: ~e = ~y , j = 1, A = [ ], F = {∅}
– While j ≤ k and k~ek ≥ th:
1. Cross correlate ~e with each of the codes. Find the code index, i, and the shifted version
(τ )
of the code, ~si i , with which the received signal has the highest correlation value.
2. Add i to the set of code indices, F .
3. Column concatenate matrix A with the correct shifted version of the code: A =
(τ ) 
A ~si i
4. Use least squares to obtain the code coefficients: ~x = (AT A)−1 AT ~y
5. Update the residual value: ~e = ~y − A~x
6. Update the counter: j = j + 1
– Stop iterating when you’ve done k iterations (i.e. you’ve reached the pre-determined sparsity
level of the signal) or when the norm of the residual drops below the threshold th. We’ll
stop at which ever one happens first.
Extra Sanity Checks
• Correlation is not commutative.
• The inner product does not depend on the coordinate system the vectors are in, it only depends
on the relative angle between these vectors and their length.

2
• For circular cross-correlation, kC~x ~y k = kC~y ~xk
• We can’t always uniquely determine our position in an n-dimensional world using n + 1 beacons
(since they might be collinear).
• For linear corr, if the signals are periodic, then they continue on to +∞ and −∞, but we’ll just
perform the linear correlation over one period, and realize that the signals outside of this period
is periodic. For circular correlation, we look at circular shifts of one period of the signal.
• Least squares is used for solving an overdetermined system and approximating a solution.
• In order to use least squares, we need the variable to be linear in the unknowns.
• If the column vectors of A are linearly dependent, then there could be multiple solutions to the
least squares problem.
• In linear least squares, the error vector is orthogonal to Col(A), but it’s not the case for any
arbitrary cost function.
• (Disc) For an orthonormal matrix A, we have AT A = I and AT = A−1
– If A ∈ RN ×N , then its columns form a basis for RN (Prove linear independence and thus
invertible).
– If A ∈ RN ×M has linearly indpendent columns. The vector ~b ∈ RN is not in the subspace
spanned by the columns of A. Then, the projection of ~b onto the subspace spanned by the
~
columns of A is projCol(A)~b = b̃ = A~x̃ = A(AT A)−1 AT ~b
~
Thus, AT (~b − projCol(A)~b) = AT (~b − b̃) = ~0
~
(b̃ − ~x is the error vector, which is orthogonal to Col(A)).
– If A ∈ RN ×M with N ≥ M , then AT A = IM ×M , which then gives proj ~b = AAT ~b
Col(A)

• If given a new cost function, f (x), to be the expression we are trying to minimize.
1. Calculate the derivative.
2. Set f 0 (x) = 0 and find the critical point.
a·~b
~
Since the second derivate of f (x) is always positive, so x = a k2
k~ is the minimum of x in the 2D
case.
• For OMP to work, the each code should be “almost” orthogonal to all of the other codes and
to all the shifted versions of itself. (Orthogonal means that the inner product of two codes is
exactly zero. “Almost” orthogonal means that the magnitude of the inner product is close to
zero.) The “gold codes” that we looked at in the homework are good examples of codes that
fulfill these properties and would work well for OMP.
• OMP: solving underdetermined A~x = ~b (more unknowns than equations) with sparse ~x.
• Matrix-matrix multiplication can be handled in blocks! Care: restrictions on the dimensions of
the matrices, i.e. W ∈ Rn×m , Y ∈ Rm×l , X ∈ Rn×p , Z ∈ Rp×l . User beware: Remember that
matrix-matrix multiplication does not, in general, commute.
 
  Y
=⇒ W | X = WY + XZ
Z

• Inverse matrices have the same eigenvectors and reciprocal eigenvalues.