CGAL For MAF
CGAL For MAF
CGAL For MAF
Conference on Digital Audio Effects (DAFX-05), Madrid, Spain, September 20-22, 2005
Traditional approaches to multichannel adaptive filtering have The multichannel lattice adaptive filtering increase the computa-
focused on solutions based on transversal or finite impulse re- tional complexity but it reduces the correlation between the chan-
sponse models. In general, an adaptive filter requires a fewer nels mitigating the effect of no-unicity problem simultaneously
number of parameters to model an unknown system, thus making that improves the convergence rate. The multichannel lattice
filters attractive in many practical applications. However, the structure is formed by a cascade of a basic cell showed in Figure
lattice forms have many advantages over the direct form imple- 3. The m and n index corresponds to the order cell and the
mentation. It is well known that the backward prediction error sample time respectively and k is the reflection coefficient in the
components of lattice predictor are orthogonal. This feature has ( m,n ) position. Due to joint process estimation setup, N − 1
been used efficiently in a joint estimation setup to improve the cells are needed where m = 1, 2, K , N − 1 . p = 1, 2, K , P chan-
convergence rate of the recursive adaptive algorithms. In the x1( n )
nels suppose.
multichannel case [1][2], the straightforward use of the lattice
predictor in each channel will result in orthogonal backward pre-
diction errors for each individual channel. Besides these structures
offer simple stability monitoring and are less sensitive to finite
precision effects. Here we present an adaptation of classical con- d ( n)
jugate gradient for lattice form implementations in the multichan-
nel adaptive filtering case. xP ( n ) y( n )
DAFX-1
Proc. of the 8th Int. Conference on Digital Audio Effects (DAFX-05), Madrid, Spain, September 20-22, 2005
error is
e2 ( n )= ⎡⎣ d ( n )− y( n ) ⎤⎦
2
(5)
The conjugate gradient algorithm is a conjugate direction method
2.1. Algorithm formulation
which is obtained by choosing the successive direction vector as a
The processing equations for the mth-cell shown in Figure 3 are
conjugate version of the successive gradient obtained as method
b( m,n )=b( m −1,n −1)+ K ( m,n )f ( m−1,n )
(1) progress. The stochastic conjugate gradient techniques for single-
f ( m,n )=f ( m −1,n )+ K * ( m,n )b( m−1,n −1)
channel lattice adaptive filtering are presented in [4]. In [5] a
where f ( m,n ) and b( m,n ) are the P×1 forward and backward
novel scheme for multichannel adaptive filtering based in conju-
prediction error vectors, respectively, and K ( m,n ) is the P× P
gate gradient techniques is described.
reflection coefficient matrix of mth-cell defined as
f1( m −1,n ) f1 ( m,n )
⎡ f1( m,n ) ⎤ ⎡ b1( m,n ) ⎤
⎢ ⎥ ⎢ ⎥
f ( m , n ) ⎥ b( m,n )= ⎢ b2 ( m,n ) ⎥
k
f ( m ,n ) = ⎢ 2
11
⎢ M ⎥ ⎢ M ⎥
⎢ ⎥ ⎢ ⎥ b1 ( m −1,n ) b1 ( m,n )
⎢⎣ f P ( m,n ) ⎥⎦ ⎢⎣bP ( m,n ) ⎥⎦ −1
k
11
(2) z k
⎡ k11( m,n ) k12 ( m,n ) L k1P ( m,n ) ⎤
k 21
12
⎢ ⎥ f2 ( m −1,n ) f 2 ( m ,n )
k ( m,n ) k22 ( m,n ) L k2 P ( m,n ) ⎥ k k
K ( m,n )= ⎢ 21
12 21
⎢ M M O M ⎥
⎢ ⎥
k
( ) ( ) ( )
22
⎣⎢ P1
k m , n k P2 m , n L k PP m , n ⎦⎥
b2 ( m −1,n ) k
b2 ( m,n )
The lattice predictor is initialized as
−1
22
z
b1(1,n )= f1(1,n )= x1( n )
b2 (1,n )= f 2 (1,n )= x2 ( n ) Figure 3: Basic two channel cell.
(3)
M Using the method of steepest descent to minimize (5) [6] with
bP (1,n )= f P (1,n )= xP ( n )
respect to the lattice coefficients results in
A more compact representation of the equation (1) and the output
of system y( n ) can be obtained µ
H( n )= H( n −1)+ e* ( n )∆( n )
S (n)
2
Β( n )= A( n )B( n −1)+ K ( n )f (1,n ) (6)
(4) λ
y( n )= H H ( n )B( n )= H( n ) A( n )B( n −1)+ H( n )K ( n )f (1,n ) K ( n )=K ( n −1)+ e* ( n )∇( n )
G2 ( n)
where H ( n ) = ⎡⎣h(1,n ) h( 2,n ) L h( N ,n ) ⎤⎦
T
is joint process where µ and λ are a convergence constants. The gradients of the
estimation coefficients vector. output with respect to the lattice coefficients are defined by
Being h ( m,n ) = ⎡⎣ h1( m,n ) h2 ( m,n ) L hP ( m,n ) ⎤⎦ .
T
∂y( n ) ∂y( n )
∆( n )= , ∇( n ) = (7)
B ( n ) = ⎡⎣b(1,n ) b( 2,n ) L b( N ,n ) ⎤⎦ ∂H( n ) ∂K ( n )
T
is a vector which
contain the backward prediction coefficient of all system, and the T
and S 2 ( n )= ⎡s 2 (1,n ) s 2 ( 2,n ) L s 2 ( N ,n ) ⎤ and
A( m,n ) defined in the bottom of this page is obtained with the ⎣ ⎦
recursive development of the equation (1). I P× P is a matrix with T
G 2 ( n )= ⎡g 2 (1,n ) g 2 ( 2,n ) L g 2 ( N ,n ) ⎤ are the adaptive
ones only in a main diagonal. ⎣ ⎦
input power estimates based on the current power level at each
2.2. Conjugate Gradient Algorithm
state and are introduced in order to normalize the convergence
Assume that the desired output for the system corresponding to
constant for maintain the same adaptive time constant and misad-
Figure 2 is d ( n ) at time n; then the instantaneous mean-square justment at each stage in the lattice.
⎡ 0 P× P 0 P× P L 0 P× P 0 P× P 0 P× P ⎤
⎢ ⎥
⎢ I P× P 0 P× P L 0 P× P 0 P× P 0 P× P ⎥
⎢ ⎥
⎢ K *
( ) ( )
1, n K 2, n I P× P L 0 P× P 0 P× P 0 P× P ⎥
⎢ K * (1,n )K ( 3,n ) K * ( 2,n )K ( 3,n ) L 0 P× P 0 P× P 0 P× P ⎥
A ( m,n ) = ⎢ ⎥
⎢ M M O M M M ⎥
⎢ * ⎥
⎢ K (1,n )K ( N −3,n ) K ( 2,n )K ( N −3,n )
*
L 0 P× P 0 P× P 0 P× P ⎥
⎢ * DAFX-2 ⎥
⎢K (1,n )K ( N − 2,n ) K ( 2,n )K ( N − 2,n )
*
L I P× P 0 P× P 0 P× P ⎥
⎢ * ⎥
⎢⎣ K (1,n )K ( N −1,n ) K ( 2,n )K ( N −1,n ) L K ( N − 2,n )K ( N −1,n ) I P× P
* *
0 P× P ⎦⎥
Proc. of the 8th Int. Conference on Digital Audio Effects (DAFX-05), Madrid, Spain, September 20-22, 2005
s 2 ( m,n )=δ s 2 ( m,n −1)+(1−δ )b 2 ( m,n ) the innovation that specifies the operation of the low-pass filter in
(8)
g 2 ( m,n )=δ g 2 ( m,n −1)+(1−δ ) ⎡f 2 ( m,n )+b 2 ( m−1,n −1) ⎤ (8) and (12). The choice of the gradient in (12) is crucial. In order
⎣ ⎦
Derived (4) can be obtained to generate more than one direction vector along which to search
for the solution vector, it is proposed that the gradient be esti-
∂y ( n )
∆( n )= =B( n ) mated as the average value of the gradient using the most recent
∂H( n )
(9) coefficients.
∂y ( n ) ⎡ ∂y ( n ) ∂y ( n ) ∂y ( n ) ⎤
∇( n ) = =⎢ L ⎥
∂K ( n ) ⎣ ∂K (1,n ) ∂K ( 2,n ) ∂K ( N ,n ) ⎦
2.2.1. Stability
∂y( n )
∇( m , n ) = The computation of the steps sizes µ and λ of the equation (11)
∂K ( m,n )
It
∂A( n ) ∂K ( n )
is very expensive in a traditional conjugate gradient.
=H H ( n ) B( n −1)+ H H ( n ) f (1,n )
∂K ( m,n ) ∂K ( m,n ) ∆ kH w k ∇ kH Vk
µ= , λ= (15)
possible obtain an recursive expression for the matrix ∇ ( m,n )
w kH ( ∆ k −φk ) VkH (∇ k −ϕk )
where φk and ϕ k are the gradients evaluated by taking a unit step
∇( m −1,n )=h( m,n )f T ( m−1,n )+b( m−1,n−1)q( m−1,n )
q( m,n )=q( m −1,n )−hT ( m −1,n )K * ( m.n ) (10) from H and K respectively [7]
k k
⎡ 0 P× P ⎤
q( 0,n )= HT ( n ) ⎢ * ⎥ ∂y ( n ) ∂y ( n )
⎢⎣ K ( n ) ⎥⎦
φk = , ϕk = (16)
∂( H k − w k ) ∂( K k − Vk )
The conjugate gradient algorithm to minimize (5) with respect to In this alternate implementation the steps sizes µ and λ was
the lattice coefficients results in an expression very similar to (6). chosen to be constant value. This algorithm is so called fast conju-
gate gradient method. The setup of these steps sizes has a great
The only difference is the introduction of the update of the coeffi-
influence in the stability of general algorithm. The lattice filter is
cients from conjugated versions of the gradient obtained recur- stable if the coefficients K ( m,n ) satisfy the condition
K ( m,n ) < 1; 1≤ m≤ N
sively. The order of this recursion is N and is necessary the gradi-
(17)
ent is estimated as the average values of the some past instantane-
It is very important check for stability during the adaptation. In
ous gradients. [4], if unstable, ignore the previous update is recommended. In
this work a reset of the algorithm has been found more properly.
∆ ∇
w1 =e* 2
and V1 =e* The equation for obtain β k and ρ in (13) take it a name of your
S G2 k
developers: Fleetcher-Reeves. Another successfully variants to
for k =1,2,K, N −1 bearing in mind are those of Polak-Ribière and Hesteness-Stiefel
respectively
H k +1 = H k + µ w k , K k +1=K k + λ Vk
∇ kH+1( ∇ k +1 −∇ k )
(11)
∆ kH+1( ∆ k +1 −∆ k )
βk = , ρk = (18)
∆ k +1=α∆ k + (1−α )∆ k −1, ∇ k +1=α∇ k + (1−α )∇ k −1 (12) ∆ kH ∆ k ∇ kH ∇ k
∆ k +1 2 ∇ 2 ∆ kH+1( ∆ k +1 −∆ k ) ∇ kH+1( ∇ k +1 −∇ k )
βk = , ρ k = k +1 (13) βk = , ρk = (19)
w kH ( ∆ k +1 −∆ k ) VkH ( ∇ k +1 −∇ k )
∆k 2 ∇k 2
Instability also occurred whenever β and ρ k exceeded unity.
w k +1=∆ k +1+ β k w k , Vk +1=∇ k +1+ ρ k Vk (14) k
For a quadratic problem, when exact expression for the gradient is
end used, the objective function is minimized as the method pro-
Here the index n has been dropped in the definitions of w , V , gresses. This ensures that β k and ρ k as defined above stays less
S , G , H , K , ∆ and ∇ . Transforming the V , G , ∇ and K than or equal to unity. In practice, the CGAL update for the
weight vectors is performed at much N − 1 times. If mean square
matrices into vectors and, concatenating it adequately in one only error satisfies certain threshold level the adaptation is interrupted
vector, operations more efficiently it is possible. 0 ≤ δ ≤ 1 and and the algorithm is reinitialized wit a pure gradient step at the
start of every iteration.
0 ≤ α ≤ 1 are forgetting factor that weighting the importance of
DAFX-3
Proc. of the 8th Int. Conference on Digital Audio Effects (DAFX-05), Madrid, Spain, September 20-22, 2005
signal value
signal value
Error Error
3.1. Single-Channel Adaptive Equalizer 0 0
−1 −1
For the testing the algorithm proposed, first, a single channel
−2 −2
application equalization shown in Figure 4 it considered. Quadra- 0 500 1000 0 500 1000
time index time index
ture phase shift keying (QPSK) adaptive equalization using a 32- Received Signal Scatter Plot Equalized Signal Scatter Plot
coefficient FIR filter. Run the adaptation for 1000 iterations. 3 3
2 2
1 1
−D
Imag[x]
Imag[y]
z 0 0
x ( n) d ( n) −1 −1
−2 −2
h ( n) w ( n) −3
−2 0 2
−3
−2 0 2
y ( n) e ( n) Real[x] Real[y]
r ( n)
Figure 6: QPSK process with a traditional CG algorithm
Figure 4: Equalization (deconvolution, inverse filtering,
In−Phase Components Quadrature Components
inverse modeling). 2 2
Desired Desired
Output Output
The delay D = 16 samples. The numerator coefficients of the plant 1 1
signal value
signal value
Error Error
(channel) h ( n ) are b = exp(j*pi/4)*[-0.7 1], the denominator 0 0
Imag[y]
signal value
signal value
Error Error
0 0
0 0
−1 −1
−1 −1 −2 −2
−3 −3
−2 −2 −2 0 2 −2 0 2
0 500 1000 0 500 1000 Real[x] Real[y]
time index time index
Received Signal Scatter Plot Equalized Signal Scatter Plot
3 3 Figure 7: QPSK process with a traditional CGAL algo-
2 2 rithm described in this paper.
1 1
Imag[x]
Imag[y]
0 0
−1 −1
3.2. Multi-Channel Adaptive Echo Cancellation
−2 −2
−3 −3 The Figure 8 show the multichannel echo cancellation scenario for
−2 0 2 −2 0 2
Real[x] Real[y] stereo case P = 2 . The nonuniqueness problem arise from the fact
that the signals in the multiple channels are linearly filtered of the
Figure 5: QPSK process with a traditional GAL algo- same source: x1( n )∗h1 = x2 ( n )∗h 2 . The input to the system is a
rithm develop by Griffiths [8]. random complex sequence with values (real and imaginaries) ±1 .
The Figure 5, Figure 6 and Figure 7 showed the results of QPSK The sequence is convolutioned in the transmission room h1 and
process equalizing with a traditional GAL (Gradient Adaptive h 2 with a stereo path simulated by two random exponentially
Lattice), CG (Conjugate Gradient) and CGAL (Conjugate Gradi-
diminishing acoustic impulse response with b = randn(L,1).* exp(-
ent Adaptive Lattice) algorithms respectively. The convergence
properties are improved considerably. The average power of the ([1:L]-1)/10) of L = 16 taps. x1( n ) and x2 ( n ) sequence are be
error signal is -88 dB reached in scarcely 20 iterations. obtained. The incoming stereo signals into a receiving room
x1( n ) and x2 ( n ) are convolutioned with the acoustic impulse
response of the receiving room g1 and g 2 for obtain the desired
response d ( n ) to which a complex white noise sequence is added
of -30 dB of power. In this example the transmission and receiving
rooms are supposed identical. The threshold level is fixed in -60
DAFX-4
Proc. of the 8th Int. Conference on Digital Audio Effects (DAFX-05), Madrid, Spain, September 20-22, 2005
dB in both cases. µ =0.03, λ =0.0002, α =0.2 and δ =0.9. The where W( n )=⎡⎣ w (1,n ) w ( 2,n ) L w( N ,n ) ⎦⎤
T
are a unique
CG algorithm use the Hesteness-Stiefel method for calculus of β k vector formed by the joint estimators
w ( m,n )=⎡⎣ w1( m,n ) w2 ( m,n ) L wP ( m,n ) ⎤⎦ . The missalig-
T
and ρ k and the CGAL use the Fleetcher-Reeves method. The
reason for this setup is that proved the best performance in both ment is a measure of correspondence of the multichannel acoustic
case. The Hesteness-Stiefel method provokes, in CGAL, certain impulse response of the transmission room and the estimated
unstability, that force to restart the adaptation with major fre- filters. For CGAL this coefficients correspond to a joint estima-
quency. However offers the major performance for the CG algo- tors. For both algorithms, in the multichannel case, is possible
rithm. obtain a good echo cancellation with a poorly missaligment. For
x1( n ) CGAL algorithm that is true inclusive in the single-channel case.
mean square error
0
h1 x2 ( n )
g1
−100
h2
dB
g2
s( n ) −200 CG
CGAL
−300
0 50 100 150 200 250 300
samples
e ( n) y ( n) 15
missaligment
CG
10 CGAL
dB 5
0
transmission room receiving room
−5
0 50 100 150 200 250 300
Figure 8: Multichannel Echo Cancellation (P=2). samples
−200
CG convergence. It is required that the residual error
e( n )= d ( n )− y( n ) achieve a small enough value in a short a time
−300
CGAL
−400
0 50 100 150 200 250 300 of possible, so that the subjective effect of uncancelled echo during
samples the beginning of a conversation is kept minimum. A common speci-
missaligment
20 fication would be to achieve convergence defined as the residual
CG
CGAL
echo attaining a value of less than 10% of this original value,
10 within 150 ms. The CGAL reaches a good mean square error in
the first samples with a mean value of -69 dB. This is due to decor-
dB
H( n )− W( n )
samples
ε ( n )= (20)
H( n ) Figure 11: Residual signal for CG and CGAL for MAEC
(P=5).
DAFX-5
Proc. of the 8th Int. Conference on Digital Audio Effects (DAFX-05), Madrid, Spain, September 20-22, 2005
5. REFERENCES
DAFX-6